Marine Data Literacy 2.0

Providing instruction for managing, converting, analyzing and displaying oceanographic station data, marine meteorological data, GIS-compatible marine and coastal data or model simulations, and mapped remote sensing imagery

 

 

 

 

Home > 1. PC Basics > 1.1 Folder Structure

1.1 General-Purpose Folder Structure for Data Files

  • Exercise Title:  General-Purpose Folder Structure for Data Files

  • Abstract:  To manage the many files used by these exercises and the products that are created, some sort of "standard" folder structure is required.  In this exercise you will create a "basic" folder structure, consisting of the top levels only (2 or 3 levels within the main folder).  During the exercises to follow, you will be asked to save files to folders in this structure.  Feel free to add subfolders or change the folder names, however you wish.

  • Just make sure to use long filenames that uniquely specify your files (in your own invented system, if you wish).  The smaller your folder structure, the more important long filenames become.  There's a link just below to a helpful article

  • Preliminary Reading:  N/A

  • Required Software:

    • Use Windows Explorer

  • Other Resources: 

  • Author:  Murray Brown /w help from friends

  • Version:  April 2013

1.  This exercise was radically upgraded in June 2011, due to comments provided by the students in the Ghent University's Erasmus Mundus MSc in Marine Biodiversity and Conservation, and also by the POGO IV Class at Bermuda Institute of Ocean Sciences
2.  This is the structure we recommend for creating a general-purpose data collection for any selected marine area.  We've tested various similar structures for many years, and this is the best one.  In the steps below, you'll make this structure.

 

  • PROJECTS

    • LIBERIA - Project files for this area

      • DATA

        • BASEMAP

          • BORDERS - Shapes for mapping

          • GEOLOGY - Sediment thickness, etc.

          • RELIEF- Bathymetry and topography

        • OCEAN

          • WOD- World Ocean Database

          • WOA - World Ocean Atlas

          • [OTHER SOURCES] - Separate folders identified by acronyms are optional here depending on your preferences

      • METADATA - Detailed information about any of these datasets, almost always in a carefully controlled format

      • PRODUCTS - Files created by the following main programs

        • HDFVIEW - Text exports, created by this program, from large grids in HDF or NetCDF

        • IDV - Maps and syntheses made by Integrated Data Viewer: synthesis program for many operational datasets and some GIS objects

        • ODV - Collections and their products made with Ocean Data View; our principal ocean station data management, analysis and display tool

          • COLLECTIONS - Individual collections of data by data type, area or other original selection criteria

          • IMAGES - Saved images of graphics

          • INTERPOLATIONS - XYZ data triplets exported from ODV's "surface mode" analyses; could also be called grids, because they are derived from grid-like interpolation schemes

          • SUBSETS - Spreadsheets of data taken from larger collections, based on temporal, spatial or other station or sample criteria; can also be a completely new collection extracted from the parent collection

          • TIMESERIES - Data analyses based mainly on data/time, not geographic location

        • OST - ODV spreadsheets created from SeaBird files by Ocean Sneaker's Tool

        • PANOPLY - Exactly positioned graphic extracts (as KML/KMZ) from HDF, NetCDF or GRIB grids

        • SAGA - Grids and vector products made by our "workhorse" public domain GIS program

          • AUXILIARY - Saved settings (properties files) for the display of shapes or grids, and other similar helper files

          • GEOIMAGES - Geo-referenced images, using either world files or KML/KMZ for positioning; includes also TIFF grid rasters (for WMS use only)

          • GRIDS - grids in Saga format, including template grids

          • PROJECTS - A special sort of file that contains lists of objects to place in one map, including the display properties of the objects

          • TABLES - tabular data from within shapes, and also created alongside them for display management

          • VECTORS - points, lines, contours, polygons, graticules, wind arrows, current arrows; all as either ESRI shapes or Google KML/KMZ

3.  Use Windows Explorer to create this folder structure (or similar).  The important thing is to make a PROJECTS folder, and a LIBERIA folder.  You can make folders for any other region of interest, such as PAGO_PAGO.
4.  This is a BAT file (or CMD file if your PC can't run BAT files) that will make the desired folders anywhere you want.  Copy the text, without any leading or trailing empty lines into a file name MAKEFOLDERS.BAT (or MAKEFOLDERS.CMD).  Save the file to the LIBERIA folder.

NOTE:  If you don't know anything about BAT or CMD files, then read the reference document cited above for more information. For your quick information:

  • MD XX - Means make a directory named XX
  • CD XX -  Means change to a directory named XX
  • CD .. - Means move up to the directory one level above

MD DATA
MD METADATA
MD PRODUCTS
CD DATA
MD OCEAN
MD BASEMAP
CD OCEAN
MD WOD
MD WOA
MD ADD_OTHER_DATA_FOLDERS_HERE_AS_NEEDED
CD ..
CD BASEMAP
MD BORDERS
MD GEOLOGY
MD RELIEF
CD ..
CD ..
CD PRODUCTS
MD HDFVIEW
MD IDV
MD ODV
MD PANOPLY
MD SAGA
CD ODV
MD COLLECTIONS
MD IMAGES
MD INTERPOLATIONS
MD SUBSETS
CD..
CD SAGA

MD PROPERTIES
MD AUXILIARY
MD GEOIMAGES
MD GRIDS
MD PROJECTS
MD TABLES
MD VECTORS
PAUSE
 

5.  Double-click on the BAT (or CMD) file to run it.  That will create the folders you need in current folder location.  Use Windows Explorer to examine the new folders to make sure the structure is like that in Panel 2, above.  If anything is missing, then simply add it manually with Windows Explorer.
6.  This method put the entire folder in the PROJECTS > LIBERIA folder, but of course you can move it anywhere you want. Just run it again, and you'll have a new empty folder structure there.