Hosted by "1PLs Agency"

Marine Data Literacy

Proudly published in the United States of America for environmental scientists around the world.  Providing instruction for managing, converting, analyzing and displaying oceanographic station data, marine meteorological data, GIS-compatible marine and coastal data, and mapped remote sensing imagery

Up
8.5 THREDDS Servers
8.6 OPeNDAP & LAS
8.7 SOS Servers
8.8 NetCDF in Saga/IDV
8.9 WMS Servers
8.10 ncWMS Server
8.11 SST Time Series
8.12 NOAA GEO-IDE

Home > 8. Access & Services > 8.6 OPeNDAP & LAS

8.6 Data Browsing/Mining in OPeNDAP and LAS Servers

1.  Open the edcconfig.xml file in your C:\EDC folder and change <CLOSE_AFTER_PROCESSING> to false.
2.  Run EDC.  Check to see that there are 3 BROWSE controls in the upper-left corner:
  • CATALOG URL - THREDDS access; covered in exercise 8.5 in this group
  • DIRECT ACCESS URL - OPeNDAP access to data
  • SENSOR OBSERVATION SERVICE - New Open Geospatial Consortium protocol; covered in exercise 8.7 in this group

3.  Examine the list of example OPeNDAP servers at the bottom of this exercise.  The server selected here is not especially important; it is simply the first one in that list.  COPY the link and place it in the DIRECT ACCESS URL space.  Then click DIRECT ACCESS.
4.  This opens a new tab, named SUBSET & PROCESS where you can download and manage data from that URL.  This may take a few minutes in some cases
5.  Examine the material available in this tab:
  • Top Row:  Image controls for the map, such as magnify/minimize, pan, area of interest, etc.
  • Middle Row-Left:  Map for spatial display and manual subsetting
  • Middle Row-Right:  Parameter selection from the aggregation
  • Bottom Row:  Controls for temporal subsetting
6.  To begin spatial selection, right-click on AOIs, and select ENTER MANUALLY.

 
7.  Enter the coordinates for the area of interest, using signed decimal degrees.  Then click ACCEPT.
8.  The current version of EDC cannot save AOIs or recall previous choices.  So don't bother to try to select anything here.
9.  Here is the map of the AOI, provided as a check of your coordinates.
10.  Examine the list of available parameters in this dataset, and select the one(s) you want to download.

At the bottom of the area, you can select a depth ("TRIM"), if depths are offered in the dataset.
11.  At the left end of the spatial control area, you can see that the "year" is 0001, which is a NetCDF convention for long-term averages of all years.  The day value of 16 is the convention for monthly averages.

The selected begin value (green) is January.
12.  At the right end, you can see that December is the end value (red).
13.  Perform some experiments to enter values that will give you the June average grid.
14.  You can check the middle of the timeline to see if you have cut the time down to ONE TIMESTEP.  [It takes some practice to feel comfortable with the time selection tools, so it's up to you to experiment and get it to work as you want.]

When you are ready click on PROCESS.
15.  When asked what filename to use, enter airtemp_jun_liberia_coads_pmel_las (or something similar).  The file will have the extension NC, and it will always be in the EDC installation folder.
16.  This small window appears, informing you of the progress of the download.
17.  If you are successful, in some cases a new tab, named DATA VIEWER will open, with a simple map of the data.  This is a visual aid to the correctness of the data extraction, and is not a regular "data product."
18.  You can check the EDC installation folder to find these new files:
  • airtemp_jun_liberia_coads_pmel_las.nc - main data product
  • airtemp_jun_liberia_coads_pmel_las.xml - metadata
19.  Just as a final check of the data product, here it is displayed in IDV, using the standard procedures for a NetCDF grid, displayed with the rainbow palette.
20.  You can follow up with the normal file-saving or file-naming work, to make sure your work is safe and identifiable.
21.  All of the above steps are valuable to know if you want to make and keep copies of the data locally.  With the Live Access Server technology, when available, you might not need to do this.  In the steps that follow, we select one LAS and show you how to get data products there, with most of the same flexibility as IDV.
22.  Examine the list of LAS servers in the final panel.  These are active HTML links, so you can open any of them directly.  For this exercise, find and open this LAS:
  • NVODS (National Virtual Oceanographic Data System

NVODS is perhaps the largest, and certainly the most reliable of all LAS's.  It contains a huge collection of links to OPeNDAP servers with climatological and historical/synoptic marine and atmospheric data analyses; this author cannot find any operational data.  NVODS is already in use in several other MDL exercises.
23.  Open the NVODS LAS and this is the initial view.  The default dataset shown here is the same one we'll use for the steps below, so we'll wait to see what it is.

In the next few panels, we'll take a tour around the LAS screen, which is probably the most important new resource you need to know about (if you haven't seen it before).
24.  In the top left corner, you can click on DATA SET to see this catalog open up.  It has 5 top levels, but we'll only use the first one.

Open the BY DATASET NAME section.
25.  Take some time to browse through this huge catalog of named datasets.  Each item opens to show sub-items, etc. etc.  There are hundreds of datasets and parameters.

Finally, scroll to the bottom to find the latest World Ocean Atlas available here, the 2005 version. 

Click on the WORLD OCEAN ATLAS 2005 1X1 DEGREE MONTHLY MEANS
26.  And even here there are more menus, because there are several variables:
  • Apparent oxygen saturation
  • Dissolved oxygen
  • Nitrate
  • Percentage oxygen saturation
  • Phosphate
  • Salinity
  • Silicate
  • Temperature

And for each one, there are multiple statistical values, as you see here for TEMPERATURE, which has 8 different items.
27.  Select TEMPERATURE ANALYZED, and this map will appear.  It is the same as the original default map.
28.  Also in the upper left corner is this set of controls (including DATA SET that you know about).  These are the main settings for the types of maps you want to make.

Check the UPDATE PLOT control to make sure that all your selections are acted upon.
29.  Just below these are the controls for geographic area.  Right now, it's set for the whole globe.
30.  Just below these you'll see the time controls for the data.  These controls automatically change their appearance for every different dataset.
31.  And in the lower left corner is a large group of settings for the exact map you're making.  These controls also change their appearance for every different dataset.

We don't have time to go through these, and they would take days to master, so work on them later as you need to make more complex data maps.
32.  Below the main data analysis map, you'll see these additional controls for the variable, the depth displayed and the time.  All of these can be selected here from drop-down menus.  These also change automatically, depending on the data.
33.  Above the data analysis map, you'll see these controls.  They are used to select what you want to do with the map(s), such as PRINT or ANIMATE.  These are just as important as the data analysis map, maybe more so, so please go through them to see what's included.  We will come back to these important controls, below.
34.  Just above the map, you'll see this block of information, called the "Annotations." 

If you want to turn this off to save space, there is a control nearby; you can see it in Panel 28.
35.  This OPeNDAP URL, in the top row of the annotations, can be used by any OPeNDAP-enabled program (such as IDV, or EDC at the top of this exercise) for direct access to the data.  This is covered extensively in the  Operational Oceanography exercises.  You should consider a personal collection of LAS sites (such as the beginner set below) and dataset URL's for specific datasets of interest.
36.  But how do you get a data map for only a specific area?  In the geographic control area, find and click SELECT REGION WITH CLICK AND DRAG.  Then draw a rectangle approximately where you want it.
37.  It takes some practice to insert the desired values so just be patient and go slowly.  This author finds that if you use this order, it usually works:

TOP > LEFT > BOTTOM > RIGHT
38.  Eventually, this map should appear, because you have automatic map updating.
39.  But how do you get data out from the LAS to use in other software?  Just above the map, click on the SAVE AS control.

This small window appears, where you can make specific selections, such as the output format, the time, the depth, etc.  For IDV you would use NetCDF, or for Saga you would use ARCGRID, for example.

You can complete this function on your own later.
40.  How can you connect the LAS data to Google Earth (GE)?  Just above the data map, click on the GOOGLE EARTH control. 

This small window appears, where you can make all sorts of changes appropriate to GE's operations.  For now, just take the default's controls by clicking OK.
41.  When this widow opens, you should check OPEN WITH and then click OK.

NOTE:  We'll come back to SAVE FILE later.
42.  Without any further questions, the map opens in GE.  Look closely and you'll see something that was not seen in the LAS.  There are point locations at approximately 3-degree intervals.  This is a very special new capability built into the latest versions of the LAS for GE exports.
43.  To see what they do, click on any one of them and you'll see these two choices.
44.  If you click on the VERTICAL PROFILE PLOT, then GE calls this graphic from the LAS server.  It's really quite nice, could be a great tool for education.

 
45.  And here is the TIME-SERIES PLOT.  An instant ecological assessment tool!  There are weather climate datasets in NVODS, so you could assemble all kinds of local climatologies very quickly, using this route.
46.  But can you save the KML for later use, or to send to someone?  Of course; just go back to Panel 41, and select SAVE FILE.
47.  Navigate to an appropriate location and save the file with the name temp_jan_liberia_0m_woa05_nvods.kml.  You can download the KML file from right here and open it in Google Earth to see how it works for yourself.

48.  Selected OPeNDAP Servers - Mix of operational and climatological data servers.  Individual files (often *.NC) or aggregations (for example *.JNL, *.NCD).  Notice that many legitimate URLs have no specific filename, just a folder location.

If these servers are in operation, and if you have copied them correctly, they can be used as follows:

  • Hard Work:  Write a complex URL script that requests the OPeNDAP server to "send" you a specific data product.  Then use IDV for the display work.

  • Medium Work: Let Environmental Data Connector (EDC) do the download work: use DIRECT ACCESS URL.  Then use IDV for the display work.

  • Light Work: Let Integrated Data Viewer (IDV) do the download and display work:  use DATA CHOOSERS > URLs
49.  OPeNDAP Servers Catalog:  A global compilation of OPeNDAP server locations, provided by OPeNDAP.org, is found at http://docs.opendap.org/index.php/Dataset_List.  The list is poorly organized and contains many dead-ends, so don't expect to easily find a URL that can simply be inserted into IDV or EDC, as above.

50.  Selected LAS Servers - LAS setups are data subset and display programs, very similar to IDV.  They provide free and open access, without any security layer.  Sometimes only a subset of the layers available through OPeNDAP.  Nearly all data sources on a good LAS have an information icon ["?" or "i"], where you can get the OPeNDAP URLs for any LAS data menu item.  If these servers are in operation, then they can be opened directly in any good browser, i.e. Firefox.

In addition to these verified sites, the following compilation of mainly unverified sites is available:

  • LAS Servers - List of  LAS sites, world-wide and in the USA

 
The exercises, notes and graphics in this website are copyrighted, and may not be copied or abstracted in any way, without my explicit permission (in writing).  Making one copy for your personal use is allowed.   Please report any copyright infringement to me. Murray Brown m.brown.nsb <at> gmail.com