Hosted by "1PLs Agency"

Marine Data Literacy 2.0

Providing instruction for managing, converting, analyzing and displaying oceanographic station data, marine meteorological data, GIS-compatible marine and coastal data or model simulations, and mapped remote sensing imagery

 

 

 

 

Home > 5. Gridded Data > 5.10 IDV Grids

5.10 Basic Grid Operations, Calculations and Subsetting in IDV

  • Exercise Title:  Basic Grid Operations, Calculations and Subsetting in IDV

  • Abstract:  In this exercise you'll see the NetCDF-related functions in IDV that correspond roughly with the grid operations functions in most GIS programs, particularly Saga.  After setting up a nice example set of grids, we find out how to export a subset for use elsewhere, using time and space limits.  Then we explore a small suite of very basic operations controls in IDV that are easy to access, and a few of these are introduced here.  Specifically in this exercise you'll find out how to construct maximum, minimum and average grids from a typical series of ocean product grids (for salinity).  And from these you can create difference grids for ecological assessments.  [A much larger and hugely complicated area is also available to you if you want to write your own formulas to work on the data.  This aspect of IDV is not considered yet in MDL.]

  • Preliminary Reading (in OceanTeacher, unless otherwise indicated):

    • N/A

  • Required Software:

  • Other Resources: 

  • Author:  Murray Brown

  • Version:  9-4-2014
1.  Copy the above XML URL for use in IDV.
2.  Run IDV.  Use VIEW > PROJECTIONS > PREDEFINED to get a global map if you don't already have one.

3.  On the dashboard select DATA CHOOSERS > GENERAL > CATALOGS.  Then enter the URL you copied into the address space.  After a few seconds, IDV will show this catalog page from the Aquarius salinity data at US NASA.

The top half (salinity data products) ranges from "daily" to "3 month" aggregations.  Select the "7 day" aggregate, and click ADD SOURCE.

.
4.  When the new data source appears in the DATA SOURCES panel, right-click on it, and select PROPERTIES.
5.  Open the TIMES tab to select approximately a complete year, from the latest product going backward.

You might be surprised to see that the catalog is not really "current" because it seems to be about 6 months behind.  [Today is 8-30-2014]
6.  On the SPATIAL SUBSET tab, draw a small box anywhere on the map (to activate the control). then enter the correct coordinates for our Liberia area.

Then click OK.
7.  Now you can make these choices to see a typical data map.  It will be for the first chosen day.

Click CREATE DISPLAY to see it.
8.  Here's the salinity map for the selected day.  You can take some time to play with this, and especially use the time control to see an animation

NOTE:  Here and in all maps, the author uses the RADAR > DBZ color palette, which is the same as the RAINBOW palette in Saga.  Click on the COLOR SHADED PLAN name to see where to make that choice.

 
9.  Here's an appropriate place to cover how to export a subset like this, defined by spatial coordinates, time (and possibly also by the "x sampling," "y sampling," and "level sampling" you see above).  We have only used spatial and temporal (time) but we could have used all 5 ways to cut down huge global grids to usable areas.  Here's what to expect:
  • The spatial, temporal, x-sampling, y-sampling and "level sampling" are all honored in the new data subset, so what you see is what you get.
  • The "sampling" in the spatial subsetting refers to reduction in file-size by taking steps through the data, such as every second or every fourth value, along both axes.  Frequently these steps can be locked together so they are the same in both the X and Y direction, but this is not always required.
10.  Find the EXPORT TO NETCDF tool, as you see here.  Then click it.
11.  Here IDV is asking you to specify the way you want to see the data.  This is almost always the same way you displayed it originally.  Find that original view and click OK.

 
12.  You will asked where to save the result, and with what name.  This seems strange at first, because it says INPUT.  This is because after you save the file there, it will be the input for what happens 2 panels below here.  Just navigate to where you want the subset, and enter a good long filename so you know what it is.  Then click OK.

13.  You are given one last chance to specify the variable to put into the file.  Select SALINITY and click OK.
14.  IDV will automatically open the subset as a new data source.  You can work with the new file or the old one, depending on what you need to do next.

This new file, already saved to disk, is usually much smaller than the original, sometimes by 2 orders of magnitude, so this method give you a good way to clean up disk space and facilitate data sharing by email.

 
15.  A test with a GRIB2 file (not shown here) confirms that if IDV can read it and display a grid, then it can export a NetCDF subset.  This would seem to provide an excellent bridge between formats.

16.  Now, we'll see how to perform grid operations and calculations with this group of data maps.  We'll be working with some very basic functions, but oddly they are found very deeply imbedded in the IDV functions.  Select DATA > FORMULAS > GRIDS > TIME STEPS > AVERAGE TIME STEPS.  This function calculates an average grid for all your selected grids.

Also hidden way down there are other controls for MAXIMUM, MINIMUM and other desirable measures.
17.  When you go into these formula functions, you will be asked many questions to confirm your exact choice of data and presentation formats.  Just be patient and try to provide appropriate answers -- if you make a mistake, you can always go back and try with different choices.  Sometimes the questions seem redundant, but just keep on trying.
18.  You are asked here to specify the type of data map to use for the average grid.  Try the same PLAN VIEW you used above for the typical example data map.  Then click OK.
19.  You are asked which parameter to display.  Select SEA SURFACE SALINITY and click OK.
20.  This new data map appears, labeled AVERAGE TIME STEPS.  The band of lower salinity water along the coast is obvious.
21.  Here is the MAX OVER TIME STEPS map, made with the second TIME STEPS function.

On a map like this (if it was much larger scale) you could look for salinities that are too high for certain organisms.
22.  Here is the MIN OVER TIME STEPS map, made with the third TIME STEPS function.

On a map like this (if it was much larger scale) you could look for salinity barriers to organisms that cannot go into low salinity areas.
23.  In MDL exercises you are usually advised to view related grids using the same color palette, for accurate visual assessment.  Here are the above 3 grids (MINIMUM, AVERAGE, MAXIMUM) using the range 31.9 (the lowest low) to 36.6 (the highest high).

24.  Now, let's see how to get the difference between the high and low grids, to see the usual annual RANGE of values.  This is an important ecological concept, because there are organisms that can adapt to high or low salinities, given enough time, but they cannot survive annual fluctuations that exceed certain threshold ranges.  All these adaptability and survival concepts are covered in Physiological Ecology, an important part of the marine biologists' graduate curricula.

25.  To difference 2 grids in IDV, on the dashboard select TOOLS > FORMULAS > MISCELLANEOUS > SIMPLE DIFFERENCE A-B.
26.  Once again, select PLAN VIEWS > COLOR SHADED PLAN VIEW.

Then click OK.
27.  Now we have to select the A and B in the formula.  We're going to work with grids in the SEA SURFACE SALINITY group (i.e. the MAX and the MIN grids), so select this group on both sides.

Then click OK.
28.  Now you're asked to select the specific grids in the SALINITY group.  On the A side, select GRIDS > TIME STEPS > MAX.  And on the B side, select MIN.

Then click OK.
29.  These two windows appear in order, repeating what you did 2 panels above,  Just make the same selections and click OK both times.

NOTE:  The author doesn't understand this repetition either, so don't ask me why.
30.  Now you can see the grid of difference values between the MAX salinity and the MIN salinity.
31.  The differences range from 0.791 to 4.1.  This is not very great, as we'd expect for open ocean area.  Satellite or model data for smaller, near-shore areas could possibly show much larger ranges, especially near large rivers or some estuaries.

You can use these sorts of difference grids directly to assess ecological exposure impacts, for key parameters, like temperature, salinity and oxygen (in deeper layers).
32.  It has been noted that these grids might be improved by "smoothing".  So here's how you can do that.  Open the dashboard for the above grid and try the different SMOOTHING methods.  Here GAUSSIAN WEIGHTED has been selected.
33.  This considerably smooths out the figure, and you can see meso-scale features that might be real.
34.  But the author urges a much more conservative approach to smoothing and suggests CRESSMAN WEIGHTED.
35.  This gives you an extremely smooth map, with onshore-offshore banding of salinity values, according well with climatologies.  Circular aperture and rectangular aperture smoothing gives much the same maps.
36.  Finally, you can select FILE > SAVE AS and navigate to PRODUCTS > IDV > XIDV and save the current data display as aquarius_salinities_exercise.xidv
37.  In the steps above, we moved through the area of IDV not well covered previously, where the user can perform operations on the data well beyond simple display.  The user is invited to explore these areas on your own.  As time permits, your instructor will try to understand all that stuff and add some key exercises.  Wish me luck.

-