Importing HDF Satellite Chlorophyll Data (Operational or Climatological) into Marine GIS: Color Web

Marine Data Literacy

Providing instruction for managing, converting, analyzing and displaying oceanographic station data, marine meteorological data, GIS-compatible marine and coastal data, and mapped remote sensing imagery

2.21 Importing HDF Satellite Surface Chlorophyll and Salinity Climatologies into Marine GIS: OceanColor Web

Exercise Title: Importing HDF Satellite Surface Chlorophyll and Salinity Climatologies into Marine GIS: OceanColor Web
Abstract: In this exercise you'll visit an old favorite data source, recently resurrected due to improvements in Saga. The data include dozens of operational and experimental ocean optics products, but we focus here on chlorophyll and salinity, as examples. Loading the rasters into Saga is extremely easy now, leading to facile management and display.
Preliminary Reading (in OceanTeacher, unless otherwise indicated):
- Remote Sensing Measurements
- Marine Parameters from Remote Sensors
- Remote Sensing Product Levels - Discussion of Levels 0-4
- Self-Describing Formats - Includes HDF
Required Software:
- Saga
Other Resources:
- US NASA Ocean Color Web - Huge archive of marine optics parameters and SST from a long series of satellites, all in a non-standard HDF format that previously required laborious conversion to ASCII before use in GIS. Latest version of Saga can accommodate these data directly.
- frame_grid_liberia_0.05deg.sgrd - Dummy 0.05-degree grid for Liberia, made in 2.5 Creating Grid Templates in Saga
- frame_grid_liberia_1deg.sgrd - Dummy 1.0-degree grid for Liberia, made in 2.5 Creating Grid Templates in Saga
Author: Murray Brown
Version: 1-13-2015

1. Perform the exercise 2.36 Obtaining Long-Term, Satellite HDF Climatologies at OceanColor Web: Temperature, Chlorophyll, Salinity to obtain data needed in this exercise. Be sure to read over all the ancillary information so you have a good idea of what you're working with and how it was obtained.

2. Run Saga.

3. Select TOOLS > IMPORT/EXPORT-GDAL > GDAL: IMPORT RASTER

4. Before you select a raster to import, scroll through the huge list of formats. The list now includes 4 HDF-related items.

Saga will use the proper algorithm to import the data, based on the actual format it encounters in the ".hdf" file you select.

5. Open the tool, and for FILES, navigate to and select the JANUARY climatology file. Leave the options not checked, then click on OK.

6. Now you have this object loaded, a grid.

7. Before we can do anything with our products, notice above that when we loaded the JANUARY grid, it has a grid system that has no geographic projection, and is merely a generic 8640 x 4320 raster. [You can see that from the coordinates along the margins.]

We need to convert ("project") it to a normal WGS84 geographic map, with latitudes and longitudes.

8. But what is the geography of the raster? Go back and look at Exercise 2.21 and examine these metadata.

Right in the middle of the metadata listing are the lines that tell us what we need to know:

We will not use the cell centers, just the cell edges
The western and southern edges of the raster and of the cell at that corner are -180 and -90, respectively
The cells are 0.041666668 degrees wide and tall.

Group size = 1
Number of attributes = 65
Product Name = A20031212013151.L3m_MC_CHL_chlor_a_4km
Sensor Name = HMODISA
Sensor =
Title = HMODISA Level-3 Standard Mapped Image
Data Center =
Station Name =
Station Latitude = 0.0
Station Longitude = 0.0
Mission =
Mission Characteristics =
Sensor Characteristics =
Product Type = other
Processing Version = 2013.1
Software Name = smigen
Software Version = 4.42
Processing Time = 2013270185633000
Input Files = A20031212013151.L3b_MC_CHL.main
[3 lines removed for brevity]
Period Start Year = 2003
Period Start Day = 121
Period End Year = 2013
Period End Day = 151
Start Time = 2003121000506662
End Time = 2013152022507750
Start Year = 2003
Start Day = 121
Start Millisec = 306662
End Year = 2013
End Day = 152
End Millisec = 8707750
Start Orbit = 5268
End Orbit = 58918
Orbit = 32093
Map Projection = Equidistant Cylindrical
Latitude Units = degrees North
Longitude Units = degrees East
Northernmost Latitude = 90.0
Southernmost Latitude = -90.0
Westernmost Longitude = -180.0
Easternmost Longitude = 180.0
Latitude Step = 0.041666668
Longitude Step = 0.041666668
SW Point Latitude = -89.979164
SW Point Longitude = -179.97917
Data Bins = 14174650
Number of Lines = 4320
Number of Columns = 8640
Parameter = Chlorophyll a concentration
Measure = Mean
Units = mg m^-3
Scaling = linear
Scaling Equation = (Slope*l3m_data) + Intercept = Parameter value
Slope = 1.0
Intercept = 0.0
Data Minimum = 0.00844
Data Maximum = 99.96249
Suggested Image Scaling Minimum = 0.01
Suggested Image Scaling Maximum = 20.0
Suggested Image Scaling Type = LOG
Suggested Image Scaling Applied = No
_lastModified = 2013270185633000
Conventions = CF-1.6
institution = NASA/GSFC OBPG

9. Now find and open the TOOLS > PROJECTION > GEOREFERENCING > DEFINE GEOREFERENCE FOR GRIDS.

For GRID SYSTEM, select the just loaded HDF system
For GRIDS, select the just loaded JANUARY raster
For DEFINITION select what you see here from all the choices.
For CELLSIZE, divide 360 by 8640 and enter the result
For LEFT, enter -180
For LOWER, enter -90

Then click OK.

10. Now a new object appears, the correctly projected raster (the upper one).

11. To check on the above conversion, open the new grid in a NEW MAP and look at the coordinates around the edge. They should be the expected latitudes and longitudes on all 4 sides.

12. Now we need to look at the contents of the file. Open a HISTOGRAM for the raster. It shows a huge amount of cells with the value -32767, a typical "no data value" in many remote sensing programs.

We need to make adjustments in Saga to ignore these values.

13. We need to make several properties settings to get the best visualization of the grid:

In the properties for the raster, set NO DATA > MINIMUM and MAXIMUM to -32767.
Also, we need to get away from black/white. So also make these settings in the properties panel.
- COLORS > COUNT > 100 > OK
- COLORS > PRESET > RAINBOW > OK
- OK to close COLORS settings

Now your properties panel should look like this. You might have to set the VALUE RANGE > MINIMUM to 0, if it has not happened automatically.

The MAXIMUM is set to an unrealistic value of 99.4, but this means that Saga has found this value somewhere in the grid.

Then click APPLY.

14. Your data are beginning faintly to appear (e.g. the Black Sea has a green pixel or 2). But they aren't clear enough.

15. Make another value histogram, and you can see the problem. Nearly all the data values are <5.

16. Go back to the properties panel, and make these adjustments:

Set VALUE RANGE > MINIMUM to 0 and MAXIMUM to something way less than 99. Try 5, for example.
Then, as we usually do for datasets that are mainly very small numbers, we set MODE > LOGARITHMIC UP.

NOTE: In the original metadata, above, there is a line that says SUGGESTED IMAGE SCALING TYPE = LOG, so we are following the path recommended by the file itself.

Then click APPLY.

17. This looks much better, although perhaps 4 might be even better for MAXIMUM, and the LOGARITHMIC STRETCH FACTOR could even be 100.

These are all adjustments that we typically must make for datasets with mainly very small numbers, like nutrients. It's completely arbitrary and up to you, the scientist, to decide.

18. And here is the value histogram now.

19. At this point, you could save the new, correctly projected grid in PRODUCTS > SAGA > GRIDS > COLORWEB > CHLORO with the filename chloro_jan_globe_modisa_ocweb_saga.sgrd (or similar).

20. Now, we need to resample this global grid down to the Liberia area.

The resolution, 0.041666667 is closest to our standard 0.05-degree dummy grid (made in 2.5 Creating Grid Templates in Saga; see above link to the grid).
Select FILE > GRID > LOAD GRID and load that grid into Saga.

21. Now, select TOOLS > GRID TOOLS > RESAMPLING.

For GRID SYSTEM, select the global system and the JANUARY chlorophyll grid.
Check PRESERVE DATA TYPE.
And for TARGET GRID SYSTEM, select GRID OR GRID SYSTEM (meaning we will specify a grid).
For GRID SYSTEM, select the 0.05-degree dummy grid for Liberia (just loaded above)

Then click OK.

22. This window opens. In most cases, you can reliably use NEAREST NEIGHBOR interpolation, the simplest method. If this gives you strange results, contact the instructor.

Then click OK.

23. Now you should see this new grid, with the name "chloro_jan_globe_modisa_ocweb_saga"

24. You should immediately replace the word globe with liberia, in the filename to avoid any confusion in the future.

25. Here is the new grid, as a MAP in Saga, and as we expect it shows only Liberia. The settings for color/palette/etc. are the same as those we selected above for the global view.

And the LEGEND has been displayed by clicking the tab of the same name in the properties panel. The highest values appear to run over the top of the palette, so you need to select a better maximum value for your data. Make sure to examine many figures so you select a single maximum value that applies to all of them.

26. To capture your work, right-click on the new grid and select SAVE GRID AS

29. Navigate to PRODUCTS > SAGA > GRIDS > COLORWEB_CHL and use the filename chloro_jan_liberia_modisa_ocweb_0.05deg_saga.sgrd for the saved product.

30. Now we'll run through the steps for the JANUARY salinity grid from Aquarius.

31. Load the grid.

32. The first item contains the data you want. The second item is just a pre-made color palette, which we don't need.

33. Here's the loaded grid in Saga. Right-click on it and use ADD TO MAP to see it.

34. Examine the map margins to see that the geographic coordinates are the undesirable 0-360º system we don't want.

We would prefer to see -180 and -90 in the lower left corner, not 0 and 0.

35. Here's the DEFINE GEOREFERENCE FOR GRIDS settings to fix the problem.

36. And here are the files we have now (reading down):

Dummy 1-degree grid for Liberia that we will need for subsampling below.
Correctly georeferenced grid; note lat/lon values
Original grid (the third one)

37. To view the new grid, with ADD TO MAP, here are the initial settings (similar to those above).

38. Here's the first view of the map, and obviously we have some data value problems. The data are just beginning to show some features in the mid-latitudes.

39. As before, we need to check the value histogram to see what's wrong.

Obviously we have a very narrow data range, 30-37, for the ocean -- viewed as a whole -- which should be used as MINIMUM and MAXIMUM values in the settings.

But as stated already, views of particular regions will probably require quite different ranges for best viewing.

40. And now you have a really nice visualization of the data, showing the surprisingly prominent dominance of the Atlantic in the salinity category.

41. Before we go on to resample the global data down to the area-of-interest (Liberia) you might want to save the global grid in the SGRD format.

42. As we did before, here's the setup for resampling the data to the Liberia area.

43. As before, we select NEAREST NEIGHBOR for interpolation.

44. Before we do anything else, check the histogram of data values to confirm that the salinities offshore Liberia cover a very narrow range, about 34-36.

45. And here's the result of using those values for MINIMUM and MAXIMUM salinity, along with the LEGEND for the map.

46. You should go back now and explore all of the not-followed opportunities, and discover the best ways to use these valuable climatologies. One very important method to explore would be the calculate difference grids between any synoptic data and the respective monthly climatology, to determine if you are seeing near-normal conditions or very unusual conditions in any particular dataset.

2.21 Importing HDF Satellite Surface Chlorophyll and Salinity Climatologies into Marine GIS: OceanColor Web