Converting Data Grids from Scaled Integers to Real Values

Marine Data Literacy

Providing instruction for managing, converting, analyzing and displaying oceanographic station data, marine meteorological data, GIS-compatible marine and coastal data, and mapped remote sensing imagery

2.22 Importing "Scaled" HDF Satellite Surface Temperature Climatologies into Marine GIS: OceanColor Web

Title: Importing "Scaled" HDF Satellite Surface Temperature Climatologies into Marine GIS: OceanColor Web
Abstract: In this exercise you'll work with the highest quality monthly climatologies available for SST; related sets of chlorophyll and salinity data are covered in Exercise 2.21. These data are provided in a slightly awkward format, requiring several steps to obtain grids usable in Saga GIS.
Preliminary Reading (in OceanTeacher, unless otherwise indicated):
- N/A
Required Software:
- Saga
Other Resources:
- US NASA Ocean Color Web - Huge archive of marine optics parameters and SST from a long series of satellites, all in a non-standard HDF format that previously required laborious conversion to ASCII before use in GIS. Latest version of Saga can accommodate these data directly.
- frame_grid_liberia_0.05deg.sgrd - Dummy 0.05-degree grid for Liberia made in Exercise 2.5
Author: Murray Brown
Version: 1-xx-2014

1. Perform the exercise 2.36 Obtaining Long-Term, Satellite HDF Climatologies at OceanColor Web: Temperature, Chlorophyll, Salinity to obtain data needed in this exercise. Be sure to read over all the ancillary information so you have a good idea of what you're working with and how it was obtained.

2. Use your unzipping program to de-compress the January file to this much larger file:

A20030012014031.L3m_MC_SST_4_jan

You have to add the filename extension, HDF, yourself, as you see here:

A20030012014031.L3m_MC_SST_4_jan.hdf

In the same way, add the extension to the other files.

3. You now have these global temperature grids in the folder LIBERIA > DATA > OCEAN > COLORWEB > SST, or perhaps all 12 months.

A20030012014031.L3m_MC_SST_4_jan.hdf
A20030012014031.L3m_MC_SST_4_apr.hdf
A20030012014031.L3m_MC_SST_4_jul.hdf
A20030012014031.L3m_MC_SST_4_oct.hdf

4. Run Saga.

5. So you use this tool to load the rasters one-at-a-time. Here the first file, January, has been selected.

Uncheck SELECT MULTIPLE BANDS
Uncheck TRANSFORMATION

Then click OK.

6. The raster is provided as a "data" grid, and a "quality" grid. Check only the data grid.

Then click OK.

7. And here you see the new grid in Saga. It has been automatically renamed as you see here.

What's wrong with this name? Saga uses the same name for all months.

8. To identify the grid, change the name back to the original HDF filename.

NOTE: How to change the name? Find the name at the top of the properties panel; change it, then click APPLY at the bottom of that panel.

Here's a set of the desired filenames you can copy and paste into Saga for that purpose.

A20030012014031.L3m_MC_SST_4_jan
A20030012014031.L3m_MC_SST_4_apr
A20030012014031.L3m_MC_SST_4_jul
A20030012014031.L3m_MC_SST_4_oct

9. Here's what you see after you change the name of the first grid

Load the grids in calendar order, and rename each one as soon as it is loaded.

10. And here you see all 4 grids loaded and renamed.

11. Use ADD TO MAP to display the first grid, January. If you examine the map, you'll find pixel coordinates, not geographic coordinates.

12. We must assign geographic coordinates, so select TOOLS > PROJECTION-GEOREFERENCING > DEFINE GEOREFERENCE FOR GRIDS.

13. For GRID SYSTEM, select the current system for the HDF rasters. Then click on the ellipsis (...) to the right of GRIDS.

14. Here you see the familiar 2-sided object selector. On the left are all 4 available rasters.

15. Use the >> control to move all 4 over to the selected side.

Then click OK.

16. Now you can complete the previous window:

DEFINITION - As you see here; but read through the choices to see how they could be used
CELLSIZE - The value here is 360/8640 or 180/4320
LEFT - The WESTERNMOST LONGITUDE from the metadata, i.e. the left edge.
LOWER - The SOUTHERNMOST LATITUDE from the metadata, i.e. the bottom edge

Then click OK.

17. These 4 new rasters (at the top), in an entirely different grid system, will appear.

18. Use ADD TO MAP to see one of the grids. Look to see if the cursor position reveals reasonable longitudes and latitudes, e.g. from -180 on the left to +180 on the right, etc.

19. If the grid location appears to be OK, then rename all 4 new grids, as you see here.

You should now delete all 4 original grids (on the bottom) to open up some memory space.

20. Now select any grid, and use HISTOGRAM to see the relation between the values and the colors.

Here is the readily apparent effect of "scaling" the data. The original data with values such as 12.345 degrees, for example, have been "scaled" to integers, for example 29168. The reason is that real values (the former) require 4 bytes (32 bits), but integer values (the latter) only require 2 bytes (16 bits referred to during file loading, above).

Scaling was a popular method to save memory some years ago, but is not so widely used now. The downside is the need to "de-scale" the data prior to use, as we are about to do now.

21. Select TOOLS > GRID CALCULUS > GRID CALCULATOR.

Begin by selecting for GRID SYSTEM the new grid system just created above.

Then click on the ellipsis (...) to select the objects.

22. Find the 4 new grids on the left, "available," side.

23. Then move them to the right, "selected," side with the >> control.

Then click OK.

24. Now you can complete the settings:

RESULT - Must be set to CREATE
FORMULA - Insert this formula, taken directly from the metadata, above
NAME - Use the name shown here, to help you identify the result
TAKE FORMULA - Do not check
USE NODATA - Do not check
DATA TYPE - As shown here

Then click OK.

25. Now, check to see a new grid, named jan_descaled has been added.

26. Use ADD TO MAP to inspect the new grid, to make sure the temperature values are within the possible range (i.e. -2 to 45, depending on location).

27. If everything looks ok, you can continue by running the same tool again, as you see here. Make sure you change these items appropriately:

RESULT - Back to CREATE
FORMULA - Change the g number
NAME - Change the month name

28. Now check to see that you have these 4 new grids (bottom 4).

29. Change the names of the new (bottom) 4 grids to the long strings you see here, for certain identification later:

sst_11mu_day_jan_global_modisa_colorweb_4km_located_descaled
sst_11mu_day_apr_global_modisa_colorweb_4km_located_descaled
sst_11mu_day_jul_global_modisa_colorweb_4km_located_descaled
sst_11mu_day_oct_global_modisa_colorweb_4km_located_descaled

30. Delete the earlier (upper) 4 grids, to save space.

31. Now, you might want to save the current shape(s) or grid(s) to make sure you don't lose anything in case of problems later on. Just name them with the above filenames (or equally descriptive ones), and use a logical folder location (such as PRODUCTS > SAGA > GRIDS > COLORWEB).

32. Now the only that remains to do is to subset the grids down to the Liberia area of interest. Load the grid frame_grid_liberia_0.05deg.

33. Open TOOLS > GRID TOOLS > RESAMPLING. Then make these choices for the first subsetting:

GRID SYSTEM - System for the new grids, with 0.041667 km resolution.
GRID - Select the Jan grid
PRESERVE DATA TYPE - Check box
TARGET GRID - Select GRID

Then click OK.

34. Now you're asked which GRID to use for a target.

GRID SYSTEM - Select the 0.05º dummy grid because it is closest to 4 km
GRID - Create

35. For INTERPOLATION METHOD just pick NEAREST NEIGHBOR for now. You can experiment later.

Then click OK.

36. This new object appears in the frame_grid_liberia system.

37. You should immediately rename it, as you see here to correspond to these actual details:

LIBERIA not GLOBAL
0.05 DEGREE not 4 KM

38. And you can use ADD TO MAP to see the grid and verify that it's a subset just for Liberia.

39. Go ahead and resample the other 3 grids and rename them properly, as you see here.

40. Just to see what they look like, here they are on a tiled map group. They're shown here with an 11-color, Saga default palette, with a value range of 23-33 degrees, roughly picked by visual inspection.

JANUARY	APRIL
JULY	OCTOBER

41. Now, you might want to save the current shape(s) or grid(s) to make sure you don't lose anything in case of problems later on. Just name them with the above filenames (or equally descriptive ones), and use a logical folder location (such as PRODUCTS > SAGA > GRIDS > COLORWEB).