Bill Sacks < sacks@ucar.edu > - 2013-09-26 21:49:39 -0600
Bugzilla Id: 1823
Bugzilla CC: dlawren@ucar.edu, muszala@ucar.edu, mvertens@ucar.edu, rfisher@ucar.edu,

With the addition of lots of inactive points in memory for the sake of dynamic landunits, the size of CLM's restart files has increased significantly; e.g., for an f09 file:

old:
-rw-rw-r-- 1 sacks ncar 808M Jun 6 15:11 clmi.I1850CRUCLM45BGC.0241-01-01.0.9x1.25_g1v6_simyr1850_c130606.nc

new:
-rw-r--r-- 1 sacks ncar 2.3G Sep 26 21:03 clmi.I1850CRUCLM45BGC.0241-01-01.0.9x1.25_g1v6_simyr1850_c130926.nc

In addition, I have now had to go through a painful interpinic process twice, when simply adding 0-weight points to CLM's internal arrays.

Thus, I propose modifying the restart file format to remove a bunch of unneeded clutter - and ideally prevent having to run interpinic whenever you change the convention of which 0-weight points are included in CLM's internal arrays.

One idea would be to only write ACTIVE points to the restart file (i.e., using the active flags at the pft, col & landunit levels). This makes sense because inactive points will generally have meaningless values anyway. This would require packing arrays into temporary, active-only arrays when writing restart files. The reverse operation would need to be done when reading restart files - read into a temporary array then unpack into memory by assigning the 1st point in the restart file to the 1st active point in memory, the 2nd point in the restart file to the 2nd active point, etc. Note that this might require doing away with some of the consistency checks (e.g., that the number of pfts on the restart file matches the number computed internally based on the surface dataset) - although with some thought we might be able to maintain equivalent checks (e.g., the number of pfts on the restart file should match the number of ACTIVE pfts computed internally).

Some thought would be needed as to how (if at all) this would require changes in interpinic - though my first thought is that nothing may need to be done in interpinic to handle this change.

Presumably the size of 1-d history files (with dov2xy = .false.) has increased similarly, but I'm not sure if we want to apply these ideas to history files, because this could mess with people's ability to post-process these files.