Skip to content

fix restart issues that arise with time averaged fields#27

Open
mvertens wants to merge 2 commits into
noresmfrom
feature/bugfix
Open

fix restart issues that arise with time averaged fields#27
mvertens wants to merge 2 commits into
noresmfrom
feature/bugfix

Conversation

@mvertens

@mvertens mvertens commented Jun 10, 2026

Copy link
Copy Markdown

Summary

Restores bit-for-bit reproducibility across restarts by reverting the
esm_history_vars default in cime_config/namelist_definition_cism.xml
from _tavg variants back to instantaneous fields for both gris and
ais ice sheets.

Root cause

In cismwrap v8 the default esm_history_vars list was changed from
instantaneous fields (e.g., acab_applied, calving_rate, total_smb_flux)
to their time-averaged variants (acab_applied_tavg, calving_rate_tavg,
total_smb_flux_tavg, etc.). With these _tavg variants in the history
list, the model no longer reproduces bit-for-bit across a restart.

The _tavg fields are auto-generated by utils/build/generate_ncvars.py
in CISM. The codegen explicitly sets load: 0 on every generated variant,
so the _tavg accumulator arrays are never written to or read from the
CISM restart file. The per-output-file scalar (outfile%total_time) that
normalizes the accumulator at write time is also reset whenever a new
output file is opened post-restart. As a result, a continuous run and a
restart run accumulate over different windows and divide by different
total_time values, producing different history-file values at the same
wall-clock time.

The prognostic state is still bit-for-bit across restart — only the
diagnostic _tavg outputs differ — but any BFB check that compares
history files flags the run as a failure.

This was masked in earlier wrapper releases because esm_history_vars
listed only instantaneous fields, which are deterministic at the output
moment and do not depend on accumulator state.

What this PR changes

  • cime_config/namelist_definition_cism.xml: for the two
    cism_evolve_this_icesheet=".true." entries (gris and ais), drop
    the _tavg suffix from all fields. frontal_melt_rate_tavg is mapped
    to the corresponding instantaneous variable melt_rate. All other v8
    additions (per-icesheet split, the extra prognostic fields like
    velnorm, btemp, tempstag, powerlaw_c, base) are kept.
  • Added an inline comment explaining the reason for the revert.

Trade-off

The history files lose time-averaging in favour of snapshot values at the
output moment. Time-averaged diagnostics can be reconstructed post-hoc, or
reintroduced after the upstream CISM _tavg restart-save issue is fixed.

Upstream issue

The proper fix lives in CISM: both the <var>_tavg arrays
(generate_ncvars.py line 126) and the outfile%total_time scalar
(libglimmer/ncdf_template.F90.in) need to be saved in the restart file
together. That work is tracked separately.

Test plan

  • Rebuild namelists (./preview_namelists) on an existing case and
    confirm esm_history_vars in cism.<icesheet>.config no longer
    contains any _tavg entries.
  • Run the 3-year continuous + restart-at-year-2 test and verify the
    year-3 history files are bit-for-bit identical between the continuous
    and restart runs.
  • Confirm prognostic answers are unchanged relative to a pre-revert
    run that didn't trigger the BFB-restart check (i.e., the model
    trajectory itself is unaffected — only what gets written to history).

@mvertens mvertens changed the title Feature/bugfix fix restart issues that arise with time averaged fields Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant