Add Python code for AMIP ancillary generation by penguian · Pull Request #141 · ACCESS-NRI/CMIP7-Input

penguian · 2026-07-03T05:37:16Z

Closes #121

See also https://github.com/ACCESS-NRI/CMIP7-Input/tree/scen7-amip
https://github.com/ACCESS-NRI/ancillary-file-science/tree/1-port-cmip7-amip-code-for-esm16-site-nci
https://code.metoffice.gov.uk/trac/roses-u/browser/d/q/8/1/9/cleanup?rev=360148
https://code.metoffice.gov.uk/trac/roses-u/browser/d/q/8/1/9/trunk?rev=360149

This pull request adds the cmip7_AM_amip_generate.py Python code for the AM_seaice_ancil_amip and AM_sst_ancil_amip tasks. This code takes the NetCDF files produced by the AM_ukesm_ancil and produces the final ancillary files. This code has also been incorporated into the https://github.com/ACCESS-NRI/CMIP7-Input/tree/scen7-amip branch, which includes all changes from https://github.com/ACCESS-NRI/CMIP7-Input/tree/127-n2-and-o3-for-esm16-h-and-vl

The code was tested by running https://code.metoffice.gov.uk/trac/roses-u/browser/d/q/8/1/9/cleanup?rev=360148 which uses the scne7-amp branch and produced the files

/g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.02/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.02/sst_amip_n96_gregorian.anc
/g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.02/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.02/seaice_amip_n96_gregorian.anc

The run logs are in /scratch/tm70/pcl851/cylc-run/u-dq819.cleanup/run10/log/job/1/AM_*_ancil_amip/

penguian · 2026-07-03T06:09:58Z

After the latest commit, I ran https://code.metoffice.gov.uk/trac/roses-u/browser/d/q/8/1/9/trunk?rev=360149 with all ancillaries other than AMIP set to false:

[pcl851@gadi-login-04 u-dq819.trunk]$ svn diff
Index: rose-suite.conf
===================================================================
--- rose-suite.conf     (revision 360149)
+++ rose-suite.conf     (working copy)
@@ -2,27 +2,27 @@
 SITE = 'nci-gadi'
 ISO_DATE_TODAY = '$(isodatetime -f CCYY.MM.DD)'
 
-ANCIL_CREATE_AEROSOL = true
+ANCIL_CREATE_AEROSOL = false
 ANCIL_CREATE_AMIP = true
-ANCIL_CREATE_CO2 = true
-ANCIL_CREATE_GHG = true
-ANCIL_CREATE_NITROGEN = true
-ANCIL_CREATE_OZONE = true
-ANCIL_CREATE_SOLAR = true
-ANCIL_CREATE_VOLCANIC = true
+ANCIL_CREATE_CO2 = false
+ANCIL_CREATE_GHG = false
+ANCIL_CREATE_NITROGEN = false
+ANCIL_CREATE_OZONE = false
+ANCIL_CREATE_SOLAR = false
+ANCIL_CREATE_VOLCANIC = false
 
 EXPS = ['AM', 'EH', 'HI', 'PI', 'PM', 'SM']
 MAIN_EXPS = ['HI', 'PI', 'SM']
 USE_EXP = {'AM': True,
-           'EH': True,
-           'HI': True,
-           'PI': True,
-           'PM': True,
-           'SM': True,}
+           'EH': False,
+           'HI': False,
+           'PI': False,
+           'PM': False,
+           'SM': False,}
 
 SCENARIOS = ['h', 'vl']
-USE_SCEN = {'h': True,
-            'vl': True,}
+USE_SCEN = {'h': False,
+            'vl': False,}
 
 TESTING = false

This produced the files

g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.03/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.03/sst_amip_n96_gregorian.anc
/g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.03/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.03/seaice_amip_n96_gregorian.anc

The logs are in /scratch/tm70/pcl851/cylc-run/u-dq819.trunk/run94/log/job/1/*AM*/01:

[pcl851@gadi-login-04 u-dq819.trunk]$ more /scratch/tm70/pcl851/cylc-run/u-dq819.trunk/run94/log/job/1/*AM*/01/job.err
::::::::::::::
/scratch/tm70/pcl851/cylc-run/u-dq819.trunk/run94/log/job/1/AM_seaice_ancil_amip/01/job.err
::::::::::::::
Using the cylc session localhost

Loading cylc/8.6.3
  Loading requirement: mosrs-setup/2.0.1
/home/851/pcl851/cylc-run/u-dq819.trunk/run94/lib/python/esm1p6_ancil/cmip7_ancil_ukesm.py:24: FutureWarning: Ignoring a datum in netCDF load for consistency with existing behaviour. In a future version of Iris, this datum will be applie
d. To apply the datum when loading, use the iris.FUTURE.datum_support flag.
  return iris.load_cube(filepath)
::::::::::::::
/scratch/tm70/pcl851/cylc-run/u-dq819.trunk/run94/log/job/1/AM_sst_ancil_amip/01/job.err
::::::::::::::
Using the cylc session localhost

Loading cylc/8.6.3
  Loading requirement: mosrs-setup/2.0.1
/home/851/pcl851/cylc-run/u-dq819.trunk/run94/lib/python/esm1p6_ancil/cmip7_ancil_ukesm.py:24: FutureWarning: Ignoring a datum in netCDF load for consistency with existing behaviour. In a future version of Iris, this datum will be applie
d. To apply the datum when loading, use the iris.FUTURE.datum_support flag.
  return iris.load_cube(filepath)

Copilot

Pull request overview

Adds a new AMIP ancillary-generation entrypoint for ESM1.6 that reads UKESM-produced NetCDF forcings, applies common coordinate fixes, and writes final UM ancillary outputs into the CMIP7 target directory structure.

Changes:

Added cmip7_AM_amip_generate.py to generate AMIP SST/sea-ice ancillaries from UKESM-produced NetCDF inputs.
Introduced cmip7_ancil_ukesm.py helper module for parsing/loading/fixing UKESM ancillary NetCDF cubes.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
CMIP7/esm1p6/atmosphere/cmip7_ancil_ukesm.py	New shared helper for UKESM NetCDF ancillary ingestion and coordinate adjustments.
CMIP7/esm1p6/atmosphere/amip/cmip7_AM_amip_generate.py	New AMIP generator wiring together parsers, UKESM load/fix, and ancillary save pathing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

penguian · 2026-07-03T06:19:02Z

The checksums have been stable since the run on 2026.06.26.

[pcl851@gadi-login-04 u-dq819.trunk]$ for var in seaice sst; do for f in /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.*/modern/amip/atmosphere/boundary_conditions/global.N96/2026.*/${var}_amip_n96_gregorian.anc;do md5sum $f;done; done
e96784e440971724c45d508a3b7cef9d  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.04.21/modern/amip/atmosphere/boundary_conditions/global.N96/2026.04.21/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.26/modern/amip/atmosphere/boundary_conditions/global.N96/2026.06.26/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.29/modern/amip/atmosphere/boundary_conditions/global.N96/2026.06.29/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.01/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.01/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.02/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.02/seaice_amip_n96_gregorian.anc
096eca3ee8f50685bc1c393036a5907a  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.03/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.03/seaice_amip_n96_gregorian.anc
0a396a15a78800f78b5108dce5e0c1d2  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.04.21/modern/amip/atmosphere/boundary_conditions/global.N96/2026.04.21/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.26/modern/amip/atmosphere/boundary_conditions/global.N96/2026.06.26/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.06.29/modern/amip/atmosphere/boundary_conditions/global.N96/2026.06.29/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.01/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.01/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.02/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.02/sst_amip_n96_gregorian.anc
0d234603969c6eead17f586b0b354736  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.03/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.03/sst_amip_n96_gregorian.anc

penguian · 2026-07-08T04:56:42Z

The changes on this branch are incorporated into the scen7-amip branch and have been tested using u-dq819/trunk.

Logs:

$ ls /scratch/tm70/pcl851/cylc-run/u-dq819.trunk/run101/log/job/1
AM_seaice_ancil_amip  AM_ukesm_ancil   AM_ukesm_taylor         git_clone_scripts  install_scripts
AM_sst_ancil_amip     AM_ukesm_regrid  git_clone_amip_scripts  install_amip

Output:

$ ls /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.08/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.08/
seaice_amip_n96_gregorian.anc  sst_amip_n96_gregorian.anc

This output is identical to yesterday's run99 output:

$ for var in seaice sst; do for f in /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.*/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.0[78]/${var}_amip_n96_gregorian.anc;do md5sum $f;done; done
53e0c4db1ae8395f9563263de2824964  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.07/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.07/seaice_amip_n96_gregorian.anc
53e0c4db1ae8395f9563263de2824964  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.08/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.08/seaice_amip_n96_gregorian.anc
bcd85ae7b9f56b6e6482db9ac0e64156  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.07/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.07/sst_amip_n96_gregorian.anc
bcd85ae7b9f56b6e6482db9ac0e64156  /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.07.08/modern/amip/atmosphere/boundary_conditions/global.N96/2026.07.08/sst_amip_n96_gregorian.anc

blimlim

Thanks @penguian, the updated function to fix the polar values looks good to me. I just had one small question

blimlim · 2026-07-08T06:47:34Z

+        cube,
+        save_dirpath,
+        args.save_filename,
+        gregorian=False,


Just wanted to check why we don't need to set the calendar to Gregorian for this one?

paul-leopardi added 2 commits June 26, 2026 17:36

Add code for saving AMIP ancils

f06d529

Try setting gregorian to False in save_ancil

6df2a0b

penguian linked an issue Jul 3, 2026 that may be closed by this pull request

Produce AMIP SST and sea-ice ancillaries for ESM1.6 #121

Open

penguian self-assigned this Jul 3, 2026

Satisfy pre-commit

fb6207d

penguian requested a review from Copilot July 3, 2026 06:10

Copilot started reviewing on behalf of penguian July 3, 2026 06:10 View session

penguian changed the title ~~121 amip~~ Add Python code for AMIP ancillary generation Jul 3, 2026

penguian requested review from MartinDix, atteggiani and blimlim July 3, 2026 06:11

Copilot AI reviewed Jul 3, 2026

View reviewed changes

Comment thread CMIP7/esm1p6/atmosphere/amip/cmip7_AM_amip_generate.py Outdated

Comment thread CMIP7/esm1p6/atmosphere/amip/cmip7_AM_amip_generate.py Outdated

Comment thread CMIP7/esm1p6/atmosphere/cmip7_ancil_ukesm.py

paul-leopardi added 2 commits July 6, 2026 15:07

Do not zero-fill. Fix poles.

629d35c

Fix the parameters in the call to save_ancil. Fix a comment

c1d5afa

penguian marked this pull request as draft July 7, 2026 11:51

Take the right mean on the right axis

3e11a99

penguian marked this pull request as ready for review July 8, 2026 04:56

blimlim approved these changes Jul 8, 2026

View reviewed changes

penguian marked this pull request as draft July 28, 2026 05:20

Navigation Menu

Uh oh!

Add Python code for AMIP ancillary generation - #141

Add Python code for AMIP ancillary generation#141
penguian wants to merge 6 commits into
mainfrom
121-amip

penguian commented Jul 3, 2026

Uh oh!

penguian commented Jul 3, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

penguian commented Jul 3, 2026

Uh oh!

penguian commented Jul 8, 2026 •

edited

Loading

Uh oh!

blimlim left a comment

Uh oh!

blimlim Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

penguian commented Jul 3, 2026

Uh oh!

penguian commented Jul 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

penguian commented Jul 3, 2026

Uh oh!

penguian commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blimlim left a comment

Choose a reason for hiding this comment

Uh oh!

blimlim Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

penguian commented Jul 8, 2026 •

edited

Loading