Reduce extraneous testing files #800

Activity

1. forsyth2

   added

   TestingFiles in `tests` modifiedFiles in `tests` modified

   on Mar 23, 2026
2. forsyth2 commented on Oct 14, 2026

   

   forsyth2

   on Oct 14, 2026

   CollaboratorAuthor

   More actions

   Remove extraneous "min case" tests

   I gave Claude the zppy/tests cfgs and main_branch_testing information as of today 10/14 (using stack #887: #871, #886, #852). Its analysis is:

   ---

   The file is large and truncated in view, so I'll look at its structure first.

   I'm partway through: I've read all the min_case cfgs and the weekly comprehensive v3 cfg, and now I'm checking how main_branch_testing and utils.py switch the weekly cfgs' options on and off.

   Still working through the last few families: I'm checking the ilamb/global_time_series unit tests, then I'll write up the full categorization.

   Of the 49 min_case templates, I'd keep 10 as still useful and treat the other 39 as already covered.

   main_branch_testing doesn't add any new cfg content. It orchestrates the four weekly templates (CFGS_TO_RUN) and flips their active_* switches through TASKS_TO_RUN in utils.py. None of the min_case cfgs are in its default run. So "covered" below means a weekly template exercises the same parameter combination when all six default tasks are on. Weekly runs are also image-checked by test_images.py, whereas the min cases are manual grep -v OK *status checks.

   The weekly templates differ in how they set infer_path_parameters and infer_section_parameters:

   • v3 and legacy: both are False there, with everything spelled out explicitly.
   • v2 and bundles: they use the defaults, which test_sections.py shows are True, so the inference behavior is exercised there.

   Still useful

   cfg	why the weekly cfgs don't cover it
   add_dependencies	It's the only one where an e3sm_diags interval spans several ts segments (10-year diags over 5-year ts). Every weekly e3sm_diags interval equals ts_num_years. The unit tests only cover the helper.
   deprecated_parameters	It's the only test of the deprecated params, it's dry-run only, and no unit test mentions them. It can go once those params are actually removed.
   global_time_series_original_8_missing_ocn	It's the only test of the skip-on-missing-dependency path. Weekly sets fail_on_dependency_skip = True and always has mpas_analysis active.
   global_time_series_viewers_all_land_variables	Borderline. It asks for plots_lnd = "all" while ts produced only 25 variables. Weekly all_lnd_var_viewer has every variable present.
   ilamb_land_only	No weekly cfg sets land_only. test_zppy_ilamb.py checks only the dependency list, not an actual run.
   tc_analysis_res	No weekly cfg sets res explicitly. The template unit tests cover radius-from-res, not the parameter pass-through.
   tc_analysis_v2_simultaneous_1/_2	It runs two identical-year tc_analysis jobs concurrently, which weekly never does. It only matters if tc_analysis still touches shared state outside its output dir. Keep the v2 pair, since v3 has the empty-dat issue.
   e3sm_diags_tc_analysis_v2_parallel	It needs more than one e3sm_diags interval with tc_analysis. Weekly v2 has one interval, and bundles has two but tc_analysis is commented out there.
   e3sm_diags_tc_analysis (v3)	For now it's the only v3/pg2 tc_analysis to e3sm_diags run, because tc_analysis is commented out of weekly v3 and bundles. Check whether it currently passes given the same empty-dat TODO.

   Caveats

   • mvm_1/_2 pairs: I treated them as equivalent to the weekly same-run mvm. The unit tests show dependencies are added only for the test years, and the reference paths are plain strings. If the pairs were meant to exercise a reference from a separate zppy run, keep one pair, for example depend_on_ts_mvm.
   • Unit tests: These per-set checks are stronger than the min cases for dependencies and path inference: test_add_climo_dependencies, test_add_ts_dependencies, and test_check_and_define_parameters.
   • Missing source: The zppy source itself wasn't in the paste. I relied on those unit tests for how dependencies and inference work.

   Things I noticed in the templates

   • streamflow_mvm_2 writes #expand_case_name# (underscore) for ref_name and short_ref_name. It won't expand, so the literal string ends up in the cfg.
   • deprecated_parameters points output and www at the zppy_weekly_comprehensive_v3_* directories, so a dry run with the same unique ID could overwrite weekly scripts.
   • The legacy weekly cfg has aactive for land_monthly_climo_merra2.
   • v2's mvm section sets ts_daily_subsection to the monthly subsection name. I haven't verified the impact, but it may matter for the tropical_subseasonal mvm dependency.
3. chengzhuzhang

   added this to the v3.4.0 milestone on Oct 19, 2026