Skip to content

Sync Jeff Arnold's water allocation/carbon/manure work with dev - #252

Merged
celray merged 31 commits into
swat-model:devfrom
tugraskan:sync/jeff-water-allocation-to-dev
Sep 3, 2026
Merged

Sync Jeff Arnold's water allocation/carbon/manure work with dev#252
celray merged 31 commits into
swat-model:devfrom
tugraskan:sync/jeff-water-allocation-to-dev

Conversation

@tugraskan

@tugraskan tugraskan commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Author

The substantive work in this PR — 25 commits — is authored by Jeff Arnold (Jeff.Arnold@agnet.tamu.edu, GitHub @arnoldjjms), from his fork arnoldjjms/swatplus_dev, verifiable directly in the commit history below.

What this PR does

Jeff's branch diverged from dev at cb442f7 and accumulated 25 commits of feature work:

  • POU/POD/POR rework of water allocation architecture
  • Pesticide modeling enhancements
  • New manure database system
  • cswat=3: lignin-based 3-pool carbon cycling
  • Residue pool changes, stored per plant type
  • Canal support in water allocation
  • Recall/multi-dataset fixes

This PR merges that work with the current tip of dev, so both sides are fully caught up.

Bug fixes carried forward from dev

While Jeff's branch was diverged, dev picked up 12 commits of its own, including four real fixes now included here:

  • Perennial root depth stuck at 25.4mm minimum at cold startphuacc_p was never initialized for perennials that start a simulation already growing, flooring root depth regardless of declared maturity.
  • Irrigation days not reset for unlimited sourcedays_irr wasn't reset when irrigating from an unlimited source, throwing off subsequent scheduling.
  • Infinite loop in mgt_operatn — a schedule whose operations all fall on the same calendar date could cycle forever; capped at one full pass through the schedule.
  • varinit performance — a daily reset was zeroing an entire all-HRU array instead of just the active HRU's row, an O(N²)-per-day cost on large basins.

The remaining 8 of those 12 dev commits were typo/comment-only fixes, also carried forward.

Conflicts resolved

6 files had real conflicts between Jeff's rework and the 12 newer dev commits. In every case Jeff's substantive code was kept as-is; dev's changes were folded in only where they were still-applicable comment/typo fixes on surviving code:

  • mgt_biomix.f90, mgt_newtillmix_cswat0.f90, mgt_newtillmix_cswat1.f90dev's fix for the >12-plant crash (c917b44) was redundant with a fix Jeff's residue-pool rework had already made the same way; kept Jeff's version (keeping both would double-allocate and crash at runtime).
  • organic_mineral_mass_module.f90 — comment-only conflict, took dev's clearer wording.
  • wallo_transfer.f90 — kept Jeff's new POU/POD object model, folded in a typo fix to the comment.
  • water_allocation_module.f90 — Jeff's rework deleted an old outside_basin_source type in favor of reusing an existing one; dev had only fixed typos inside that now-deleted block. Kept Jeff's design, folded in the one typo fix that still applied.

Full writeup with the actual conflict diffs: tugraskan/swatplus_ug#156. Raw conflict view (GitHub's native conflict UI, unresolved) against Jeff's own main: arnoldjjms/swatplus_dev#9.

Scope

refdata/ and jupyter_notebooks/carbon.ipynb changes unrelated to this work (stale local-checkout drift, not new data — verified no new files were added, only deletions/modifications relative to dev) were reverted to match dev exactly, so this PR is scoped to the .f90 source changes plus one doc typo fix. Also swept the branch for every misspelling from dev's "Fix typos" commit to make sure none were silently reintroduced by the merge — found and fixed one (alrreadyalready, in dead/commented-out lines in res_hydro.f90). Compiles clean with gfortran/CMake: 0 warnings, 0 errors.

🤖 Generated with Claude Code

Jeff.Arnold and others added 27 commits November 19, 2025 11:58
…ultiple channels within a water allication object.
- Switch transplant data file from "transplant.plt" to "transplant.ops"
- Clarify comments for `phuacc` and `pop` in plant data module
- Add check for positive `trn_cur` in channel control logic to prevent invalid water allocation loops
Introduces cswat=3 option for three-pool residue decomposition using lignin fractions, with new subroutines for surface/soil residue and tillage mixing. Adds plant data structures for above/below-ground partitioning, tracks C above 300mm and live root fractions, and updates output files accordingly. Expands decision/action system for snow and reservoir irrigation demand. Implements new reservoir release schemes (Doell, Hanazaki, HYPE) with rolling inflow/demand memory. Refactors soil test adjustment for better weighted averaging. Generalizes cswat logic, improves table reader utility, and fixes various bugs and formatting issues.
Introduce new manure database and organic/mineral attribute structures, with file readers for both. Update constituent and salt application logic to use new manure type checks. Refactor transplant and irrigation management output for clarity. Enhance debug build safety with Fortran bounds checking and adjust static linking for GNU compilers. Minor cleanups throughout.
Expanded support for tracking and transferring constituent (nutrient, salt, contaminant) mass in water allocation, canal, and aquifer routines. Refactored data structures for clarity, added new arrays for constituent mass in allocation objects, and updated canal seepage and aquifer loss logic. Improved fertilizer/manure data handling and application, updated file formats, and streamlined soil carbon cycling code. All relevant routines and file readers updated for consistency and clarity. These changes ensure constituent mass is tracked alongside water throughout the system.

Refactor water allocation: new POU/POD/POR structure

Major overhaul of the water allocation (WALLO) system:
- Introduced new Fortran types for Points of Use (POU), Points of Delivery (POD), and Points of Return (POR), replacing the old transfer object structure.
- Implemented daily/monthly/yearly/average annual hydrograph and constituent mass tracking for each POU/POD/POR.
- Integrated decision tables for dynamic allocation of water rights, duties, and source/return fractions.
- Refactored wallo_control, wallo_withdraw, and related routines to use the new structure, with improved logic for withdrawal, delivery, and return.
- Updated all relevant modules and input/output routines to support the new format.
- Enhanced handling of out-of-basin sources, canals, water towers, and aquifers as PODs, with new fields for limits and losses.
- Improved recall file handling and integration.
- Updated fertilizer/manure logic to use new manure_om database with explicit C/N/P fractions.
- Improved erosion and C-factor calculations and outputs.
- Numerous bug fixes, code cleanups, and documentation updates.
Major overhaul of the water allocation (WALO) system:
- Replaces legacy wallo (water rights object) structure with modular POU/POD/POR (Point of Use/Delivery/Return) system.
- Adds new input files (`point_of_use.wro`, `point_of_deliv.wro`) and updates all read, allocation, and initialization routines.
- Refactors main control flow, moving start-of-day logic to `wallo_start_day` and updating all transfer, treatment, and use routines.
- Removes legacy wallo routines and output, updating all references and output logic.
- Updates constituent and salt irrigation routines to use new structure.
- Improves modularity, maintainability, and extensibility of water allocation code.
- Added `pest_tempdecay_module` for temperature-adjusted decay rates.
- Updated `pest_decay.f90` to handle temperature effects and metabolites.
- Fixed mass leaks in `smp_filter.f90` and `smp_grass_wway.f90`.
- Corrected pesticide mixing logic in tillage modules.
- Improved pesticide application and root uptake calculations.
- Introduced `harvest_sink` and `resistance_n` parameters for pesticide behavior.
- Enhanced output tracking with new fields for harvest and plant death.
- Updated channel pesticide routing with dynamic partitioning factors.
- Addressed double-counting in decay and metabolite formation.
- Improved pesticide handling during harvest for sink and non-sink cases.
- Added floodplain area (`fp_km2`) to channel morphology outputs.
- Fixed unit inconsistencies, improved readability, and removed redundant code.
Refactored and modularized water allocation code to improve readability, maintainability, and scalability. Introduced new data structures (`irrigation_water_transfer`, `pou_irrigation`, `hru_irrigation`) to handle HRU-specific irrigation demands, applied water, runoff, and efficiency. Integrated decision tables (`dtbl_lum`, `dtbl_flo`) for dynamic irrigation demand and allocation rates.

Added new subroutines (`wallo_pou_deliv`, `wallo_return`, `water_hru_irr_read`) to manage irrigation and water delivery processes. Updated file handling logic to support new input formats and added checks for file existence. Improved hydrograph outputs and constituent mass calculations to account for irrigation water quality.

Fixed bugs in POD/POU handling and minimum storage checks. Optimized array allocation for large-scale simulations. Enhanced documentation and comments for better code understanding.
…cation-to-dev

# Conflicts:
#	src/mgt_biomix.f90
#	src/mgt_newtillmix_cswat0.f90
#	src/mgt_newtillmix_cswat1.f90
#	src/organic_mineral_mass_module.f90
#	src/wallo_transfer.f90
#	src/water_allocation_module.f90
doc/VSCode_Codespace.md, jupyter_notebooks/carbon.ipynb, and refdata/Osu_1hru/lum.dtl
only carried upstream's typo-fix commit through the merge - no functional change,
out of scope for this sync. Keeping the branch focused on the .f90 source changes.
@tugraskan tugraskan closed this Aug 25, 2026
These 63 files under refdata/ (plus carbon.ipynb) had drifted from dev
during Jeff's working commits - all deletions/modifications, zero new
files, and confirmed absent at the fork point (cb442f7). Looks like an
out-of-date or gitignore-affected local checkout got swept up in a
broad commit, not intentional changes tied to the water allocation
work. Restoring dev's checked-in versions removes that unrelated drift.
Missed by the merge: Jeff's branch independently touched this line elsewhere
(recall_db -> recall refactor nearby), so git's auto-merge silently kept his
version of the comment, which fixed "substraction" but not "already" on its
own. Both instances are in commented-out (dead) code - no functional change.
Jeff's branch had added ignore rules for manure_db.frt, manure_om.frt,
and mgt_out.txt, which is why those refdata files showed as deleted -
inconsistent now that the earlier refdata restore brought them back
as tracked files.
@tugraskan tugraskan reopened this Aug 25, 2026
@tugraskan
tugraskan requested a review from celray August 25, 2026 19:46
@celray

celray commented Aug 25, 2026

Copy link
Copy Markdown
Member

Running tests on various datasets before merging.

@celray

celray commented Sep 3, 2026

Copy link
Copy Markdown
Member

PR 252 merge notes

finished checking PR 252. PR head 795d2d9 on dev e05177f.

  • Jeff's last commit (3634090, 25 Aug) is carried unchanged. Nothing of his was lost.
  • PR head vs Jeff's commit: 18 files, and every hunk is one of dev's 12 commits (days_irr reset, mgt_operatn loop guard, varinit row reset, perennial phuacc_p init, typos).
  • The three tillage files: Jeff already allocates mix_org%rsd by plant count, so dropping dev's copy of that fix was right.
  • Old wallo routines are gone. Only dead external :: names remain.
  • file.cio: basin line (with carbon.bsn) read the same as dev. Water rights line now reads two names; datasets write three columns, so no shift.

Recall fix

dev (and 62.0.0) can't read a normal recall.rec at all since Jeff's January commit 72206bc; any dataset with recall objects segfaults in command.f90:335. Jeff's fix puts the 61.0.2.11 reader back. Works.

Tests (ifx 2026 Release; gfortran 16.2 Release for Osu_1hru and Ames_sub1)

dataset dev PR
Osu_1hru ok ifx: ok, water balance ~1% off dev (his residue/carbon changes). gfortran: segfault on day 1 in mgt_newtillmix_cswat0 (item 9)
Ames_sub1 (carbon.bsn, 46 yr) ok ok, wb/nb identical, carbon+yield last digit. gfortran: ok, water balance identical to the ifx run
HUC8 02070009, 30 yearly recalls, wallo off segfault day 1 ok, 65 s
same, with old Wichita wallo files hangs day 1 segfault wallo_treatment.f90:24
robit (plugin output 4 Aug: canals, tower, wtp, 2 uses, osrc, irrigation district) not run segfault wallo_return.f90:50 without wtps_wuses.wal; with it, Release builds crash between day 1 and the first yearly print (wallo_pou_output.f90), location varies by build; bounds-checked build stops day 1 at wallo_withdraw.f90:36; irrigation POU also out of bounds at actions.f90:143

Recall numbers check out: 5927.658 m3/d in, 2163595 m3/yr out (2169523 in the
leap year).

The Wichita files are an older format and lack wtps_wuses.wal, so that crash only says missing wallo inputs segfault instead of erroring. Need a dataset in the current format to test the wallo model itself.

Follow-up PRs should address these:

  1. water_allocation_read.f90:38,192: .not. i_exist .or. name /= "null" is inverted, creates an empty file called null. Use i_exist .and. ....
  2. recall_read.f90:113: hard-coded "recall.rec", always-true test, creates an empty recall.rec. Use in_rec%recall_rec.
  3. wallo_wtps_wuses.f90:25: always-true test, creates empty wtps_wuses.wal and leaves wal_tr_omd etc. unallocated, so any wtp/use POU segfaults. Fix the test and error out cleanly when the file is missing.
  4. time_control.f90:476 writes bsn_sedbud to unit 3152 unconditionally, but it's only opened when there are channels. Leaves fort.3152 on Ames_sub1.
  5. Irrigation POU crashes in actions.f90:143. The irr_demand action indexes pcom(j)%dtbl(idtbl) with the global dtbl_lum id, but that array is sized and ordered by the HRU's own land-use tables (hru_dtbl_actions_init.f90:36). A wallo irrigation table like irr_str8_dmd that is not in the HRU's lum list is out of bounds. Also wallo_start_day.f90:124 overwrites the summed irrigation demand with rate_max unconditionally. Later PR, Jeff's design call.
  6. wallo_withdraw.f90:36 (osrc case) writes poud_om(pou_num)%pod(ipod) with the global POD number instead of the POU-local slot pod_num. Bounds-checked Debug build stops there on robit (index 4, bound 1); Release runs past it and later segfaults in wallo_pou_output.f90 at the yearly print, most likely from that overwrite. Same family: wallo_control.f90:28 loops do ipou = 1, pod(ipod)%pous and uses the counter as a POU number; it should use pod(ipod)%pou(ipous)%num.
  7. water_use_read.f90:101 has its exit inside the do iwuse loop, so only the first water use is read; the outer do then re-enters and consumes the remaining rows as the title, count and header lines, so a file with four or more uses reaches allocate (wuse(imax)) again and aborts before day 1 (ifx error 151 at line 44, reproduced on robit with five uses), and one with two or three exits silently with uses 2 and up empty. water_treatment_read.f90 lost the outer exit altogether (dev has it at line 79); it completes its do iwtp loop, so on re-entry it is at end of file and survives unless the file has trailing lines, in which case it re-allocates and aborts too. Both come from Jeff's reader rework, not the conflict resolution.
  8. recall_db%iorg_min is never assigned (only declared, recall_module.f90:13) but wallo_start_day.f90:54, wallo_treatment.f90:45 and wallo_use.f90:41 read it, so an outside source, treatment plant or use with recall concentrations resolves to recall(0). Read side of the outside-source path; item 6 is the write side.
  9. mgt_newtillmix_cswat0.f90:170 csmix%pest(:) = 0. on an array that pesticide_init only allocates when the project has pesticides. Any project with carbon 0, tillage and no pesticides: segfault under gfortran (Osu_1hru dies on the first tillage, gdb puts it in mgt_newtillmix_cswat0; dev built the same way runs it), no visible effect under ifx even with checks on. Added in Jeff's May pesticide commit, only in the cswat0 routine. Matters because the aarch64 production build is gfortran. Guard with if (allocated(csmix%pest)) or if (cs_db%num_pests > 0).
  10. Tidy: dead externals, write (10108,*) debug line in monthly recall, and the tillage allocate guard: dev's if (allocated(mix_org%rsd)) deallocate is gone and in mgt_newtillmix_cswat0.f90 the deallocates sit inside if (dtil > 0.) while the allocate does not, so a zero-depth tillage entry would allocate twice on the next call. Latent, no shipped tillage database has one.

Summary: what needs fixing after the merge

The merge itself is fine and the recall fix is needed now. The new water allocation code is not usable end to end yet. Two areas stand out; item 9 is a gfortran-only segfault on ordinary projects and should go first; the rest are reader and output fixes (items 1 to 4, 7 and 8 above).

Irrigation (item 5). Any POU of type irr crashes on day 1. The irr_demand action in actions.f90 indexes the HRU's per-table counters with the global decision table id, so a wallo irrigation table that is not in the HRU's land-use list is out of bounds. wallo_start_day.f90 also throws away the summed HRU demand and uses rate_max regardless. Until this is fixed, water allocation can only be tested without irrigation POUs.

Interbasin transfer, outside sources (item 6 and wallo_start_day). The osrc path is where the out-of-bounds write sits (wallo_withdraw.f90:36, global POD number used as POU-local slot), and in wallo_start_day.f90 the constant and decision-table cases take the outside source concentrations from wuse_om_efflu instead of osrc_om. The withdrawal also uses the recall values as mass without a concentration-to-mass step. Fix: read from osrc_om, convert concentration to mass on withdrawal, and accumulate into %pod(pod_num).

Test dataset for both: robit_wallo.zip below (wtps_wuses.wal included).

Test datasets

Inputs only, model outputs stripped, each with a README of what to expect.

  • huc8_02070009_recall.zip (5.4 MB, md5 6a4ff5864d137d4557e5600c49215782): CEAP HUC8, 4222 HRUs, 1997 to 2000, 30 yearly point-source recalls, water allocation off. Segfaults on dev and 62.0.0, runs on PR 252. (https://office.chawanda.com/s/rLJ2TqqwjG6nmcw)
  • robit_wallo.zip (0.6 MB, md5 96fd166deddcce6a3cb1027454a7905d): plugin-written water allocation set (canals, tower, wtp, uses, outside source, irrigation district) plus the wtps_wuses.wal the model needs and no-irrigation variants of pou.wro/pod.wro. Reproduces the irrigation and interbasin crashes above. (https://office.chawanda.com/s/QRm22o89qH5EZtQ)

@celray
celray merged commit 277c238 into swat-model:dev Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants