Skip to content

Feat/nvtx markers in clocks single - #78

Closed
JorgeG94 wants to merge 5 commits into
MOM6-GPU:dev/gpufrom
JorgeG94:feat/nvtx_markers_in_clocks_single
Closed

Feat/nvtx markers in clocks single#78
JorgeG94 wants to merge 5 commits into
MOM6-GPU:dev/gpufrom
JorgeG94:feat/nvtx_markers_in_clocks_single

Conversation

@JorgeG94

Copy link
Copy Markdown

No description provided.

edoyango and others added 5 commits July 3, 2026 14:18
)

* block k in ppm_reconstruction

* add error for -ve block size

* remove nvf inline and replace with intel forceinline

Significantly improves performance of blocked zonal/meridional_mass_flux at -O2

* change default block sizes at compile time

if __NVCOMPILER_OPENMP_GPU macro is defined (to be replaced at a later time),
set default_n?block to 0. Additionally, in general, block size of 0 means the
block size is dynamic to the computational domain.
The following have been added to Roquet and Wright equations of state:

    module procedure calculate_density_3d
    module procedure calculate_stanley_density_2d
    module procedure calculate_density_derivs_3d
    module procedure calculate_density_second_derivs_2d

These were extracted from a larger pull request supporting pressure
density integrals (NOAA-GFDL#156)
This incorporates several changes related to the block-based reproducing
sum implementation, and mirrors a pull request to dev/gfdl.

* Reproducing sum now divides domains into "blocks" such that each block
  does not exceed the maximum number of values before the carry
  accumulator may overflow.

* Several hard-coded parameters are replaced with Fortran intrinsics.

* Some parameters are renamed for convenience (esp. ni -> efp_digits)

* A max-domain warning was removed from the unit tests, since this now
  would be automatically divided into blocks.
* Block k in CorAdCalc

Add CORAD_NKBLOCK and restructure CorAdCalc around k-blocks so CPU builds keep layer-wise cache behavior while GPU builds can expose k as part of the do concurrent parallel space. Convert per-layer scratch arrays and gradKE to operate on the active k-block, while retaining serial wrappers for OBC, WENO, and KE_UP3 paths that still need dedicated GPU ports.

* Rename kstart/kend to k_start/k_end in CorAdCalc

* Rename CORAD_NKBLOCK to CORIOLIS_ADV_NKBLOCK

* Iterate k-block loops on the block-local index directly

* Port the CORIOLIS_EN_DIS uh/vh-ratio loop to do concurrent

---------

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: OpenAI Codex <codex@openai.com>
Record each clock's name as cpu_clock_id() registers it, then open an NVTX
range in cpu_clock_begin() and close it in cpu_clock_end(). Every existing
cpu_clock_id() name becomes a named range in an nsys timeline with no
call-site changes.

The range is opened before, and closed after, the mpp clock so that it
encloses it. Begin and end test the same condition, so starts and ends stay
balanced for handles that were never named or that fall outside the table.

The code is guarded behind -DMOM_USE_NVTX, which is undefined by default:
without it the preprocessed source is byte-identical to before and no extra
library is linked. Profiling builds need -DMOM_USE_NVTX and -cudalib=nvtx.

Applied to both the FMS1 and FMS2 infra shims, which are byte-identical to
each other. This also covers the halo and message-passing clocks, which
MOM_domain_infra begins and ends through this module directly rather than
through the MOM_cpu_clock framework wrapper.

Based on Edward Yang's ae67665 ("add nvtx markers to clocks", branch
benchmark_ALE_nvtx_clocks), which introduced the name-table approach and the
range nesting; this adds the build guard so non-NVHPC builds are unaffected.

Co-authored-by: Edward Yang <edward.yang@anu.edu.au>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JorgeG94 JorgeG94 closed this Jul 15, 2026
@JorgeG94
JorgeG94 deleted the feat/nvtx_markers_in_clocks_single branch July 15, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants