[Issue]: hipMemGetInfo() gives about 3GB less memory than expected in WSL2 vs Windows native

[Diablo-D3]

Problem Description

hipMemGetInfo() gives about 3GB less memory than expected in WSL2 vs Windows native

Originally reported as ggml-org/llama.cpp#23999, but it has become clear it is a ROCm/HIP issue. Tested with bc1fdab.

@yann-amd asked me to open a ticket here after triaging it over there.

Operating System

Debian 13

CPU

9800x3D

GPU

7900 XTX

ROCm Version

ROCm 7.2.1

ROCm Component

No response

Steps to Reproduce

#include <cstdio>
#include <hip/hip_runtime.h>

int main() {
    size_t total_mem;
    size_t free_mem;

    hipDevice_t device;
    hipGetDevice(&device);

    hipMemGetInfo(&free_mem, &total_mem);
    printf("%zu %zu\n", free_mem / 1024 / 1024, total_mem / 1024 / 1024);

    return 0;
}

hipcc hipmemgetinfo.cpp -o hipmemgetinfo && ./hipmemgetinfo

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

rocminfo --support output

WSL environment detected.
=====================
HSA System Attributes
=====================
Runtime Version:         1.18
Runtime Ext Version:     1.15
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE
System Endianness:       LITTLE
Mwaitx:                  DISABLED
XNACK enabled:           NO
DMAbuf Support:          YES
VMM Support:             YES

==========
HSA Agents
==========
*******
Agent 1
*******
  Name:                    AMD Ryzen 7 9800X3D 8-Core Processor
  Uuid:                    CPU-XX
  Marketing Name:          AMD Ryzen 7 9800X3D 8-Core Processor
  Vendor Name:             CPU
  Feature:                 None specified
  Profile:                 FULL_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        0(0x0)
  Queue Min Size:          0(0x0)
  Queue Max Size:          0(0x0)
  Queue Type:              MULTI
  Node:                    0
  Device Type:             CPU
  Cache Info:
    L1:                      49152(0xc000) KB
  Chip ID:                 0(0x0)
  Cacheline Size:          64(0x40)
  BDFID:                   0
  Internal Node ID:        0
  Compute Unit:            16
  SIMDs per CU:            0
  Shader Engines:          0
  Shader Arrs. per Eng.:   0
  Memory Properties:
  Features:                None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: FINE GRAINED
      Size:                    24408140(0x174704c) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Recommended Granule:4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 2
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    24408140(0x174704c) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Recommended Granule:4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 3
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    24408140(0x174704c) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Recommended Granule:4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 4
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    24408140(0x174704c) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Recommended Granule:4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
  ISA Info:
*******
Agent 2
*******
  Name:                    gfx1100
  Uuid:                    GPU-735ef1a0b8160f95
  Marketing Name:          AMD Radeon RX 7900 XTX
  Vendor Name:             AMD
  Feature:                 KERNEL_DISPATCH
  Profile:                 BASE_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        128(0x80)
  Queue Min Size:          64(0x40)
  Queue Max Size:          131072(0x20000)
  Queue Type:              MULTI
  Node:                    1
  Device Type:             GPU
  Cache Info:
    L1:                      32(0x20) KB
    L2:                      6144(0x1800) KB
    L3:                      98304(0x18000) KB
  Chip ID:                 29772(0x744c)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   2431
  BDFID:                   768
  Internal Node ID:        1
  Compute Unit:            96
  SIMDs per CU:            2
  Shader Engines:          6
  Shader Arrs. per Eng.:   2
  Coherent Host Access:    FALSE
  Memory Properties:
  Features:                KERNEL_DISPATCH
  Fast F16 Operation:      TRUE
  Wavefront Size:          32(0x20)
  Workgroup Max Size:      1024(0x400)
  Workgroup Max Size per Dimension:
    x                        1024(0x400)
    y                        1024(0x400)
    z                        1024(0x400)
  Max Waves Per CU:        32(0x20)
  Max Work-item Per CU:    1024(0x400)
  Grid Max Size:           4294967295(0xffffffff)
  Grid Max Size per Dimension:
    x                        2147483647(0x7fffffff)
    y                        65535(0xffff)
    z                        65535(0xffff)
  Max fbarriers/Workgrp:   32
  Packet Processor uCode:: 632
  SDMA engine uCode::      27
  IOMMU Support::          None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    25105704(0x17f1528) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Recommended Granule:2048KB
      Alloc Alignment:         4KB
      Allocatable:             FALSE
      Alloc Granule:           0KB
      Alloc Recommended Granule:0KB
      Alloc Alignment:         0KB
      Accessible by all:       FALSE
  ISA Info:
    ISA 1
      Name:                    amdgcn-amd-amdhsa--gfx1100
      Machine Models:          HSA_MACHINE_MODEL_LARGE
      Profiles:                HSA_PROFILE_BASE
      Default Rounding Mode:   NEAR
      Default Rounding Mode:   NEAR
      Fast f16:                TRUE
      Workgroup Max Size:      1024(0x400)
      Workgroup Max Size per Dimension:
        x                        1024(0x400)
        y                        1024(0x400)
        z                        1024(0x400)
      Grid Max Size:           4294967295(0xffffffff)
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)
        y                        65535(0xffff)
        z                        65535(0xffff)
      FBarrier Max Size:       32
    ISA 2
      Name:                    amdgcn-amd-amdhsa--gfx11-generic
      Machine Models:          HSA_MACHINE_MODEL_LARGE
      Profiles:                HSA_PROFILE_BASE
      Default Rounding Mode:   NEAR
      Default Rounding Mode:   NEAR
      Fast f16:                TRUE
      Workgroup Max Size:      1024(0x400)
      Workgroup Max Size per Dimension:
        x                        1024(0x400)
        y                        1024(0x400)
        z                        1024(0x400)
      Grid Max Size:           4294967295(0xffffffff)
      Grid Max Size per Dimension:
        x                        2147483647(0x7fffffff)
        y                        65535(0xffff)
        z                        65535(0xffff)
      FBarrier Max Size:       32

Additional Information

Values consistent with ReBAR on and off, have not tested with Above 4GB Decoding off.

[fcui-amd]

There seems no much gap

[Diablo-D3]

Your results are what I expect, and are the same results I am getting from natively on Windows.

What version of ROCm are you testing with?

[fcui-amd]

rocm 7.2, 7.13, and the latest works just the same.

[Diablo-D3]

With this, I get 21615 24517, and (24517 - 21615) / 1024 is a little over 2.8gb, which ... isn't close to the 3.6GB VRAM in use, but could explain it (which means ROCm natively in Windows is calculating free different than ROCm via librocdxg does).

Also, your card has 32GB, yet your Windows says 30, and my card has 24GB and my Windows says 24.

I'm confused at why we're getting oddly different numbers, your W7800 is a 7900XTX with 2x VRAM and should be otherwise identical to mine.

[fcui-amd]

The gap is not reproducible in my side.
as for the 32GB vram reported as 30 GB, it's related with

[Diablo-D3]

Ahh, I always wondered how the Pro cards implemented ECC.

So, if you consume all of your VRAM, does the value change at all for you? It doesn't natively in Windows.

If I quit every single thing I can kill, I get:

and the test program produces 23365 24517.

I would love to use this exact test case on Windows natively, but I can't figure out the right invocation for hipcc, & 'C:\Program Files\AMD\ROCm\7.1\bin\hipcc' hipmemgetinfo.cpp -o hipmemgetinfo.exe isn't correct, and I'm a Linux programmer, not a Windows one: it can't find the stdlib headers, and it already has C:\Program Files\AMD\ROCm\7.1\lib\clang\21\include in the include path, according to -v.

[fcui-amd]

to build hip_avail test on windows, you could install hipsdk according to https://rocm.docs.amd.com/projects/install-on-windows/en/latest/install/install.html
then

$env:PATH +=" ;C:\\Program Files\\AMD\\ROCm\\7.1\\bin"

then hipcc would be in env PATH

and to consume all vram:

[Diablo-D3]

I have the HIP SDK already installed in Windows, but adding it to PATH doesn't fix this.

PS> hipcc hipmemgetinfo.cpp -o hipmemgetinfo.exe -v
clang version 21.0.0git (git@github.com:Compute-Mirrors/llvm-project 5dcc622b51ecd499912c1062ce2b0ecda60d8e93)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\AMD\ROCm\7.1\bin
Found HIP installation: C:\Program Files\AMD\ROCm\7.1\, version 7.1.51803
 "C:\\Program Files\\AMD\\ROCm\\7.1\\bin\\clang.exe" -cc1 -triple amdgcn-amd-amdhsa -aux-triple x86_64-pc-windows-msvc -Werror=atomic-alignment -emit-obj -mincremental-linker-compatible -dumpdir hipmemgetinfo.exe- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hipmemgetinfo.cpp -mrelocation-model pic -pic-level 2 -fhalf-no-semantic-interposition -mframe-pointer=none -relaxed-aliasing -fno-rounding-math -mconstructor-aliases -aux-target-cpu x86-64 -fcuda-is-device -fno-threadsafe-statics -mllvm -amdgpu-internalize-symbols -fcuda-allow-variadic-functions -fvisibility=hidden -fapply-global-visibility-to-externs -mlink-builtin-bitcode "C:\\Program Files\\AMD\\ROCm\\7.1\\amdgcn\\bitcode\\ocml.bc" -mlink-builtin-bitcode "C:\\Program Files\\AMD\\ROCm\\7.1\\amdgcn\\bitcode\\ockl.bc" -mlink-builtin-bitcode "C:\\Program Files\\AMD\\ROCm\\7.1\\amdgcn\\bitcode\\oclc_daz_opt_off.bc" -mlink-builtin-bitcode "C:\\Program Files\\AMD\\ROCm\\7.1\\amdgcn\\bitcode\\oclc_unsafe_math_off.bc" -mlink-builtin-bitcode "C:\\Program Files\\AMD\\ROCm\\7.1\\amdgcn\\bitcode\\oclc_finite_only_off.bc" -mlink-builtin-bitcode "C:\\Program Files\\AMD\\ROCm\\7.1\\amdgcn\\bitcode\\oclc_correctly_rounded_sqrt_on.bc" -mlink-builtin-bitcode "C:\\Program Files\\AMD\\ROCm\\7.1\\amdgcn\\bitcode\\oclc_wavefrontsize64_on.bc" -mlink-builtin-bitcode "C:\\Program Files\\AMD\\ROCm\\7.1\\amdgcn\\bitcode\\oclc_isa_version_906.bc" -mlink-builtin-bitcode "C:\\Program Files\\AMD\\ROCm\\7.1\\amdgcn\\bitcode\\oclc_abi_version_600.bc" -target-cpu gfx906 -debugger-tuning=gdb "-fdebug-compilation-dir=C:\\Users\\diablo\\Desktop" -v "-fcoverage-compilation-dir=C:\\Users\\diablo\\Desktop" -resource-dir "C:\\Program Files\\AMD\\ROCm\\7.1\\lib\\clang\\21" -internal-isystem "C:\\Program Files\\AMD\\ROCm\\7.1\\lib\\clang\\21\\include\\cuda_wrappers" -idirafter "C:\\Program Files\\AMD\\ROCm\\7.1\\include" -include __clang_hip_runtime_wrapper.h -internal-isystem "C:\\Program Files\\AMD\\ROCm\\7.1\\lib\\clang\\21\\include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 10.0/VC/include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 9.0/VC/include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 9.0/VC/PlatformSDK/Include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 8/VC/include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 8/VC/PlatformSDK/Include" -internal-isystem "C:\\Program Files\\AMD\\ROCm\\7.1\\lib\\clang\\21\\include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 10.0/VC/include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 9.0/VC/include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 9.0/VC/PlatformSDK/Include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 8/VC/include" -internal-isystem "C:/Program Files/Microsoft Visual Studio 8/VC/PlatformSDK/Include" -O3 -fdeprecated-macro -fno-autolink -ferror-limit 19 -fmessage-length=80 -fhip-new-launch-api -fms-extensions -fms-compatibility -fms-compatibility-version=19.33 -std=c++14 -fskip-odr-check-in-gmf -fdelayed-template-parsing -fcxx-exceptions -fexceptions -fcolor-diagnostics -vectorize-loops -vectorize-slp -cuid=6a19c853b1f7d141 -fcuda-allow-variadic-functions -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o "C:\\Users\\diablo\\AppData\\Local\\Temp\\hipmemgetinfo-gfx906-38adcc.o" -x hip hipmemgetinfo.cpp
clang -cc1 version 21.0.0git based upon LLVM 21.0.0git default target x86_64-pc-windows-msvc
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 10.0/VC/include"
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 9.0/VC/include"
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 9.0/VC/PlatformSDK/Include"
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 8/VC/include"
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 8/VC/PlatformSDK/Include"
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 10.0/VC/include"
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 9.0/VC/include"
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 9.0/VC/PlatformSDK/Include"
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 8/VC/include"
ignoring nonexistent directory "C:/Program Files/Microsoft Visual Studio 8/VC/PlatformSDK/Include"
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/include"
ignoring duplicate directory "C:\Program Files\AMD\ROCm\7.1\lib\clang\21\include"
ignoring duplicate directory "C:\Program Files\AMD\ROCm\7.1\lib\clang\21\include"
#include "..." search starts here:
#include <...> search starts here:
 C:\Program Files\AMD\ROCm\7.1\lib\clang\21\include\cuda_wrappers
 C:\Program Files\AMD\ROCm\7.1\lib\clang\21\include
 C:\Program Files\AMD\ROCm\7.1\include
End of search list.
In file included from <built-in>:1:
In file included from C:\Program Files\AMD\ROCm\7.1\lib\clang\21\include\__clang_hip_runtime_wrapper.h:111:
C:\Program Files\AMD\ROCm\7.1\lib\clang\21\include\cuda_wrappers\cmath:30:2: error:
      "Could not find standard C++ header 'cmath'. Add -v to your compilation
      command to check the include paths being searched. You may need to install
      the appropriate standard C++ library package corresponding to the search
      path."
   30 | #error "Could not find standard C++ header 'cmath'. Add -v to your compi...
      |  ^
In file included from <built-in>:1:
C:\Program Files\AMD\ROCm\7.1\lib\clang\21\include\__clang_hip_runtime_wrapper.h:112:10: fatal error:
      'cstdlib' file not found
  112 | #include <cstdlib>
      |          ^~~~~~~~~
2 errors generated when compiling for gfx906.
failed to execute:""C:\Program Files\AMD\ROCm\7.1\bin\clang.exe"   -O3 --driver-mode=g++ -O3 -fuse-ld=lld --ld-path="C:\Program Files\AMD\ROCm\7.1\bin/lld-link.exe" -Llib --hip-link  -x hip "hipmemgetinfo.cpp" -o "hipmemgetinfo.exe" -v"

If I was on Linux, I'd be thinking I was missing libstdc++ for my distro.

Do I need to install something from the Visual Studio build tools as well? https://rocm.docs.amd.com/projects/install-on-windows/en/latest/install/install.html doesn't say that I need to.

[fcui-amd]

Yes, I think you need to install C++ support via Visual Studio Installer.
Install Desktop development with C++, including MSVC v143 and the Windows 10/11 SDK.

[Diablo-D3]

Okay, that worked. hipcc hipmemgetinfo.cpp -o hipmemgetinfo.exe did what it was supposed to.

With GPU VRAM almost filled up...

hipMemGetInfo() on native Windows now says 24410 24560, which is the value I expected. In WSL2, it says 6313 24517.

[fcui-amd]

I think the WSL number is the accurate one here. On native Windows your VRAM is already almost fully used, yet hipMemGetInfo() still reports free ≈ 24410. In WSL, 6313 reflects what's actually left when the GPU is busy. total matches on both sides.

[Diablo-D3]

Applications seem to be depend on it not being correct. They want to know, basically, what is the largest allocation they can make that fits on the card, even if it shoves other apps out of VRAM.

CUDA, afaict, also reports maximum allocation size, not correct one, but I don't have any Nvidia hardware to check with.

[fcui-amd]

This is expected — free just means different things on each platform:

WSL2 (librocdxg) reports physical free VRAM (total minus what's actually resident on the card, system-wide). This matches native Linux ROCm/HIP: if one process reserves 3 GB, another process sees free drop by ~3 GB.
Windows native HIP reports a per-process budget. WDDM memory is virtualized and pageable, so a fresh process sees free ≈ total regardless of what else is on the card — the same model NVIDIA uses on WSL.
We intentionally keep WSL2 consistent with native Linux HIP, not Windows native. So the 6313 in WSL2 is the expected value (real physical occupancy), and the 24410 on Windows reflects the budget model — both are correct, they just measure different things.