Changelog — what each release changed, newest first. High level only: the
things that affect someone using uCache; the commit history has the detail.
Heading size is how much the release matters: # a landmark, ## new
capability, ### fixes and refinements.
Versions are SemVer: x changes something you depend on, y adds capability,
z fixes and refines. Prebuilt EL9 packages are attached to each release.
Across every release: a newer release reads a cache written by an older one.
Recompression stays off unless you turn it on. Nothing is published unless you
run ucache publish. Any XRootD 5.6 or newer 5.x client works; 6.x from
v0.21.0.
- Bug fix, clean job exit: a job can end while uCache is still finishing work in the background.
- Local file caching: files on local, network, or mounted
drives can now be cached using
root://local//abs/path. - Improved cache isolation: if no new data is accessed by the
analysis, minor changes in how ROOT reads the data won't
require access to the origin. Enabled by default
(
complete_chunks). - Slow cache disk protection: if the cache disk is too slow for writing, uCache waits up to 2 seconds, then continues without caching. It logs the issue and resumes caching when the disk catches up.
- The freshness window is set as a duration
(
revalidate_after, default7d, replacingrevalidate_seconds) and applies to local files too.
- Failover: when a file is read through a redirector (as in CMS AAA) and a site stalls, fails or serves a damaged copy, uCache switches to another copy if one is available.
tried=in a URL accepts patterns:tried=*.example.orgblocks every server matching it.max_read_fraction = 0reads files straight from the origin and caches nothing.- A file read through different sites can be cached once. With
logical_name_prefixes = /store/mc/set, these two URLs are one cache entry:root://eoscms.cern.ch//eos/cms/store/mc/Run3Summer22EENanoAODv12/TTToHcToWAToMuMu_MHc-100_MA-60_MultiLepFilter_TuneCP5_13p6TeV_madgraph-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_postEE_v6-v2/2820000/03fd6a3a-e792-408b-a497-2607ffca2ace.rootroot://cms-xrd-global.cern.ch//store/mc/Run3Summer22EENanoAODv12/TTToHcToWAToMuMu_MHc-100_MA-60_MultiLepFilter_TuneCP5_13p6TeV_madgraph-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_postEE_v6-v2/2820000/03fd6a3a-e792-408b-a497-2607ffca2ace.root
- Recompressing files as they are first read (
recompress = on) now makes later reads as fast as recompressing them withucache recompress, with less memory than in 1.3; the first read takes close to what it takes with the cache off, for RNTuple as for TTree. - Replicas built by
ucache recompressin earlier releases are not used; running it again rebuilds them from what is cached. in_use_secondsreplacesevict_protect_secondsandmap_expiry_seconds;recompress_reclaimis gone.
- On macOS, cached data takes no more disk space than its size. A cache
made by an earlier version is only fixed once you clear it (
ucache clear);ucache doctortells you whether yours needs it. ucache clearempties the cache at once and frees its disk space in the background;--waitwaits for it.ucache statusshows the disk space the cache takes, asdudoes.
- The macOS tarball, which 1.3.3 lacks; otherwise the same as 1.3.3.
XRD_LOGLEVEL=Debugshows uCache's messages in the client's log; a log with a warning starts with a line naming the version and cache directory.- Bug fixes:
- Jobs reading one file with many threads no longer report CRC mismatches; caches that show them heal after the upgrade.
- With
recompress = on, a damaged cached page no longer stops the job.
ucache publishsends more about each run, so reports can compare jobs and machines.- Instruction counts are recorded on macOS and are correct on shared machines.
- Bug fixes:
recompress = onworks on the 2015 CMS open data.- uCache warns when a file cannot be recompressed on its first read.
ucache summaryanducache historyshow the gain after a first run over an empty cache and a second run; a separate run with the cache off is no longer needed.ucache recompressuses all cores by default (--jobs Nto use fewer).- Bug fixes:
- Python process pools (
multiprocessing,concurrent.futures) work with uCache. - Copying a recompressed file with fsspec or your own script gives the original file, or an error telling you to copy with the cache off.
- Python process pools (
- With
recompress = on, a file's replica is built while it is first read; no separateucache recompresspass is needed. - Copies —
xrdcp,hadd,rootcp, gfal, ROOT'sTFile::Cpand similar — read straight from the origin and leave the cache unchanged. - Added
max_read_fraction: a job that reads most of a large file reads it from the origin instead of caching it; default 25%. - Prebuilt binaries for macOS on Apple silicon: one tarball that uses the XRootD client of the ROOT or XRootD you already have.
- The package includes
ucache_demo.C: a dimuon mass analysis over 100 CMS open-data files that shows the cache working, cold and warm, through the byte cache and the replicas. - One package serves XRootD 5 and 6 clients: it carries a build of the plugin for each, and the client loads its own.
- The RPM no longer requires the XRootD client; uCache uses whichever client the process has loaded.
XRD_PLUGIN=<dir>/libXrdClUCache.sowithUCACHE_DIRturns uCache on with every default and no config file.- The RPM installs a recommended configuration at
/usr/share/xrd-ucache/ucache.conf.
- Added
prefetch: read-ahead for TTree reads, so a first pass waits less on the origin; off by default, on withprefetch = on.
- Multi-process readers such as coffea now fill the cache and appear in
ucache summaryanducache historythe same way a single process does. - uproot 5 with its default transport reads cached files.
- Added
ucache publish: submit benchmark records and run history for analysis and recommendations; data is anonymized. - Added
ucache identity: view or set the identity used to link published records across machines and browsers. - Added
announce: uCache identifies itself to XRootD servers so sites can see traffic that comes through a cache; on by default, off withannounce = off.
- Changed cache data format to support cache gain monitoring.
- Added
ucache summary: total and latest-run time savings. - Added
ucache history: per-run duration, bytes read, cache tier, and measured gain. ucache stats --resetnow archives records instead of deleting them, preserving gain baselines.
- XRootD 6 clients are supported — the plugin builds against and loads into 6 as well as 5.
- A running job no longer evicts its own working set: an entry is not an eviction candidate while it was read within the last day.
- When the cache is full of such entries it stops caching new files rather than
evicting data the job still needs, and
ucache statussays so. Reads keep working, uncached.evict_protect_seconds = 0restores the old behaviour. - Recompression accepts ZLIB sources by default as well as LZMA, so it no longer declines a ZLIB dataset without being told to.
- The plugin, the CLI and the tests build and run on macOS (Apple silicon,
MacPorts), and uCache serves
root://reads there. ucache benchmeasures macOS storage too, and reports how that measurement differs from the Linux one — the two platforms cannot bypass the page cache the same way.- Faster checksums on 64-bit Arm.
- Building against XRootD 6 stops at configure with an explanation, rather than failing deep inside the compiler.
- A warm pass over a recompressed RNTuple file no longer re-reads anything from the origin.
- RNTuple files can be recompressed into the replica tier — same switch, same behaviour as TTree files.
- Large reads are cached again: their edge pages were being dropped, and an oversized vector-read element was refused outright.
ucache benchmeasures a cold fill through uCache's own code rather than an imitation of it.- A reference describing every measurement
ucache benchmakes, and how to read it.
ucache benchrecords the device it measured and attributes the run's IO to it, so a number can be traced to the hardware that produced it.
ucache benchmeasures at several stream counts, split into datasheet-comparable and workload-shaped groups, with every run appended to a log.--threadsis required rather than guessed: it is the concurrency your jobs actually run at, which no tool can infer.
- A failed recompression build is reported and exits non-zero;
--strictfor callers who want it enforced. - The number of background recompression jobs is set by the caller.
- Reads of cached data issue one request per contiguous run of pages instead of one per page, on both cache tiers.
- Background recompression respects the eviction floor, so it can no longer fill the cache and start evicting the data your job is reading.
- When it builds nothing, it says which reason applies.
- Pages are verified against their checksums before being recompressed.
- ROOT files that grew past 2 GB after their key list was written can be recompressed; earlier releases refused them.
- Prebuilt EL9 packages (RPM and relocatable tarball) attached to each release, built against the oldest supported client so they load under any newer one.
ucache doctordetects an XRootD client too old for the plugin; previously it simply did not load, and nothing said why.
- First public release: a transparent per-user read cache for
root://data — an XrdCl plugin that caches the pages your jobs read onto local disk, and aucachecommand to set it up, inspect it and clean it. - Fail-open by construction: any cache problem degrades to a normal uncached read.
- Optional recompression rewrites cached files into a form that is faster to read back.