Skip to content

Latest commit

 

History

History
220 lines (191 loc) · 10.6 KB

HISTORY.md

File metadata and controls

220 lines (191 loc) · 10.6 KB

Changelog — what each release changed, newest first. High level only: the things that affect someone using uCache; the commit history has the detail. Heading size is how much the release matters: # a landmark, ## new capability, ### fixes and refinements.

Versions are SemVer: x changes something you depend on, y adds capability, z fixes and refines. Prebuilt EL9 packages are attached to each release.

Across every release: a newer release reads a cache written by an older one. Recompression stays off unless you turn it on. Nothing is published unless you run ucache publish. Any XRootD 5.6 or newer 5.x client works; 6.x from v0.21.0.

v1.5.2 — 2026-10-23

  • Bug fix, clean job exit: a job can end while uCache is still finishing work in the background.

v1.5.1 — 2026-10-22

  • Local file caching: files on local, network, or mounted drives can now be cached using root://local//abs/path.
  • Improved cache isolation: if no new data is accessed by the analysis, minor changes in how ROOT reads the data won't require access to the origin. Enabled by default (complete_chunks).
  • Slow cache disk protection: if the cache disk is too slow for writing, uCache waits up to 2 seconds, then continues without caching. It logs the issue and resumes caching when the disk catches up.
  • The freshness window is set as a duration (revalidate_after, default 7d, replacing revalidate_seconds) and applies to local files too.

v1.5.0 — 2026-10-18

  • Failover: when a file is read through a redirector (as in CMS AAA) and a site stalls, fails or serves a damaged copy, uCache switches to another copy if one is available.
  • tried= in a URL accepts patterns: tried=*.example.org blocks every server matching it.
  • max_read_fraction = 0 reads files straight from the origin and caches nothing.
  • A file read through different sites can be cached once. With logical_name_prefixes = /store/mc/ set, these two URLs are one cache entry: root://eoscms.cern.ch//eos/cms/store/mc/Run3Summer22EENanoAODv12/TTToHcToWAToMuMu_MHc-100_MA-60_MultiLepFilter_TuneCP5_13p6TeV_madgraph-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_postEE_v6-v2/2820000/03fd6a3a-e792-408b-a497-2607ffca2ace.root root://cms-xrd-global.cern.ch//store/mc/Run3Summer22EENanoAODv12/TTToHcToWAToMuMu_MHc-100_MA-60_MultiLepFilter_TuneCP5_13p6TeV_madgraph-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_postEE_v6-v2/2820000/03fd6a3a-e792-408b-a497-2607ffca2ace.root

v1.4.0 — 2026-10-05

  • Recompressing files as they are first read (recompress = on) now makes later reads as fast as recompressing them with ucache recompress, with less memory than in 1.3; the first read takes close to what it takes with the cache off, for RNTuple as for TTree.
  • Replicas built by ucache recompress in earlier releases are not used; running it again rebuilds them from what is cached.
  • in_use_seconds replaces evict_protect_seconds and map_expiry_seconds; recompress_reclaim is gone.

v1.3.5 — 2026-09-30

  • On macOS, cached data takes no more disk space than its size. A cache made by an earlier version is only fixed once you clear it (ucache clear); ucache doctor tells you whether yours needs it.
  • ucache clear empties the cache at once and frees its disk space in the background; --wait waits for it.
  • ucache status shows the disk space the cache takes, as du does.

v1.3.4 — 2026-09-29

  • The macOS tarball, which 1.3.3 lacks; otherwise the same as 1.3.3.

v1.3.3 — 2026-09-29

  • XRD_LOGLEVEL=Debug shows uCache's messages in the client's log; a log with a warning starts with a line naming the version and cache directory.
  • Bug fixes:
    • Jobs reading one file with many threads no longer report CRC mismatches; caches that show them heal after the upgrade.
    • With recompress = on, a damaged cached page no longer stops the job.

v1.3.2 — 2026-09-29

  • ucache publish sends more about each run, so reports can compare jobs and machines.
  • Instruction counts are recorded on macOS and are correct on shared machines.
  • Bug fixes:
    • recompress = on works on the 2015 CMS open data.
    • uCache warns when a file cannot be recompressed on its first read.

v1.3.1 — 2026-09-27

  • ucache summary and ucache history show the gain after a first run over an empty cache and a second run; a separate run with the cache off is no longer needed.
  • ucache recompress uses all cores by default (--jobs N to use fewer).
  • Bug fixes:
    • Python process pools (multiprocessing, concurrent.futures) work with uCache.
    • Copying a recompressed file with fsspec or your own script gives the original file, or an error telling you to copy with the cache off.

v1.3.0 — 2026-09-26

  • With recompress = on, a file's replica is built while it is first read; no separate ucache recompress pass is needed.
  • Copies — xrdcp, hadd, rootcp, gfal, ROOT's TFile::Cp and similar — read straight from the origin and leave the cache unchanged.
  • Added max_read_fraction: a job that reads most of a large file reads it from the origin instead of caching it; default 25%.
  • Prebuilt binaries for macOS on Apple silicon: one tarball that uses the XRootD client of the ROOT or XRootD you already have.
  • The package includes ucache_demo.C: a dimuon mass analysis over 100 CMS open-data files that shows the cache working, cold and warm, through the byte cache and the replicas.
  • One package serves XRootD 5 and 6 clients: it carries a build of the plugin for each, and the client loads its own.
  • The RPM no longer requires the XRootD client; uCache uses whichever client the process has loaded.
  • XRD_PLUGIN=<dir>/libXrdClUCache.so with UCACHE_DIR turns uCache on with every default and no config file.
  • The RPM installs a recommended configuration at /usr/share/xrd-ucache/ucache.conf.

v1.2.0 — 2026-09-22

  • Added prefetch: read-ahead for TTree reads, so a first pass waits less on the origin; off by default, on with prefetch = on.

v1.1.1 — 2026-09-16

  • Multi-process readers such as coffea now fill the cache and appear in ucache summary and ucache history the same way a single process does.
  • uproot 5 with its default transport reads cached files.

v1.1.0 — 2026-09-07

  • Added ucache publish: submit benchmark records and run history for analysis and recommendations; data is anonymized.
  • Added ucache identity: view or set the identity used to link published records across machines and browsers.
  • Added announce: uCache identifies itself to XRootD servers so sites can see traffic that comes through a cache; on by default, off with announce = off.

v1.0.0 — 2026-09-02

  • Changed cache data format to support cache gain monitoring.
  • Added ucache summary: total and latest-run time savings.
  • Added ucache history: per-run duration, bytes read, cache tier, and measured gain.
  • ucache stats --reset now archives records instead of deleting them, preserving gain baselines.

v0.21.0 — 2026-08-21

  • XRootD 6 clients are supported — the plugin builds against and loads into 6 as well as 5.
  • A running job no longer evicts its own working set: an entry is not an eviction candidate while it was read within the last day.
  • When the cache is full of such entries it stops caching new files rather than evicting data the job still needs, and ucache status says so. Reads keep working, uncached. evict_protect_seconds = 0 restores the old behaviour.
  • Recompression accepts ZLIB sources by default as well as LZMA, so it no longer declines a ZLIB dataset without being told to.

v0.20.0 — 2026-08-16

  • The plugin, the CLI and the tests build and run on macOS (Apple silicon, MacPorts), and uCache serves root:// reads there.
  • ucache bench measures macOS storage too, and reports how that measurement differs from the Linux one — the two platforms cannot bypass the page cache the same way.
  • Faster checksums on 64-bit Arm.
  • Building against XRootD 6 stops at configure with an explanation, rather than failing deep inside the compiler.

v0.19.1 — 2026-08-13

  • A warm pass over a recompressed RNTuple file no longer re-reads anything from the origin.

v0.19.0 — 2026-08-13

  • RNTuple files can be recompressed into the replica tier — same switch, same behaviour as TTree files.
  • Large reads are cached again: their edge pages were being dropped, and an oversized vector-read element was refused outright.

v0.18.4 — 2026-08-12

  • ucache bench measures a cold fill through uCache's own code rather than an imitation of it.
  • A reference describing every measurement ucache bench makes, and how to read it.

v0.18.3 — 2026-08-10

  • ucache bench records the device it measured and attributes the run's IO to it, so a number can be traced to the hardware that produced it.

v0.18.2 — 2026-08-10

  • ucache bench measures at several stream counts, split into datasheet-comparable and workload-shaped groups, with every run appended to a log.
  • --threads is required rather than guessed: it is the concurrency your jobs actually run at, which no tool can infer.

v0.18.1 — 2026-07-27

  • A failed recompression build is reported and exits non-zero; --strict for callers who want it enforced.
  • The number of background recompression jobs is set by the caller.

v0.18.0 — 2026-07-27

  • Reads of cached data issue one request per contiguous run of pages instead of one per page, on both cache tiers.

v0.17.5 — 2026-07-26

  • Background recompression respects the eviction floor, so it can no longer fill the cache and start evicting the data your job is reading.
  • When it builds nothing, it says which reason applies.
  • Pages are verified against their checksums before being recompressed.

v0.17.4 — 2026-07-25

  • ROOT files that grew past 2 GB after their key list was written can be recompressed; earlier releases refused them.

v0.17.3 — 2026-07-22

  • Prebuilt EL9 packages (RPM and relocatable tarball) attached to each release, built against the oldest supported client so they load under any newer one.
  • ucache doctor detects an XRootD client too old for the plugin; previously it simply did not load, and nothing said why.

v0.17.2 — 2026-07-22

  • First public release: a transparent per-user read cache for root:// data — an XrdCl plugin that caches the pages your jobs read onto local disk, and a ucache command to set it up, inspect it and clean it.
  • Fail-open by construction: any cache problem degrades to a normal uncached read.
  • Optional recompression rewrites cached files into a form that is faster to read back.