Skip to content

Commit 2484ced

Browse files
authored
Update Storage plugin to version 1.0.3
- Removed unnecessary entry processing and unnecessary cache invalidations
2 parents 4d89ace + c9922a1 commit 2484ced

3 files changed

Lines changed: 387 additions & 166 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,12 @@
5353
- Re-registering the same email address replaces older, still-open pending tokens.
5454
- Old confirmation links for the same address are thereby invalidated.
5555
- If a local blocklist is missing, a download is attempted before the first form processing.
56-
- ReadMore plugin update zu version 1.0.4
56+
- ReadMore plugin update to version 1.0.4
5757
- Revised visibility layer ([#939](https://github.com/flatpressblog/flatpress/pull/939))
58+
- Staroge plugin update to 1.0.3
59+
- Removed unnecessary entry processing and unnecessary cache invalidations. ([#943](https://github.com/flatpressblog/flatpress/pull/943))
60+
- Mediamanger plugin update to version 2.0.2
61+
- Incremental, file-based usage index. ([#942](https://github.com/flatpressblog/flatpress/pull/942))
5862

5963
### Bugfixes
6064
- PhotoSwipe plugin update to version 2.0.7

‎docs/FlatPress_APCu_Cache_Overview.md‎

Lines changed: 38 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -560,13 +560,13 @@ Medium–High in widgets-heavy setups. Calendar widgets are often present on eve
560560
- `fp:storage:aggregate:vN`
561561
- `fp:storage:dirsize:<channel>[:nth][:ncc]:vN`
562562
- `fp:storage:quota:vN`
563-
- `fp:storage:dirsize:v1:<sha1(root)>` (FlatPress folder total size)
563+
- `fp:storage:dirsize:root:<sha1(root)>:vN` (FlatPress folder total size)
564564

565565
**File:** `fp-plugins/storage/plugin.storage.php`
566566

567567
**What is cached:**
568568

569-
- Storage aggregates (entries/comments counters, top lists, etc.).
569+
- Storage aggregates (entry/comment counters and byte sizes, plus the optional Top-10 comment list).
570570
- Directory size computations:
571571
- Per storage channel (e.g. `images`, `attachs`), optionally excluding `.thumbs` (`:nth`)
572572
and/or `.captions.conf` (`:ncc`).
@@ -585,20 +585,40 @@ Medium–High in widgets-heavy setups. Calendar widgets are often present on eve
585585
- APCu: `fp:storage:quota:vN` with default TTL 3600s (payload includes `ts`).
586586
- File fallback: `fp-content/cache/storage.quota.json` with TTL = `$ttl` (based on file mtime).
587587
- FlatPress folder total size:
588-
- APCu: `fp:storage:dirsize:v1:<sha1(root)>` with TTL 120s.
588+
- APCu: `fp:storage:dirsize:root:<sha1(root)>:vN` with TTL 120s.
589589
- File fallback: `fp-content/cache/storage.dirsize.json` with TTL 120s.
590590

591+
**Cold rebuild path:**
592+
593+
- The Storage panel resolves the aggregate cache and FlatPress-root-size cache before traversing the filesystem.
594+
- If both caches are cold and `CONTENT_DIR` is below `BASE_DIR`, one `RecursiveDirectoryIterator` traversal of the FlatPress root computes both result sets.
595+
- If only the aggregate is cold, only `CONTENT_DIR` is streamed.
596+
- If only the root-size cache is cold, only the root-size metric is collected.
597+
- Entry and comment files are counted and sized directly from `SplFileInfo`; the Storage scanner does **not** call `bdb_parse_entry()` and does not materialize a complete `fs_pathlister` array.
598+
- Symbolic links are not followed. `RecursiveIteratorIterator::CATCH_GET_CHILD` and per-file exception handling preserve best-effort behavior on restrictive/shared-host filesystems.
599+
591600
**Invalidation:**
592601

593-
- Namespaced via `plugin_storage_cache_ns()` reading `fp:storage:v`.
594-
- Version bump via `plugin_storage_cache_bump()` (`apcu_incr('fp:storage:v', …)`) is triggered by:
595-
- `publish_post`, `delete_post`, `comment_save`, `comment_delete`
596-
- File invalidation:
597-
- `plugin_storage_cache_bump()` deletes **only** `storage.aggregate.json` (best effort).
598-
Other file fallbacks rely on their TTL and are refreshed on demand.
602+
- APCu data is namespaced through `plugin_storage_cache_ns()` and `fp:storage:v`.
603+
- The namespace value is memoized for the current request so an in-progress scan remains tied to the generation it started with. After a successful bump, the mutation request refreshes its own memoized namespace immediately.
604+
- `plugin_storage_cache_bump()` is bound only to post-success hooks:
605+
- `entry_saved`
606+
- `entry_deleted`
607+
- `comment_saved`
608+
- `comment_deleted`
609+
- The file fallbacks `storage.aggregate.json` and `storage.dirsize.json` are purged on every bump **even when APCu is unavailable**.
610+
- When APCu is available, the same bump increments `fp:storage:v`, invalidating aggregate, root-size, channel-size and quota APCu keys by generation.
611+
- Channel JSON fallbacks (`storage.dirsize.*.json`) and `storage.quota.json` are not purged by content mutations; they retain their own TTL-based refresh because entry/comment writes do not change image/attachment content or hosting quota limits directly.
612+
613+
**Write safety:**
614+
615+
- Before an APCu-backed cold scan is persisted, the Storage plugin rechecks the generation it started with. If a concurrent successful mutation bumped `fp:storage:v`, that scan result is used only for the current response and is **not** written into the newer APCu generation or its JSON fallback.
616+
- Aggregate and root-size JSON fallbacks are written through `io_write_file()`, i.e. temp-file + rename atomic replacement.
617+
- Cache write or cache purge failures are best effort and never participate in the success/failure decision of entry or comment writes.
618+
- APCu remains optional. With APCu disabled, the same Storage metrics and invalidation semantics continue to work through the JSON fallbacks; cross-request generation detection is not available without a shared in-memory backend, so the short file TTL remains the final stale-data bound for external/concurrent changes that bypass normal hooks.
599619

600620
**Impact:**
601-
Low–Medium. Mostly relevant for admin and diagnostics; not on every frontend request.
621+
Medium–High for large installations in the Storage admin panel. The warm path stays cache-backed, while a complete aggregate/root cache miss avoids per-entry parsing, full path-list materialization and a second full FlatPress-tree traversal.
602622

603623
---
604624

@@ -1020,10 +1040,13 @@ Some features use a **dual-layer cache** (APCu + file fallback) to stay fast eve
10201040
- Invalidation: version bump (`fp:calendar:v`) plus file purge (`calendar-*.html`) on `plugin_calendar_cache_bump()`.
10211041

10221042
- **Storage** (`fp-plugins/storage/plugin.storage.php`)
1023-
- Aggregate JSON: `fp-content/cache/storage.aggregate.json` (purged on `plugin_storage_cache_bump()` when APCu is on).
1043+
- Aggregate JSON: `fp-content/cache/storage.aggregate.json`.
1044+
- FlatPress root-size JSON: `fp-content/cache/storage.dirsize.json`.
1045+
- Both files are purged by `plugin_storage_cache_bump()` after successful entry/comment mutations, regardless of APCu availability.
10241046
- Additional JSON fallbacks:
1025-
- `storage.dirsize.*.json`, `storage.quota.json`, `storage.dirsize.json`
1026-
(TTL-based; refreshed on demand; not globally purged on version bumps).
1047+
- `storage.dirsize.*.json`, `storage.quota.json`
1048+
(TTL-based; refreshed on demand; not purged by entry/comment cache bumps).
1049+
- Storage JSON writes use the core atomic `io_write_file()` path.
10271050

10281051
- **Mastodon** (`fp-plugins/mastodon/plugin.mastodon.php`)
10291052
- Durable state: `fp-content/plugin_mastodon/state.json`.
@@ -1104,7 +1127,7 @@ The following table summarizes each logical cache group:
11041127
| BBCode | `fp:bbcode:*` | No | Parser/img/meta mtimes, TTL 300–7200s | Medium–High |
11051128
| Archives | `fp:archives:v`, `fp:archives:list*`, `fp:archives:html*` | **Yes** | `plugin_archives_cache_bump()` + PrettyURLs bump | Medium |
11061129
| Calendar | `fp:calendar:v`, `calendar:*:vN` | **Yes** | `plugin_calendar_cache_bump()` + PrettyURLs bump | Medium–High |
1107-
| Storage plugin | `fp:storage:v`, `fp:storage:aggregate*`, `fp:storage:dirsize*`, `fp:storage:quota*` | No | Storage rescan + TTL | Low–Medium |
1130+
| Storage plugin | `fp:storage:v`, `fp:storage:aggregate*`, `fp:storage:dirsize*`, `fp:storage:quota*` | No | Post-success hooks + generation bump + JSON purge/TTL | Medium–High |
11081131
| Mastodon instance snapshot | `fp:mastodon:instance_document:<sha1(instance_url)>` | No | TTL 900s, `instance_url` change, snapshot refresh | Low–Medium |
11091132
| Media Manager usage index | `mediamanager:usage-index:v1` | No | `entry_saved`/`entry_deleted`, dirty recovery | High (large sites) |
11101133
| Mastodon scheduler summary | core `fp:io:*` for `scheduler-state.json` | No | File mtime/size via core I/O, rebuilt from `state.json` when stale | Medium |
@@ -1163,7 +1186,7 @@ For completeness, the following logical prefixes are used by FlatPress `1.6.dev`
11631186
- `fp:statics:list:`
11641187
- `fp:storage:aggregate`
11651188
- `fp:storage:dirsize:`
1166-
- `fp:storage:dirsize:v1:`
1189+
- `fp:storage:dirsize:root:`
11671190
- `fp:storage:quota`
11681191
- `fp:storage:v`
11691192
- `fp:admin:setup_hide_report`

0 commit comments

Comments
 (0)