Repository navigation
Conversation
harmony7
force-pushed
the
fix/s3-mode-fixes
branch
from
October 20, 2026 12:13
07ee1e8 to
3e47a4b
Compare
harmony7
requested review from
a team and
kailan
and removed request for
a team
October 20, 2026 12:16
After a collection is deleted, the service can continue to serve its cached responses until they expire. collections delete now purges the surrogate key <publishId>-<collection> after the delete, with the same options and rules as publish-content: --fastly-service-id, --purge-environment, and the API token check before anything is deleted. The purge loop moves from publish-content to purgeCollection() in util/purge.ts, so that both commands use it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In S3 mode, the commands ignored --local for storage. publish-content uploaded to the bucket, and collections delete and clean deleted from it. There is no local simulation of S3 storage. The S3 storage provider builder now throws an error if --local is set, before it prints or connects to anything. All commands that accept --local (publish-content, collections list, delete, promote, update-expiration, and clean) load the provider through this builder, so they all stop with exit code 1. --help and the README now tell that --local is for KV Store mode only. The new e2e test needs no network and no credentials. It uses an endpoint where nothing listens, so a command that connects fails with ECONNREFUSED. Fixes #111. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
publish-content scans up to 16 files at the same time. Files with the same content checked the variant map before the first of them set it, because the map was set only after the compression and the hash. Thus each of these files compressed the content again (mostly visible for Brotli), and added the same keys to the upload batch. Several files could also write the same variant file at the same time, and the hash could read a file that another write had truncated. The new OncePerKey helper keeps one promise for each variant storage key. The first file starts the work and stores the promise before its first await. The other files await the same promise. Only the first file adds the key to the batch, and the variant file is written one time. publish-content --local now shows the number of variants to upload, because it does not list the keys in storage and showed no count before. Tests: a unit test for OncePerKey, and an e2e test (no network, no credentials) that publishes 40 files with the same content and 1 other file with --local. It expects 2 Brotli and 2 gzip compressions and 6 variants to upload. Before the fix, it had many more compressions. Fixes #112. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
harmony7
force-pushed
the
fix/s3-mode-fixes
branch
from
October 22, 2026 00:17
3e47a4b to
6520212
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three fixes for 8.0, before the first stable release with S3-compatible storage. Each fix is one commit.
This PR replaces #110. Its commit is the first commit here, with no changes.
Closes #111.
Closes #112.
PR Guide:
review-113.pdf
1. Purge the collection after
collections delete(from #110)After a collection is deleted, the service can continue to serve its cached responses until they expire.
collections deletenow purges the surrogate key<publishId>-<collection>, the same aspublish-contentdoes after a publish.publish-content:--fastly-service-id(thenFASTLY_SERVICE_ID, thenservice_idinfastly.toml),--purge-environment, and the API token. If the CLI finds a Service ID but no API token, it stops before it deletes anything. If it finds no Service ID, or with--local, it skips the purge. If the purge fails, it shows a warning, and the command completes.purgeCollection()insrc/cli/util/purge.ts. Both commands use it.collections deletefinds a Service ID, it now needs an API token that can purge that service. MIGRATING.md says this.2.
--localin S3 mode stops with an error (#111)In S3 mode, the commands ignored
--localfor storage.publish-content --localuploaded to the bucket, andcollections delete --localandclean --localdeleted from it. There is no local simulation of S3 storage.--localis set, before it prints or connects to anything. All six commands that accept--localload the provider through this builder, so they all stop with exit code 1:--local is available only in KV Store mode. In S3 mode, the commands always use the bucket in static-publish.rc.js.--local(it has onlys3:publish), so it does not change.--localhad one effect before:publish-contentskipped the purge. The CHANGELOG tells users who used it for that what to do.--helpand the README now tell that--localis for KV Store mode only.3. Prepare each variant of the same content one time (#112)
publish-contentscans up to 16 files at the same time. Files with the same content checked the variant map before the first of them set it. Thus each of these files compressed the content again (mostly visible for Brotli), and added the same keys to the upload batch (in S3 mode, onePutObjectfor each duplicate). Several files could also write the same variant file at the same time.OncePerKeyhelper (src/cli/util/variants.ts) keeps one promise for each variant storage key. The first file starts the work and stores the promise before its firstawait. The other files await the same promise. Only the first file adds the key to the batch.publish-content --localnow shows the number of variants to upload. Before, local mode showed no count, because it does not list the keys in storage.Tests
npm test: 77 pass. New: unit tests forpurgeCollection()(from Purge the collection after collections delete #110) and forOncePerKey.test/e2e/s3-local-flag.test.js: each of the six commands with--localin S3 mode stops before it connects. The endpoint is a port where nothing listens, so a command that connects fails withECONNREFUSED. Without the fix, all six fail.test/e2e/same-content.test.js: a--localpublish of 40 files with the same content and 1 other file. It expects 2 Brotli and 2 gzip compressions, and 6 variants to upload. Without the fix, there are more compressions.test/e2e/kv-store.test.jsandtest/e2e/s3.test.jswith real test resources: 24 pass.deployed.test.jswas not run.style.css. After the first publish, these tests compare the number of variants to upload with the number of file keys in storage. Before the fix, both tests fail (23 uploads for 17 keys).collections delete --localwith a Service ID stops with the--localerror before it connects to storage.🤖 Generated with Claude Code