Add an auto_packaging switch so entries are packaged only on Update Package (0.21.5)

CHANGELOG.md

@@ -5,6 +5,17 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
+## [0.21.5] - 2026-10-26
+
+### Added
+
+- `auto_packaging` in the webhook secret (`packages.autoPackaging` in a profile config, synced by `sync-secrets`). The setup wizard asks "Package entries automatically on every entry event?" with the package settings; answering No turns it off. When `"false"`, `entry.created` and `entry.updated.fields` events and new Canvases no longer queue a packaging run: the Canvas is drawn from the entry's current package, or shows "Package Not Created", with the time it was drawn in the footer, and an entry is packaged only when a user clicks **Update Package** or accepts its review. Review events still run so that acceptance seals the package and reopening unseals it. Defaults to `"true"`, so existing deployments are unchanged. The secret is re-read within about a minute, so no redeploy is needed. An unparseable value is a configuration error (#395, #392)
+- The setup wizard keeps an existing secret's `auto_packaging` value when it rebuilds a profile from the secret, so re-running the wizard does not turn automatic packaging back on
+
+### Fixed
+
+- The App Canvas says "Package Not Created" for an entry whose package does not exist yet, instead of showing catalog and sync links to a package that isn't there
+
 ## [0.21.4] - 2026-10-13
 
 ### Changed

README.md

@@ -43,6 +43,22 @@ To put each scientist's notebook packages in the same namespace as their raw-dat
 - Existing packages are not moved when the prefix changes. An entry's Canvas looks for its package under the new name, which is created the next time the entry is packaged.
 - If the bucket's Quilt workflow sets a `handle_pattern`, it must accept the new namespaces.
 
+#### Turning Automatic Packaging Off
+
+By default every supported entry event (`v2.entry.created`, `v2.entry.updated.fields`, `v2.entry.updated.reviewRecord`) and every new Canvas queues a packaging run.
+To package entries only when a scientist asks for it, answer **No** to the setup wizard's "Package entries automatically on every entry event?" prompt.
+That stores `packages.autoPackaging: false` in the profile config and syncs it to the webhook secret as `auto_packaging: "false"`; you can also edit either one directly (`npx @quiltdata/benchling-webhook sync-secrets` pushes a profile change to the secret).
+
+With automatic packaging off:
+
+- `v2.entry.created` and `v2.entry.updated.fields` events are acknowledged and skipped; nothing is queued.
+- `v2.entry.updated.reviewRecord` events still run, because accepting a review seals the package and reopening the review unseals it (see [Accepted Reviews](#accepted-reviews)). Accepting a review therefore still packages the entry.
+- A new Canvas shows the entry's current package if one exists, otherwise "Package Not Created", together with the linked packages, and its footer shows when it was drawn. It never waits on "Updating...".
+- **Update Package** on the Canvas still queues a packaging run, so it is the only other way an entry is packaged.
+
+The secret is re-read within about a minute, so the switch takes effect without a redeploy.
+Packages already created stay where they are. Turning it back on packages an entry the next time Benchling sends an event for it.
+
 #### Package Metadata
 
 The package metadata is the package's `entry.json`. Besides the entry's IDs, name, creator, authors, and timestamps, it includes the entry's schema `fields` and `customFields`. Benchling keys these by display name, so the webhook re-keys them in snake_case: "Experiment Type" becomes `experiment_type` and "ELN-ID #" becomes `eln_id`. Each key maps to the field's `displayValue`, the text the scientist sees in Benchling: a dropdown option's or linked entity's name, not its ID. Custom fields have no `displayValue`, so they map to their `value`. Both keys are always present, as `{}` when the entry has none. The package's `entry_data.json` keeps the raw entry, with each field's display name, `value`, `type`, and so on.
@@ -243,7 +259,7 @@ The wizard will guide you through:
 1. **Catalog discovery** - Detect your Quilt catalog configuration
 2. **Stack validation** - Extract settings from your CloudFormation stack
 3. **Credential collection** - Enter Benchling app credentials
-4. **Package settings** - Configure bucket, package prefix (see [Package Naming](#package-naming)), metadata key, and optional Quilt workflow
+4. **Package settings** - Configure bucket, package prefix (see [Package Naming](#package-naming)), metadata key, optional Quilt workflow, and whether entries are packaged automatically (see [Turning Automatic Packaging Off](#turning-automatic-packaging-off))
 5. **Deployment mode selection**:
    - **Integrated**: Uses your Quilt stack's built-in webhook, if any
    - **Standalone**: Deploys a separate webhook stack for testing

bin/commands/setup-wizard.ts

@@ -77,6 +77,7 @@ export interface SetupWizardOptions {
     pkgPrefix?: string;
     pkgKey?: string;
     workflow?: string;
+    autoPackaging?: boolean;
     logLevel?: string;
     webhookAllowList?: string;
 }
@@ -310,6 +311,7 @@ export async function runSetupWizard(options: SetupWizardOptions = {}): Promise<
             pkgPrefix: options.pkgPrefix,
             pkgKey: options.pkgKey,
             workflow: options.workflow,
+            autoPackaging: options.autoPackaging,
             logLevel: options.logLevel,
             webhookAllowList: options.webhookAllowList,
         });

bin/commands/sync-secrets.ts

@@ -300,6 +300,7 @@ function buildSecretValue(config: ProfileConfig, clientSecret: string): string {
         pkg_prefix: config.packages.prefix,
         pkg_key: config.packages.metadataKey,
         ...(config.packages.workflow ? { workflow: config.packages.workflow } : {}),
+        auto_packaging: config.packages.autoPackaging === false ? "false" : "true",
         log_level: config.logging?.level || "INFO",
         webhook_allow_list: config.security?.webhookAllowList || "",
         enable_webhook_verification: config.security?.enableVerification !== false ? "true" : "false",

docker/app-manifest.yaml

@@ -2,7 +2,7 @@ manifestVersion: 1
 info:
   name: nightly-quilttest-com
   description: Packaging Benchling Notebooks as Quilt packages
-  version: 0.21.4
+  version: 0.21.5
 features:
   - name: Quilt Connector
     id: quilt_entry

docker/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "benchling-quilt-integration"
-version = "0.21.4"
+version = "0.21.5"
 description = "Benchling-Quilt Integration Webhook Service"
 license = {text = "Apache-2.0"}
 authors = [

docker/src/app.py

@@ -710,12 +710,13 @@ def _send_updating_canvas_best_effort(payload: Payload) -> None:
                 error=str(exc),
             )
 
-    def _send_bucketless_final_canvas_best_effort(payload: Payload) -> None:
-        """Start the final bucketless Canvas render after the quick acknowledgment.
+    def _send_final_canvas_best_effort(payload: Payload) -> None:
+        """Start the final Canvas render after the quick acknowledgment.
 
-        Bucketless mode has no packaging workflow to produce a later package
-        revision event, so the Canvas handler must render the final linked-package
-        view itself. Run it asynchronously to keep the webhook response fast.
+        Used when no packaging request is queued (bucketless mode, or automatic
+        packaging turned off), so no later package-revision event will redraw
+        the Canvas: render the current package and linked-package view now.
+        Run it asynchronously to keep the webhook response fast.
         """
         if not payload.canvas_id or benchling is None or config is None:
             return
@@ -745,6 +746,22 @@ def _bucketless_response(payload: Payload, message: str) -> JSONResponse:
             status_code=202,
         )
 
+    def _manual_packaging_response(payload: Payload, message: str) -> JSONResponse:
+        logger.info(
+            "Automatic packaging is off; skipped package creation",
+            entry_id=payload.entry_id,
+            event_type=payload.event_type,
+        )
+        return JSONResponse(
+            {
+                "entry_id": payload.entry_id,
+                "status": "SKIPPED",
+                "mode": "manual",
+                "message": message,
+            },
+            status_code=202,
+        )
+
     async def _handle_event_impl(request: Request, _verified: None = None):
         """Shared event webhook handling implementation."""
         try:
@@ -774,11 +791,17 @@ async def _handle_event_impl(request: Request, _verified: None = None):
                 )
                 _send_updating_canvas_best_effort(payload)
                 if not config.s3_bucket_name:
-                    _send_bucketless_final_canvas_best_effort(payload)
+                    _send_final_canvas_best_effort(payload)
                     return _bucketless_response(
                         payload,
                         "Bucketless mode does not create a default package for canvas events.",
                     )
+                if not config.auto_packaging:
+                    _send_final_canvas_best_effort(payload)
+                    return _manual_packaging_response(
+                        payload,
+                        "Automatic packaging is off; the Canvas shows the current package.",
+                    )
                 publish_packaging_request(entry_packager.sqs_client, packaging_queue_url, payload)
 
                 logger.info("Canvas update initiated from /event endpoint", canvas_id=payload.canvas_id)
@@ -804,6 +827,13 @@ async def _handle_event_impl(request: Request, _verified: None = None):
                     payload,
                     "Bucketless mode does not create a default package for entry events.",
                 )
+            # A review event seals the package on acceptance and unseals it when the
+            # review is reopened, so it must run even when automatic packaging is off.
+            if not config.auto_packaging and payload.event_type != "v2.entry.updated.reviewRecord":
+                return _manual_packaging_response(
+                    payload,
+                    "Automatic packaging is off; click Update Package on the entry's Canvas to package it.",
+                )
 
             publish_packaging_request(entry_packager.sqs_client, packaging_queue_url, payload)
 
@@ -948,11 +978,17 @@ async def _handle_canvas_impl(request: Request, _verified: None = None):
 
             _send_updating_canvas_best_effort(payload)
             if not config.s3_bucket_name:
-                _send_bucketless_final_canvas_best_effort(payload)
+                _send_final_canvas_best_effort(payload)
                 return _bucketless_response(
                     payload,
                     "Bucketless mode does not create a default package for canvas initialization.",
                 )
+            if not config.auto_packaging:
+                _send_final_canvas_best_effort(payload)
+                return _manual_packaging_response(
+                    payload,
+                    "Automatic packaging is off; the Canvas shows the current package.",
+                )
             sqs_message_id = publish_packaging_request(entry_packager.sqs_client, packaging_queue_url, payload)
 
             logger.info(
@@ -1193,13 +1229,14 @@ def handle_refresh_canvas(payload, button_id, benchling, config):
         webhook was reconfigured and restarted with a bucket after this canvas
         was drawn), transition to the bucketed flow — enqueue a packaging
         request to create the default package and render the standard canvas.
-        Otherwise, re-run the bucketless linked-package lookup.
+        Otherwise, or when automatic packaging is off, re-render the Canvas
+        from the current package and linked-package lookup without enqueuing.
         """
         logger.info("Refresh canvas requested", entry_id=payload.entry_id, button_id=button_id)
 
         _send_updating_canvas_best_effort(payload)
 
-        if config.s3_bucket_name:
+        if config.s3_bucket_name and config.auto_packaging:
             # A bucket was configured after this (bucketless) canvas was drawn:
             # initiate default package creation instead of re-rendering bucketless.
             assert entry_packager is not None
@@ -1221,7 +1258,7 @@ def handle_refresh_canvas(payload, button_id, benchling, config):
                 status_code=202,
             )
 
-        _send_bucketless_final_canvas_best_effort(payload)
+        _send_final_canvas_best_effort(payload)
         return JSONResponse({"status": "ACCEPTED", "message": "Refreshing canvas..."}, status_code=202)
 
     def handle_view_metadata(payload, button_id, benchling, config):

docker/src/canvas.py

@@ -17,7 +17,7 @@
 from . import canvas_blocks as blocks
 from . import canvas_formatting as fmt
 from .config import Config
-from .package_files import PackageFile, PackageFileFetcher
+from .package_files import PackageFile, PackageFileFetcher, is_not_found
 from .package_naming import MissingPlaceholderError, entry_package_name
 from .package_query import ELN_ENTRY_KEY, PackageQuery
 from .packages import Package
@@ -245,12 +245,15 @@ def _make_markdown_content(self) -> str:
         if unnamed:
             content = self._make_unnamed_markdown(unnamed)
         elif self.config.s3_bucket_name:
-            content = fmt.format_package_header(
-                package_name=self.package_name,
-                display_id=self.entry.display_id,
-                catalog_url=self.catalog_url,
-                sync_url=self.sync_uri(),
-            )
+            if self._package_exists():
+                content = fmt.format_package_header(
+                    package_name=self.package_name,
+                    display_id=self.entry.display_id,
+                    catalog_url=self.catalog_url,
+                    sync_url=self.sync_uri(),
+                )
+            else:
+                content = fmt.format_package_not_found(self.package_name)
             content += self._packaging_status_markdown()
         else:
             content = (
@@ -329,6 +332,26 @@ def _packaging_status_markdown(self, sealed: bool = False) -> str:
             )
         return ""
 
+    def _package_exists(self) -> bool:
+        """Whether the entry's package has a latest revision.
+
+        Only a definite not-found says no. Any other error is logged and the
+        package is rendered as existing, as it was before this check.
+        """
+        try:
+            self._package_file_fetcher.get_package_top_hash(self.package_name)
+        except ClientError as exc:
+            if is_not_found(exc):
+                return False
+            logger.warning(
+                "Could not check whether the package exists", package_name=self.package_name, error=str(exc)
+            )
+        except Exception as exc:  # noqa: BLE001 - render as before
+            logger.warning(
+                "Could not check whether the package exists", package_name=self.package_name, error=str(exc)
+            )
+        return True
+
     def _load_seal(self) -> Optional[Tuple[str, dict]]:
         """Return the entry package's sealed top hash and seal, or None if it is unsealed.
 
@@ -837,8 +860,11 @@ def _handle(self) -> None:
         """Handle Canvas webhook payload."""
         try:
             logger.info("Updating canvas", canvas_id=self.canvas_id, entry_id=self.entry_id)
+            # With no packaging request queued (bucketless, or automatic packaging
+            # off) no package-revision event will follow to stamp the footer, so
+            # this render is the final one.
             updated_at = None
-            if not self.config.s3_bucket_name:
+            if not self.config.s3_bucket_name or not self.config.auto_packaging:
                 updated_at = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
             self.update_canvas(updated_at=updated_at)
 

docker/src/config.py

@@ -51,6 +51,7 @@ class Config:
     enable_webhook_verification: bool = True
     pkg_prefix: str = ""
     workflow: str = ""
+    auto_packaging: bool = True
     quilt_write_role_arn: str = ""
     quilt_iceberg_database: str = ""
 
@@ -114,6 +115,7 @@ def __post_init__(self):
         self.s3_prefix = "benchling"
         self.package_key = "experiment_id"
         self.pkg_prefix = "benchling"
+        self.auto_packaging = True
 
         # Application configuration
         self.app_env = os.getenv("APP_ENV", os.getenv("ENVIRONMENT", "production"))
@@ -147,6 +149,7 @@ def __post_init__(self):
                 '  "pkg_key": "experiment_id",\n'
                 '  "user_bucket": "s3-bucket-name (optional)",\n'
                 '  "log_level": "INFO",\n'
+                '  "auto_packaging": "true",\n'
                 '  "enable_webhook_verification": "true"\n'
                 "}\n"
             )
@@ -312,6 +315,7 @@ def apply_benchling_secrets(self, secret_data: BenchlingSecretData) -> None:
             self.pkg_prefix = self.s3_prefix
             self.package_key = secret_data.pkg_key or "experiment_id"
             self.workflow = secret_data.workflow or ""
+            self.auto_packaging = secret_data.auto_packaging
 
             # Security configuration from secret, unless env var explicitly disables it
             env_override = os.getenv("ENABLE_WEBHOOK_VERIFICATION", "").lower()

docker/src/config_schema.py

@@ -183,6 +183,10 @@ class BenchlingSecret(BaseModel):
     enable_webhook_verification: str = Field(
         "true", description="Enable Lambda authorizer for webhook signature verification (API Gateway)"
     )
+    workflow: Optional[str] = Field(None, description="Quilt workflow name for package creation (optional)")
+    auto_packaging: str = Field(
+        "true", description="Package entries automatically on entry and Canvas events; 'false' for Update Package only"
+    )
     queue_url: Optional[str] = Field(None, description="SQS queue URL (optional)")
 
     class Config:
@@ -197,12 +201,12 @@ def validate_log_level(cls, v: str) -> str:
             raise ValueError(f"Invalid log level: {v}. Must be one of {valid_levels}")
         return v.upper()
 
-    @field_validator("enable_webhook_verification")
+    @field_validator("enable_webhook_verification", "auto_packaging")
     @classmethod
-    def validate_webhook_verification(cls, v: str) -> str:
-        """Validate webhook verification flag"""
+    def validate_bool_flag(cls, v: str) -> str:
+        """Validate a 'true'/'false' flag"""
         if v.lower() not in ["true", "false"]:
-            raise ValueError(f"Invalid webhook verification flag: {v}. Must be 'true' or 'false'")
+            raise ValueError(f"Invalid flag value: {v}. Must be 'true' or 'false'")
         return v.lower()