Skip to content

Commit 424354f

Browse files
author
stlc-bot
committed
fix(scrape): reuse legacy caches across all output formats (#1197)
Stainless-Generated-From: 3bf3428
1 parent 7de59fc commit 424354f

4 files changed

Lines changed: 7 additions & 7 deletions

File tree

‎src/ServiceContracts/WebContract.php‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -220,7 +220,7 @@ public function extractStyleguide(
220220
* @param string $url the URL to scrape
221221
* @param ImageParams|ImageParamsShape $imageParams Image options. Requires formats.images: true.
222222
* @param MarkdownParams|MarkdownParamsShape $markdownParams Markdown options. Requires formats.markdown: true.
223-
* @param int $maxAgeMs Maximum age for the entire capture, including bytes. Defaults to 1 day; 0 fetches fresh. Captures with hosted image files refresh after 23 hours.
223+
* @param int $maxAgeMs Maximum age of each cached output. Defaults to 1 day; 0 fetches fresh and updates the requested outputs. Compatible outputs are shared with the individual scrape endpoints. Image results with hosted files refresh after 23 hours; other outputs retain their own freshness.
224224
* @param ParseParams|ParseParamsShape $parseParams Required when formats.parse is true.
225225
* @param ScreenshotParams|ScreenshotParamsShape $screenshotParams Screenshot options. Requires formats.screenshot: true.
226226
* @param SharedParams|SharedParamsShape $sharedParams Shared browser and content settings. Content filters leave screenshots and original bytes unchanged.

‎src/Services/WebRawService.php‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -306,7 +306,7 @@ public function extractStyleguide(
306306
/**
307307
* @api
308308
*
309-
* Capture the requested formats from one page visit. Shared settings apply once. HTML-only requests use the existing fast acquisition path. One credit per capture, or two with browser actions; PDF OCR adds one credit per recovered page. Original response bytes and screenshots are limited to 20 MiB each, screenshots to 40 megapixels, and the combined browser capture to 60 MiB.
309+
* Reuse cached outputs independently and capture missing formats in one page visit. Each cache key includes only the settings that affect that output. HTML is shared with Markdown and parsed fields. Cached outputs can come from different visits within maxAgeMs; use 0 for a fresh capture. HTML-only requests use the existing fast acquisition path. One credit per request, including cache hits, or two with browser actions; PDF OCR adds one credit per recovered page on fresh extraction. Original response bytes and screenshots are limited to 20 MiB each, screenshots to 40 megapixels, and the combined browser capture to 60 MiB.
310310
*
311311
* @param array{
312312
* formats: Formats|FormatsShape,

‎src/Services/WebService.php‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -332,13 +332,13 @@ public function extractStyleguide(
332332
/**
333333
* @api
334334
*
335-
* Capture the requested formats from one page visit. Shared settings apply once. HTML-only requests use the existing fast acquisition path. One credit per capture, or two with browser actions; PDF OCR adds one credit per recovered page. Original response bytes and screenshots are limited to 20 MiB each, screenshots to 40 megapixels, and the combined browser capture to 60 MiB.
335+
* Reuse cached outputs independently and capture missing formats in one page visit. Each cache key includes only the settings that affect that output. HTML is shared with Markdown and parsed fields. Cached outputs can come from different visits within maxAgeMs; use 0 for a fresh capture. HTML-only requests use the existing fast acquisition path. One credit per request, including cache hits, or two with browser actions; PDF OCR adds one credit per recovered page on fresh extraction. Original response bytes and screenshots are limited to 20 MiB each, screenshots to 40 megapixels, and the combined browser capture to 60 MiB.
336336
*
337337
* @param Formats|FormatsShape $formats Outputs to return. Enable at least one; omitted formats are false.
338338
* @param string $url the URL to scrape
339339
* @param ImageParams|ImageParamsShape $imageParams Image options. Requires formats.images: true.
340340
* @param MarkdownParams|MarkdownParamsShape $markdownParams Markdown options. Requires formats.markdown: true.
341-
* @param int $maxAgeMs Maximum age for the entire capture, including bytes. Defaults to 1 day; 0 fetches fresh. Captures with hosted image files refresh after 23 hours.
341+
* @param int $maxAgeMs Maximum age of each cached output. Defaults to 1 day; 0 fetches fresh and updates the requested outputs. Compatible outputs are shared with the individual scrape endpoints. Image results with hosted files refresh after 23 hours; other outputs retain their own freshness.
342342
* @param ParseParams|ParseParamsShape $parseParams Required when formats.parse is true.
343343
* @param ScreenshotParams|ScreenshotParamsShape $screenshotParams Screenshot options. Requires formats.screenshot: true.
344344
* @param SharedParams|SharedParamsShape $sharedParams Shared browser and content settings. Content filters leave screenshots and original bytes unchanged.

‎src/Web/WebScrapeParams.php‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
use ContextDev\Web\WebScrapeParams\Zdr;
1919

2020
/**
21-
* Capture the requested formats from one page visit. Shared settings apply once. HTML-only requests use the existing fast acquisition path. One credit per capture, or two with browser actions; PDF OCR adds one credit per recovered page. Original response bytes and screenshots are limited to 20 MiB each, screenshots to 40 megapixels, and the combined browser capture to 60 MiB.
21+
* Reuse cached outputs independently and capture missing formats in one page visit. Each cache key includes only the settings that affect that output. HTML is shared with Markdown and parsed fields. Cached outputs can come from different visits within maxAgeMs; use 0 for a fresh capture. HTML-only requests use the existing fast acquisition path. One credit per request, including cache hits, or two with browser actions; PDF OCR adds one credit per recovered page on fresh extraction. Original response bytes and screenshots are limited to 20 MiB each, screenshots to 40 megapixels, and the combined browser capture to 60 MiB.
2222
*
2323
* @see ContextDev\Services\WebService::scrape()
2424
*
@@ -74,7 +74,7 @@ final class WebScrapeParams implements BaseModel
7474
public ?MarkdownParams $markdownParams;
7575

7676
/**
77-
* Maximum age for the entire capture, including bytes. Defaults to 1 day; 0 fetches fresh. Captures with hosted image files refresh after 23 hours.
77+
* Maximum age of each cached output. Defaults to 1 day; 0 fetches fresh and updates the requested outputs. Compatible outputs are shared with the individual scrape endpoints. Image results with hosted files refresh after 23 hours; other outputs retain their own freshness.
7878
*/
7979
#[Optional]
8080
public ?int $maxAgeMs;
@@ -235,7 +235,7 @@ public function withMarkdownParams(
235235
}
236236

237237
/**
238-
* Maximum age for the entire capture, including bytes. Defaults to 1 day; 0 fetches fresh. Captures with hosted image files refresh after 23 hours.
238+
* Maximum age of each cached output. Defaults to 1 day; 0 fetches fresh and updates the requested outputs. Compatible outputs are shared with the individual scrape endpoints. Image results with hosted files refresh after 23 hours; other outputs retain their own freshness.
239239
*/
240240
public function withMaxAgeMs(int $maxAgeMs): self
241241
{

0 commit comments

Comments
 (0)