Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 86 additions & 32 deletions doc/async-rest-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,45 @@
Goal: keep blocking BeyondWords API requests off the hot paths — page render and
the save request.

## Done (this PR)
## How it works

### Read caching

The editor-dropdown reads (`Client::get_languages()`, `get_voices()`,
`get_summarization_settings()`, `get_summarization_settings_templates()`,
`get_video_settings()`, `get_video_settings_templates()`) now cache successful
responses in transients (`Client::cached_get()`, 15-min TTL). These power both
the block-editor REST proxies and the classic-editor render, so both get fewer
and more resilient API calls.
The editor-dropdown reads go through `Client::cached_get()`, which stores the
decoded response in a transient:
[src/api/class-client.php](../src/api/class-client.php).

- `Client::get_languages()`
- `Client::get_voices()` (and `get_voice()`, which filters that list)
- `Client::get_project()`
- `Client::get_video_settings()`
- `Client::get_summarization_settings_templates()`
- `Client::get_video_settings_templates()`

These power both the block-editor REST proxies and the classic-editor render, so
both get fewer and more resilient API calls.

- The cache key is salted with the project ID + API key, so changing either
invalidates implicitly — no flush needed, which matters on object-cache hosts
(e.g. VIP) where `_transient_*` rows can't be enumerated.
- Only 2xx array responses are cached; errors retry on the next call.
- Successful 2xx array responses are cached for `Client::CACHE_TTL` (15
minutes).
- Failures are negative-cached: an empty array is stored for the shorter
`Client::CACHE_TTL_ON_ERROR` (2 minutes). Because `cached_get()` returns any
value it finds (`false !== $cached`), that stored `[]` short-circuits the next
fetch, so an unreachable API is probed at most once per interval rather than
on every render.
- Requests use `Client::DEFAULT_REQUEST_TIMEOUT` (3 seconds), except voices,
which uses `Client::VOICES_REQUEST_TIMEOUT` (8 seconds) — it is the one slow
endpoint, and the default would abandon (and then negative-cache) many
cold-cache fetches.

### Background create/update (VIP only)

The blocking `create_audio`/`update_audio` call on `save_post` made saving slow.
`Sync::on_add_or_update_post()` now defers `generate_audio_for_post()` to a
WP-Cron single event (`Sync::GENERATE_AUDIO_CRON_HOOK`) so the save request
returns immediately.
The blocking `create_audio`/`update_audio` call on `wp_after_insert_post` made
saving slow. `Sync::on_add_or_update_post()` defers `generate_audio_for_post()`
to a WP-Cron single event (`Sync::GENERATE_AUDIO_CRON_HOOK`) so the save request
returns immediately: [src/post/class-sync.php](../src/post/class-sync.php).

- **Gated to WordPress VIP only** (`Sync::is_async_generation_enabled()`), detected
by VIP-only cron symbols (`Automattic\WP\Cron_Control\Main`,
Expand All @@ -33,16 +50,51 @@ returns immediately.
WP-Cron from a real system cron, so the deferred job fires promptly.
- Overridable via the `beyondwords_async_generate_audio` filter.
- The cron job re-runs `generate_audio_for_post()`, which re-checks eligibility
and captures the API response → writes `content_id` / `preview_token` / voice
meta exactly as the synchronous path does. No webhook/poll needed.
and passes the API response to `process_response()` → writes
`beyondwords_project_id`, `beyondwords_content_id` and
`beyondwords_preview_token` exactly as the synchronous path does. Language and
voice meta are deliberately not copied back from the response: those keys hold
explicit editor choices. No webhook/poll needed.
- Error meta is written when the job runs (≈1 cron tick later) rather than
inline, so the editor surfaces an API error on its next load.

## Deferred (follow-up)
### Background delete (VIP only)

The trash and permanent-delete handlers use the same mechanism.
`Sync::on_trash_post()` and `Sync::on_delete_post()` call
`delete_audio_for_post_or_defer()`, which on VIP schedules
`Sync::DELETE_AUDIO_CRON_HOOK` via `schedule_audio_deletion()` and off VIP
deletes inline.

- The cron event carries the BeyondWords project + content IDs, not a post ID:
the meta is wiped (trash) or the post row is gone (permanent delete) by the
time the job runs. `Sync::delete_audio_by_ids()` handles the event and calls
`Client::delete_audio_by_ids()`.
- Both handlers first call `unschedule_audio_generation()` to drop any pending
generate event for the post.
- The save-time "delete content" branch in `Sync::on_add_or_update_post()`
(`beyondwords_delete_content` meta) still deletes synchronously.

### Bulk generation

### Classic-editor async reads
`Sync::bulk_generate_audio_for_posts()` backs the posts-list "Generate audio"
bulk action ([src/posts-list/class-bulk-edit.php](../src/posts-list/class-bulk-edit.php)).
It sets `beyondwords_generate_audio` meta on every selected post, then:

The block editor is already fully async (it populates dropdowns from the
- On VIP, queues one cron event per post and returns immediately.
- Off VIP, generates inline up to `Sync::BULK_GENERATE_SYNC_LIMIT` (10) — each
post is a blocking API call, so the cap keeps the batch inside execution
limits. Posts without a content ID are processed first, so re-running the
action makes forward progress. The remainder is reported as a `deferred`
count and surfaced by `Notices::deferred_notice()`
([src/posts-list/class-notices.php](../src/posts-list/class-notices.php));
their generate flag is already set, so re-running the action completes them.

## Known limitations

### Classic-editor server-side reads

The block editor is fully async (it populates dropdowns from the
`beyondwords/v1` REST proxies via React). The classic editor still calls the API
**server-side during metabox render**:

Expand All @@ -51,19 +103,21 @@ The block editor is already fully async (it populates dropdowns from the
- `SettingsFields::render_format_section()` → `get_video_settings_templates()` + `get_video_settings()`

Read caching (above) means these only hit the network on a cold cache, so the
per-render blocking is largely mitigated. Fully removing it means rendering empty
`<select>`s and populating them from the REST proxies in `classic-metabox.js`
(extending the pattern `select-voice/classic-metabox.js` already uses for voices
on language change, to also run on load and to cover templates/sizes).

Deferred because classic-on-VIP is not a priority and the rewrite touches the
render + JS of both components plus their Cypress specs. The REST proxies it
needs already exist (`/languages`, `/languages/{code}/voices`,
per-render blocking is largely mitigated.

Removing it entirely is partly done.
[src/editor/components/select-voice/classic-metabox.js](../src/editor/components/select-voice/classic-metabox.js)
already fetches from the REST proxies — `hydrate()` re-fetches the saved
language's voices on load, and `applyProjectDefaultLanguage()` fetches
`/projects/{id}` when Customize is switched on — so the model/voice dropdowns
could be rendered empty and filled client-side.
The language list has no client-side fetch, and
[src/editor/components/settings-fields/classic-metabox.js](../src/editor/components/settings-fields/classic-metabox.js)
does no fetching at all, so its template and video-size dropdowns would need one
written from scratch.

Not a priority: classic-on-VIP is rare and the rewrite touches the render + JS
of both components plus their Cypress specs. The REST proxies it needs already
exist (`/languages`, `/languages/{code}/voices`,
`/summarization-settings-templates`, `/video-settings-templates`,
`/projects/{id}/video-settings`).

### Background delete

The save-time "delete content" branch and the trash/delete handlers still call
the API synchronously. Lower priority (delete isn't on the common save path), but
could move to the same VIP cron mechanism.
`/projects/{id}`, `/projects/{id}/video-settings`).
102 changes: 91 additions & 11 deletions doc/code-quality-checks.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,27 +6,107 @@ before every commit.
> When somebody commits changes, GrumPHP will run some tests on the committed
> code. If the tests fail, you won't be able to commit your changes.

The GrumPHP config file is at [`grumphp.yml`](../grumphp.yml).
The GrumPHP config file is at [`grumphp.yml`](../grumphp.yml). It declares a
list of tasks and a set of testsuites, and each git hook runs one named
testsuite: the `commit-msg` hook runs `git_commit_msg`, and the `pre-commit`
hook runs `git_pre_commit`. Tasks that are not in the testsuite for a hook do
not run on commit.

The checks are:
`stop_on_failure` is enabled, so the run halts at the first failing task.

## What runs on commit

The `git_commit_msg` testsuite runs one task:

- `git_commit_message`: Cap the width of the commit subject line
(`max_subject_width: 120`) and do not require the subject to be capitalised
(`enforce_capitalized_subject: false`). `max_body_width` is not set in
`grumphp.yml`, so GrumPHP's default of 72 applies and every body line wider
than that fails the hook

The `git_pre_commit` testsuite runs five tasks:

- `git_commit_message`: Require a brief commit message (under 120 chars)
- `composer`: Validate `composer.json`
- `composer_normalize`: Keep `composer.json` nice and tidy
- `composer_normalize`: Keep `composer.json` nice and tidy (4-space indent)
- `phpcs`: Run [PHP_CodeSniffer](https://github.com/squizlabs/PHP_CodeSniffer)
using the config at [`.phpcs.xml`](../.phpcs.xml), to the *WordPress-VIP-Go* standard.
- `phplint`: Check source files for syntax errors
- `test_phpunit`: PHPUnit tests
- `coverage_check`: PHPUnit code coverage
over the plugin source using the config at [`.phpcs.xml`](../.phpcs.xml)
- `lint_plugins`: Run PHP_CodeSniffer over the companion plugins in
[`plugins/`](../plugins) via the `lint:plugins` composer script
- `phplint`: Check source files for syntax errors, excluding `vendor` and
`plugins`

If you see a big "nope" then fix the reported problems before attempting to
commit your code again.

## Bypassing the checks
### The `phpcs` ruleset

[`.phpcs.xml`](../.phpcs.xml) scans [`src`](../src),
[`speechkit.php`](../speechkit.php), [`uninstall.php`](../uninstall.php) and
[`index.php`](../index.php). It layers four WordPress rulesets in increasing
order of strictness — *WordPress-Core*, *WordPress-Docs*, *WordPress-Extra* and
*WordPress-VIP-Go* — plus *VariableAnalysis* for unused, undefined and
re-declared variables. Later rulesets override earlier ones.

On top of those it sets some project conventions (short array syntax, K&R brace
placement, the `speechkit` text domain) and silences a set of rules, many of
them PHPDoc requirements. Most are grouped under a comment giving the
reason — legacy code under `src/` that predates the strict ruleset, and PHPDoc
rules whose retroactive satisfaction would be a sprint of mechanical typing
without functional value — and a few later ones carry their own note. Two
array-spacing sniffs
(`WordPress.Arrays.ArrayDeclarationSpacing.AssociativeArrayFound` and
`NormalizedArrays.Arrays.ArrayBraceSpacing.SpaceBeforeArrayCloserSingleLine`)
are silenced without a stated reason.

The `phpcs` task ignores `/tests/*`, `/plugins/*` and
`/wp-tests-config-sample.php`. The `plugins/` directory is covered separately by
`lint_plugins`, which uses [`plugins/.phpcs.xml`](../plugins/.phpcs.xml) — a
smaller ruleset based on *WordPress-VIP-Go* alone.

## What exists but is not run on commit

Two further tasks are declared in [`grumphp.yml`](../grumphp.yml) but are not
members of any testsuite:

- `test_phpunit`: runs the `test` composer script (PHPUnit, then the coverage
check)
- `coverage_check`: runs the `test:coverage-check` composer script, which reads
`tests/phpunit/_report/clover.xml` and fails below 85% coverage

Because they are in no testsuite, neither git hook runs them. They only run when
GrumPHP is invoked with no `--testsuite` option (`./vendor/bin/grumphp run`),
which runs every declared task. In practice PHPUnit is run directly — see
[running-tests.md](running-tests.md).

## Bypassing the checks

Adding `-n` as an argument to your `git commit` command will skip the GrumPHP
checks for the current commit on your development machine, but
**GrumPHP always runs in the GitHub Actions workflow**.
checks for the current commit on your development machine, but the checks also
run in the GitHub Actions workflow, so skipping them locally only defers the
failure.

## What runs in GitHub Actions

[`.github/workflows/main.yml`](../.github/workflows/main.yml) does not run all
of GrumPHP. The `grumphp` job runs a single testsuite:

```sh
./vendor/bin/grumphp run --testsuite code_quality
```

The `code_quality` testsuite holds the same five tasks as `git_pre_commit`:
`composer`, `composer_normalize`, `phpcs`, `lint_plugins` and `phplint`.

The remaining checks are separate jobs in the same workflow:

- `lint-js`: ESLint over `src` via `npm run lint:js`, after asserting that
prettier resolves to wp-prettier
- `jest-tests`: Jest unit tests via `npm run test:unit`
- `phpunit-tests`: PHPUnit across a PHP 8.0 / 8.5 matrix, followed by
`composer test:coverage-check`
- `plugin-check`: the WordPress plugin-check action, run against the built
plugin ZIP
- `cypress-tests`: Cypress end-to-end tests against a real WordPress install

We urge you to fix any reported problems locally before pushing your commits.

Expand Down
47 changes: 41 additions & 6 deletions doc/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,42 @@ npm install
npm run build
```

## 5. Install PHP dependencies
## 5. Get PHP & Composer

The next step runs Composer on your host machine, so you need a host PHP and a
host Composer. [composer.json](../composer.json) requires `php >=8.0`.

[Get Composer](https://getcomposer.org/download/). On macOS both are available
via Homebrew:

```bash
brew install php composer
```

## 6. Install PHP dependencies

Run Composer on your host machine, not inside a container:

```bash
npm run composer -- install
composer install
```

## 6. Start wp-env
This must happen before wp-env starts. `wp-env start` activates every plugin
listed in `.wp-env.json`, including this one, and
[speechkit.php](../speechkit.php) requires `vendor/autoload.php` at load time,
so activation fatals if `vendor/` is missing.

The `npm run composer` script is only usable once the environment exists — it
is `wp-env run cli ... composer`, so it dispatches into a container that
`npm run env:start` has not created yet. Use it for later Composer commands,
such as regenerating the classmap autoloader after adding, removing or renaming
files in `src/`:

```bash
npm run composer -- dump-autoload
```

## 7. Start wp-env

Ensure that Docker is running, then start both environments:

Expand Down Expand Up @@ -74,13 +103,19 @@ WordPress development site started at http://localhost:8889/

Well done, you should now have a functional wp-env development environment for our plugin.

Log into WordPress admin and activate our plugin to get started.
The plugin is already active: wp-env activates every plugin listed in
`.wp-env.json`, and `./` (this repo) is one of them.

Log into WordPress admin and go to Settings → BeyondWords to enter the API key
and Project ID that connect the plugin to your BeyondWords project. See
[src/settings/class-settings.php](../src/settings/class-settings.php).

* The default WordPress credentials are Username: `admin`, Password: `password`
* The default database credentials are: Username: `root`, Password: `password`

Before you push any commits in Git our [automated code quality checks](../doc/code-quality-checks.md)
need to pass. See [running tests](../doc/running-tests.md) to get the tests running in your wp-env.
Our [automated code quality checks](code-quality-checks.md) run on every commit
via a git hook, and need to pass before the commit is accepted. See
[running tests](running-tests.md) to get the tests running in your wp-env.

## Further reading

Expand Down
Loading
Loading