Skip to content

feat(steam): add the Steam storefront as a metadata provider - #4241

Open
sdornan wants to merge 8 commits into
rommapp:masterfrom
sdornan:claude/steam-metadata-provider
Open

feat(steam): add the Steam storefront as a metadata provider#4241
sdornan wants to merge 8 commits into
rommapp:masterfrom
sdornan:claude/steam-metadata-provider

Conversation

@sdornan

@sdornan sdornan commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Description

Adds the Steam storefront as a metadata provider, covering the win, linux and mac platforms.

This is groundwork for #4227 (ProtonDB compatibility badges). ProtonDB has no official API, but it does expose one stable undocumented endpoint that its own frontend uses:

GET https://www.protondb.com/api/v1/reports/summaries/{steam_appid}.json
-> { bestReportedTier, confidence, score, tier, total, trendingTier }

The blocker is the key: everything ProtonDB serves is keyed by Steam app ID, and RomM stores none. ProtonDB has no name search of its own — its search box proxies Steam's storefront search. So Steam lands first, and it earns its place independently: the store payload carries title, description, portrait capsule art, screenshots, genres, developers, publishers, release date and Metacritic score, with no API key. ProtonDB then becomes a small follow-up PR that reads steam_id and makes one cached call.

How it works

Two keyless endpoints, the ones backing the store's own search box and app pages:

Purpose Endpoint
name -> appid store.steampowered.com/api/storesearch
appid -> metadata store.steampowered.com/api/appdetails

Valve documents neither, so the handler is defensive throughout:

  • Opt-in behind STEAM_API_ENABLED (default false), matching HowLongToBeat and Flashpoint.
  • Paced at 0.6 rps with 429 backoff — the storefront starts refusing at roughly 200 requests per 5 minutes per IP, and a scan spends two requests per game.
  • Platform-gated on the slug, so retro platforms never spend a request looking for a title that cannot be there.
  • Type-filtered. The search index carries DLC, soundtracks and tools alongside games; hits are filtered by type and then confirmed against the app page, which is the only place a demo or video is distinguishable from a game.
  • Escapable. A (steam-12345) filename tag pins a match by hand, and the app ID is editable in Edit ROM (which refetches, like the other providers).
  • Every failure path degrades to "no match" rather than aborting a scan.

Covers use Steam's portrait library_600x900 capsule, which is the right shape for RomM, falling back to the landscape header when an app has no capsule.

SteamMetadata reuses the key names the other providers already use for shared values (genres, companies, game_modes, total_rating, first_release_date), so wiring Steam into the generated_* facet columns later is a SQL change with no stored-row reshaping.

Deliberately out of scope

  • ProtonDB itself — separate PR, closes [Feature] Add ProtonDB as a scraping provider #4227.
  • The facet columns. Steam is not added to the generated_genres / generated_companies / generated_game_modes / generated_first_release_date / generated_average_rating COALESCE lists from migration 0098. Those are STORED generated columns, so changing them forces a full rebuild of the roms table on upgrade for every user, including the large majority with no PC games. Until then Steam's genres, companies, game modes and release date are persisted but surface nowhere: metadatum reads the roms_metadata view over those generated columns, so they reach neither the details page nor the gallery filter dropdowns. What Steam contributes today is the name, summary, cover, screenshots and the provider rating chip, which reads steam_metadata.total_rating directly. Worth batching with a future migration that already rebuilds the table.
  • IGDB external_games. IGDB returns Steam app IDs there and it is currently stubbed as UnimplementedEntity. Wiring it up would give exact IDs whenever IGDB already matched, roughly halving Steam search traffic. Clean follow-up now that the handler exists.
  • v1. Per the freeze in CLAUDE.md, Steam does not appear in the v1 Edit ROM dialog or details page. Flagging because HowLongToBeat predates that freeze and does appear there, so the two providers differ in v1. Happy to mirror it if preferred.

Notes for reviewers

  • The migration is purely additive: roms.steam_id, roms.steam_metadata, an index, and the roms_facets mirror column (with the trigger set recreated, following migration 0103).
  • The logo ships as SVG rather than PNG like its siblings — Wikimedia's thumbnailer cannot rasterise this file, and the vector reads better at the 20px chip size.
  • scrappers in the asset path is a pre-existing repo-wide misspelling (192 occurrences across 27 files); this PR matches it rather than renaming. Glad to fix it separately if wanted.

Testing

  • Full backend suite: 2970 passed, 0 failed.
  • New tests: tests/handler/metadata/test_steam_handler.py (18) and tests/adapters/services/test_steam.py (8), covering match, DLC/non-game rejection, weak-match rejection, platform gating, filename tag, rate-limit retry and backoff.
  • Migration verified upgrade -> downgrade -> re-upgrade on MariaDB 10.11 and PostgreSQL 16, including a functional check that the recreated triggers mirror steam_id into roms_facets on both insert and update.
  • Verified against the live API: Cyberpunk 2077 -> 1091500, Hollow Knight -> 367520, Baldurs Gate 3 -> Baldur's Gate 3 (apostrophe-insensitive). SNES ROM skipped without a request, junk name rejected, (steam-220) tag honoured, cover fallback confirmed on app 15100 (no portrait capsule).
  • npm run typecheck, npm run build, trunk fmt && trunk check, and both i18n checkers pass.

Not verified: the frontend vitest suite. All 65 test files fail at setup with localStorage undefined in src/locales/index.ts. This reproduces identically on a clean tree at master, so it is pre-existing tooling breakage in my environment (happy-dom 20.10.2, matching the lockfile) rather than anything in this branch — but it does mean the v2 component tests never ran here. Worth a second pair of eyes.

i18n

Five new keys, translated into all 18 locales. check_i18n_locales.py and check_i18n_sorted.py both pass.

AI assistance disclosure

This PR was written with substantial AI assistance (Claude Opus 5 via Claude Code). The API research, architecture decisions, implementation, tests, translations and verification were all AI-produced under human direction and review. Scoping decisions (full provider vs. ID resolver, platform coverage, matching strategy, deferring the facet migration) were made by the human author.

Checklist

  • I've tested the changes locally
  • I've updated relevant comments
  • I've assigned reviewers for this PR
  • I've added unit tests that cover the changes

@gantoine gantoine added the on-hold Pending further research or blocked by another issue label Aug 19, 2026
@sdornan
sdornan force-pushed the claude/steam-metadata-provider branch from 500bcbb to 68803a8 Compare August 21, 2026 22:25
@sdornan
sdornan marked this pull request as ready for review August 21, 2026 22:44
Copilot AI lite review requested due to automatic review settings August 21, 2026 22:44
@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds an opt-in Steam storefront metadata provider for Windows, Linux, and macOS ROMs, including schema persistence, scan orchestration, configuration, API contracts, and v2 UI support.

  • Searches Steam by title or an explicit filename tag and stores app metadata and artwork.
  • Adds Steam fields to ROM and facet persistence through migration 0108.
  • Exposes provider configuration, health, filtering, editing, and localized frontend presentation.
  • The UPDATE scan path currently fails to preserve an existing Steam match by ID.

Confidence Score: 4/5

The PR should not merge until UPDATE scans preserve and refetch the existing Steam app ID rather than replacing it through filename search.

An UPDATE of an already matched or manually assigned Steam ROM can resolve a different storefront result and overwrite the persisted provider identity and associated metadata.

Files Needing Attention: backend/handler/scan_handler.py

Important Files Changed

Filename Overview
backend/handler/scan_handler.py Integrates Steam into provider fetching and priority merging, but UPDATE scans search by filename instead of refreshing the existing Steam ID.
backend/handler/metadata/steam_handler.py Implements platform gating, explicit tags, fuzzy matching, type validation, metadata extraction, and cover fallback.
backend/adapters/services/steam.py Adds paced storefront requests with timeout, rate-limit retry, and defensive empty-result handling.
backend/alembic/versions/0108_add_steam_metadata.py Adds Steam columns and consistently extends the MariaDB/PostgreSQL facet synchronization triggers.
backend/endpoints/roms/init.py Adds Steam ID lookup, manual editing, metadata refetching, and unmatch behavior.
backend/models/rom.py Adds Steam persistence and facet coverage fields to the ORM model.
frontend/src/v2/composables/useScanProviders/index.ts Integrates Steam into the shared v2 scan-provider selection model.
frontend/src/v2/utils/metadataProviders.ts Adds Steam provider metadata used by v2 presentation surfaces.

Fix all with Greploop Fix All in Claude Code

Prompt To Fix All With AI
### Issue 1
backend/handler/scan_handler.py:708
**Steam refresh replaces persisted identity**

When an UPDATE scan processes a ROM with an existing or manually selected Steam ID, this path performs another fuzzy filename search instead of fetching that ID, causing a different sufficiently similar result to replace the stored Steam app and its metadata.

```suggestion
            if scan_type == ScanType.UPDATE and rom.steam_id:
                return await meta_steam_handler.get_rom_by_id(rom.steam_id)
            return await meta_steam_handler.get_rom(rom_attrs["fs_name"], platform.slug)
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "chore(steam): tighten comments and the f..." | Re-trigger Greptile

Comment thread backend/handler/scan_handler.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in Steam storefront metadata provider for Windows, Linux, and macOS, including rate-limited lookup, persistence, scanning, and v2 UI integration.

Changes:

  • Adds Steam API matching, metadata extraction, artwork, and throttling.
  • Integrates Steam into schemas, migrations, scans, configuration, and generated types.
  • Adds v2 UI support, translations, branding, documentation, and tests.

Reviewed changes

Copilot reviewed 89 out of 98 changed files in this pull request and generated 4 comments.

Show a summary per file
File Review summary
frontend/src/v2/views/Settings/ScanSettings.vue Reviewed; no final comments.
frontend/src/v2/views/Settings/MetadataSources.vue Reviewed; no final comments.
frontend/src/v2/views/Settings/MetadataSources.test.ts Reviewed; no final comments.
frontend/src/v2/utils/metadataProviders.ts Reviewed; no final comments.
frontend/src/v2/tokens/index.ts Reviewed; no final comments.
frontend/src/v2/styles/tokens.css Reviewed; no final comments.
frontend/src/v2/composables/useScanProviders/index.ts Reviewed; no final comments.
frontend/src/v2/components/Scan/ScanInfoDialog.vue Nit (3 votes): Move Steam from specificProviders to generalProviders.
frontend/src/v2/components/GameDetails/RelatedGameCard.vue Reviewed; no final comments.
frontend/src/v2/components/GameDetails/providers.ts Reviewed; no final comments.
frontend/src/v2/components/EditRom/MetadataIdSection.vue Moderate (2 votes): Add Steam to the ProviderConfig registry so raw metadata is available.
frontend/src/v2/components/Auth/SetupStepMetadata.vue Reviewed; no final comments.
frontend/src/stores/heartbeat.ts Reviewed; no final comments.
frontend/src/locales/zh_TW/setup.json Reviewed; no final comments.
frontend/src/locales/zh_TW/settings.json Reviewed; no final comments.
frontend/src/locales/zh_TW/rom.json Reviewed; no final comments.
frontend/src/locales/zh_CN/setup.json Reviewed; no final comments.
frontend/src/locales/zh_CN/settings.json Reviewed; no final comments.
frontend/src/locales/zh_CN/rom.json Reviewed; no final comments.
frontend/src/locales/tr_TR/setup.json Reviewed; no final comments.
frontend/src/locales/tr_TR/settings.json Reviewed; no final comments.
frontend/src/locales/tr_TR/rom.json Reviewed; no final comments.
frontend/src/locales/ru_RU/setup.json Reviewed; no final comments.
frontend/src/locales/ru_RU/settings.json Reviewed; no final comments.
frontend/src/locales/ru_RU/rom.json Reviewed; no final comments.
frontend/src/locales/ro_RO/setup.json Reviewed; no final comments.
frontend/src/locales/ro_RO/settings.json Reviewed; no final comments.
frontend/src/locales/ro_RO/rom.json Reviewed; no final comments.
frontend/src/locales/pt_BR/setup.json Reviewed; no final comments.
frontend/src/locales/pt_BR/settings.json Reviewed; no final comments.
frontend/src/locales/pt_BR/rom.json Reviewed; no final comments.
frontend/src/locales/pl_PL/setup.json Reviewed; no final comments.
frontend/src/locales/pl_PL/settings.json Reviewed; no final comments.
frontend/src/locales/pl_PL/rom.json Reviewed; no final comments.
frontend/src/locales/ko_KR/setup.json Reviewed; no final comments.
frontend/src/locales/ko_KR/settings.json Reviewed; no final comments.
frontend/src/locales/ko_KR/rom.json Reviewed; no final comments.
frontend/src/locales/ja_JP/setup.json Reviewed; no final comments.
frontend/src/locales/ja_JP/settings.json Reviewed; no final comments.
frontend/src/locales/ja_JP/rom.json Reviewed; no final comments.
frontend/src/locales/it_IT/setup.json Reviewed; no final comments.
frontend/src/locales/it_IT/settings.json Reviewed; no final comments.
frontend/src/locales/it_IT/rom.json Reviewed; no final comments.
frontend/src/locales/hu_HU/setup.json Reviewed; no final comments.
frontend/src/locales/hu_HU/settings.json Reviewed; no final comments.
frontend/src/locales/hu_HU/rom.json Reviewed; no final comments.
frontend/src/locales/fr_FR/setup.json Reviewed; no final comments.
frontend/src/locales/fr_FR/settings.json Reviewed; no final comments.
frontend/src/locales/fr_FR/rom.json Reviewed; no final comments.
frontend/src/locales/es_ES/setup.json Reviewed; no final comments.
frontend/src/locales/es_ES/settings.json Reviewed; no final comments.
frontend/src/locales/es_ES/rom.json Reviewed; no final comments.
frontend/src/locales/en_US/setup.json Reviewed; no final comments.
frontend/src/locales/en_US/settings.json Reviewed; no final comments.
frontend/src/locales/en_US/rom.json Reviewed; no final comments.
frontend/src/locales/en_GB/setup.json Reviewed; no final comments.
frontend/src/locales/en_GB/settings.json Reviewed; no final comments.
frontend/src/locales/en_GB/rom.json Reviewed; no final comments.
frontend/src/locales/de_DE/setup.json Reviewed; no final comments.
frontend/src/locales/de_DE/settings.json Reviewed; no final comments.
frontend/src/locales/de_DE/rom.json Reviewed; no final comments.
frontend/src/locales/cs_CZ/setup.json Reviewed; no final comments.
frontend/src/locales/cs_CZ/settings.json Reviewed; no final comments.
frontend/src/locales/cs_CZ/rom.json Reviewed; no final comments.
frontend/src/locales/bg_BG/setup.json Reviewed; no final comments.
frontend/src/locales/bg_BG/settings.json Reviewed; no final comments.
frontend/src/locales/bg_BG/rom.json Reviewed; no final comments.
frontend/src/__generated__/models/SteamPlatforms.ts Reviewed; no final comments.
frontend/src/__generated__/models/SimpleRomSchema.ts Reviewed; no final comments.
frontend/src/__generated__/models/RomSteamMetadata.ts Reviewed; no final comments.
frontend/src/__generated__/models/MetadataSourcesDict.ts Reviewed; no final comments.
frontend/src/__generated__/models/DetailedRomSchema.ts Reviewed; no final comments.
frontend/src/__generated__/models/Body_update_rom_api_roms__id__put.ts Reviewed; no final comments.
frontend/src/__generated__/index.ts Reviewed; no final comments.
frontend/assets/scrappers/steam.svg Reviewed; no final comments.
examples/config.example.yml Reviewed; no final comments.
examples/config.es-de.example.yml Reviewed; no final comments.
env.template Reviewed; no final comments.
docs/BACKEND_ARCHITECTURE.md Reviewed; no final comments.
backend/watcher.py Reviewed; no final comments.
backend/tests/handler/metadata/test_steam_handler.py Reviewed; no final comments.
backend/tests/adapters/services/test_steam.py Reviewed; no final comments.
backend/tasks/scheduled/scan_library.py Reviewed; no final comments.
backend/models/rom.py Reviewed; no final comments.
backend/handler/scan_handler.py Reviewed; no final comments.
backend/handler/metadata/steam_handler.py Moderate (4 votes): Apply the same type == "game" check in get_rom_by_id so pinned DLCs, demos, and videos cannot be stored as games.

Moderate (2 votes): Add a rendering fallback or include Steam in facet derivation so persisted Steam genres, companies, and release dates appear on the details page.
backend/handler/metadata/__init__.py Reviewed; no final comments.
backend/handler/database/roms_handler.py Reviewed; no final comments.
backend/endpoints/roms/__init__.py Reviewed; no final comments.
backend/endpoints/responses/rom.py Reviewed; no final comments.
backend/endpoints/responses/heartbeat.py Reviewed; no final comments.
backend/endpoints/heartbeat.py Reviewed; no final comments.
backend/config/config_manager.py Reviewed; no final comments.
backend/config/__init__.py Reviewed; no final comments.
backend/alembic/versions/0108_add_steam_metadata.py Reviewed; no final comments.
backend/adapters/services/steam.py Reviewed; no final comments.
backend/adapters/services/steam_types.py Reviewed; no final comments.
Files not reviewed (7)
  • frontend/src/generated/index.ts: Generated file
  • frontend/src/generated/models/Body_update_rom_api_roms__id__put.ts: Generated file
  • frontend/src/generated/models/DetailedRomSchema.ts: Generated file
  • frontend/src/generated/models/MetadataSourcesDict.ts: Generated file
  • frontend/src/generated/models/RomSteamMetadata.ts: Generated file
  • frontend/src/generated/models/SimpleRomSchema.ts: Generated file
  • frontend/src/generated/models/SteamPlatforms.ts: Generated file
Suppressed comments (8)

backend/alembic/versions/0108_add_steam_metadata.py:133

  • steam_metadata is only persisted here, while RomMetadata is a thin view over STORED generated columns whose COALESCE lists in migration 0098 do not include this new blob. A Steam-only match therefore returns empty metadatum.genres, companies, game_modes, and first_release_date, so the v2 Overview grid cannot display the shared Steam metadata. Add a migration that updates the generated expressions, or explicitly merge Steam values into the details response.
        batch_op.add_column(
            sa.Column(
                "steam_metadata",
                sa.JSON().with_variant(
                    postgresql.JSONB(astext_type=sa.Text()), "postgresql"
                ),
                nullable=True,
            ),

backend/alembic/versions/0108_add_steam_metadata.py:149

  • The new _MIRRORED_COLUMNS_AFTER path is what keeps steam_id in roms_facets, but the existing facet tests only assert the pre-Steam provider IDs. Add insert/update assertions for steam_id; otherwise a trigger or backfill regression can pass the full suite while Steam filtering and coverage silently stop working.
    _recreate_triggers(_MIRRORED_COLUMNS_AFTER)

backend/handler/database/roms_handler.py:2633

  • Adding steam_id to direct metadata lookup is not enough for grouped galleries. The group_by_meta_id subquery and its coalesce partition below still omit this column, so two ROMs that share only a Steam app ID are not treated as siblings, unlike ROMs matched by the other providers. Add steam_id to both parts of that deduplication query, including the platform-aware partition key.
        steam_id: int | None = None,

backend/handler/scan_handler.py:708

  • The UPDATE condition explicitly selects ROMs with an existing rom.steam_id, but this still searches rom_attrs["fs_name"]. A renamed file or ambiguous title can attach a different app, or lose the existing match, during refresh. Mirror the Flashpoint branch and refetch by ID for UPDATE, and for UNMATCHED when an existing ID has missing metadata.
            if (scan_type == ScanType.UPDATE and rom.steam_id) or (

backend/models/rom.py:365

  • This new identity is not included in the group_by_meta_id partition, whose COALESCE chain currently stops at flashpoint_id. Two ROMs for the same Steam app on the same platform remain separate gallery entries unless another provider also matched, so add steam_id to the partition and its covering-index/migration coverage.
    steam_id: Mapped[int | None] = mapped_column(Integer(), default=None)

backend/models/rom.py:794

  • Adding Steam to the provider-column map does not update the general matched filter: _filter_by_matched() still stops at flashpoint_id. A ROM matched only by Steam therefore is returned as unmatched and excluded from matched=true; add Rom.steam_id.isnot(None) to that predicate as well.
    "steam": Rom.steam_id,

backend/tests/handler/metadata/test_steam_handler.py:205

  • get_rom_by_id() is the direct-ID path used by filename tags and Edit ROM, but this test only covers a missing app. Add a case where the details payload has type set to demo or another non-game and assert no match, so the direct path remains protected by the same regression coverage as name matching.
async def test_get_rom_by_id_missing_app():
    handler, service = _handler(details=None)

    assert await handler.get_rom_by_id(999999999) == {"steam_id": None}

frontend/src/v2/components/GameDetails/providers.ts:98

  • Steam covers are served from shared.akamai.steamstatic.com, but OverviewTab maps cover hosts separately and has no entry for this hostname. When Steam supplies the cover, the overview's cover-art attribution is blank even though Steam appears in the data-source list. Add the Steam CDN host to COVER_HOSTS.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +213 to +215
if not details:
log.debug("Could not find Steam app %s", steam_id)
return SteamRom(steam_id=None)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description of the behaviour is accurate: get_rom_by_id builds from any payload appdetails returns successfully, with no type check.

I would rather not add the check, though, and this is a judgement call worth a maintainer opinion rather than something I should just decide.

The two checks guard different things. The type == "game" filter in _search_and_match exists to reject search noise, because storesearch returns DLC, soundtracks and tools alongside games and the matcher has to pick one unaided. get_rom_by_id is only reached by an explicit human decision: a (steam-12345) filename tag, or an ID typed into Edit ROM. Both exist as escape hatches for when the automatic matcher gets it wrong, so having them refuse the ID a user just pinned would defeat the point. Steam also types some things a user would reasonably want to pin as something other than game, and its type field is undocumented like the rest of the payload.

The one path that is not a direct human action is the UPDATE/UNMATCHED refetch in resolve_steam_rom, but the ID it passes came from either a match that already cleared the type check or a manual pin, so it inherits the same reasoning.

If the preference is to guard it anyway, I would rather log a warning and still honour the ID than drop the match, so a mis-pinned DLC is visible in the scan log instead of silently failing to match. Happy to add that if you would like it.

Comment on lines +47 to +50
first_release_date: NotRequired[int]
genres: NotRequired[list[str]]
companies: NotRequired[list[str]]
game_modes: NotRequired[list[str]]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right about the behaviour, and it means the PR description was wrong. I traced it: metadatum is the roms_metadata view, which reads the generated_* columns, and generated_genres in migration 0098 COALESCEs manual, igdb, moby, ss, launchbox, ra, flashpoint and gamelist. Not steam.

So Steam genres, companies, game modes and release date are persisted and surface nowhere. The description claimed they "show on the details page but do not feed the gallery filter dropdowns", which is wrong on both halves of the sentence. I have corrected it. What Steam actually contributes today is the name, summary, cover, screenshots and the provider rating chip, which reads steam_metadata.total_rating through providers.ts.

I am not taking either fix here, and want to be explicit about why rather than leave it implied:

  • Adding Steam to the 0098 COALESCE lists is deliberately out of scope. Those are STORED generated columns, so touching them rebuilds the whole roms table on upgrade for every user, including the large majority with no PC games. It is worth batching with a future migration that already rebuilds the table.
  • A rendering fallback that reads steam_metadata.genres when metadatum.genres is empty would be the only provider-specific special case in the overview, and would have to be unpicked when the facet migration lands.

The shared key names still earn their place: they make that later migration a SQL change with no reshaping of stored rows. They just do not pay off yet, and the description should have said so.

Comment thread frontend/src/v2/components/EditRom/MetadataIdSection.vue
Comment thread frontend/src/v2/components/Scan/ScanInfoDialog.vue Outdated
gantoine pushed a commit that referenced this pull request Aug 22, 2026
The "general catalogs vs specialised sources vs hash proxies" split was
declared four separate times, each with its own shape: the scan
composable's key sets, and the provider arrays in the setup wizard's
metadata step, the settings view and the scan info dialog. Adding Steam
in #4241 filed it as specialised in three of them while the composable
treated it as general, and the fix took two passes because the first
sweep only found two of the three stragglers.

Provider slug to group now lives in one map, and each surface derives
its sections from it instead of declaring them, so the three near
identical markup blocks per surface collapse into one loop. Per-surface
presentation data (logos, locale keys, websites, docs URLs, heartbeat
wiring, section icons) stays where it is used. The hash matcher key
list and union are derived from the proxy group too.

Rendered providers, their order and their fields are unchanged; rows
and sections gained data-provider / data-group attributes as test
hooks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
sdornan and others added 8 commits August 22, 2026 13:35
Steam covers the PC platforms RomM otherwise has thin metadata for, and
its store payload is provider-grade on its own: title, description,
portrait capsule art, screenshots, genres, developers, publishers,
release date and Metacritic score, with no API key.

Valve documents no storefront API, so this reads the two endpoints that
back the store's own search box and app pages, paced well under the ~200
requests per 5 minutes they tolerate. Every failure path degrades to "no
match" rather than aborting a scan, and the provider is opt-in behind
STEAM_API_ENABLED, matching HowLongToBeat and Flashpoint.

Matching is gated on the platform slug so the three PC platforms are the
only ones that spend a request. The storefront's search index carries
DLC, soundtracks and tools alongside games, so hits are filtered by type
and then confirmed against the app page, which is the only place a demo
or a video is distinguishable from a game. A (steam-12345) filename tag
pins a match by hand when the title is ambiguous.

SteamMetadata reuses the key names the other providers already use for
the values they share, so wiring Steam into the generated facet columns
later is a SQL change with no stored-row reshaping.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds Steam to the provider chip registry, the game-details provider card
(linking to the store page), scan and metadata-source settings, the setup
wizard, and manual ID entry in Edit ROM. Its Metacritic score feeds the
existing rating normalisation, which already grades each provider on its
own scale.

The logo ships as an SVG rather than a PNG like its siblings: Wikimedia's
thumbnailer cannot rasterise this file, and the vector reads better at
the 20px chip size anyway.

v1 is left alone, so Steam does not appear in the v1 Edit ROM dialog or
details page.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Steam was in useScanProviders' GENERAL_PROVIDER_KEYS but under the
specialised sources in the setup wizard and the settings page, so the
scan dialog and those two screens disagreed about what it is. It ships a
full game record, like the other catalogs, so it moves there.

Covers the split with a test, since nothing asserted it before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Trims the comments and docstrings to the repo's one-to-two-line rule,
drops an unreachable `or ""` behind a truthiness guard, and guards the
roms_facets column drop with if_exists to match the add in upgrade().

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An UPDATE scan re-ran the fuzzy filename search even when the ROM already
carried a Steam ID, so a different app scoring above the similarity floor
could replace a match a user had pinned by hand. Refetch the stored ID
instead, as moby, ss, sgdb and flashpoint already do.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…table

The ID-vs-filename decision sat in a closure inside scan_rom, out of reach
of a unit test. Lifts it to a module-level resolver, as LaunchBox already
does, and covers both scan types plus the stored-ID case.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…dit tab

ScanInfoDialog kept its own copy of the provider split and still listed
Steam under the single-dimension sources, and EditRomDialog's provider
registry had no Steam entry, so a matched ROM offered no raw-metadata tab
even though the endpoint already accepts raw_steam_metadata.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Master took 0108 for `roms_primary_region` while this branch was open, and
two revisions off 0107 leave alembic with two heads and no upgrade path.
Chains after it instead. That revision only touches `roms`, so the facet
trigger rebuild here still has the column set 0103 left behind.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sdornan
sdornan force-pushed the claude/steam-metadata-provider branch from 0560c97 to 4ab6d38 Compare August 22, 2026 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

on-hold Pending further research or blocked by another issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add ProtonDB as a scraping provider

3 participants