feat: add Fireworks AI plugin for usage tracking and billing metrics#359
feat: add Fireworks AI plugin for usage tracking and billing metrics#359sayuru-akash wants to merge 5 commits into
Conversation
- Implemented the Fireworks AI plugin to track serverless usage, prompt tokens, generated tokens, monthly spend, and budget. - Added API calls to list accounts and quotas, and export billing metrics. - Integrated macOS Keychain and environment variable support for API key management. - Created a detailed documentation file for the Fireworks AI plugin. - Added SVG icon for the Fireworks AI plugin. - Developed comprehensive tests to ensure functionality and error handling.
sayuru-akash
requested review from
davidarny,
robinebers and
validatedev
as code owners
🤖 Augment PR SummarySummary: Adds a new Fireworks AI provider plugin so OpenUsage can show Fireworks account health, spend/budget, and serverless token usage. Changes:
Technical Notes: Billing exports use UTC date-only windows ("Last 30 days") and the UI falls back to spend/budget when token exports are unavailable. 🤖 Was this summary useful? React with 👍 or 👎 |
![augmentcode[bot]](https://avatars.githubusercontent.com/in/1027498?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 59d328eacb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Adds a new Fireworks AI provider plugin to OpenUsage, including host/runtime support for pulling usage/billing metrics and documentation so users can configure authentication and understand reported metrics.
Changes:
- Added a Fireworks AI plugin (manifest, implementation, icon) that queries Fireworks account/quota APIs and optionally computes rolling serverless token usage via
firectl billing export-metrics. - Extended the Tauri plugin host API to expose
host.fireworks.exportBillingMetrics()to plugins, whitelistFIREWORKS_API_KEY, and redact Fireworks account IDs from logged URLs. - Added provider docs, README provider list entry, and Vitest coverage for the new plugin + host changes.
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src-tauri/src/plugin_engine/runtime.rs | Calls the new Fireworks wrapper patch during plugin probe setup. |
| src-tauri/src/plugin_engine/host_api.rs | Adds env allowlist entry, URL redaction for /accounts/{id}, Fireworks host API injection, and firectl runner integration + tests. |
| README.md | Adds Fireworks AI to the provider list. |
| plugins/test-helpers.js | Extends the mock host context with host.fireworks.exportBillingMetrics. |
| plugins/fireworks-ai/plugin.test.js | Comprehensive tests for auth resolution, quota parsing, billing export integration, and error paths. |
| plugins/fireworks-ai/plugin.json | New plugin manifest defining Fireworks AI lines/labels/scopes and metadata. |
| plugins/fireworks-ai/plugin.js | Fireworks AI plugin implementation (accounts, quotas, billing CSV parsing, output formatting). |
| plugins/fireworks-ai/icon.svg | Provider icon asset. |
| docs/providers/fireworks-ai.md | Setup + behavior documentation for the Fireworks AI provider. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
6 issues found across 9 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src-tauri/src/plugin_engine/host_api.rs">
<violation number="1" location="src-tauri/src/plugin_engine/host_api.rs:31">
P1: Global allowlisting of `FIREWORKS_API_KEY` exposes a sensitive credential to all plugins because env access is not scoped per plugin.</violation>
<violation number="2" location="src-tauri/src/plugin_engine/host_api.rs:2062">
P1: Fireworks API key is passed in child-process argv (`--api-key <secret>`), creating credential exposure risk via process inspection/log surfaces.</violation>
<violation number="3" location="src-tauri/src/plugin_engine/host_api.rs:2072">
P1: `Command::output()` has no timeout — if `firectl billing export-metrics` hangs (DNS/network issues), the probe thread blocks indefinitely and the batch never finishes. Add bounded wait/kill logic similar to how other external command runners in this codebase handle subprocess timeouts.</violation>
</file>
<file name="README.md">
<violation number="1" location="README.md:34">
P2: README documents Fireworks metrics as TPS, but the plugin and provider docs expose prompt/generated token totals, creating a user-facing contract mismatch.</violation>
</file>
<file name="plugins/fireworks-ai/plugin.js">
<violation number="1" location="plugins/fireworks-ai/plugin.js:36">
P3: `lastFinite()` iterates forward and returns the **first** finite number, not the last. The name is misleading and will confuse future maintainers. Rename to `firstFinite` or change the loop to iterate backwards.</violation>
<violation number="2" location="plugins/fireworks-ai/plugin.js:302">
P2: Zero-token billing exports are incorrectly marked as non-ok, causing valid zero usage to be shown as unavailable/error state.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.
| "MINIMAX_CN_API_KEY", | ||
| "SYNTHETIC_API_KEY", | ||
| "PI_CODING_AGENT_DIR", | ||
| "FIREWORKS_API_KEY", |
There was a problem hiding this comment.
P1: Global allowlisting of FIREWORKS_API_KEY exposes a sensitive credential to all plugins because env access is not scoped per plugin.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src-tauri/src/plugin_engine/host_api.rs, line 31:
<comment>Global allowlisting of `FIREWORKS_API_KEY` exposes a sensitive credential to all plugins because env access is not scoped per plugin.</comment>
<file context>
@@ -28,6 +28,7 @@ const WHITELISTED_ENV_VARS: [&str; 16] = [
"MINIMAX_CN_API_KEY",
"SYNTHETIC_API_KEY",
"PI_CODING_AGENT_DIR",
+ "FIREWORKS_API_KEY",
];
</file context>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 59d328eacb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Hi, @validatedev @robinebers , Just checking in on this PR, because I didn’t fully understand why it’s still open, as I can see the Fireworks AI plugin has already been added in the latest release, even though this PR hasn’t been merged. Could we review this and either merge it if it’s still relevant or close it if it’s not relevant? Feel free to let me know if you need any assistance on my end as well. Thanks |
|
Quick check on #359, happy to help! |
robinebers
left a comment
robinebers
left a comment
There was a problem hiding this comment.
Hey! This is Rob's AI reviewer. Thanks for the contribution!
Fireworks AI plugin looks solid overall. The plugin shape, icon, env whitelist, redaction test, and plugin tests all line up well. Three things to fix before merge:
-
Add a timeout around
firectl—exportBillingMetricsruns the Fireworks CLI with.output(), so iffirectlhangs, the whole provider probe can hang too. Please add a timeout like the existingccusagerunner uses. -
Document the new host helper — this PR adds
ctx.host.fireworks.exportBillingMetrics, butdocs/plugins/api.mddoes not explain it yet. Since this is new plugin API surface, please add a short section for it. -
Fix the README wording — the README says
prompt/generated TPS, but the plugin shows token totals, not tokens-per-second. Please change it toprompt/generated tokens.
Everything else looks good.
|
@codex review |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 9 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e141abcc50
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Hi, @validatedev @robinebers , Just pinging an update on this PR, cause I did some updates and pushed. You may check. Thanks |
Description
This pull request introduces a new provider plugin for Fireworks AI, enabling users to track their Fireworks AI usage directly within OpenUsage.
The integration is built on Fireworks AI's official non-browser surfaces and does not depend on browser login or session state. It fetches account details and quotas from the Fireworks API, and uses the official
firectl billing export-metricspath to compute rolling token usage for serverless inference.The provider currently calculates and displays the most truthful metrics available from the validated account shape:
Authentication is supported via both macOS Keychain and environment variables, ensuring secure and flexible API key management.
A complete plugin manifest, documentation, icon assets, shared host/runtime support, and comprehensive test coverage have also been included.
Fireworks AI Integration
What Else We Can Do In The Future
Related Issue
Fixes #352
Type of Change
Screenshot
Testing
bun run buildand it succeededbun run testand all tests passbun tauri devChecklist
mainbranchSummary by cubic
Adds a
fireworks-aiprovider plugin to track Fireworks usage and billing in OpenUsage with secure CLI-backed token totals and clear fallbacks. Includes a hostfireworks.exportBillingMetricshelper with a temporary~/.fireworks/auth.ini, date-only windows, URL redaction, and a 15s timeout.fireworks-aiplugin: lists accounts/quotas fromhttps://api.fireworks.ai/v1, selects a healthy account; plan tier inferred from budget cap; API-key auth via macOS Keychain orFIREWORKS_API_KEY.fireworks.exportBillingMetricsusingfirectl billing export-metrics; respects zero-value windows; falls back to token quotas or spend-only with clear reasons; detects missing runner.firectlwith a temporary~/.fireworks/auth.ini(no key on argv) and enforces a 15s timeout to avoid hangs.FIREWORKS_API_KEY, redacts/accounts/{id}in URLs, exposesfireworks.exportBillingMetricswith a JS wrapper; added docs, tests, icon, manifest, and README entry.Written for commit 8676648. Summary will update on new commits.