diff --git a/AGENTS.md b/AGENTS.md index 96f5962..11addf9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -293,6 +293,19 @@ Current endpoints: - `GET /test/token-pilot/budget`: exercises budget enabled/limit behavior when budget beans are present. - `GET /actuator/prometheus`: validates actuator/prometheus exposure. +Demo profile endpoints (run with `--spring.profiles.active=demo`): + +- `GET /test/token-pilot/demo`: describes the no-key, in-memory provider and scenario links. +- `GET /test/token-pilot/demo/run`: runs context fit/block, atomic reservation concurrency, + idempotency, release, and success/failure/unknown reconciliation scenarios. +- `GET /test/token-pilot/demo/{context-fit|context-block|budget-concurrency|idempotency|release|reconciliation-success|reconciliation-failure|reconciliation-unknown}`: + runs one deterministic scenario. + +The demo profile uses `DemoChatModel` only; it makes no external provider calls. The +sample app's `DEMO_RUNBOOK.md` documents the Gradle, Prometheus/Grafana, and scenario +verification flow. The provisioned dashboard uses Token Pilot-owned `tokenpilot_*` +meters and deliberately does not use legacy `ai_token_*` meters. + Test-only E2E endpoint: - `GET /test/token-pilot/chat`: exercises the Spring AI `ChatClient` advisor path with a fake/mock provider or documented real provider setup. @@ -463,6 +476,10 @@ Stage and deploy a Central release: - Added root publication aggregation and a release-candidate verification gate that stages every public module and checks the complete artifact/signature set for the selected version. +- Added a `demo` Spring profile to the sample app with a deterministic in-memory + ChatModel, public admission/reservation/idempotency/reconciliation scenarios, + integration coverage, a Token Pilot Prometheus/Grafana dashboard, and a + deterministic verification runbook. - Added isolated external Core/Starter consumer verification with fresh cache, dependency evidence, negative artifact/version scenarios, and a protected 0.1.0 release-candidate workflow. diff --git a/README.md b/README.md index a0cf04f..15ad431 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # TokenPilot -> One Java gateway to control LLM providers, routing, fallback, tokens, and costs. +> Java LLM control and accounting with optional provider adapters. TokenPilot helps Java applications control an LLM call before it is sent and reconcile its cost after it completes. It brings token estimation, context @@ -57,6 +57,41 @@ Spring AI `ChatClient` calls, TokenPilot now performs conservative preflight, claims one provider dispatch, and reconciles provider-reported actual usage against the reservation-time pricing snapshot. +## Documentation + +- [10-minute quickstart](docs/QUICKSTART.md) — choose Core or Starter and run the first verification path. +- [Configuration reference](docs/CONFIGURATION.md) — properties, defaults, conditions, and failure modes. +- [Metrics reference](docs/METRICS.md) — Token Pilot-owned meters and legacy compatibility policy. +- [Sample app runbook](docs/SAMPLE_RUNBOOK.md) — local app, Prometheus, Grafana, and troubleshooting commands. +- [Deterministic demo runbook](token-pilot-sample-app/DEMO_RUNBOOK.md) — eight admission, reservation, idempotency, release, and reconciliation scenarios. +- [Release procedure](docs/RELEASE.md) — staging, signing, external consumer, and Central Portal gates. +- [30-day MVP cutline](docs/30_DAY_MVP_REPORT.md) and [post-MVP evolution plan](docs/EVOLUTION_PLAN.md). + +## Sample demo + +The sample app exposes a deterministic `demo` profile for checking the control +and accounting lifecycle from HTTP. It covers context admission, atomic budget +reservation, idempotency, release, successful reconciliation, and pending +reconciliation results. + +```bash +./gradlew --no-daemon :token-pilot-sample-app:bootRun \ + --args='--spring.profiles.active=demo' + +curl -s http://localhost:8080/test/token-pilot/demo/run | jq +``` + +The demo returns eight `PASS` scenario results and a snapshot of the current +Token Pilot-owned metrics. Prometheus and Grafana can be started with: + +```bash +docker compose -f token-pilot-sample-app/docker-compose.yml up --build -d +``` + +See [QUICKSTART.md](docs/QUICKSTART.md) for the ten-minute path and +[SAMPLE_RUNBOOK.md](docs/SAMPLE_RUNBOOK.md) for endpoint and troubleshooting +details. + ## Micrometer metrics When a `MeterRegistry` is available, the Spring Boot starter publishes @@ -183,11 +218,10 @@ Boot, Spring AI, Micrometer, or Reactor dependencies. Spring Boot 3, Spring AI 1.x, other Spring Boot/Spring AI patch combinations, and older Java runtimes are not part of the 0.1.0 support guarantee. -The verified Spring AI path is the synchronous `ChatClient` call lifecycle with -a fake provider, including preflight blocking, atomic reservation, dispatch, -and estimate/actual reconciliation. Real-provider compatibility, chunk -accounting, streaming cancellation, and partial-usage reconciliation are not -included in this compatibility claim. +The Spring AI path includes the synchronous `ChatClient` call lifecycle with +preflight blocking, atomic reservation, dispatch, and estimate/actual +reconciliation. Chunk accounting, streaming cancellation, and partial-usage +reconciliation remain outside the current lifecycle. If a provider returns a model different from the request pricing snapshot, TokenPilot keeps the estimate as `PRICING_RECONCILIATION_REQUIRED` instead of diff --git a/docs/30_DAY_MVP_REPORT.md b/docs/30_DAY_MVP_REPORT.md new file mode 100644 index 0000000..bf7e676 --- /dev/null +++ b/docs/30_DAY_MVP_REPORT.md @@ -0,0 +1,46 @@ +# 30-day MVP report + +This document is the active MVP cutline for Token Pilot 0.1.0. It describes +verified repository behavior, not a promise that every artifact is already +published. + +## Current release-candidate scope + +| Area | Status | +| --- | --- | +| Usage normalization and token breakdown invariants | Implemented and unit tested | +| Exact monetary arithmetic and currency validation | Implemented with `BigDecimal` and explicit currency | +| Missing-pricing policy and immutable pricing snapshot | Implemented | +| Versioned model catalog and context admission | Implemented | +| UTF-8 byte heuristic estimator | Implemented; `TEXT_ONLY`, heuristic, not exact BPE | +| Atomic in-memory budget reservation | Implemented | +| Estimate/actual reconciliation | Implemented, including pending liability and duplicate-safe callbacks | +| Threshold notification events | Implemented as process-local best-effort delivery | +| Token Pilot-owned Micrometer metrics | Implemented with bounded default tags | +| Spring AI 2.0 non-streaming lifecycle | Implemented and fake-provider E2E tested | +| Sample app deterministic demo profile | Implemented with eight HTTP scenarios and Prometheus/Grafana verification | +| Core-only and Starter publication paths | Build and generated consumer gates present; final staging evidence required | +| Durable stores/outbox/replay | Post-MVP | +| Advanced streaming reconciliation | Post-MVP | + +## Release exit criteria + +- `./gradlew test` passes. +- Publication metadata, MIT license metadata, Java 25 bytecode, and dependency boundaries pass verification. +- `token-pilot-core` resolves from an external consumer without Spring, Spring AI, Micrometer, or Reactor runtime dependencies. +- `token-pilot-starter` resolves with a provider selected by the application and does not bundle a provider implementation. +- Core and Starter use the same release version. +- The default Token Pilot metrics do not expose raw user, tenant, request, reservation, or model identifiers. +- The sample app can expose health, smoke, deterministic ledger, and Prometheus checks without an API key. +- Release documentation does not describe staging artifacts as a Maven Central release. + +## Explicit cutline + +The following are deliberately outside the 0.1.0 MVP: + +- Exact provider BPE/tokenizer assets. +- Full media, tools, schemas, and structured-output admission. +- Streaming chunk accounting and cancellation reconciliation. +- Durable reservation/accounting/notification delivery. +- Provider routing, retries, fallback, circuit breakers, and gateway HTTP runtime. +- Multi-tenant administration and durable operational stores. diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md new file mode 100644 index 0000000..112051f --- /dev/null +++ b/docs/CONFIGURATION.md @@ -0,0 +1,180 @@ +# Token Pilot configuration reference + +All Token Pilot Spring Boot properties use the `token-pilot` prefix. The +defaults below are supplied by `TokenPilotProperties` and the owner-specific +auto-configurations. + +## Root and pricing + +| Property | Default | Meaning | +| --- | --- | --- | +| `token-pilot.enabled` | `true` | Enables the Token Pilot auto-configuration graph. | +| `token-pilot.pricing.plans` | `[]` | Pricing plans bound into the in-memory pricing registry. | +| `token-pilot.pricing.plans[].model-id` | required | Model ID or exact alias resolved by the pricing registry. | +| `token-pilot.pricing.plans[].currency` | required | ISO-4217 currency code used by the plan. | +| `token-pilot.pricing.plans[].rates.PROMPT` | plan-defined | Rate for ordinary input tokens. | +| `token-pilot.pricing.plans[].rates.COMPLETION` | plan-defined | Rate for ordinary output tokens. | +| `token-pilot.pricing.plans[].rates.REASONING` | optional | Rate for reasoning output tokens. | +| `token-pilot.pricing.plans[].rates.CACHE_READ_PROMPT` | optional | Rate for cached input tokens read from cache. | +| `token-pilot.pricing.plans[].rates.CACHE_CREATION_PROMPT` | optional | Rate for input tokens written to cache. | + +Rates use `BigDecimal` internally. A missing plan or rate is not silently +converted into a priced zero. The configured missing-pricing policy determines +whether the provider boundary fails closed or records an unpriced result. + +```yaml +token-pilot: + pricing: + plans: + - model-id: gpt-4o-mini + currency: USD + rates: + PROMPT: 0.00015 + COMPLETION: 0.00060 +``` + +## Spring AI adapter + +| Property | Default | Meaning | +| --- | --- | --- | +| `token-pilot.spring-ai.default-model-id` | unset | Fallback model when the request/provider does not expose a model. | +| `token-pilot.spring-ai.default-reserved-output-tokens` | unset | Fallback output reservation when request options and a resolver provide none. | +| `token-pilot.spring-ai.framing-headroom-tokens` | `0` | Additional conservative headroom for known request framing or advisor additions. | + +Resolution order for reserved output is: + +```text +ChatOptions.maxTokens → ReservedOutputTokensResolver → configured default +``` + +Non-positive resolver values are rejected. The current request admission scope +is `TEXT_ONLY`; media, tool messages or schemas, structured-output +augmentation, and enforcement-enabled streaming are rejected before +reservation when they cannot be represented by the supported contract. + +## Metrics + +| Property | Default | Meaning | +| --- | --- | --- | +| `token-pilot.metrics.enabled` | `true` | Enables Token Pilot-owned meters when a `MeterRegistry` exists. | +| `token-pilot.metrics.tag-whitelist` | `[]` | User tag keys allowed only on the legacy compatibility publisher. | +| `token-pilot.metrics.legacy-ai-token-metrics-enabled` | `false` | Opts into the legacy `ai.token.*` meters. | + +The default Token Pilot-owned meters use bounded domain values and do not add +raw model, tenant, user, request, reservation, or idempotency identifiers. +`tag-whitelist` limits keys, not the number of distinct values supplied by an +application. + +```yaml +token-pilot: + metrics: + enabled: true + tag-whitelist: [] + legacy-ai-token-metrics-enabled: false +``` + +## Budget + +| Property | Default | Meaning | +| --- | --- | --- | +| `token-pilot.budget.enabled` | `false` | Enables the in-memory atomic reservation and accounting store. | +| `token-pilot.budget.monthly-limit` | `10.00` | Monthly limit for each resolved budget key. | +| `token-pilot.budget.policy-id` | `default-monthly` | Policy identity included in the budget key. | +| `token-pilot.budget.target-type` | `tenant` | Target namespace included in the budget key. | +| `token-pilot.budget.target-tag-key` | `tenant_id` | Request tag used to resolve the target ID. | +| `token-pilot.budget.fallback-target-id` | unset | Explicit fallback when the target tag is absent. Without it, resolution fails closed. | +| `token-pilot.budget.currency` | `USD` | Currency of the monthly limit. | +| `token-pilot.budget.zone-id` | `UTC` | IANA zone used to define monthly window boundaries. | + +When enforcement is enabled, the safe preflight upper bound is reserved before +provider dispatch. Actual usage later moves liability into committed cost, or +into a pending reconciliation state when usage is unavailable or needs a +separate pricing snapshot. + +```yaml +token-pilot: + budget: + enabled: true + policy-id: default-monthly + target-type: tenant + target-tag-key: tenant_id + fallback-target-id: shared + monthly-limit: 10.00 + currency: USD + zone-id: Asia/Seoul +``` + +Budget state is process-local and in-memory. It does not provide a durable +store, restart replay, or multi-instance atomicity. + +## Notification + +| Property | Default | Meaning | +| --- | --- | --- | +| `token-pilot.notification.enabled` | `false` | Enables threshold event deduplication and notification lifecycle events. | + +Notification auto-configuration requires a `BudgetNotificationHandler` bean +and an enabled budget accounting graph: + +```java +@Component +final class BudgetAlertHandler implements BudgetNotificationHandler { + @Override + public void handle(BudgetNotificationEvent event) { + // Application-owned email, Slack, Webhook, or audit integration. + } +} +``` + +Handler failures are isolated from the provider response and accounting +transition. Delivery is process-local, best-effort, and at-most-once. + +## Sample `demo` profile + +The sample app's `demo` profile supplies a deterministic in-memory `ChatModel`, +enables budget enforcement, and exposes the eight scenario endpoints described +in [QUICKSTART.md](QUICKSTART.md). Its effective configuration is: + +```yaml +token-pilot: + enabled: true + pricing: + plans: + - model-id: gpt-4o-2024-08-06 + currency: USD + rates: + PROMPT: 0.00015 + COMPLETION: 0.00060 + spring-ai: + default-model-id: gpt-4o-2024-08-06 + default-reserved-output-tokens: 16 + framing-headroom-tokens: 0 + budget: + enabled: true + policy-id: demo-monthly + target-type: tenant + target-tag-key: tenant_id + fallback-target-id: demo-shared + monthly-limit: 0.001 + currency: USD + zone-id: UTC + metrics: + enabled: true + legacy-ai-token-metrics-enabled: false + tag-whitelist: [] +``` + +Start it with: + +```bash +./gradlew --no-daemon :token-pilot-sample-app:bootRun \ + --args='--spring.profiles.active=demo' +``` + +## Auto-configuration rules + +- User-defined beans win over defaults through `@ConditionalOnMissingBean`. +- Optional integrations require their classpath and feature conditions. +- Budget and notification beans are absent by default. +- A custom budget store used for enforcement must also expose the accounting capability; startup fails rather than silently falling back to ledger-only behavior. +- The starter does not select an LLM provider or require an API key by itself. diff --git a/docs/EVOLUTION_PLAN.md b/docs/EVOLUTION_PLAN.md new file mode 100644 index 0000000..54fbee3 --- /dev/null +++ b/docs/EVOLUTION_PLAN.md @@ -0,0 +1,36 @@ +# Evolution plan + +Token Pilot is starting as a framework-independent Java LLM control and +accounting core with optional adapters. The long-term direction is a Java-native +embeddable gateway SDK and, later, an optional standalone runtime. + +## Post-MVP workstreams + +### 1. Exact token intelligence + +- Add exact byte-level BPE implementations and versioned encoding assets. +- Expand Unicode and provider corpus coverage. +- Benchmark estimators with JMH and make tokenizer compatibility explicit. +- Preserve the existing safe-upper-bound contract while exact implementations mature. + +### 2. Provider-independent gateway slice + +- Define provider-neutral request, response, usage, and error contracts. +- Add an optional OpenAI-compatible HTTP gateway surface. +- Keep the core accounting and budget interfaces reusable by direct SDK consumers and gateway callers. + +### 3. Reliable routing and execution + +- Add retries, fallback, circuit breakers, and provider attempt identity. +- Reconcile cost and usage at attempt level. +- Define streaming cancellation and partial-usage semantics before enabling enforcement for streaming calls. + +### 4. Durable multi-tenant operations + +- Add durable reservation, accounting, notification, and audit stores. +- Support multi-instance atomicity and restart replay. +- Add tenant policy administration and security hardening. +- Add OpenTelemetry integration where it complements, rather than duplicates, Spring AI telemetry. + +These workstreams must not be presented as implemented features until their +contracts, tests, and external-consumer evidence land. diff --git a/docs/METRICS.md b/docs/METRICS.md new file mode 100644 index 0000000..7493a7c --- /dev/null +++ b/docs/METRICS.md @@ -0,0 +1,110 @@ +# Metrics reference + +Token Pilot metrics are an operational view of control and accounting events. +The exact monetary source of truth remains the ledger's `BigDecimal` values. +Micrometer counters use `double` internally. + +## Token Pilot-owned meters + +| Meter | Prometheus name | Type | Tags | Published when | +| --- | --- | --- | --- | --- | +| `tokenpilot.cost.total` | `tokenpilot_cost_total_currency_total` | Counter | `currency` | A usage-based reservation reconciliation is newly applied as `COMMITTED`. | +| `tokenpilot.preflight.requests` | `tokenpilot_preflight_requests_total` | Counter | `decision`, `reason` | A core context admission decision is emitted. | +| `tokenpilot.budget.reservations` | `tokenpilot_budget_reservations_total` | Counter | `state` | A reservation evaluation result is emitted. | +| `tokenpilot.reconciliation.error.tokens` | `tokenpilot_reconciliation_error_tokens` | DistributionSummary | `direction` | A newly committed reconciliation has an estimate/actual token delta. | +| `tokenpilot.reconciliation.outcomes` | `tokenpilot_reconciliation_outcomes_reconciliations_total` | Counter | `outcome`, `reason` | A reconciliation transition is newly applied. | +| `tokenpilot.pricing.missing` | `tokenpilot_pricing_missing_total` | Counter | `policy` | Missing pricing is observed at the provider boundary. | +| `tokenpilot.listener.failures` | `tokenpilot_listener_failures_total` | Counter | `listener`, `phase` | An optional accounting listener fails and is isolated. | +| `tokenpilot.notification.events` | `tokenpilot_notification_events_total` | Counter | `outcome`, `threshold` | A notification lifecycle result is emitted. | + +The default user tag whitelist is empty. Domain enum values are normalized to +lowercase except for registered ISO currency codes, which retain values such +as `USD`. + +## Reconciliation semantics + +`tokenpilot.cost.total` consumes only newly applied actual commits. Duplicate +or reused callbacks do not increase cost again. When actual usage is +unavailable, Token Pilot emits a `reconciliation_required` outcome rather than +recording zero cost or zero token error. + +For committed reconciliations, `direction` means: + +- `underestimate`: actual input plus output tokens exceed the estimate plus reserved output. +- `overestimate`: actual usage is below the estimate plus reserved output. +- `exact`: the totals match. + +Listener failures are observed after the accounting transition. They do not +roll back the transition, stop later listeners, or trigger durable redelivery. + +## Legacy compatibility meters + +The following meters are disabled in starter/autoconfigure by default: + +- `ai.token.usage.total` +- `ai.token.usage.distribution` +- `ai.token.cost.total` + +Enable them explicitly during migration: + +```yaml +token-pilot: + metrics: + legacy-ai-token-metrics-enabled: true + tag-whitelist: [tenant_id] +``` + +This compatibility path may overlap Spring AI Observability and allows +application-provided tag values. It is planned for removal in 0.2.0. New +dashboards should use Spring AI's standard provider telemetry for latency, +trace, and provider token usage, and use `tokenpilot.*` for control and +accounting outcomes. + +## Demo verification + +Start the sample app with the `demo` profile, then run the aggregate endpoint: + +```bash +./gradlew --no-daemon :token-pilot-sample-app:bootRun \ + --args='--spring.profiles.active=demo' +curl -s http://localhost:8080/test/token-pilot/demo/run | jq +``` + +The response should contain eight `PASS` scenarios. The budget concurrency +scenario emits one admitted request and seven `blocked` reservation results; +the successful reconciliation scenario emits `committed`, while the failure +and unknown-usage scenarios emit `reconciliation_required`. + +## Prometheus checks + +Expose the Actuator endpoint in a Spring Boot application: + +```yaml +management: + endpoints: + web: + exposure: + include: prometheus,health +``` + +Then inspect the output: + +```bash +curl -fsS http://localhost:8080/actuator/prometheus | grep 'tokenpilot_' +``` + +The provisioned Grafana dashboard reads the same names from +`token-pilot-sample-app/grafana/provisioning/dashboards/token-pilot-dashboard.json`. +Start it with: + +```bash +docker compose -f token-pilot-sample-app/docker-compose.yml up --build -d +``` + +Micrometer's Prometheus naming convention converts dots to underscores and +adds the appropriate counter or summary suffix. Always verify the rendered +Prometheus name from the endpoint before copying a query into a dashboard. + +Do not put prompt text, completion text, request IDs, reservation IDs, raw +provider model strings, or unbounded tenant/user values into metric names or +default tags. diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md new file mode 100644 index 0000000..96228dd --- /dev/null +++ b/docs/QUICKSTART.md @@ -0,0 +1,238 @@ +# Token Pilot 10-minute quickstart + +Token Pilot has two entry paths. Use the first one when the application needs +framework-independent contracts, and the second one when it already uses +Spring Boot and Spring AI. + +| Path | Use it when | Runtime boundary | +| --- | --- | --- | +| `token-pilot-core` | You need token estimation and context admission without Spring | No Spring, Spring AI, Micrometer, or Reactor runtime dependency | +| `token-pilot-starter` | You use Spring Boot and Spring AI | Adds Token Pilot auto-configuration and optional adapters; the application selects its provider | + +The current compatibility baseline is Java 25, Spring Boot 4.1.0, and Spring +AI 2.0.0. The repository default is `0.0.1-SNAPSHOT`; use `0.1.0` only for the +release-candidate staging flow in [RELEASE.md](RELEASE.md). + +## 1. Core path + +Publish the local snapshot from the repository root: + +```bash +./gradlew publishToMavenLocal +``` + +Then add the framework-independent artifact to a plain Java application: + +```gradle +repositories { + mavenLocal() + mavenCentral() +} + +dependencies { + implementation 'cloud.token-pilot:token-pilot-core:0.0.1-SNAPSHOT' +} +``` + +The core API exposes a UTF-8 byte heuristic estimator, a versioned model +catalog, and a context admission contract: + +```java +import io.tokenpilot.core.CoreComponents; +import io.tokenpilot.core.TokenBudget; +import io.tokenpilot.core.TokenEstimator; +import io.tokenpilot.core.domain.BudgetResult; +import io.tokenpilot.core.domain.TokenCountResult; + +public final class CoreExample { + public static void main(String[] args) { + TokenEstimator estimator = CoreComponents.utf8ByteHeuristicTokenEstimator(); + TokenBudget budget = CoreComponents.tokenBudget( + CoreComponents.defaultModelRegistry() + ); + + TokenCountResult input = estimator.estimate("hello"); + BudgetResult result = budget.check("gpt-4o-mini", input, 256); + + System.out.println("status=" + result.status()); + System.out.println("reason=" + result.reason()); + System.out.println("model=" + result.canonicalModelId().orElse("unknown")); + System.out.println("estimator=" + result.estimatorDescriptor()); + } +} +``` + +The estimator returns `TEXT_ONLY` and `exact=false`. Its +`safeUpperBoundTokens` value is the conservative value used for context +admission; it is not an exact tokenizer result. Use +`TokenBudget.requireFits(...)` at a provider boundary when every non-`FITS` +result must stop execution. + +Admission results are: + +- `FITS`: the safe input upper bound plus reserved output fits the registered context window. +- `EXCEEDS`: the safe upper bound or reserved output exceeds the context window. +- `INDETERMINATE`: the model, scope, tokenizer compatibility, or token count is insufficient to authorize the call. + +## 2. Starter and sample demo + +For a Spring Boot application, add the Token Pilot starter and choose a Spring +AI provider in the application: + +```gradle +repositories { + mavenLocal() + mavenCentral() +} + +dependencies { + implementation 'cloud.token-pilot:token-pilot-starter:0.0.1-SNAPSHOT' + implementation 'org.springframework.ai:spring-ai-starter-model-openai:2.0.0' +} +``` + +The repository sample app has a deterministic `demo` profile. It starts with +no application secret and exposes the control/accounting flow as JSON: + +```bash +./gradlew --no-daemon :token-pilot-sample-app:bootRun \ + --args='--spring.profiles.active=demo' +``` + +In another terminal: + +```bash +curl -s http://localhost:8080/test/token-pilot/demo | jq +curl -s http://localhost:8080/test/token-pilot/demo/run | jq +``` + +The aggregate endpoint runs these eight scenarios: + +| Scenario | Expected evidence | +| --- | --- | +| `context-fit` | `FITS`, provider invocations `0` | +| `context-block` | `EXCEEDS`, provider invocations `0` | +| `budget-concurrency` | one admitted request, seven blocked requests, one provider invocation | +| `idempotency` | duplicate command reuses the same key and dispatches once | +| `release` | reservation is released before provider dispatch | +| `reconciliation-success` | actual usage becomes `COMMITTED` | +| `reconciliation-failure` | provider failure becomes `RECONCILIATION_REQUIRED` | +| `reconciliation-unknown` | missing usage becomes `RECONCILIATION_REQUIRED` | + +The response includes `runId`, `scenarios`, and a current `metrics` snapshot. +Run one scenario directly when a focused result is needed: + +```bash +curl -s http://localhost:8080/test/token-pilot/demo/context-block | jq +curl -s http://localhost:8080/test/token-pilot/demo/budget-concurrency | jq +curl -s http://localhost:8080/test/token-pilot/demo/idempotency | jq +curl -s http://localhost:8080/test/token-pilot/demo/reconciliation-failure | jq +``` + +The complete sample procedure is in +[`token-pilot-sample-app/DEMO_RUNBOOK.md`](../token-pilot-sample-app/DEMO_RUNBOOK.md). + +## 3. Prometheus and Grafana + +Keep the sample app running and start the monitoring services: + +```bash +docker compose -f token-pilot-sample-app/docker-compose.yml up --build -d +``` + +Then use: + +- Prometheus targets: +- Prometheus query UI: +- Grafana: + +The provisioned dashboard is named `Token Pilot Demo`. The Prometheus target +scrapes `host.docker.internal:8080/actuator/prometheus`, so run the sample app +on the host before starting the containers. + +Check the rendered meters directly: + +```bash +curl -s http://localhost:8080/actuator/prometheus \ + | rg '^tokenpilot_(preflight|budget|cost|reconciliation)' +``` + +Stop the monitoring services after verification: + +```bash +docker compose -f token-pilot-sample-app/docker-compose.yml down +``` + +## 4. Starter configuration + +The smallest application configuration selects a model, pricing, and the +reserved output fallback: + +```yaml +spring: + ai: + openai: + api-key: ${OPENAI_API_KEY} + +token-pilot: + enabled: true + pricing: + plans: + - model-id: gpt-4o-mini + currency: USD + rates: + PROMPT: 0.00015 + COMPLETION: 0.00060 + spring-ai: + default-model-id: gpt-4o-mini + default-reserved-output-tokens: 256 + metrics: + enabled: true + legacy-ai-token-metrics-enabled: false +``` + +For a supported non-streaming `ChatClient` call, the adapter applies: + +```text +request adaptation +→ TEXT_ONLY scope and framing preflight +→ pricing snapshot resolution +→ context and budget admission +→ atomic reservation when budget enforcement is enabled +→ provider dispatch +→ usage normalization +→ actual reconciliation and Token Pilot metrics +``` + +The current request admission scope is `TEXT_ONLY`. Media, tool messages or +schemas, structured-output augmentation, and enforcement-enabled streaming +are rejected before reservation when they cannot be represented by the +supported contract. + +## 5. Optional budget and notification + +Budget enforcement and notifications are disabled by default. Enable them with +an explicit target or fallback target: + +```yaml +token-pilot: + budget: + enabled: true + policy-id: default-monthly + target-type: tenant + target-tag-key: tenant_id + fallback-target-id: shared + monthly-limit: 10.00 + currency: USD + zone-id: UTC + notification: + enabled: true +``` + +Notification delivery is application-owned. Implement +`BudgetNotificationHandler` as a Spring bean when threshold events should be +sent to email, Slack, Webhook, or another application service. + +For property defaults, conditional bean rules, and failure behavior, continue +with [CONFIGURATION.md](CONFIGURATION.md). For the meter names and Prometheus +queries, see [METRICS.md](METRICS.md). diff --git a/docs/RELEASE.md b/docs/RELEASE.md new file mode 100644 index 0000000..cbb7695 --- /dev/null +++ b/docs/RELEASE.md @@ -0,0 +1,141 @@ +# Release and publication procedure + +Token Pilot publishes seven library modules with one shared version: + +- `token-pilot-core` +- `token-pilot-spring-ai` +- `token-pilot-micrometer` +- `token-pilot-budget` +- `token-pilot-notification` +- `token-pilot-autoconfigure` +- `token-pilot-starter` + +The sample app is not a published module. The current user-facing Spring +artifact is `token-pilot-starter`; do not use the unpublished name +`token-pilot-spring-ai-starter`. + +## Version policy + +The default build version is `0.0.1-SNAPSHOT`. A release candidate is selected +explicitly: + +```bash +./gradlew properties -PprojectVersion=0.1.0 | grep '^version:' +``` + +Do not describe `0.1.0` as published until a consumer resolves it from the +intended repository. Staging verification and Central production publication +are separate gates. + +## Secretless verification + +These checks should run in ordinary CI without signing or publication +credentials: + +```bash +./gradlew test +./gradlew verifyPublicationMetadata verifyCompatibilityMatrix +./gradlew verifyCoreConsumer +./gradlew verifyMicrometerConsumer +./gradlew verifyPublishedIntegrationConsumer +``` + +The generated consumers are temporary external-style projects. They verify the +Core runtime boundary, optional Micrometer owners, the Spring AI adapter, and +the Starter with an explicitly selected provider. + +## Local snapshot consumption + +```bash +./gradlew publishToMavenLocal +``` + +Use `mavenLocal()` only for local verification. A consumer must still depend on +the published coordinates rather than using Gradle `project(...)` substitution +when validating publication metadata. + +## Staging a release candidate + +First validate the intended version and generated metadata: + +```bash +./gradlew verifyPublicationMetadata \ + -PprojectVersion=0.1.0 +``` + +Then create signed staging artifacts: + +```bash +./gradlew publishAllPublicationsToStagingRepository \ + -PprojectVersion=0.1.0 +``` + +The staging repository is module-local under each published module's +`build/staging-deploy` directory. The release build must contain binary, +sources, Javadoc, POM, and Gradle module metadata artifacts. + +## Central Portal deployment + +JReleaser targets the Central Publisher Portal namespace +`cloud.token-pilot`. Deployment requires credentials and signing material +provided outside the repository: + +```bash +export CENTRAL_USERNAME="..." +export CENTRAL_PASSWORD="..." +export SIGNING_KEY_FILE="/secure/path/private-key.asc" +export SIGNING_PUBLIC_KEY_FILE="/secure/path/public-key.asc" +export SIGNING_PASSWORD="..." + +./gradlew jreleaserDeploy -PprojectVersion=0.1.0 +``` + +The following values are supported through Gradle properties or environment +variables: + +| Purpose | Gradle property | Environment variable | +| --- | --- | --- | +| Central username | `centralUsername` | `CENTRAL_USERNAME` | +| Central password | `centralPassword` | `CENTRAL_PASSWORD` | +| Signing secret key file | `signingKeyFile` | `SIGNING_KEY_FILE` | +| Signing public key file | `signingPublicKeyFile` | `SIGNING_PUBLIC_KEY_FILE` | +| Signing passphrase | `signingPassword` | `SIGNING_PASSWORD` | +| GitHub release token | `githubToken` | `GITHUB_TOKEN` | + +Never commit credentials, armored keys, or API keys. A non-snapshot release +requires signing material; a missing secret should fail clearly rather than +silently producing an unsigned release. + +## GitHub Packages snapshots + +Snapshots can be published separately when the repository credentials are +available: + +```bash +./gradlew publish \ + -PmavenRepoUrl=https://maven.pkg.github.com/tokenpliot/tokenpilot \ + -PmavenRepoUsername="$GITHUB_ACTOR" \ + -PmavenRepoPassword="$GITHUB_TOKEN" +``` + +## Final consumption gate + +Before announcing a release, verify both paths from clean external consumers: + +```gradle +dependencies { + implementation 'cloud.token-pilot:token-pilot-core:0.1.0' +} +``` + +```gradle +dependencies { + implementation 'cloud.token-pilot:token-pilot-starter:0.1.0' + implementation 'org.springframework.ai:spring-ai-starter-model-openai:2.0.0' +} +``` + +The Core consumer must resolve without Spring, Spring AI, Micrometer, or +Reactor runtime dependencies. The Starter consumer must not receive a +provider implementation transitively from Token Pilot. Only after this gate +passes may `0.1.0` be described as released. diff --git a/docs/SAMPLE_RUNBOOK.md b/docs/SAMPLE_RUNBOOK.md new file mode 100644 index 0000000..b757034 --- /dev/null +++ b/docs/SAMPLE_RUNBOOK.md @@ -0,0 +1,181 @@ +# Sample app runbook + +`token-pilot-sample-app` is the repository verification application for the +starter, auto-configuration, control lifecycle, and Token Pilot-owned metrics. +Use the default profile for the basic endpoints and the `demo` profile for the +deterministic admission, reservation, and reconciliation flow. + +## Prerequisites + +- Java 25 +- Docker Desktop when Prometheus and Grafana are needed +- `curl` and `jq` for readable JSON output + +## Run automated verification + +Run the sample app tests from the repository root: + +```bash +./gradlew --no-daemon :token-pilot-sample-app:test +``` + +The complete repository gate is: + +```bash +./gradlew --no-daemon test +``` + +The tests cover preflight blocking, atomic reservation, actual usage +reconciliation, pricing misses, unsupported request scope, listener isolation, +and enforcement-enabled streaming rejection. `DemoScenarioE2ETest` covers the +HTTP contract for all eight demo scenarios. + +## Default profile + +Start the sample app without an active profile: + +```bash +./gradlew --no-daemon :token-pilot-sample-app:bootRun +``` + +The server listens on port `8080`. Verify the starter and Actuator endpoints: + +```bash +curl -fsS http://localhost:8080/test/token-pilot/smoke +curl -fsS http://localhost:8080/test/token-pilot/beans +curl -fsS http://localhost:8080/test/token-pilot/record +curl -fsS http://localhost:8080/actuator/health +curl -fsS http://localhost:8080/actuator/prometheus \ + | rg '^tokenpilot_' +``` + +The default profile enables Token Pilot and Token Pilot-owned metrics, loads +the `gpt-4o-mini` pricing example, and leaves budget and notification +enforcement disabled. + +## Deterministic demo profile + +Stop the default app if it is running, then start the `demo` profile: + +```bash +./gradlew --no-daemon :token-pilot-sample-app:bootRun \ + --args='--spring.profiles.active=demo' +``` + +The demo profile uses the in-memory `DemoChatModel` and does not require an +application secret. Check the index first: + +```bash +curl -s http://localhost:8080/test/token-pilot/demo | jq +``` + +Run the complete scenario set: + +```bash +curl -s http://localhost:8080/test/token-pilot/demo/run | jq '{ + runId, + scenarios: [.scenarios[] | { + scenario, + status, + providerInvocations: .evidence.providerInvocations, + admittedRequests: .evidence.admittedRequests, + blockedRequests: .evidence.blockedRequests, + accountingState: .evidence.accountingState, + budget: .evidence.budget + }], + metrics +}' +``` + +Expected results: + +| Scenario | Expected result | +| --- | --- | +| `context-fit` | `PASS`, admission `FITS`, provider invocations `0` | +| `context-block` | `PASS`, admission `EXCEEDS`, provider invocations `0` | +| `budget-concurrency` | `PASS`, one admitted request, seven blocked requests, one provider invocation | +| `idempotency` | `PASS`, the repeated key dispatches once | +| `release` | `PASS`, the reservation is released before dispatch | +| `reconciliation-success` | `PASS`, accounting state `COMMITTED` | +| `reconciliation-failure` | `PASS`, accounting state `RECONCILIATION_REQUIRED` | +| `reconciliation-unknown` | `PASS`, accounting state `RECONCILIATION_REQUIRED` | + +Run a focused scenario through its individual endpoint: + +```bash +curl -s http://localhost:8080/test/token-pilot/demo/context-fit | jq +curl -s http://localhost:8080/test/token-pilot/demo/context-block | jq +curl -s http://localhost:8080/test/token-pilot/demo/budget-concurrency | jq +curl -s http://localhost:8080/test/token-pilot/demo/idempotency | jq +curl -s http://localhost:8080/test/token-pilot/demo/release | jq +curl -s http://localhost:8080/test/token-pilot/demo/reconciliation-success | jq +curl -s http://localhost:8080/test/token-pilot/demo/reconciliation-failure | jq +curl -s http://localhost:8080/test/token-pilot/demo/reconciliation-unknown | jq +``` + +The endpoint list and the effective profile configuration are also maintained +in [`token-pilot-sample-app/DEMO_RUNBOOK.md`](../token-pilot-sample-app/DEMO_RUNBOOK.md). + +## Prometheus and Grafana + +Keep the app running and start the monitoring containers from the repository +root: + +```bash +docker compose -f token-pilot-sample-app/docker-compose.yml up --build -d +``` + +Open the following addresses: + +- Prometheus targets: +- Prometheus query UI: +- Grafana: + +The Prometheus target is `token-pilot-backend` and scrapes +`host.docker.internal:8080/actuator/prometheus`. Run `/test/token-pilot/demo/run` +first, then inspect the current `tokenpilot_*` meters: + +```bash +curl -s http://localhost:8080/actuator/prometheus \ + | rg '^tokenpilot_(preflight|budget|cost|reconciliation)' +``` + +The Grafana container provisions the `Token Pilot Demo` dashboard from +`token-pilot-sample-app/grafana/provisioning/dashboards/token-pilot-dashboard.json`. +It is configured with the Prometheus data source and uses the current +Token Pilot-owned meter names. + +Stop the monitoring containers when finished: + +```bash +docker compose -f token-pilot-sample-app/docker-compose.yml down +``` + +## Reset and troubleshooting + +The demo budget store and Micrometer counters are in memory. Restart the +sample app to begin with a clean scenario state. + +### `ledgerAdvisor` is absent + +Confirm that the starter is on the classpath and `token-pilot.enabled=true`. +The sample app should not create the advisor manually. + +### Budget beans are absent + +The default profile intentionally disables enforcement. Use the `demo` profile, +or set `token-pilot.budget.enabled=true` and provide a target tag or +`fallback-target-id`. + +### No `tokenpilot_*` meters appear + +Check that Actuator and the Prometheus registry are present, +`token-pilot.metrics.enabled=true`, and that a relevant event has occurred. A +cost meter appears after a newly applied usage-based reservation commit; a +legacy cost-only ledger record is not sufficient. + +### The dashboard shows no data + +Check Prometheus `/targets` first. The target must be `UP`, and the sample app +must be reachable at port `8080`. Query the rendered names directly in +Prometheus before changing a dashboard panel. diff --git a/token-pilot-sample-app/DEMO_RUNBOOK.md b/token-pilot-sample-app/DEMO_RUNBOOK.md new file mode 100644 index 0000000..2376c1f --- /dev/null +++ b/token-pilot-sample-app/DEMO_RUNBOOK.md @@ -0,0 +1,95 @@ +# Token Pilot demo runbook + +이 데모는 `demo` Spring profile에서만 활성화되는 네트워크 없는 가짜 +`ChatModel`을 사용합니다. 별도 자격 증명이나 외부 호출 없이도 context admission, +atomic reservation, idempotency, release, actual reconciliation을 반복해서 확인할 수 +있습니다. + +## 1. 앱 실행 + +프로젝트 루트에서 실행합니다. + +```bash +./gradlew --no-daemon :token-pilot-sample-app:bootRun \ + --args='--spring.profiles.active=demo' +``` + +앱은 `http://localhost:8080`에서 대기합니다. 별도 API key는 필요하지 않습니다. + +## 2. 한 번에 시나리오 실행 + +먼저 데모 상태와 시나리오 목록을 보여줍니다. + +```bash +curl -s http://localhost:8080/test/token-pilot/demo | jq +``` + +8개 시나리오를 실행하고 핵심 검증 필드만 출력합니다. + +```bash +curl -s http://localhost:8080/test/token-pilot/demo/run | jq '{ + runId, + scenarios: [.scenarios[] | { + scenario, + status, + providerInvocations: .evidence.providerInvocations, + admittedRequests: .evidence.admittedRequests, + blockedRequests: .evidence.blockedRequests, + accountingState: .evidence.accountingState, + budget: .evidence.budget + }], + metrics +}' +``` + +핵심 기대 결과는 다음과 같습니다. + +- `context-fit` → `FITS`, provider 호출 `0` +- `context-block` → `EXCEEDS`, provider 호출 `0` +- `budget-concurrency` → `admittedRequests=1`, `blockedRequests=7`, `providerInvocations=1` +- `idempotency` → `sameIdempotencyKey=true`, provider 호출 `1` +- `reconciliation-success` → `COMMITTED` +- `reconciliation-failure`, `reconciliation-unknown` → `RECONCILIATION_REQUIRED` + +개별 장면을 다시 찍고 싶을 때는 다음 endpoint를 사용합니다. + +```bash +curl -s http://localhost:8080/test/token-pilot/demo/context-block | jq +curl -s http://localhost:8080/test/token-pilot/demo/budget-concurrency | jq +curl -s http://localhost:8080/test/token-pilot/demo/idempotency | jq +curl -s http://localhost:8080/test/token-pilot/demo/reconciliation-failure | jq +``` + +## 3. Prometheus와 Grafana 실행 + +앱은 호스트에서 실행하고 monitoring container만 띄웁니다. + +```bash +docker compose -f token-pilot-sample-app/docker-compose.yml up --build -d +``` + +확인할 주소는 다음과 같습니다. + +- Prometheus target: `http://localhost:9090/targets` → `token-pilot-backend`가 `UP` +- Prometheus query: `http://localhost:9090/graph` +- Grafana: `http://localhost:3000` → `Token Pilot Demo` dashboard 선택 + +Prometheus는 `host.docker.internal:8080/actuator/prometheus`를 scrape하도록 구성되어 +있습니다. 앱의 `/run`을 실행한 뒤 10초 정도 기다리면 Grafana stat과 time-series에 +`tokenpilot_*` 값이 표시됩니다. + +```bash +curl -s http://localhost:8080/actuator/prometheus \ + | rg '^tokenpilot_(preflight|budget|cost|reconciliation)' +``` + +검증이 끝나면 monitoring container를 정리합니다. + +```bash +docker compose -f token-pilot-sample-app/docker-compose.yml down +``` + +## 4. 상태 초기화 + +앱을 종료한 뒤 다시 `bootRun`하면 in-memory budget과 Micrometer counter가 +초기화됩니다. diff --git a/token-pilot-sample-app/docker-compose.yml b/token-pilot-sample-app/docker-compose.yml index cf872eb..b658184 100644 --- a/token-pilot-sample-app/docker-compose.yml +++ b/token-pilot-sample-app/docker-compose.yml @@ -10,12 +10,13 @@ services: grafana: build: ./grafana container_name: grafana + depends_on: + - prometheus ports: - "3000:3000" environment: - PROMETHEUS_URL=http://prometheus:9090 - GF_AUTH_ANONYMOUS_ENABLED=true - GF_AUTH_ANONYMOUS_ORG_NAME=Main Org. - # 👇 여기 아래 두 줄을 집중해서 수정/추가해 주세요! - - GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer # 기존 Admin에서 Viewer로 변경 (면접관용) - - GF_AUTH_DISABLE_LOGIN_FORM=false # 로그인 창을 살려둠 (내 작업용 숨겨진 통로) \ No newline at end of file + - GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer + - GF_AUTH_DISABLE_LOGIN_FORM=true # 별도 로그인 없이 대시보드 조회 diff --git a/token-pilot-sample-app/grafana/provisioning/dashboards/token-pilot-dashboard.json b/token-pilot-sample-app/grafana/provisioning/dashboards/token-pilot-dashboard.json index 305ffc7..14540f5 100644 --- a/token-pilot-sample-app/grafana/provisioning/dashboards/token-pilot-dashboard.json +++ b/token-pilot-sample-app/grafana/provisioning/dashboards/token-pilot-dashboard.json @@ -1,3475 +1,498 @@ { - "__inputs": [ - { - "name": "DS_PROMETHEUS", - "label": "Prometheus", - "description": "", - "type": "datasource", - "pluginId": "prometheus", - "pluginName": "Prometheus" - } - ], - "__elements": {}, - "__requires": [ - { - "type": "panel", - "id": "gauge", - "name": "Gauge", - "version": "" - }, - { - "type": "grafana", - "id": "grafana", - "name": "Grafana", - "version": "13.0.1+security-01" - }, - { - "type": "panel", - "id": "piechart", - "name": "Pie chart", - "version": "" - }, - { - "type": "datasource", - "id": "prometheus", - "name": "Prometheus", - "version": "1.0.0" - }, - { - "type": "panel", - "id": "stat", - "name": "Stat", - "version": "" - }, - { - "type": "panel", - "id": "timeseries", - "name": "Time series", - "version": "" - } - ], "annotations": { - "list": [ - { - "builtIn": 1, - "datasource": { - "type": "grafana", - "uid": "grafana" - }, - "enable": true, - "hide": true, - "iconColor": "rgba(0, 211, 255, 1)", - "name": "Annotations & Alerts", - "target": { - "limit": 100, - "matchAny": false, - "tags": [], - "type": "dashboard" - }, - "type": "dashboard" - } - ] + "list": [] }, - "description": "Spring Boot Dashboard adapted from https://grafana.com/grafana/dashboards/10280-microservices-spring-boot-2-1/", "editable": true, "fiscalYearStartMonth": 0, - "graphTooltip": 0, + "graphTooltip": 1, "links": [], - "liveNow": false, "panels": [ { - "collapsed": true, + "datasource": { + "type": "prometheus", + "uid": "prometheus-uid" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 1 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, "gridPos": { - "h": 1, - "w": 24, + "h": 5, + "w": 6, "x": 0, "y": 0 }, - "id": 101, - "panels": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "decimals": 1, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - } - ], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "s" - } - }, - "gridPos": { - "h": 3, - "w": 6, - "x": 0, - "y": 1 - }, - "id": 52, - "maxDataPoints": 100, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "horizontal", - "percentChangeColorMode": "standard", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "process_uptime_seconds{application=\"$application\", instance=\"$instance\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 2, - "legendFormat": "", - "metric": "", - "range": true, - "refId": "A", - "step": 14400 - } - ], - "title": "Uptime", - "type": "stat" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "decimals": 1, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - } - ], - "max": 100, - "min": 0, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "rgba(50, 172, 45, 0.97)", - "value": 0 - }, - { - "color": "rgba(237, 129, 40, 0.89)", - "value": 70 - }, - { - "color": "rgba(245, 54, 54, 0.9)", - "value": 90 - } - ] - }, - "unit": "percent" - } - }, - "gridPos": { - "h": 6, - "w": 5, - "x": 6, - "y": 1 - }, - "id": 58, - "maxDataPoints": 100, - "options": { - "barShape": "flat", - "barWidthFactor": 0.5, - "effects": { - "barGlow": false, - "centerGlow": false, - "gradient": false - }, - "endpointMarker": "point", - "minVizHeight": 75, - "minVizWidth": 75, - "orientation": "horizontal", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "segmentCount": 1, - "segmentSpacing": 0.3, - "shape": "gauge", - "showThresholdLabels": false, - "showThresholdMarkers": true, - "sizing": "auto", - "sparkline": false, - "textMode": "auto" - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "sum(jvm_memory_used_bytes{application=\"$application\", instance=\"$instance\", area=\"heap\", namespace=\"$Namespace\"})*100/sum(jvm_memory_max_bytes{application=\"$application\",instance=\"$instance\", area=\"heap\", namespace=\"$Namespace\"})", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "", - "range": true, - "refId": "A", - "step": 14400 - } - ], - "title": "Heap Used", - "type": "gauge" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "decimals": 1, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - }, - { - "options": { - "from": -1e+32, - "result": { - "text": "N/A" - }, - "to": 0 - }, - "type": "range" - } - ], - "max": 100, - "min": 0, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "rgba(50, 172, 45, 0.97)", - "value": 0 - }, - { - "color": "rgba(237, 129, 40, 0.89)", - "value": 70 - }, - { - "color": "rgba(245, 54, 54, 0.9)", - "value": 90 - } - ] - }, - "unit": "percent" - } - }, - "gridPos": { - "h": 6, - "w": 5, - "x": 11, - "y": 1 - }, - "id": 60, - "maxDataPoints": 100, - "options": { - "barShape": "flat", - "barWidthFactor": 0.5, - "effects": { - "barGlow": false, - "centerGlow": false, - "gradient": false - }, - "endpointMarker": "point", - "minVizHeight": 75, - "minVizWidth": 75, - "orientation": "horizontal", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "segmentCount": 1, - "segmentSpacing": 0.3, - "shape": "gauge", - "showThresholdLabels": false, - "showThresholdMarkers": true, - "sizing": "auto", - "sparkline": false, - "textMode": "auto" - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "sum(jvm_memory_used_bytes{application=\"$application\", instance=\"$instance\", area=\"nonheap\", namespace=\"$Namespace\"})*100/sum(jvm_memory_max_bytes{application=\"$application\",instance=\"$instance\", area=\"nonheap\", namespace=\"$Namespace\"})", - "format": "time_series", - "intervalFactor": 2, - "legendFormat": "", - "range": true, - "refId": "A", - "step": 14400 - } - ], - "title": "Non-Heap Used", - "type": "gauge" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "locale" - } - }, - "gridPos": { - "h": 6, - "w": 8, - "x": 16, - "y": 1 - }, - "id": 66, - "options": { - "alertThreshold": true, - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "process_files_open_files{application=\"$application\", instance=\"$instance\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Open Files", - "range": true, - "refId": "A" - } - ], - "title": "Process Open Files", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - } - ], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "dateTimeAsIso" - } - }, - "gridPos": { - "h": 3, - "w": 6, - "x": 0, - "y": 4 - }, - "id": 56, - "maxDataPoints": 100, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "horizontal", - "percentChangeColorMode": "standard", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "process_start_time_seconds{application=\"$application\", instance=\"$instance\", namespace=\"$Namespace\"}*1000", - "format": "time_series", - "intervalFactor": 2, - "legendFormat": "", - "metric": "", - "range": true, - "refId": "A", - "step": 14400 - } - ], - "title": "Start time", - "type": "stat" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - } - }, - "gridPos": { - "h": 7, - "w": 12, - "x": 0, - "y": 7 - }, - "id": 95, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "system_cpu_usage{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "System CPU Usage", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "process_cpu_usage{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Process CPU Usage", - "range": true, - "refId": "B" - } - ], - "title": "CPU Usage", - "type": "timeseries" - }, + "id": 1, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - } - }, - "gridPos": { - "h": 7, - "w": 12, - "x": 12, - "y": 7 - }, - "id": 96, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "system_load_average_1m{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Load Average [1m]", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "system_cpu_count{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "CPU Core Size", - "range": true, - "refId": "B" - } - ], - "title": "Load Average", - "type": "timeseries" + "editorMode": "code", + "expr": "sum(tokenpilot_preflight_requests_total{application=\"token-pilot-sample-app\",decision=\"fits\"})", + "legendFormat": "FITS", + "refId": "A" } ], - "title": "Basic Statistics", - "type": "row" + "title": "Context FITS", + "type": "stat" }, { - "collapsed": true, - "gridPos": { - "h": 1, - "w": 24, - "x": 0, - "y": 1 + "datasource": { + "type": "prometheus", + "uid": "prometheus-uid" }, - "id": 102, - "panels": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "bytes" - } - }, - "gridPos": { - "h": 8, - "w": 24, - "x": 0, - "y": 2 - }, - "id": 85, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "repeat": "memory_pool_heap", - "repeatDirection": "h", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_memory_used_bytes{instance=\"$instance\", application=\"$application\", id=\"$memory_pool_heap\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Used", - "range": true, - "refId": "C" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_memory_committed_bytes{instance=\"$instance\", application=\"$application\", id=\"$memory_pool_heap\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Commited", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_memory_max_bytes{instance=\"$instance\", application=\"$application\", id=\"$memory_pool_heap\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Max", - "range": true, - "refId": "B" - } - ], - "title": "$memory_pool_heap (heap)", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "bytes" - } - }, - "gridPos": { - "h": 8, - "w": 24, - "x": 0, - "y": 10 - }, - "id": 88, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "repeat": "memory_pool_nonheap", - "repeatDirection": "h", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_memory_used_bytes{instance=\"$instance\", application=\"$application\", id=\"${memory_pool_nonheap:raw}\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Used", - "range": true, - "refId": "C" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_memory_committed_bytes{instance=\"$instance\", application=\"$application\", id=\"${memory_pool_nonheap:raw}\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Commited", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_memory_max_bytes{instance=\"$instance\", application=\"$application\", id=\"${memory_pool_nonheap:raw}\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Max", - "range": true, - "refId": "B" - } - ], - "title": "$memory_pool_nonheap (non-heap)", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - } - }, - "gridPos": { - "h": 7, - "w": 12, - "x": 0, - "y": 26 - }, - "id": 82, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_buffer_memory_used_bytes{instance=\"$instance\", application=\"$application\", id=\"direct\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Used Bytes", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_buffer_total_capacity_bytes{instance=\"$instance\", application=\"$application\", id=\"direct\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Capacity Bytes", - "range": true, - "refId": "B" - } - ], - "title": "Direct Buffers", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - } - }, - "gridPos": { - "h": 8, - "w": 12, - "x": 12, - "y": 26 - }, - "id": 68, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_threads_daemon_threads{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Daemon", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_threads_live_threads{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Live", - "range": true, - "refId": "B" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_threads_peak_threads{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Peak", - "range": true, - "refId": "C" - } - ], - "title": "Threads", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "bytes" - } - }, - "gridPos": { - "h": 8, - "w": 12, - "x": 0, - "y": 33 - }, - "id": 78, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 1 + } + ] }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(jvm_gc_memory_allocated_bytes_total{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "allocated", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(jvm_gc_memory_promoted_bytes_total{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "promoted", - "range": true, - "refId": "B" - } - ], - "title": "Memory Allocate/Promote", - "type": "timeseries" + "unit": "short" }, + "overrides": [] + }, + "gridPos": { + "h": 5, + "w": 6, + "x": 6, + "y": 0 + }, + "id": 2, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - } - }, - "gridPos": { - "h": 7, - "w": 12, - "x": 12, - "y": 34 - }, - "id": 83, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_buffer_memory_used_bytes{instance=\"$instance\", application=\"$application\", id=\"mapped\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Used Bytes", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "jvm_buffer_total_capacity_bytes{instance=\"$instance\", application=\"$application\", id=\"mapped\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Capacity Bytes", - "range": true, - "refId": "B" - } - ], - "title": "Mapped Buffers", - "type": "timeseries" + "editorMode": "code", + "expr": "sum(tokenpilot_preflight_requests_total{application=\"token-pilot-sample-app\",decision=\"exceeds\"})", + "legendFormat": "BLOCKED", + "refId": "A" } ], - "title": "JVM Statistics - Memory", - "type": "row" + "title": "Context BLOCK", + "type": "stat" }, { - "collapsed": true, - "gridPos": { - "h": 1, - "w": 24, - "x": 0, - "y": 2 + "datasource": { + "type": "prometheus", + "uid": "prometheus-uid" }, - "id": 103, - "panels": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "locale" - } - }, - "gridPos": { - "h": 10, - "w": 12, - "x": 0, - "y": 3 - }, - "id": 74, - "options": { - "legend": { - "calcs": [ - "mean", - "max", - "min", - "sum" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "blue", + "value": null + } + ] }, - "pluginVersion": "9.5.1", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(jvm_gc_pause_seconds_count{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{action}} [{{cause}}]", - "range": true, - "refId": "A" - } - ], - "title": "GC Count", - "type": "timeseries" + "unit": "short" }, + "overrides": [] + }, + "gridPos": { + "h": 5, + "w": 6, + "x": 12, + "y": 0 + }, + "id": 3, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "s" - } - }, - "gridPos": { - "h": 10, - "w": 12, - "x": 12, - "y": 3 - }, - "id": 76, - "options": { - "legend": { - "calcs": [ - "mean", - "max", - "min", - "sum" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "9.5.1", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(jvm_gc_pause_seconds_sum{instance=\"$instance\", application=\"$application\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{action}} [{{cause}}]", - "range": true, - "refId": "A" - } - ], - "title": "GC Stop the World Duration", - "type": "timeseries" + "editorMode": "code", + "expr": "sum(tokenpilot_budget_reservations_total{application=\"token-pilot-sample-app\",state=\"created\"})", + "legendFormat": "CREATED", + "refId": "A" } ], - "title": "JVM Statistics - GC", - "type": "row" + "title": "Reservations CREATED", + "type": "stat" }, { - "collapsed": true, - "gridPos": { - "h": 1, - "w": 24, - "x": 0, - "y": 3 + "datasource": { + "type": "prometheus", + "uid": "prometheus-uid" }, - "id": 104, - "panels": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - } - ], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "none" - } - }, - "gridPos": { - "h": 4, - "w": 4, - "x": 0, - "y": 4 - }, - "id": 44, - "maxDataPoints": 100, - "options": { - "colorMode": "none", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "horizontal", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "textMode": "auto" - }, - "pluginVersion": "9.5.1", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "hikaricp_connections{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "", - "range": true, - "refId": "A" - } - ], - "title": "Connections Size", - "type": "stat" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "normal" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - } - }, - "gridPos": { - "h": 8, - "w": 20, - "x": 4, - "y": 4 - }, - "id": 36, - "options": { - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "9.5.1", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "hikaricp_connections_active{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Active", - "range": true, - "refId": "B" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "hikaricp_connections_idle{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Idle", - "range": true, - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "hikaricp_connections_pending{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Pending", - "range": true, - "refId": "C" - } - ], - "title": "Connections", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - } - ], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green" - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "none" - } - }, - "gridPos": { - "h": 4, - "w": 4, - "x": 0, - "y": 8 - }, - "id": 46, - "maxDataPoints": 100, - "options": { - "colorMode": "none", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "horizontal", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "textMode": "auto" - }, - "pluginVersion": "9.5.1", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "hikaricp_connections_timeout_total{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\", namespace=\"$Namespace\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "", - "range": true, - "refId": "A" - } - ], - "title": "Connection Timeout Count", - "type": "stat" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "gridPos": { - "h": 6, - "w": 8, - "x": 0, - "y": 12 - }, - "id": 38, - "options": { - "alertThreshold": true - }, - "pluginVersion": "9.5.1", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "expr": "hikaricp_connections_creation_seconds_sum{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\"} / hikaricp_connections_creation_seconds_count{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Creation Time", - "refId": "A" - } - ], - "title": "Connection Creation Time", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "gridPos": { - "h": 6, - "w": 8, - "x": 8, - "y": 12 - }, - "id": 42, - "options": { - "alertThreshold": true + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "orange", + "value": null + } + ] }, - "pluginVersion": "9.5.1", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "expr": "hikaricp_connections_usage_seconds_sum{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\"} / hikaricp_connections_usage_seconds_count{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Usage Time", - "refId": "A" - } - ], - "title": "Connection Usage Time", - "type": "timeseries" + "unit": "short" }, + "overrides": [] + }, + "gridPos": { + "h": 5, + "w": 6, + "x": 18, + "y": 0 + }, + "id": 4, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "gridPos": { - "h": 6, - "w": 8, - "x": 16, - "y": 12 - }, - "id": 40, - "options": { - "alertThreshold": true - }, - "pluginVersion": "9.5.1", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "expr": "hikaricp_connections_acquire_seconds_sum{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\"} / hikaricp_connections_acquire_seconds_count{instance=\"$instance\", application=\"$application\", pool=\"$hikaricp\"}", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "Acquire Time", - "refId": "A" - } - ], - "title": "Connection Acquire Time", - "type": "timeseries" + "editorMode": "code", + "expr": "sum(tokenpilot_budget_reservations_total{application=\"token-pilot-sample-app\",state=\"blocked\"})", + "legendFormat": "BLOCKED", + "refId": "A" } ], - "title": "Database Connection Pool HikariCP Statistics", - "type": "row" + "title": "Reservations BLOCKED", + "type": "stat" }, { - "collapsed": true, + "datasource": { + "type": "prometheus", + "uid": "prometheus-uid" + }, + "fieldConfig": { + "defaults": { + "decimals": 6, + "unit": "currencyUSD" + }, + "overrides": [] + }, "gridPos": { - "h": 1, - "w": 24, + "h": 5, + "w": 8, "x": 0, - "y": 4 + "y": 5 }, - "id": 105, - "panels": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "none" - } - }, - "gridPos": { - "h": 7, - "w": 24, - "x": 0, - "y": 5 - }, - "id": 4, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [], - "displayMode": "table", - "placement": "right", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(http_server_requests_seconds_count{instance=\"$instance\", application=\"$application\", uri!~\".*(prometheus|health).*\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{method}} [{{status}}] - {{uri}}", - "range": true, - "refId": "A" - } - ], - "title": "Request Count", - "type": "timeseries" - }, + "id": 5, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "s" - } - }, - "gridPos": { - "h": 7, - "w": 24, - "x": 0, - "y": 12 - }, - "id": 2, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "max", - "min" - ], - "displayMode": "table", - "placement": "right", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(http_server_requests_seconds_sum{instance=\"$instance\", application=\"$application\", exception=\"none\", uri!~\".*(prometheus|health).*\", namespace=\"$Namespace\"}[$__rate_interval]) / irate(http_server_requests_seconds_count{instance=\"$instance\", application=\"$application\", exception=\"none\",uri!~\".*(prometheus|health).*\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{method}} [{{status}}] - {{uri}}", - "range": true, - "refId": "A" - } - ], - "title": "Response Time", - "type": "timeseries" + "editorMode": "code", + "expr": "sum(tokenpilot_cost_total_currency_total{application=\"token-pilot-sample-app\",currency=\"USD\"})", + "legendFormat": "USD", + "refId": "A" } ], - "title": "HTTP Statistics", - "type": "row" + "title": "Actual Cost", + "type": "stat" }, { - "collapsed": true, + "datasource": { + "type": "prometheus", + "uid": "prometheus-uid" + }, + "fieldConfig": { + "defaults": { + "unit": "short" + }, + "overrides": [] + }, "gridPos": { - "h": 1, - "w": 24, - "x": 0, + "h": 5, + "w": 8, + "x": 8, "y": 5 }, - "id": 106, - "panels": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "none" - } - }, - "gridPos": { - "h": 7, - "w": 12, - "x": 0, - "y": 6 - }, - "id": 6, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min", - "sum" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "alias": "", - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(logback_events_total{instance=\"$instance\", application=\"$application\", level=\"info\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "info", - "range": true, - "rawSql": "SELECT\n $__time(time_column),\n value1\nFROM\n metric_table\nWHERE\n $__timeFilter(time_column)\n", - "refId": "A" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "logback_events_total{instance=\"$instance\", application=\"$application\"}", - "hide": true, - "legendFormat": "__auto", - "range": true, - "refId": "B" - } - ], - "title": "INFO logs", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "none" - } - }, - "gridPos": { - "h": 7, - "w": 12, - "x": 12, - "y": 6 - }, - "id": 10, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min", - "sum" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "alias": "", - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(logback_events_total{instance=\"$instance\", application=\"$application\", level=\"error\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "error", - "range": true, - "rawSql": "SELECT\n $__time(time_column),\n value1\nFROM\n metric_table\nWHERE\n $__timeFilter(time_column)\n", - "refId": "A" - } - ], - "title": "ERROR logs", - "type": "timeseries" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "none" - } - }, - "gridPos": { - "h": 7, - "w": 8, - "x": 0, - "y": 13 - }, - "id": 14, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min", - "sum" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "alias": "", - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(logback_events_total{instance=\"$instance\", application=\"$application\", level=\"warn\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "warn", - "range": true, - "rawSql": "SELECT\n $__time(time_column),\n value1\nFROM\n metric_table\nWHERE\n $__timeFilter(time_column)\n", - "refId": "A" - } - ], - "title": "WARN logs", - "type": "timeseries" - }, + "id": 6, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "none" - } - }, - "gridPos": { - "h": 7, - "w": 8, - "x": 8, - "y": 13 - }, - "id": 16, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min", - "sum" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } + "editorMode": "code", + "expr": "sum(tokenpilot_reconciliation_outcomes_reconciliations_total{application=\"token-pilot-sample-app\",outcome=\"committed\"})", + "legendFormat": "COMMITTED", + "refId": "A" + } + ], + "title": "Reconciliation COMMITTED", + "type": "stat" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus-uid" + }, + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "orange", + "value": null + }, + { + "color": "red", + "value": 1 + } + ] }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "alias": "", - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(logback_events_total{instance=\"$instance\", application=\"$application\", level=\"debug\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "debug", - "range": true, - "rawSql": "SELECT\n $__time(time_column),\n value1\nFROM\n metric_table\nWHERE\n $__timeFilter(time_column)\n", - "refId": "A" - } - ], - "title": "DEBUG logs", - "type": "timeseries" + "unit": "short" }, + "overrides": [] + }, + "gridPos": { + "h": 5, + "w": 8, + "x": 16, + "y": 5 + }, + "id": 7, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "none" - } - }, - "gridPos": { - "h": 7, - "w": 8, - "x": 16, - "y": 13 - }, - "id": 20, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [ - "mean", - "lastNotNull", - "max", - "min", - "sum" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "alias": "", - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "irate(logback_events_total{instance=\"$instance\", application=\"$application\", level=\"trace\", namespace=\"$Namespace\"}[$__rate_interval])", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "trace", - "range": true, - "rawSql": "SELECT\n $__time(time_column),\n value1\nFROM\n metric_table\nWHERE\n $__timeFilter(time_column)\n", - "refId": "A" - } - ], - "title": "TRACE logs", - "type": "timeseries" + "editorMode": "code", + "expr": "sum(tokenpilot_reconciliation_outcomes_reconciliations_total{application=\"token-pilot-sample-app\",outcome=\"reconciliation_required\"})", + "legendFormat": "RECONCILIATION_REQUIRED", + "refId": "A" } ], - "title": "Logback Statistics", - "type": "row" + "title": "Reconciliation REQUIRED", + "type": "stat" }, { - "collapsed": true, + "datasource": { + "type": "prometheus", + "uid": "prometheus-uid" + }, + "fieldConfig": { + "defaults": { + "unit": "short" + }, + "overrides": [] + }, "gridPos": { - "h": 1, - "w": 24, + "h": 8, + "w": 12, "x": 0, - "y": 6 + "y": 10 }, - "id": 107, - "panels": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "currencyUSD" - } - }, - "gridPos": { - "h": 9, - "w": 8, - "x": 0, - "y": 7 - }, - "id": 97, - "options": { - "barShape": "flat", - "barWidthFactor": 0.3, - "effects": { - "barGlow": false, - "centerGlow": false, - "gradient": true - }, - "endpointMarker": "point", - "minVizHeight": 75, - "minVizWidth": 75, - "orientation": "auto", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "segmentCount": 1, - "segmentSpacing": 0.3, - "shape": "gauge", - "showThresholdLabels": false, - "showThresholdMarkers": false, - "sizing": "auto", - "sparkline": true, - "textMode": "auto" - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "ai_token_cost_total_currency_total", - "legendFormat": "__auto", - "range": true, - "refId": "A" - } - ], - "title": "총 청구 비용", - "type": "gauge" - }, - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "barWidthFactor": 0.6, - "drawStyle": "line", - "fillOpacity": 25, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "auto", - "showValues": false, - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "percent" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": 0 - }, - { - "color": "red", - "value": 80 - } - ] - } - } - }, - "gridPos": { - "h": 9, - "w": 8, - "x": 8, - "y": 7 - }, - "id": 98, - "options": { - "annotations": { - "clustering": -1, - "multiLane": false - }, - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "ai_token_usage_total_tokens_total", - "legendFormat": "__auto", - "range": true, - "refId": "A" - } - ], - "title": "총 사용 토큰 수", - "type": "timeseries" - }, + "id": 8, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - } - } - } - }, - "gridPos": { - "h": 9, - "w": 8, - "x": 16, - "y": 7 - }, - "id": 99, - "options": { - "displayLabels": [ - "percent" - ], - "legend": { - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "pieType": "donut", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "sort": "desc", - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "sum by (token_type) (ai_token_usage_total_tokens_total)", - "legendFormat": "__auto", - "range": true, - "refId": "A" - } - ], - "title": "Prompt vs Completion 비율 (입력/출력 토큰 비율)", - "type": "piechart" + "editorMode": "code", + "expr": "sum by (state) (rate(tokenpilot_budget_reservations_total{application=\"token-pilot-sample-app\"}[1m]))", + "legendFormat": "reservation {{state}}/s", + "refId": "A" + } + ], + "title": "Reservation decisions / sec", + "type": "timeseries" + }, + { + "datasource": { + "type": "prometheus", + "uid": "prometheus-uid" + }, + "fieldConfig": { + "defaults": { + "unit": "short" }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 10 + }, + "id": 9, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "none" + } + }, + "targets": [ { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - } - } - } - }, - "gridPos": { - "h": 9, - "w": 8, - "x": 0, - "y": 16 - }, - "id": 100, - "options": { - "legend": { - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "pieType": "pie", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "sort": "desc", - "tooltip": { - "hideZeros": false, - "mode": "single", - "sort": "none" - } - }, - "pluginVersion": "13.0.1+security-01", - "targets": [ - { - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "editorMode": "code", - "expr": "sum by (model) (ai_token_usage_distribution_tokens_count)", - "legendFormat": "__auto", - "range": true, - "refId": "A" - } - ], - "title": "API Calls by Model (모델별 호출 횟수)", - "type": "piechart" + "editorMode": "code", + "expr": "sum by (outcome) (rate(tokenpilot_reconciliation_outcomes_reconciliations_total{application=\"token-pilot-sample-app\"}[1m]))", + "legendFormat": "reconciliation {{outcome}}/s", + "refId": "A" } ], - "title": "Spring AI Token", - "type": "row" + "title": "Accounting outcomes / sec", + "type": "timeseries" + }, + { + "gridPos": { + "h": 6, + "w": 24, + "x": 0, + "y": 18 + }, + "id": 10, + "options": { + "content": "### Recording flow\n\n1. Call `/test/token-pilot/demo/run` once.\n2. Show `context-fit` / `context-block` and provider invocation counts.\n3. Show `budget-concurrency` with `admittedRequests=1`, `blockedRequests=7`, `providerInvocations=1`.\n4. Refresh this dashboard to show current `tokenpilot_*` meters.\n\nThis dashboard intentionally uses Token Pilot-owned low-cardinality metrics only.", + "mode": "markdown" + }, + "title": "How to record", + "type": "text" } ], - "preload": false, - "refresh": "", - "schemaVersion": 42, + "refresh": "10s", + "schemaVersion": 39, "tags": [ - "app" + "token-pilot", + "demo" ], "templating": { - "list": [ - { - "allowCustomValue": true, - "current": {}, - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "definition": "label_values(jvm_info,application)", - "includeAll": false, - "label": "Application", - "name": "application", - "options": [], - "query": { - "query": "label_values(jvm_info,application)", - "refId": "PrometheusVariableQueryEditor-VariableQuery" - }, - "refresh": 2, - "regex": "", - "regexApplyTo": "value", - "sort": 1, - "type": "query" - }, - { - "allowCustomValue": true, - "current": {}, - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "definition": "label_values(jvm_info,namespace)", - "includeAll": false, - "name": "Namespace", - "options": [], - "query": { - "query": "label_values(jvm_info,namespace)", - "refId": "PrometheusVariableQueryEditor-VariableQuery" - }, - "refresh": 2, - "regex": "", - "regexApplyTo": "value", - "type": "query" - }, - { - "allowCustomValue": true, - "current": {}, - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "definition": "label_values(jvm_info{application=\"$application\", namespace=\"$Namespace\"},instance)", - "includeAll": false, - "label": "Instance", - "name": "instance", - "options": [], - "query": { - "query": "label_values(jvm_info{application=\"$application\", namespace=\"$Namespace\"},instance)", - "refId": "PrometheusVariableQueryEditor-VariableQuery" - }, - "refresh": 2, - "regex": "", - "regexApplyTo": "value", - "sort": 1, - "type": "query" - }, - { - "allowCustomValue": true, - "current": {}, - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "definition": "", - "includeAll": false, - "label": "HikariCP-Pool", - "name": "hikaricp", - "options": [], - "query": { - "query": "label_values(hikaricp_connections{instance=\"$instance\", application=\"$application\"}, pool)", - "refId": "Prometheus-hikaricp-Variable-Query" - }, - "refresh": 2, - "regex": "", - "regexApplyTo": "value", - "sort": 1, - "type": "query" - }, - { - "allowCustomValue": true, - "current": {}, - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "definition": "label_values(jvm_memory_used_bytes{application=\"$application\", instance=\"$instance\", area=\"heap\"},id)", - "includeAll": true, - "label": "Memory Pool (heap)", - "name": "memory_pool_heap", - "options": [], - "query": { - "query": "label_values(jvm_memory_used_bytes{application=\"$application\", instance=\"$instance\", area=\"heap\"},id)", - "refId": "PrometheusVariableQueryEditor-VariableQuery" - }, - "refresh": 2, - "regex": "", - "regexApplyTo": "value", - "sort": 1, - "type": "query" - }, - { - "allowCustomValue": true, - "current": {}, - "datasource": { - "type": "prometheus", - "uid": "prometheus-uid" - }, - "definition": "label_values(jvm_memory_used_bytes{application=\"$application\", instance=\"$instance\", area=\"nonheap\"},id)", - "includeAll": true, - "label": "Memory Pool (nonheap)", - "name": "memory_pool_nonheap", - "options": [], - "query": { - "query": "label_values(jvm_memory_used_bytes{application=\"$application\", instance=\"$instance\", area=\"nonheap\"},id)", - "refId": "PrometheusVariableQueryEditor-VariableQuery" - }, - "refresh": 2, - "regex": "", - "regexApplyTo": "value", - "sort": 1, - "type": "query" - } - ] + "list": [] }, "time": { - "from": "now-1h", + "from": "now-15m", "to": "now" }, - "timepicker": { - "refresh_intervals": [ - "5s", - "10s", - "30s", - "1m", - "5m", - "15m", - "30m", - "1h", - "2h", - "1d" - ] - }, + "timepicker": {}, "timezone": "browser", - "title": "Spring Boot 3.x Statistics", - "uid": "spring_boot_21", - "version": 1, - "weekStart": "" -} \ No newline at end of file + "title": "Token Pilot Demo", + "uid": "token-pilot-demo", + "version": 1 +} diff --git a/token-pilot-sample-app/prometheus/prometheus.yml b/token-pilot-sample-app/prometheus/prometheus.yml index 2da243e..e3cddd0 100644 --- a/token-pilot-sample-app/prometheus/prometheus.yml +++ b/token-pilot-sample-app/prometheus/prometheus.yml @@ -5,6 +5,6 @@ scrape_configs: - job_name: 'token-pilot-backend' metrics_path: '/actuator/prometheus' static_configs: - # ECS용(localhost)과 로컬용(host.docker)을 둘 다 적습니다! - # 각 환경에서 접속되는 곳에서만 알아서 데이터를 빨아옵니다. - - targets: ['localhost:8080', 'host.docker.internal:8080'] \ No newline at end of file + # demo profile의 Spring Boot 앱은 호스트에서 실행하고, + # Prometheus는 host.docker.internal을 통해 scrape합니다. + - targets: ['host.docker.internal:8080'] diff --git a/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoChatModel.java b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoChatModel.java new file mode 100644 index 0000000..9561e42 --- /dev/null +++ b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoChatModel.java @@ -0,0 +1,117 @@ +package io.tokenpilot.sample; + +import org.springframework.ai.chat.metadata.ChatResponseMetadata; +import org.springframework.ai.chat.metadata.DefaultUsage; +import org.springframework.ai.chat.metadata.Usage; +import org.springframework.ai.chat.model.ChatModel; +import org.springframework.ai.chat.model.ChatResponse; +import org.springframework.ai.chat.model.Generation; +import org.springframework.ai.chat.prompt.ChatOptions; +import org.springframework.ai.chat.prompt.Prompt; +import org.springframework.ai.model.tool.ToolCallingChatOptions; + +import java.util.List; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicInteger; + +/** + * 녹화와 로컬 검증을 위한 네트워크 없는 Spring AI provider입니다. + * + *

기본 프로필에는 등록되지 않으며 {@code demo} 프로필에서만 사용됩니다. + * provider usage, 실패, usage 누락, 동시 dispatch 대기를 모두 결정적으로 재현할 수 + * 있어 API key 없이도 adapter와 accounting lifecycle을 확인할 수 있습니다.

+ */ +public final class DemoChatModel implements ChatModel { + + private static final Usage DEFAULT_USAGE = new DefaultUsage(8, 3); + + private final AtomicInteger invocationCount = new AtomicInteger(); + private final AtomicBoolean blockNextCall = new AtomicBoolean(); + + private volatile RuntimeException nextFailure; + private volatile boolean usageAvailable = true; + private volatile CountDownLatch providerEntered = new CountDownLatch(0); + private volatile CountDownLatch providerRelease = new CountDownLatch(0); + + @Override + public ChatResponse call(Prompt prompt) { + invocationCount.incrementAndGet(); + + RuntimeException failure = nextFailure; + if (failure != null) { + throw failure; + } + + if (blockNextCall.compareAndSet(true, false)) { + CountDownLatch entered = providerEntered; + CountDownLatch release = providerRelease; + entered.countDown(); + try { + if (!release.await(10, TimeUnit.SECONDS)) { + throw new IllegalStateException("demo provider release timed out"); + } + } + catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + throw new IllegalStateException("demo provider wait interrupted", interrupted); + } + } + + ChatResponseMetadata.Builder metadata = ChatResponseMetadata.builder() + .model("gpt-4o-2024-08-06"); + if (usageAvailable) { + metadata.usage(DEFAULT_USAGE); + } + + return new ChatResponse( + List.of(new Generation(new org.springframework.ai.chat.messages.AssistantMessage( + "Token Pilot demo response" + ))), + metadata.build() + ); + } + + @Override + public ChatOptions getOptions() { + return ToolCallingChatOptions.builder().build(); + } + + public int invocationCount() { + return invocationCount.get(); + } + + public void failNextCall() { + nextFailure = new IllegalStateException("demo provider failure"); + } + + public void omitUsageNextCall() { + usageAvailable = false; + } + + public void blockNextProviderCall() { + providerEntered = new CountDownLatch(1); + providerRelease = new CountDownLatch(1); + blockNextCall.set(true); + } + + public boolean awaitProviderEntry(long timeout, TimeUnit unit) + throws InterruptedException { + return providerEntered.await(timeout, unit); + } + + public void releaseProvider() { + providerRelease.countDown(); + } + + public void reset() { + releaseProvider(); + invocationCount.set(0); + blockNextCall.set(false); + nextFailure = null; + usageAvailable = true; + providerEntered = new CountDownLatch(0); + providerRelease = new CountDownLatch(0); + } +} diff --git a/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoConfiguration.java b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoConfiguration.java new file mode 100644 index 0000000..bfc4b71 --- /dev/null +++ b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoConfiguration.java @@ -0,0 +1,31 @@ +package io.tokenpilot.sample; + +import org.springframework.ai.chat.client.ChatClient; +import org.springframework.ai.chat.client.ChatClientBuilderCustomizer; +import org.springframework.ai.chat.model.ChatModel; +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Profile; + +/** API key 없이 sample app을 실행하는 demo profile의 Spring AI 구성입니다. */ +@Configuration(proxyBeanMethods = false) +@Profile("demo") +public class DemoConfiguration { + + @Bean + DemoChatModel demoChatModel() { + return new DemoChatModel(); + } + + @Bean + ChatClient.Builder demoChatClientBuilder( + ChatModel chatModel, + ObjectProvider customizers + ) { + ChatClient.Builder builder = ChatClient.builder(chatModel); + customizers.orderedStream() + .forEach(customizer -> customizer.customize(builder)); + return builder; + } +} diff --git a/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoController.java b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoController.java new file mode 100644 index 0000000..a452354 --- /dev/null +++ b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoController.java @@ -0,0 +1,87 @@ +package io.tokenpilot.sample; + +import org.springframework.context.annotation.Profile; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.RestController; + +import java.util.List; +import java.util.Map; + +/** {@code demo} 프로필에서만 노출되는 검증용 HTTP API입니다. */ +@RestController +@Profile("demo") +public class DemoController { + + private final DemoScenarioService scenarioService; + + public DemoController(DemoScenarioService scenarioService) { + this.scenarioService = scenarioService; + } + + @GetMapping("/test/token-pilot/demo") + public Map index() { + return Map.of( + "status", "ok", + "profile", "demo", + "provider", "in-memory DemoChatModel", + "networkCall", false, + "apiKeyRequired", false, + "run", "/test/token-pilot/demo/run", + "scenarios", List.of( + "/test/token-pilot/demo/context-fit", + "/test/token-pilot/demo/context-block", + "/test/token-pilot/demo/budget-concurrency", + "/test/token-pilot/demo/idempotency", + "/test/token-pilot/demo/release", + "/test/token-pilot/demo/reconciliation-success", + "/test/token-pilot/demo/reconciliation-failure", + "/test/token-pilot/demo/reconciliation-unknown" + ) + ); + } + + @GetMapping("/test/token-pilot/demo/run") + public DemoRunReport runAll() { + return scenarioService.runAll(); + } + + @GetMapping("/test/token-pilot/demo/context-fit") + public DemoScenarioResult contextFit() { + return scenarioService.contextFit(); + } + + @GetMapping("/test/token-pilot/demo/context-block") + public DemoScenarioResult contextBlock() { + return scenarioService.contextBlock(); + } + + @GetMapping("/test/token-pilot/demo/budget-concurrency") + public DemoScenarioResult budgetConcurrency() { + return scenarioService.budgetConcurrency(); + } + + @GetMapping("/test/token-pilot/demo/idempotency") + public DemoScenarioResult idempotency() { + return scenarioService.idempotency(); + } + + @GetMapping("/test/token-pilot/demo/release") + public DemoScenarioResult release() { + return scenarioService.release(); + } + + @GetMapping("/test/token-pilot/demo/reconciliation-success") + public DemoScenarioResult reconciliationSuccess() { + return scenarioService.reconciliationSuccess(); + } + + @GetMapping("/test/token-pilot/demo/reconciliation-failure") + public DemoScenarioResult reconciliationFailure() { + return scenarioService.reconciliationFailure(); + } + + @GetMapping("/test/token-pilot/demo/reconciliation-unknown") + public DemoScenarioResult reconciliationUnknown() { + return scenarioService.reconciliationUnknown(); + } +} diff --git a/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoRunReport.java b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoRunReport.java new file mode 100644 index 0000000..10bcc7e --- /dev/null +++ b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoRunReport.java @@ -0,0 +1,22 @@ +package io.tokenpilot.sample; + +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Collections; + +/** 모든 핵심 control/accounting 시나리오를 한 번에 실행한 demo 결과입니다. */ +public record DemoRunReport( + String runId, + String profile, + String provider, + String note, + List scenarios, + Map metrics +) { + + public DemoRunReport { + scenarios = List.copyOf(scenarios); + metrics = Collections.unmodifiableMap(new LinkedHashMap<>(metrics)); + } +} diff --git a/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoScenarioResult.java b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoScenarioResult.java new file mode 100644 index 0000000..a54dbdf --- /dev/null +++ b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoScenarioResult.java @@ -0,0 +1,18 @@ +package io.tokenpilot.sample; + +import java.util.LinkedHashMap; +import java.util.Map; +import java.util.Collections; + +/** 화면에서 바로 읽을 수 있는 단일 demo 시나리오 결과입니다. */ +public record DemoScenarioResult( + String scenario, + String status, + String summary, + Map evidence +) { + + public DemoScenarioResult { + evidence = Collections.unmodifiableMap(new LinkedHashMap<>(evidence)); + } +} diff --git a/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoScenarioService.java b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoScenarioService.java new file mode 100644 index 0000000..305309f --- /dev/null +++ b/token-pilot-sample-app/src/main/java/io/tokenpilot/sample/DemoScenarioService.java @@ -0,0 +1,667 @@ +package io.tokenpilot.sample; + +import io.micrometer.core.instrument.Counter; +import io.micrometer.core.instrument.MeterRegistry; +import io.tokenpilot.autoconfigure.TokenPilotProperties; +import io.tokenpilot.budget.AtomicBudgetStateStore; +import io.tokenpilot.budget.BudgetDecision; +import io.tokenpilot.budget.BudgetEvaluator; +import io.tokenpilot.budget.BudgetReservationRequest; +import io.tokenpilot.budget.BudgetReservationResult; +import io.tokenpilot.budget.BudgetSnapshot; +import io.tokenpilot.budget.IdempotencyKey; +import io.tokenpilot.budget.ReservationState; +import io.tokenpilot.budget.ReservationTransition; +import io.tokenpilot.core.ModelRegistry; +import io.tokenpilot.core.TokenBudget; +import io.tokenpilot.core.TokenEstimator; +import io.tokenpilot.core.domain.AdmissionStatus; +import io.tokenpilot.core.domain.BudgetResult; +import io.tokenpilot.core.domain.Cost; +import io.tokenpilot.core.domain.TokenCountAccuracy; +import io.tokenpilot.core.domain.TokenCountResult; +import io.tokenpilot.core.domain.TokenCountScope; + +import org.springframework.ai.chat.client.ChatClient; +import org.springframework.ai.chat.client.ChatClientResponse; +import org.springframework.ai.chat.prompt.ChatOptions; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.annotation.Profile; +import org.springframework.stereotype.Service; + +import java.math.BigDecimal; +import java.util.ArrayList; +import java.util.Currency; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.Future; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; +import java.util.concurrent.atomic.AtomicInteger; + +/** + * 반복 실행할 수 있는 framework-independent control/accounting 시나리오입니다. + * 모든 provider 호출은 {@link DemoChatModel}로만 전달되며 외부 네트워크를 사용하지 않습니다. + */ +@Service +@Profile("demo") +public class DemoScenarioService { + + private static final String MODEL_ID = "gpt-4o-2024-08-06"; + private static final String CATALOG_VERSION = "openai-2026-08-14"; + private static final Currency USD = Currency.getInstance("USD"); + private static final int COMPETITOR_COUNT = 7; + private static final long REQUEST_OUTPUT_TOKENS = 16; + private static final String CONCURRENCY_PROMPT = + "Hold one conservative reservation while other requests compete. " + + "x".repeat(5_000); + + private final ChatClient.Builder chatClientBuilder; + private final DemoChatModel demoChatModel; + private final AtomicBudgetStateStore budgetStateStore; + private final BudgetEvaluator budgetEvaluator; + private final ModelRegistry modelRegistry; + private final TokenEstimator tokenEstimator; + private final TokenBudget tokenBudget; + private final TokenPilotProperties properties; + private final MeterRegistry meterRegistry; + private final AtomicInteger runSequence = new AtomicInteger(); + + @Autowired + public DemoScenarioService( + ChatClient.Builder chatClientBuilder, + DemoChatModel demoChatModel, + AtomicBudgetStateStore budgetStateStore, + BudgetEvaluator budgetEvaluator, + ModelRegistry modelRegistry, + TokenEstimator tokenEstimator, + TokenBudget tokenBudget, + TokenPilotProperties properties, + MeterRegistry meterRegistry + ) { + this.chatClientBuilder = chatClientBuilder; + this.demoChatModel = demoChatModel; + this.budgetStateStore = budgetStateStore; + this.budgetEvaluator = budgetEvaluator; + this.modelRegistry = modelRegistry; + this.tokenEstimator = tokenEstimator; + this.tokenBudget = tokenBudget; + this.properties = properties; + this.meterRegistry = meterRegistry; + } + + public synchronized DemoRunReport runAll() { + String runId = nextRunId(); + List results = List.of( + contextFit(runId), + contextBlock(runId), + budgetConcurrency(runId), + idempotency(runId), + release(runId), + reconciliation(runId, ReconciliationMode.SUCCESS), + reconciliation(runId, ReconciliationMode.PROVIDER_FAILURE), + reconciliation(runId, ReconciliationMode.USAGE_UNKNOWN) + ); + return new DemoRunReport( + runId, + "demo", + "in-memory DemoChatModel", + "외부 provider 호출과 API key 없이 재현한 보수적 admission·예약·회계 흐름입니다.", + results, + metricSnapshot() + ); + } + + public synchronized DemoScenarioResult contextFit() { + return contextFit(nextRunId()); + } + + public synchronized DemoScenarioResult contextBlock() { + return contextBlock(nextRunId()); + } + + public synchronized DemoScenarioResult budgetConcurrency() { + return budgetConcurrency(nextRunId()); + } + + public synchronized DemoScenarioResult idempotency() { + return idempotency(nextRunId()); + } + + public synchronized DemoScenarioResult release() { + return release(nextRunId()); + } + + public synchronized DemoScenarioResult reconciliationSuccess() { + return reconciliation(nextRunId(), ReconciliationMode.SUCCESS); + } + + public synchronized DemoScenarioResult reconciliationFailure() { + return reconciliation(nextRunId(), ReconciliationMode.PROVIDER_FAILURE); + } + + public synchronized DemoScenarioResult reconciliationUnknown() { + return reconciliation(nextRunId(), ReconciliationMode.USAGE_UNKNOWN); + } + + private DemoScenarioResult contextFit(String runId) { + String text = "A short request stays inside the versioned model context window."; + BudgetResult result = checkContext(text, 64); + Map evidence = contextEvidence(result); + evidence.put("runId", runId); + evidence.put("providerInvocations", 0); + evidence.put("requestText", text); + boolean passed = result.status() == AdmissionStatus.FITS; + return new DemoScenarioResult( + "context-fit", + passed ? "PASS" : "FAIL", + passed + ? "보수적 REQUEST 상한으로 provider 호출 전 FITS를 확인했습니다." + : "짧은 요청이 예상과 다르게 context admission에 실패했습니다.", + evidence + ); + } + + private DemoScenarioResult contextBlock(String runId) { + String text = "context overflow ".repeat(10_000); + BudgetResult result = checkContext(text, 64); + Map evidence = contextEvidence(result); + evidence.put("runId", runId); + evidence.put("providerInvocations", 0); + evidence.put("inputCharacters", text.length()); + evidence.put("requestTextIncluded", false); + boolean passed = result.status() == AdmissionStatus.EXCEEDS; + return new DemoScenarioResult( + "context-block", + passed ? "PASS" : "FAIL", + passed + ? "보수적 safe upper bound가 context를 넘겨 provider 전에 차단했습니다." + : "큰 입력이 예상과 다르게 context admission을 통과했습니다.", + evidence + ); + } + + private DemoScenarioResult budgetConcurrency(String runId) { + demoChatModel.reset(); + String tenant = "demo-concurrency-" + runId; + List competitorOutcomes = new ArrayList<>(); + CallOutcome primaryOutcome = CallOutcome.failed("NOT_STARTED", "primary call did not finish"); + + try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { + try { + demoChatModel.blockNextProviderCall(); + Future primary = executor.submit( + () -> invokeChat( + tenant, + "request-" + runId + "-primary", + "attempt-" + runId + "-primary", + "idempotency-" + runId + "-primary", + CONCURRENCY_PROMPT, + REQUEST_OUTPUT_TOKENS + ) + ); + boolean providerEntered = demoChatModel.awaitProviderEntry(5, TimeUnit.SECONDS); + + if (!providerEntered) { + throw new IllegalStateException("demo provider did not enter in time"); + } + + List> competitors = new ArrayList<>(); + for (int index = 0; index < COMPETITOR_COUNT; index++) { + String suffix = runId + "-competitor-" + index; + competitors.add(executor.submit( + () -> invokeChat( + tenant, + "request-" + suffix, + "attempt-" + suffix, + "idempotency-" + suffix, + CONCURRENCY_PROMPT, + REQUEST_OUTPUT_TOKENS + ) + )); + } + + // 경쟁 요청들은 첫 예약이 IN_FLIGHT인 동안 예약 단계에서 종료되어야 합니다. + for (Future competitor : competitors) { + competitorOutcomes.add(await(competitor)); + } + demoChatModel.releaseProvider(); + primaryOutcome = await(primary); + } + finally { + demoChatModel.releaseProvider(); + } + } + catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + return failedScenario( + "budget-concurrency", + "동시 예약 검증 중 thread가 중단되었습니다.", + Map.of("error", "INTERRUPTED") + ); + } + catch (RuntimeException failure) { + return failedScenario( + "budget-concurrency", + "동시 예약 시나리오가 완료되지 않았습니다.", + Map.of("error", sanitize(failure.getMessage())) + ); + } + + long blocked = competitorOutcomes.stream() + .filter(outcome -> outcome.state().equals("BLOCKED")) + .count(); + boolean passed = primaryOutcome.state().equals("COMMITTED") + && blocked == COMPETITOR_COUNT + && demoChatModel.invocationCount() == 1; + Map evidence = new LinkedHashMap<>(); + evidence.put("primary", primaryOutcome.state()); + evidence.put("competitors", competitorOutcomes.stream() + .map(CallOutcome::state) + .toList()); + evidence.put("admittedRequests", 1); + evidence.put("blockedRequests", blocked); + evidence.put("providerInvocations", demoChatModel.invocationCount()); + evidence.put("expectedProviderInvocations", 1); + evidence.put("budget", budgetSnapshot(tenant)); + return new DemoScenarioResult( + "budget-concurrency", + passed ? "PASS" : "FAIL", + passed + ? "원자적 예약으로 1건만 provider에 도달하고 나머지는 BLOCKED 되었습니다." + : "동시 예약 결과가 원자적 admission 기대와 다릅니다.", + evidence + ); + } + + private DemoScenarioResult idempotency(String runId) { + demoChatModel.reset(); + String tenant = "demo-idempotency-" + runId; + String requestId = "request-" + runId; + String attemptId = "attempt-" + runId; + String idempotencyKey = "idempotency-" + runId; + CallOutcome firstOutcome = CallOutcome.failed("NOT_STARTED", "first call did not finish"); + CallOutcome duplicateOutcome = CallOutcome.failed("NOT_STARTED", "duplicate call did not finish"); + + try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { + try { + demoChatModel.blockNextProviderCall(); + Future first = executor.submit( + () -> invokeChat( + tenant, + requestId, + attemptId, + idempotencyKey, + "The same idempotency key must not dispatch twice.", + REQUEST_OUTPUT_TOKENS + ) + ); + if (!demoChatModel.awaitProviderEntry(5, TimeUnit.SECONDS)) { + throw new IllegalStateException("demo provider did not enter in time"); + } + Future duplicate = executor.submit( + () -> invokeChat( + tenant, + requestId, + attemptId, + idempotencyKey, + "The same idempotency key must not dispatch twice.", + REQUEST_OUTPUT_TOKENS + ) + ); + duplicateOutcome = await(duplicate); + demoChatModel.releaseProvider(); + firstOutcome = await(first); + } + finally { + demoChatModel.releaseProvider(); + } + } + catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + return failedScenario( + "idempotency", + "멱등성 검증 중 thread가 중단되었습니다.", + Map.of("error", "INTERRUPTED") + ); + } + catch (RuntimeException failure) { + return failedScenario( + "idempotency", + "멱등성 시나리오가 완료되지 않았습니다.", + Map.of("error", sanitize(failure.getMessage())) + ); + } + + boolean passed = firstOutcome.state().equals("COMMITTED") + && duplicateOutcome.state().equals("DUPLICATE_NOT_DISPATCHED") + && demoChatModel.invocationCount() == 1; + Map evidence = new LinkedHashMap<>(); + evidence.put("first", firstOutcome.state()); + evidence.put("duplicate", duplicateOutcome.state()); + evidence.put("sameRequestId", true); + evidence.put("sameIdempotencyKey", true); + evidence.put("providerInvocations", demoChatModel.invocationCount()); + evidence.put("budget", budgetSnapshot(tenant)); + return new DemoScenarioResult( + "idempotency", + passed ? "PASS" : "FAIL", + passed + ? "같은 멱등성 키의 재시도는 예약을 재사용하지만 provider를 중복 호출하지 않았습니다." + : "멱등성 키 재사용 결과가 기대와 다릅니다.", + evidence + ); + } + + private DemoScenarioResult release(String runId) { + String tenant = "demo-release-" + runId; + Cost amount = Cost.of(new BigDecimal("0.0001"), USD); + BudgetDecision decision = budgetEvaluator.evaluate( + Map.of("tenant_id", tenant), + amount + ); + String requestId = "request-release-" + runId; + BudgetReservationRequest request = new BudgetReservationRequest( + decision.key(), + decision.limit(), + amount, + requestId, + new IdempotencyKey("idempotency-release-" + runId), + MODEL_ID, + "default", + CATALOG_VERSION, + Optional.empty(), + Optional.empty() + ); + BudgetReservationResult reservation = budgetStateStore.checkAndReserve(request); + ReservationTransition transition = budgetStateStore.releaseBeforeDispatch( + reservation.reservationId() + ); + Map evidence = new LinkedHashMap<>(); + evidence.put("reservationStatus", reservation.status().name()); + evidence.put("transitionStatus", transition.status().name()); + evidence.put("resultingState", transition.resultingState().name()); + evidence.put("providerInvocations", 0); + evidence.put("budget", budgetSnapshot(tenant)); + boolean passed = reservation.status().name().equals("CREATED") + && transition.resultingState() == ReservationState.RELEASED + && transition.status().isApplied(); + return new DemoScenarioResult( + "release", + passed ? "PASS" : "FAIL", + passed + ? "provider dispatch 전 예약 실패를 안전하게 release 했습니다." + : "dispatch 전 release 결과가 기대와 다릅니다.", + evidence + ); + } + + private DemoScenarioResult reconciliation( + String runId, + ReconciliationMode mode + ) { + demoChatModel.reset(); + String tenant = "demo-reconciliation-" + mode.name().toLowerCase() + "-" + runId; + if (mode == ReconciliationMode.PROVIDER_FAILURE) { + demoChatModel.failNextCall(); + } + else if (mode == ReconciliationMode.USAGE_UNKNOWN) { + demoChatModel.omitUsageNextCall(); + } + + CallOutcome outcome = invokeChat( + tenant, + "request-reconciliation-" + runId, + "attempt-reconciliation-" + runId, + "idempotency-reconciliation-" + mode.name().toLowerCase() + "-" + runId, + "Reconcile this deterministic demo request.", + REQUEST_OUTPUT_TOKENS + ); + BudgetSnapshot snapshot = readBudgetSnapshot(tenant); + Map budget = budgetEvidence(snapshot); + String accountingState = mode == ReconciliationMode.SUCCESS + ? "COMMITTED" + : "RECONCILIATION_REQUIRED"; + boolean pending = snapshot.pendingReconciliationLiability().value().signum() > 0; + boolean committed = snapshot.committedCost().value().signum() > 0; + boolean passed = switch (mode) { + case SUCCESS -> outcome.state().equals("COMMITTED") && committed && !pending; + case PROVIDER_FAILURE -> outcome.state().equals("PROVIDER_FAILED") && pending && !committed; + case USAGE_UNKNOWN -> outcome.state().equals("COMMITTED") && pending && !committed; + }; + Map evidence = new LinkedHashMap<>(); + evidence.put("providerOutcome", outcome.state()); + evidence.put("providerDetail", outcome.detail()); + evidence.put("providerResponse", outcome.response()); + evidence.put("accountingState", accountingState); + evidence.put("providerInvocations", demoChatModel.invocationCount()); + evidence.put("budget", budget); + return new DemoScenarioResult( + "reconciliation-" + mode.pathName, + passed ? "PASS" : "FAIL", + passed + ? mode.summary + : "provider 결과와 accounting 상태가 기대와 다릅니다.", + evidence + ); + } + + private CallOutcome invokeChat( + String tenant, + String requestId, + String attemptId, + String idempotencyKey, + String text, + long maxTokens + ) { + try { + ChatClientResponse response = chatClientBuilder.clone() + .build() + .prompt() + .user(text) + .options(ChatOptions.builder() + .model(MODEL_ID) + .maxTokens((int) maxTokens)) + .advisors(advisors -> advisors + .param("tenant_id", tenant) + .param("tokenpilot.request.id", requestId) + .param("tokenpilot.attempt.id", attemptId) + .param("tokenpilot.idempotency.key", new IdempotencyKey(idempotencyKey))) + .call() + .chatClientResponse(); + String responseText = response.chatResponse() + .getResult() + .getOutput() + .getText(); + return new CallOutcome("COMMITTED", "provider response returned", responseText); + } + catch (RuntimeException failure) { + String message = sanitize(failure.getMessage()); + String state = message.contains("budget reservation rejected") + ? "BLOCKED" + : message.contains("reservation dispatch was not claimed") + ? "DUPLICATE_NOT_DISPATCHED" + : message.contains("demo provider failure") + ? "PROVIDER_FAILED" + : "FAILED"; + return CallOutcome.failed(state, message); + } + } + + private CallOutcome await(Future future) + throws InterruptedException { + try { + return future.get(10, TimeUnit.SECONDS); + } + catch (ExecutionException executionFailure) { + Throwable cause = executionFailure.getCause(); + return CallOutcome.failed( + "TASK_FAILED", + sanitize(cause == null ? executionFailure.getMessage() : cause.getMessage()) + ); + } + catch (TimeoutException timeout) { + return CallOutcome.failed("TIMEOUT", "demo scenario future timed out"); + } + } + + private BudgetResult checkContext(String text, long reservedOutputTokens) { + String framed = "USER:" + text.length() + ":" + text + "\n"; + TokenCountResult textResult = tokenEstimator.estimate(framed); + TokenCountResult requestResult; + if (textResult.isUnavailable()) { + requestResult = TokenCountResult.unavailable( + textResult.unavailableReason().orElseThrow(), + TokenCountScope.REQUEST, + textResult.estimatorDescriptor(), + textResult.tokenizationBasis() + ); + } + else { + requestResult = TokenCountResult.counted( + textResult.tokens().orElseThrow(), + Math.addExact( + textResult.safeUpperBoundTokens().orElseThrow(), + properties.getSpringAi().getFramingHeadroomTokens() + ), + TokenCountAccuracy.HEURISTIC, + TokenCountScope.REQUEST, + textResult.estimatorDescriptor(), + textResult.tokenizationBasis() + ); + } + return tokenBudget.check(MODEL_ID, requestResult, reservedOutputTokens); + } + + private Map contextEvidence(BudgetResult result) { + Map evidence = new LinkedHashMap<>(); + evidence.put("status", result.status().name()); + evidence.put("reason", result.reason().name()); + evidence.put("model", result.canonicalModelId().orElse(MODEL_ID)); + evidence.put("inputEstimatedTokens", result.inputEstimatedTokens().isPresent() + ? result.inputEstimatedTokens().getAsLong() + : "unavailable"); + evidence.put("inputSafeUpperBoundTokens", result.inputSafeUpperBoundTokens().isPresent() + ? result.inputSafeUpperBoundTokens().getAsLong() + : "unavailable"); + evidence.put("reservedOutputTokens", result.reservedOutputTokens()); + evidence.put("maxContextTokens", result.maxContextTokens().isPresent() + ? result.maxContextTokens().getAsLong() + : "unavailable"); + evidence.put("remainingTokens", result.remainingTokens().isPresent() + ? result.remainingTokens().getAsLong() + : "not-provided-for-rejected-admission"); + evidence.put("estimator", result.estimatorDescriptor().estimatorId() + + "@" + result.estimatorDescriptor().estimatorVersion()); + evidence.put("tokenizationBasis", result.tokenizationBasis().id()); + evidence.put("catalogVersion", modelRegistry.find(MODEL_ID) + .map(model -> model.catalogVersion()) + .orElse("unknown")); + evidence.put("scope", "REQUEST"); + return evidence; + } + + private Map budgetSnapshot(String tenant) { + return budgetEvidence(readBudgetSnapshot(tenant)); + } + + private BudgetSnapshot readBudgetSnapshot(String tenant) { + BudgetDecision decision = budgetEvaluator.evaluate(Map.of("tenant_id", tenant)); + return budgetStateStore.snapshot(decision.key(), decision.limit()); + } + + private Map budgetEvidence(BudgetSnapshot snapshot) { + Map result = new LinkedHashMap<>(); + result.put("committedCost", CostBoundaryFormatter.format(snapshot.committedCost())); + result.put("activeReservedCost", CostBoundaryFormatter.format(snapshot.activeReservedCost())); + result.put("pendingReconciliationLiability", + CostBoundaryFormatter.format(snapshot.pendingReconciliationLiability())); + result.put("effectiveUsage", CostBoundaryFormatter.format(snapshot.effectiveUsage())); + result.put("remaining", CostBoundaryFormatter.format(snapshot.remaining())); + result.put("limit", CostBoundaryFormatter.format(snapshot.limit())); + result.put("activeReservationCount", snapshot.activeReservationIds().size()); + return result; + } + + private Map metricSnapshot() { + return Map.of( + "preflightFits", counter( + "tokenpilot.preflight.requests", "decision", "fits", "reason", "within_context" + ), + "preflightExceeds", counter( + "tokenpilot.preflight.requests", "decision", "exceeds", "reason", "context_exceeded" + ), + "reservationsCreated", counter( + "tokenpilot.budget.reservations", "state", "created" + ), + "reservationsReused", counter( + "tokenpilot.budget.reservations", "state", "reused" + ), + "reservationsBlocked", counter( + "tokenpilot.budget.reservations", "state", "blocked" + ), + "committedCostUsd", counter( + "tokenpilot.cost.total", "currency", "USD" + ), + "reconciliationCommitted", counter( + "tokenpilot.reconciliation.outcomes", "outcome", "committed", "reason", "actual_usage_reported" + ), + "reconciliationRequired", counter( + "tokenpilot.reconciliation.outcomes", "outcome", "reconciliation_required", "reason", "actual_usage_unavailable" + ) + ); + } + + private double counter(String name, String... tags) { + Counter counter = meterRegistry.find(name).tags(tags).counter(); + return counter == null ? 0.0 : counter.count(); + } + + private DemoScenarioResult failedScenario( + String scenario, + String summary, + Map evidence + ) { + return new DemoScenarioResult(scenario, "FAIL", summary, evidence); + } + + private String nextRunId() { + return "run-" + runSequence.incrementAndGet(); + } + + private static String sanitize(String value) { + if (value == null || value.isBlank()) { + return "unspecified"; + } + String normalized = value.replaceAll("\\s+", " "); + return normalized.length() <= 160 + ? normalized + : normalized.substring(0, 157) + "..."; + } + + private record CallOutcome(String state, String detail, String response) { + + static CallOutcome failed(String state, String detail) { + return new CallOutcome(state, detail, "not-returned"); + } + } + + private enum ReconciliationMode { + SUCCESS("success", "provider usage를 actual로 commit했습니다."), + PROVIDER_FAILURE("failure", "provider 실패 후 estimate liability를 reconciliation 대기로 보존했습니다."), + USAGE_UNKNOWN("unknown", "provider 응답은 반환됐지만 usage 불명으로 reconciliation 대기에 보존했습니다."); + + private final String pathName; + private final String summary; + + ReconciliationMode(String pathName, String summary) { + this.pathName = pathName; + this.summary = summary; + } + } +} diff --git a/token-pilot-sample-app/src/main/resources/application-demo.yml b/token-pilot-sample-app/src/main/resources/application-demo.yml new file mode 100644 index 0000000..b56f8e7 --- /dev/null +++ b/token-pilot-sample-app/src/main/resources/application-demo.yml @@ -0,0 +1,38 @@ +server: + port: 8080 + +management: + endpoints: + web: + exposure: + include: prometheus, health + metrics: + tags: + application: token-pilot-sample-app + +token-pilot: + enabled: true + pricing: + plans: + - model-id: gpt-4o-2024-08-06 + currency: USD + rates: + PROMPT: 0.00015 + COMPLETION: 0.00060 + spring-ai: + default-model-id: gpt-4o-2024-08-06 + default-reserved-output-tokens: 16 + framing-headroom-tokens: 0 + budget: + enabled: true + policy-id: demo-monthly + target-type: tenant + target-tag-key: tenant_id + fallback-target-id: demo-shared + monthly-limit: 0.001 + currency: USD + zone-id: UTC + metrics: + enabled: true + legacy-ai-token-metrics-enabled: false + tag-whitelist: [] diff --git a/token-pilot-sample-app/src/test/java/io/tokenpilot/sample/DemoScenarioE2ETest.java b/token-pilot-sample-app/src/test/java/io/tokenpilot/sample/DemoScenarioE2ETest.java new file mode 100644 index 0000000..5e880d6 --- /dev/null +++ b/token-pilot-sample-app/src/test/java/io/tokenpilot/sample/DemoScenarioE2ETest.java @@ -0,0 +1,96 @@ +package io.tokenpilot.sample; + +import org.junit.jupiter.api.Test; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.context.ActiveProfiles; + +import java.io.IOException; +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; + +import static org.assertj.core.api.Assertions.assertThat; + +/** API key 없이 실행되는 제출용 demo profile의 HTTP 계약을 검증합니다. */ +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) +@ActiveProfiles("demo") +class DemoScenarioE2ETest { + + private final HttpClient httpClient = HttpClient.newHttpClient(); + + @LocalServerPort + private int port; + + @Test + void demoIndexDeclaresDeterministicNoKeyProvider() throws Exception { + HttpResponse response = get("/test/token-pilot/demo"); + + assertThat(response.statusCode()).isEqualTo(200); + assertThat(response.body()) + .contains("\"status\":\"ok\"") + .contains("\"profile\":\"demo\"") + .contains("\"provider\":\"in-memory DemoChatModel\"") + .contains("\"networkCall\":false") + .contains("\"apiKeyRequired\":false") + .contains("/test/token-pilot/demo/run"); + } + + @Test + void demoRunCoversAdmissionReservationIdempotencyAndReconciliation() throws Exception { + HttpResponse response = get("/test/token-pilot/demo/run"); + + assertThat(response.statusCode()).isEqualTo(200); + assertThat(response.body()) + .contains("\"scenario\":\"context-fit\"") + .contains("\"scenario\":\"context-block\"") + .contains("\"scenario\":\"budget-concurrency\"") + .contains("\"scenario\":\"idempotency\"") + .contains("\"scenario\":\"release\"") + .contains("\"scenario\":\"reconciliation-success\"") + .contains("\"scenario\":\"reconciliation-failure\"") + .contains("\"scenario\":\"reconciliation-unknown\"") + .contains("\"blockedRequests\":7") + .contains("\"expectedProviderInvocations\":1") + .contains("\"sameIdempotencyKey\":true") + .contains("\"accountingState\":\"RECONCILIATION_REQUIRED\"") + .contains("\"reconciliationRequired\""); + assertThat(count(response.body(), "\"status\":\"PASS\"")) + .isEqualTo(8); + } + + @Test + void demoPublishesCurrentTokenPilotMetricsOnly() throws Exception { + get("/test/token-pilot/demo/run"); + + HttpResponse response = get("/actuator/prometheus"); + + assertThat(response.statusCode()).isEqualTo(200); + assertThat(response.body()) + .contains("tokenpilot_preflight_requests_total") + .contains("tokenpilot_budget_reservations_total") + .contains("tokenpilot_reconciliation_outcomes_reconciliations_total") + .contains("tokenpilot_cost_total_currency_total") + .doesNotContain("ai_token_usage_total") + .doesNotContain("ai_token_cost_total"); + } + + private HttpResponse get(String path) throws IOException, InterruptedException { + HttpRequest request = HttpRequest.newBuilder() + .uri(URI.create("http://localhost:" + port + path)) + .GET() + .build(); + return httpClient.send(request, HttpResponse.BodyHandlers.ofString()); + } + + private static int count(String value, String needle) { + int count = 0; + int offset = 0; + while ((offset = value.indexOf(needle, offset)) >= 0) { + count++; + offset += needle.length(); + } + return count; + } +}