Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Token Pilot의 제품 포지션은 framework-independent Java LLM control and ac
| `token-pilot-notification` | Atomic accounting integration implemented | Commit, reconciliation-required, late reconciliation, and reservation BLOCK results produce process-local atomically deduplicated threshold events with isolated handlers, lifecycle observation, and a sanitized error hook; durable delivery remains |
| `token-pilot-autoconfigure` | Basic implementation complete | Owner-specific optional bean graphs wire pricing, atomic budget/accounting, Spring AI, notification, and metrics while preserving budget-disabled ledger-only compatibility and `ChatClientBuilderCustomizer` |
| `token-pilot-starter` | Basic implementation complete | Thin final user entrypoint that brings runtime modules together |
| `token-pilot-sample-app` | Basic E2E complete | Direct ledger metrics and fake Spring AI lifecycle E2E cover preflight BLOCK, reservation, reconciliation, request-scope rejection, Advisor ordering, listener isolation, and enforcement streaming rejection without an API key |
| `token-pilot-sample-app` | Basic E2E complete | Direct ledger metrics, deterministic fake-provider demo E2E, and an opt-in OpenAI provider smoke path cover the sample integration; live provider compatibility still requires an explicit API-key run |

## Current Work Focus

Expand Down Expand Up @@ -306,6 +306,13 @@ sample app's `DEMO_RUNBOOK.md` documents the Gradle, Prometheus/Grafana, and sce
verification flow. The provisioned dashboard uses Token Pilot-owned `tokenpilot_*`
meters and deliberately does not use legacy `ai_token_*` meters.

OpenAI smoke profile endpoints (run with `--spring.profiles.active=openai-smoke`):

- `GET /test/token-pilot/openai-smoke`: performs one real OpenAI Chat Completions call,
returns normalized usage and accounting evidence, and requires `OPENAI_API_KEY`.
- `OpenAiSmokeE2ETest`: is skipped by default and only runs when both
`RUN_OPENAI_SMOKE=true` and `OPENAI_API_KEY` are present.

Test-only E2E endpoint:

- `GET /test/token-pilot/chat`: exercises the Spring AI `ChatClient` advisor path with a fake/mock provider or documented real provider setup.
Expand Down Expand Up @@ -369,7 +376,7 @@ The active checklist is in `docs/30_DAY_MVP_REPORT.md`; detailed long-term works
- A response model that differs from the reservation pricing snapshot is not committed at the request model price; it remains pending with `PRICING_RECONCILIATION_REQUIRED` for later settlement.
- Pricing-mismatch pending events preserve provider actual usage and response model metadata. Consumers may call the explicit `reconcileLateActual(command, responsePricingSnapshot)` API with an immutable response-model pricing snapshot; model, currency, rate, state, and duplicate-callback validation remain fail-closed.
- The repository, README, JReleaser configuration, and every published module POM use the MIT License. `verifyPublicationMetadata` guards this release contract and ensures the sample app is not published.
- Sample app E2E uses a fake Spring AI `ChatModel`; real provider API behavior is not yet verified.
- Sample app's default/demo E2E uses a fake Spring AI `ChatModel`; the OpenAI smoke path is opt-in and is not a CI/live compatibility guarantee until the guarded test is run with a real key.
- `token-pilot-spring-ai-starter` does not exist in the current build; never use it as an install instruction until implemented and published.
- Maven Central release consumption must be re-verified for both core and starter paths before announcing `0.1.0`.
- Preflight cost bounds must use one immutable pricing snapshot from calculation through reservation and reconciliation; resolving a mutable registry again by model/policy identifiers can mix prices from different requests.
Expand Down Expand Up @@ -471,6 +478,11 @@ Stage and deploy a Central release:

## Update History

### 2026-08-27

- Added the sample app's application-owned OpenAI starter dependency, isolated `openai-smoke` profile, usage/accounting evidence endpoint, and key-gated live JUnit smoke test; default and demo profiles explicitly disable provider auto-configuration so deterministic tests remain unchanged.
- Documented the sequential demo → OpenAI smoke procedure, environment-only API key handling, model/pricing requirements, and Prometheus evidence.

### 2026-08-26

- Added root publication aggregation and a release-candidate verification gate
Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ against the reservation-time pricing snapshot.
- [Metrics reference](docs/METRICS.md) — Token Pilot-owned meters and legacy compatibility policy.
- [Sample app runbook](docs/SAMPLE_RUNBOOK.md) — local app, Prometheus, Grafana, and troubleshooting commands.
- [Deterministic demo runbook](token-pilot-sample-app/DEMO_RUNBOOK.md) — eight admission, reservation, idempotency, release, and reconciliation scenarios.
- [English demo runbook](token-pilot-sample-app/DEMO_RUNBOOK_EN.md) — the same demo and OpenAI smoke procedure in English.
- The same [sample app runbook](docs/SAMPLE_RUNBOOK.md) includes the opt-in OpenAI provider smoke path.
- [Release procedure](docs/RELEASE.md) — staging, signing, external consumer, and Central Portal gates.
- [30-day MVP cutline](docs/30_DAY_MVP_REPORT.md) and [post-MVP evolution plan](docs/EVOLUTION_PLAN.md).

Expand Down Expand Up @@ -92,6 +94,12 @@ See [QUICKSTART.md](docs/QUICKSTART.md) for the ten-minute path and
[SAMPLE_RUNBOOK.md](docs/SAMPLE_RUNBOOK.md) for endpoint and troubleshooting
details.

The sample app's `openai-smoke` profile uses the application-selected OpenAI
starter and exposes `/test/token-pilot/openai-smoke`. It requires
`OPENAI_API_KEY`, makes a real provider call, and reports normalized usage plus
the resulting Token Pilot accounting state. The guarded live test is skipped
unless `RUN_OPENAI_SMOKE=true` is set.

## Micrometer metrics

When a `MeterRegistry` is available, the Spring Boot starter publishes
Expand Down
24 changes: 24 additions & 0 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,30 @@ is `TEXT_ONLY`; media, tool messages or schemas, structured-output
augmentation, and enforcement-enabled streaming are rejected before
reservation when they cannot be represented by the supported contract.

## Sample OpenAI smoke profile

The repository sample app selects an actual provider only when the
`openai-smoke` profile is active. The provider dependency is application-owned:

```gradle
implementation 'cloud.token-pilot:token-pilot-starter:0.0.1-SNAPSHOT'
implementation 'org.springframework.ai:spring-ai-starter-model-openai:2.0.0'
```

The profile reads `OPENAI_API_KEY` and `OPENAI_MODEL` from the environment and
uses a versioned catalog entry by default. Its Token Pilot pricing values are
example rates per 1,000 tokens and should be replaced with the rates for the
selected provider model before using the result as a financial record.

The sample endpoint is:

```text
GET /test/token-pilot/openai-smoke
```

See [`docs/SAMPLE_RUNBOOK.md`](SAMPLE_RUNBOOK.md) for the demo-to-smoke
sequence and the guarded live test command.

## Metrics

| Property | Default | Meaning |
Expand Down
40 changes: 39 additions & 1 deletion docs/QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,45 @@ curl -s http://localhost:8080/test/token-pilot/demo/reconciliation-failure | jq
The complete sample procedure is in
[`token-pilot-sample-app/DEMO_RUNBOOK.md`](../token-pilot-sample-app/DEMO_RUNBOOK.md).

## 3. Prometheus and Grafana
## 3. OpenAI provider smoke

The sample app also includes an opt-in path that uses the real OpenAI provider.
Stop the `demo` profile first, then set the key in the environment. The call
can incur provider charges, so keep the prompt short and never commit the key.

```bash
export OPENAI_API_KEY='your OpenAI API key'
export OPENAI_MODEL='gpt-4o-mini-2024-07-18'

./gradlew --no-daemon :token-pilot-sample-app:bootRun \
--args='--spring.profiles.active=openai-smoke'
```

In another terminal:

```bash
curl -sS --get http://localhost:8080/test/token-pilot/openai-smoke \
--data-urlencode 'prompt=Reply with one short sentence confirming the provider is reachable.' \
| jq
```

The expected evidence is `status=PASS`, a reported usage source, and
`accountingState=COMMITTED`. The default model is the versioned model already
known by the sample catalog. If `OPENAI_MODEL` is changed, its catalog entry
and pricing plan must also match the response model.

The live test is guarded and skipped by the normal test suite:

```bash
RUN_OPENAI_SMOKE=true \
./gradlew --no-daemon :token-pilot-sample-app:test \
--tests io.tokenpilot.sample.OpenAiSmokeE2ETest
```

Run the demo first, stop it, then start `openai-smoke`; both profiles use port
`8080` and must not run concurrently.

## 4. Prometheus and Grafana

Keep the sample app running and start the monitoring services:

Expand Down
54 changes: 53 additions & 1 deletion docs/SAMPLE_RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,59 @@ curl -s http://localhost:8080/test/token-pilot/demo/reconciliation-unknown | jq
```

The endpoint list and the effective profile configuration are also maintained
in [`token-pilot-sample-app/DEMO_RUNBOOK.md`](../token-pilot-sample-app/DEMO_RUNBOOK.md).
in the [Korean demo runbook](../token-pilot-sample-app/DEMO_RUNBOOK.md) and the
[English demo runbook](../token-pilot-sample-app/DEMO_RUNBOOK_EN.md).

## OpenAI provider smoke profile

Stop the demo app before starting this profile. The sample app selects the
OpenAI provider only for `openai-smoke`; the default and `demo` profiles keep
provider auto-configuration disabled so fake-provider tests remain deterministic.

Set the API key in the environment. Do not put it in a source file or commit it.
The default model is the versioned catalog entry
`gpt-4o-mini-2024-07-18`; each request is a real provider call and may incur
provider charges.

```bash
export OPENAI_API_KEY='your OpenAI API key'
export OPENAI_MODEL='gpt-4o-mini-2024-07-18'

./gradlew --no-daemon :token-pilot-sample-app:bootRun \
--args='--spring.profiles.active=openai-smoke'
```

In another terminal, run one short request:

```bash
curl -sS --get http://localhost:8080/test/token-pilot/openai-smoke \
--data-urlencode 'prompt=Reply with one short sentence confirming the provider is reachable.' \
| jq
```

The response is a pass when it reports `status=PASS`, a non-`UNAVAILABLE`
`usageSource`, and `accountingState=COMMITTED`. It also returns normalized
input/output/total tokens, the newly applied cost metric delta, and the budget
snapshot. Inspect the rendered meters with:

```bash
curl -sS http://localhost:8080/actuator/prometheus \
| rg '^tokenpilot_(preflight|budget|cost|reconciliation)'
```

The opt-in JUnit smoke test uses the same endpoint and is guarded by both
`RUN_OPENAI_SMOKE=true` and `OPENAI_API_KEY`:

```bash
RUN_OPENAI_SMOKE=true \
./gradlew --no-daemon :token-pilot-sample-app:test \
--tests io.tokenpilot.sample.OpenAiSmokeE2ETest
```

To run both paths, execute `/test/token-pilot/demo/run` while the `demo` profile
is active, stop that process, then restart with `openai-smoke` and call the
smoke endpoint. They intentionally share port `8080` and should not run at the
same time.

## Prometheus and Grafana

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import org.springframework.ai.chat.client.ChatClientBuilderCustomizer;

/**
* ChatClient.Builder에 LedgerAdvisor를 자동으로 주입하는 커스터마이저.
* Customizer that automatically injects LedgerAdvisor into ChatClient.Builder.
*/
public class LedgerChatClientBuilderCustomizer implements ChatClientBuilderCustomizer {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@
import java.util.Objects;

/**
* 개별 AI 모델에 대한 가격 설정 정보.
* Pricing configuration for an individual AI model.
*
* @param modelId AI 모델 식별자 (예: gpt-4o, Claude-3-5-sonnet). 이 값은 ChatResponse의 model_id와 매칭되어야 합니다.
* @param rates 토큰 타입별 1,000(1K) 토큰당 가격 설정. (PROMPT, COMPLETION 등)
* @param currency 가격 정산에 사용할 통화 단위 (기본값: USD).
* @param modelId AI model identifier, such as gpt-4o or Claude-3-5-sonnet. This value must match the ChatResponse model_id.
* @param rates per-1,000 (1K) token price by token type, such as PROMPT or COMPLETION
* @param currency currency used for cost settlement, defaulting to USD
*/
public record PricingPlanProperties(
String modelId,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;

/** Token Pilot의 framework-independent core 자동 설정입니다. */
/** Auto-configuration for the Token Pilot framework-independent core. */
@AutoConfiguration
@ConditionalOnProperty(
prefix = "token-pilot",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

import java.time.Clock;

/** Optional budget module의 bean graph를 격리하는 자동 설정입니다. */
/** Auto-configuration isolating the optional budget module bean graph. */
@AutoConfiguration(after = {
TokenPilotAutoConfiguration.class,
TokenPilotBudgetMetricsAutoConfiguration.class
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;

/** Atomic reservation/accounting metrics의 조건부 자동 설정입니다. */
/** Conditional auto-configuration for atomic reservation/accounting metrics. */
@AutoConfiguration(after = TokenPilotCoreMetricsAutoConfiguration.class)
@ConditionalOnClass({
MeterRegistry.class,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@
import java.util.Objects;

/**
* Token Pilot 설정을 optional budget 정책으로 변환하는 공개 진입점입니다.
* Public entry point for converting Token Pilot settings into an optional budget policy.
*
* <p>budget 타입을 {@link TokenPilotProperties}에서 분리해 budget 모듈이 없는
* core-only 자동 설정도 class loading에 실패하지 않도록 합니다.</p>
* <p>Budget types are kept separate from {@link TokenPilotProperties} so
* core-only auto-configuration can load when the budget module is absent.</p>
*/
public final class TokenPilotBudgetPolicyFactory {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;

/** Core control events와 legacy ledger metrics의 조건부 자동 설정입니다. */
/** Conditional auto-configuration for core control events and legacy ledger metrics. */
@AutoConfiguration(after = TokenPilotAutoConfiguration.class, afterName = {
"org.springframework.boot.micrometer.metrics.autoconfigure.MetricsAutoConfiguration",
"org.springframework.boot.micrometer.metrics.autoconfigure.CompositeMeterRegistryAutoConfiguration"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;

/** Optional notification module의 bean graph를 격리하는 자동 설정입니다. */
/** Auto-configuration isolating the optional notification module bean graph. */
@AutoConfiguration(after = {
TokenPilotAutoConfiguration.class,
TokenPilotNotificationMetricsAutoConfiguration.class
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;

/** Budget notification lifecycle metrics의 조건부 자동 설정입니다. */
/** Conditional auto-configuration for budget notification lifecycle metrics. */
@AutoConfiguration(after = TokenPilotCoreMetricsAutoConfiguration.class)
@ConditionalOnClass({
MeterRegistry.class,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import java.util.Set;

/**
* Token Pilot의 설정을 담당하는 프로퍼티 클래스.
* Configuration properties for Token Pilot.
*/
@ConfigurationProperties(prefix = "token-pilot")
public class TokenPilotProperties {
Expand Down Expand Up @@ -186,7 +186,7 @@ public void setTargetTagKey(String targetTagKey) {
}

/**
* 대상 tag가 없을 때 사용할 명시적 fallback입니다. 미설정 시 평가는 fail-closed 됩니다.
* Explicit fallback used when the target tag is absent. Evaluation fails closed when it is unset.
*/
public String getFallbackTargetId() {
return fallbackTargetId;
Expand All @@ -205,7 +205,7 @@ public void setCurrency(String currency) {
}

/**
* 월별 budget window 경계를 계산하는 IANA ZoneId입니다. 기본값은 UTC입니다.
* IANA ZoneId used to calculate monthly budget-window boundaries. The default is UTC.
*/
public String getZoneId() {
return zoneId;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;

/** Optional Spring AI adapter의 bean graph를 격리하는 자동 설정입니다. */
/** Auto-configuration isolating the optional Spring AI adapter bean graph. */
@AutoConfiguration(after = {
TokenPilotAutoConfiguration.class,
TokenPilotBudgetAutoConfiguration.class,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1104,7 +1104,7 @@ int evaluations() {
}
}

// 테스트용 no-op handler - 실제 알림 전송 없이 빈 등록 여부만 검증
// Test-only no-op handler; verifies bean registration without sending notifications.
@Configuration(proxyBeanMethods = false)
static class FakeBudgetNotificationHandlerConfiguration {
@Bean
Expand Down
Original file line number Diff line number Diff line change
@@ -1,35 +1,36 @@
package io.tokenpilot.budget;

/**
* 예약 생성 이후 회계 상태 변경 명령의 결과입니다.
* Result of an accounting state-change command after reservation creation.
*
* <p>{@link #APPLIED}만 새로운 회계 변경이 적용되었음을 뜻합니다. 나머지 결과는 기존 상태와
* 금액을 바꾸지 않았음을 뜻하므로 호출자는 예외 메시지나 저장소 내부 구현에 의존하지 않고
* 재시도, 충돌 처리, 입력 보정을 결정할 수 있습니다.</p>
* <p>Only {@link #APPLIED} means that a new accounting change was applied. All
* other results leave the existing state and amounts unchanged, allowing callers
* to decide on retries, conflict handling, or input correction without depending
* on exception messages or store internals.</p>
*/
public enum AccountingTransitionStatus {
/** 요청한 회계 변경이 새로 적용되었습니다. */
/** The requested accounting change was newly applied. */
APPLIED,

/** 같은 회계 명령과 값이 이미 적용되어 기존 결과가 재사용되었습니다. */
/** The same accounting command and values were already applied, so the existing result was reused. */
REUSED,

/** 기존에 적용된 종료 명령 또는 값이 새 요청과 충돌합니다. */
/** An already applied terminal command or value conflicts with the new request. */
CONFLICT,

/** 대상 예약을 찾을 수 없습니다. */
/** The target reservation was not found. */
NOT_FOUND,

/** 명령 금액의 통화가 예약 통화와 다릅니다. */
/** The command amount uses a different currency from the reservation. */
CURRENCY_MISMATCH,

/** 명령 인자가 계약을 만족하지 않습니다. */
/** The command arguments do not satisfy the contract. */
INVALID_ARGUMENT,

/** 현재 예약 상태에서는 요청한 전이가 허용되지 않습니다. */
/** The requested transition is not allowed from the current reservation state. */
NOT_ALLOWED;

/** 새로운 회계 변경이 적용되었는지 반환합니다. */
/** Returns whether a new accounting change was applied. */
public boolean isApplied() {
return this == APPLIED;
}
Expand Down
Loading
Loading