diff --git a/.gitignore b/.gitignore index ec8b4a2..5593d06 100644 --- a/.gitignore +++ b/.gitignore @@ -362,3 +362,7 @@ MigrationBackup/ # Fody - auto-generated XML schema FodyWeavers.xsd /CoverageReport + +# Local environment variables +.env +.env.local diff --git a/Milvaion.slnx b/Milvaion.slnx index 3f778c2..ec5db20 100644 --- a/Milvaion.slnx +++ b/Milvaion.slnx @@ -85,6 +85,7 @@ + @@ -165,7 +166,13 @@ - + + + + + + + diff --git a/README.md b/README.md index 6a488ce..eb45d96 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,8 @@

A distributed job scheduling system built on .NET 10 +
+Already on Hangfire or Quartz.NET? Keep it — add Milvaion monitoring in two lines.

@@ -60,6 +62,53 @@ Milvaion solves these problems by **completely separating scheduling from execut --- +## Already Running Hangfire or Quartz.NET? + +**You don't have to migrate to adopt Milvaion.** + +Milvaion plugs into the scheduler you already run. Your scheduler keeps owning triggers, storage and cron expressions. Your job code doesn't change. Milvaion adds one real-time dashboard across every service, plus persisted execution history, metrics and alerting. + +```csharp +// Hangfire - your existing setup stays exactly as it is +builder.Services.AddMilvaionHangfireIntegration(builder.Configuration); +builder.Services.AddHangfire((sp, config) => config.UseMilvaion(sp)); + +// Quartz.NET - your existing setup stays exactly as it is +builder.Services.AddMilvaionQuartzIntegration(builder.Configuration); +builder.Services.AddQuartz(q => q.UseMilvaion(builder.Services)); +``` + +That's the whole integration. + +### The problem this solves + +A typical .NET estate ends up with background jobs scattered across a dozen services — each with its own dashboard behind its own URL and its own auth. Nobody can answer *"which jobs failed last night?"* without opening every one of them, and history vanishes when the retention window rolls over. + +| Before | After | +|--------|-------| +| One dashboard per service | One dashboard for the whole estate | +| History limited by Hangfire/Quartz retention | Full execution history in PostgreSQL | +| Logs only in each app's own sink | Real-time log streaming per execution | +| Failures noticed when someone complains | Multi-channel alerts on failure and timeout | +| No cross-service metrics | Success rate, duration and EPM per job | + +### The adoption path + +1. **Add monitoring** — two lines per application. No migration window, no change to job code. +2. **Get visibility** — every Hangfire and Quartz.NET job appears in the dashboard, flagged as external. +3. **Migrate selectively, or never** — when one job outgrows in-process execution, move just that job to a Milvaion worker. Everything else keeps running untouched. + +Plenty of teams stop at step 2. That's a perfectly good outcome. + +| Scheduler | Package | Status | +|-----------|---------|--------| +| **Hangfire** | `Milvasoft.Milvaion.Sdk.Worker.Hangfire` | ✅ Available | +| **Quartz.NET** | `Milvasoft.Milvaion.Sdk.Worker.Quartz` | ✅ Available | + +📖 **[Full Integration Guide →](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/external-schedulers)** + +--- + ## Features ![Milvaion Real Time](https://portal.milvasoft.com/assets/images/executions-4b5918b7fca1b603f54be133c7880397.gif) @@ -105,26 +154,28 @@ Milvaion solves these problems by **completely separating scheduling from execut - **Maintenance Worker** - Milvaion self data warehouse cleanup and archival ### External Scheduler Integration -Already using **Quartz.NET** or **Hangfire**? Keep your existing scheduler and gain Milvaion's monitoring capabilities: - -| Scheduler | Package | Status | -|-----------|---------|--------| -| **Quartz.NET** | `Milvasoft.Milvaion.Sdk.Worker.Quartz` | ✅ Available | -| **Hangfire** | `Milvasoft.Milvaion.Sdk.Worker.Hangfire` | ✅ Available | +Milvaion also monitors jobs running in **Quartz.NET** and **Hangfire** without replacing them — see [Already Running Hangfire or Quartz.NET?](#already-running-hangfire-or-quartznet) above. -```csharp -// Quartz.NET Integration -builder.Services.AddMilvaionQuartzIntegration(builder.Configuration); -builder.Services.AddQuartz(q => q.UseMilvaion(builder.Services)); +### MCP Server +Point Claude Code, Cursor or GitHub Copilot at Milvaion and ask about your jobs in plain language: -// Hangfire Integration -builder.Services.AddMilvaionHangfireIntegration(builder.Configuration); -builder.Services.AddHangfire((sp, config) => config.UseMilvaion(sp)); +```json +{ + "mcpServers": { + "milvaion": { + "type": "http", + "url": "https://milvaion.yourcompany.com/mcp", + "headers": { "X-ApiKey": "your-api-key" } + } + } +} ``` -External jobs appear in Milvaion dashboard with full monitoring, metrics, and execution history - without changing your existing scheduler setup. +> *"Which jobs failed last night and why?"* + +More than forty tools — reading, triggering, pausing, editing and deleting — each gated by the same permissions used everywhere else. A read-only api key gives an assistant full visibility and no ability to change anything. Milvaion is the data source here; it never calls a language model and stores no model provider keys. -[For more information...](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/external-schedulers) +📖 **[MCP Server Guide →](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/mcp-server)** --- @@ -378,6 +429,9 @@ Each workflow can also configure **Max Step Retries** and a **Timeout** (auto-ca | Document | Description | |----------|-------------| | [Introduction](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/introduction) | What is Milvaion, when to use it | +| [Hangfire & Quartz.NET Integration](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/external-schedulers) | Keep your existing scheduler, add Milvaion monitoring | +| [Api Keys](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/api-keys) | Credentials for CI pipelines, scripts and MCP clients | +| [MCP Server](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/mcp-server) | Connect Claude Code, Cursor or Copilot to Milvaion | | [Quick Start](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/quick-start) | Get running in under 10 minutes | | [Core Concepts](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/core-concepts) | Architecture and key terms | | [Your First Worker](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/your-first-worker) | Create a custom worker | @@ -397,6 +451,7 @@ Each workflow can also configure **Max Step Retries** and a **Timeout** (auto-ca | [Architecture](./docs/githubdocs/ARCHITECTURE.md) | Technical architecture deep-dive | | [Development](./docs/githubdocs/DEVELOPMENT.md) | Development environment setup | | [Worker SDK](./docs/githubdocs/WORKER-SDK.md) | Worker SDK reference | +| [MCP Server](./docs/githubdocs/MCP-SERVER.md) | MCP server and api key auth internals | | [Security](./SECURITY.md) | Security policies | --- diff --git a/docs/githubdocs/MCP-SERVER.md b/docs/githubdocs/MCP-SERVER.md new file mode 100644 index 0000000..fd05df4 --- /dev/null +++ b/docs/githubdocs/MCP-SERVER.md @@ -0,0 +1,260 @@ +# MCP Server + +This document covers the internals of Milvaion's Model Context Protocol server and the api key authentication it depends on. For user-facing setup instructions see the [MCP Server](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/mcp-server) and [Api Keys](https://portal.milvasoft.com/docs/1.0.1/open-source-libs/milvaion/api-keys) portal docs. + +## Table of Contents + +- [Overview](#overview) +- [Api Key Authentication](#api-key-authentication) +- [MCP Server Wiring](#mcp-server-wiring) +- [Authorization Inside Tools](#authorization-inside-tools) +- [Adding a Tool](#adding-a-tool) +- [Design Decisions](#design-decisions) +- [Testing](#testing) + +--- + +## Overview + +Milvaion is an MCP **server**: it exposes its own data as tools that an MCP client (Claude Code, Cursor, Copilot) can call. Milvaion never talks to a language model and holds no model provider credentials. + +```text +MCP client Milvaion.Api + │ + │ POST /mcp (JSON-RPC, X-ApiKey header) + ▼ +ApiKeyAuthenticationHandler ──▶ ApiKeyStore (Redis cache ─▶ Postgres) + │ ClaimsPrincipal with permissions as role claims + ▼ +MapMcp endpoint ──▶ tool method ──▶ McpPermissionGuard.Require(...) + │ + ▼ + IMediator ──▶ existing CQRS query +``` + +### Source Layout + +| Path | Contains | +|------|----------| +| `src/Milvaion.Api/Mcp/MilvaionJobTools.cs` | Job and occurrence tools, including CRUD | +| `src/Milvaion.Api/Mcp/MilvaionOpsTools.cs` | Worker, workflow and dashboard tools | +| `src/Milvaion.Api/Mcp/MilvaionDiagnosticsTools.cs` | Dead letter failures and the activity log | +| `src/Milvaion.Api/Mcp/MilvaionInsightTools.cs` | Metric reports, infrastructure health, configuration | +| `src/Milvaion.Api/Mcp/MilvaionPrompts.cs` | Prompt templates for diagnosis workflows | +| `src/Milvaion.Api/Mcp/McpPermissionGuard.cs` | Per-tool permission enforcement | +| `src/Milvaion.Api/AppStartup/McpExtensions.cs` | Registration and endpoint mapping | +| `src/Milvaion.Api/Utils/ApiKeyAuthenticationHandler.cs` | Authentication scheme | +| `src/Milvaion.Api/Utils/KeyHelper.cs` | Key generation and signature validation | +| `src/Milvaion.Api/Services/ApiKeyStore.cs` | Cached key lookup and cache invalidation | +| `src/Milvaion.Api/Services/ApiKeyGenerator.cs` | `IApiKeyGenerator` implementation | +| `src/Milvaion.Application/Features/ApiKeys/` | Api key CQRS features | + +Dependency: `ModelContextProtocol.AspNetCore`. + +--- + +## Api Key Authentication + +### Key Format + +A key is a JWT signed with `MilvaionConfig.ApiKey.Secret` (HMAC-SHA256), carrying: + +| Claim | Meaning | +|-------|---------| +| `jti` | Id of the `MilvaionApiKey` record | +| `kv` | Version of the signing secret | +| `exp` | Expiry, omitted for keys that never expire | + +The key itself is **never persisted**. Only `MaskedKey` — the trailing characters — is stored, purely so a user can match a listed key against the one in their configuration. + +### Why a Record Lookup + +Signature validation alone would be enough to prove a key is authentic, but a self-contained token cannot be revoked. Authentication therefore always resolves `jti` to a `MilvaionApiKey` row and checks `RevokedAt`, `ExpiresAt` and `KeyVersion` before accepting the request. + +That lookup is cached in **Redis**, not in process memory. With several API replicas an in-process cache would leave a revoked key working on every replica that did not happen to handle the revoke request. `RevokeApiKeyCommandHandler` and `DeleteApiKeyCommandHandler` call `IApiKeyCacheInvalidator.InvalidateAsync` so revocation takes effect immediately across the cluster. + +If Redis is unavailable, `ApiKeyStore` falls back to the database rather than failing authentication — a cache outage should not lock every integration out. + +### Permissions as Role Claims + +`AuthAttribute` derives from `AuthorizeAttribute` and matches on `Roles`. `ApiKeyAuthenticationHandler` therefore emits each granted permission as a `ClaimTypes.Role` claim, exactly as the login token does: + +```csharp +claims.AddRange(apiKey.Permissions.Select(p => new Claim(ClaimTypes.Role, p))); +claims.Add(new Claim(GlobalConstant.UserTypeClaimName, nameof(UserType.Manager))); +``` + +The consequence is that **every existing `[Auth(PermissionCatalog...)]` attribute already works for api key callers** with no change. Had this been implemented as an action filter instead of an authentication scheme, authorization would have had to be reimplemented separately for machine callers. + +The default authorization policy names both schemes: + +```csharp +options.DefaultPolicy = new AuthorizationPolicyBuilder( + JwtBearerDefaults.AuthenticationScheme, + ApiKeyAuthenticationDefaults.AuthenticationScheme) + .RequireAuthenticatedUser() + .Build(); +``` + +### LastUsedAt Throttling + +`LastUsedAt` would otherwise mean a database write on every request. `ApiKeyStore.ShouldWriteLastUsedAsync` uses a Redis `SET ... NX EX` as both throttle and lock: whoever sets the key owns that interval. Default interval is five minutes, configurable through `ApiKeyAuthenticationOptions.LastUsedWriteInterval`. + +Failures here are swallowed and logged. Bookkeeping must never fail an otherwise valid request. + +### `ApiAuthAttribute` + +`[ApiAuth]` restricts an endpoint to api key callers only, for endpoints that should not be reachable from a browser session. `[Auth]` accepts both. All parsing and validation lives in the authentication handler; the attribute only selects the scheme and applies the permission requirement. + +--- + +## MCP Server Wiring + +Registration is in `McpExtensions.AddMilvaionMcp`: + +```csharp +services.AddMcpServer(options => { /* ServerInfo, ServerInstructions */ }) + .WithHttpTransport(options => options.Stateless = true) + .WithToolsFromAssembly(); +``` + +`WithToolsFromAssembly` discovers every `[McpServerToolType]` class and registers each `[McpServerTool]` method. Tool classes are resolved from DI per request, so constructor injection of scoped services works. + +Mapping is in `McpExtensions.MapMilvaionMcp`: + +```csharp +app.MapMcp("/mcp").RequireAuthorization(); +``` + +> **Ordering matters.** `MapMilvaionMcp()` must be called before `MapFallbackToFile("index.html")` in `Program.cs`, or the SPA fallback swallows `/mcp`. + +### ServerInstructions + +`ServerInstructions` is sent to the client at initialize and shapes how the model uses the server. Milvaion's explains the job/occurrence distinction, steers diagnosis towards `list_failures` rather than `list_occurrences`, and tells the model to confirm before triggering anything. In practice this affects tool selection accuracy more than any individual tool description. + +--- + +## Authorization Inside Tools + +MCP tools are not MVC actions, so `[Auth]` never runs for them. Authentication is still enforced at the endpoint, but the per-permission check has to happen in the tool body: + +```csharp +_guard.Require(PermissionCatalog.ScheduledJobManagement.List); +``` + +`McpPermissionGuard` reads `HttpContext.User`, honours `App.SuperAdmin`, and throws `McpException` otherwise. The message names the missing permission on purpose — an agent told exactly what it lacks stops retrying and can report something actionable. + +`Has(permission)` is available for trimming optional detail out of a response instead of failing the whole call. + +--- + +## Adding a Tool + +1. Add a method to an existing `[McpServerToolType]` class, or create a new one — `WithToolsFromAssembly` finds it either way. +2. Call `_guard.Require(...)` **first**, before any work. +3. Delegate to the existing MediatR query or command. Do not reimplement data access; a change to filtering or projection should show up identically in the dashboard, the REST API and MCP. +4. Write a `[Description]` aimed at a model, not a developer: say when to reach for this tool rather than another one. +5. Add XML docs to match the rest of the codebase. + +```csharp +/// +/// Gets scheduled jobs. +/// +/// Free text search over job names and types. +/// +/// Paged job list with the total count. +[McpServerTool(Name = "list_jobs")] +[Description("Lists scheduled jobs in Milvaion. Use this to find a job's id before calling other tools.")] +public async Task ListJobsAsync( + [Description("Optional free text search over job names and types.")] string searchTerm = null, + CancellationToken cancellationToken = default) +{ + _guard.Require(PermissionCatalog.ScheduledJobManagement.List); + ... +} +``` + +### Conventions + +- **Tool names** are `snake_case` and verb-first: `list_jobs`, `get_occurrence`, `trigger_job`. +- **Annotations are mandatory.** Set `ReadOnly = true` on anything that only reads, `Destructive = true` on anything irreversible, `Idempotent = true` on state setters. Clients use these to decide what to auto-approve; an unannotated destructive tool looks identical to a list call. +- **Filter, do not paginate.** Every list tool should expose the filters a person would reach for - id, status, owner, date bounds - built through `BuildDateRangeCriterias`, `AddEqualityCriteria` and `ToFilterRequest` in `MilvaionJobTools`. Making the model page through thousands of rows to find ten is slow and expensive. +- **Sort explicitly.** The handlers do not all default their sort order, so a query without `Sorting` returns rows in whatever order the plan produces. Match whatever the dashboard sends for the same list, or the model and the user will be looking at different data. +- **Bound the response.** Anything that can grow without limit - logs especially - needs a cap and a note saying it was truncated. Blowing the context window is a silent failure: the answer just gets vague and expensive. +- **Page sizes** are clamped to `_maxPageSize` (100). A model asking for 10,000 rows should get 100, not an error. +- **Not-found** throws `McpException` with a message naming the id, rather than returning null. +- **Write tools** require their own permission and never bypass domain safeguards — `trigger_job` always dispatches with `Force = false`. +- **Attribution**: anything with a side effect stamps the calling key into the reason, so history distinguishes machine from human. + +--- + +## Design Decisions + +### Why stateless HTTP transport + +Milvaion is routinely deployed with several API replicas behind a load balancer. Stateful mode would require sticky sessions. Stateless rules out server-to-client requests (sampling, elicitation, roots), none of which these tools need. + +### Why `update_job` takes plain arguments + +`UpdateScheduledJobCommand` uses `UpdateProperty` to separate "not supplied" from "set to null". Exposing that shape to a model would mean asking it to emit `{"value": "...", "isUpdated": true}` per field, which is both awkward to describe and easy to get wrong. + +The tool takes nullable plain arguments instead and builds the wrappers in C#: null means unchanged. The cost is that a field cannot be cleared through MCP. That asymmetry is deliberate — a model accidentally blanking a cron expression is a worse outcome than not being able to blank one on purpose. + +### Why `set_job_active` exists separately + +Pausing a job is the most common intervention during an incident and it should not require a partial update payload. A dedicated tool also gives the description room to steer the model towards pausing rather than deleting, which is what users almost always mean by "stop this job". + +### Why workflow authoring is absent + +There is no `create_workflow` or `update_workflow`. A workflow is a directed graph with conditions, merge nodes and data mappings between steps; expressing one as tool arguments would be a large nested payload with plenty of ways to produce a graph that is syntactically valid and semantically wrong. Reading, triggering, cancelling and deleting workflows are all available — authoring stays in the visual builder. + +### Why the write tools are shaped around intent + +Rather than mirroring the REST surface one-to-one, the write tools are named after what a person wants: `set_job_active` rather than a generic update, `resolve_failures` rather than an update with six optional fields. Narrow tools are easier for a model to select correctly and give far less room for it to do something adjacent to what was asked. + +### Why tools delegate to MediatR + +Every tool sends an existing query. This keeps filtering, projection, permission semantics and localisation identical across all three entry points, and means new tools are cheap — most are a permission check plus a `Send`. + +--- + +## Testing + +`/mcp` speaks JSON-RPC over HTTP, so it can be exercised without an MCP client. + +```bash +# Expect 401 +curl -i -X POST http://localhost:5000/mcp \ + -H "Content-Type: application/json" \ + -H "Accept: application/json, text/event-stream" \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' + +# Tool list +curl -X POST http://localhost:5000/mcp \ + -H "Content-Type: application/json" \ + -H "Accept: application/json, text/event-stream" \ + -H "X-ApiKey: $MILVAION_API_KEY" \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' + +# Tool call +curl -X POST http://localhost:5000/mcp \ + -H "Content-Type: application/json" \ + -H "Accept: application/json, text/event-stream" \ + -H "X-ApiKey: $MILVAION_API_KEY" \ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_overview","arguments":{}}}' +``` + +Worth covering explicitly: + +- A key **without** `ScheduledJobManagement.Trigger` calling `trigger_job` must fail. This is the guarantee the read-only key story rests on. +- A **revoked** key must stop working immediately, not after the cache expires. +- A key issued under an older `ApiKey.Version` must be rejected with the retired-secret message. + +--- + +## Further Reading + +- [MCP C# SDK](https://csharp.sdk.modelcontextprotocol.io/) +- [Model Context Protocol specification](https://modelcontextprotocol.io/specification/) +- [Architecture](ARCHITECTURE.md) +- [Development](DEVELOPMENT.md) diff --git a/docs/portaldocs/00-guide.md b/docs/portaldocs/00-guide.md index 0311e42..b960612 100644 --- a/docs/portaldocs/00-guide.md +++ b/docs/portaldocs/00-guide.md @@ -5,6 +5,9 @@ sidebar_position: 0 description: Documentation guide of Milvaion. --- +

+ Milvaion +

# Milvaion Documentation @@ -16,39 +19,46 @@ Welcome to the official Milvaion documentation. This documentation will help you | Document | Description | |----------|-------------| -| [01-introduction.md](01-introduction.md) | What is Milvaion, when to use it, comparison with alternatives | -| [02-quick-start.md](02-quick-start.md) | Get running locally in under 10 minutes | -| [03-core-concepts.md](03-core-concepts.md) | Understand the architecture and key terms | +| [Introduction](01-introduction.md) | What is Milvaion, when to use it, comparison with alternatives | +| [Quick Start](02-quick-start.md) | Get running locally in under 10 minutes | +| [Core Concepts](03-core-concepts.md) | Understand the architecture and key terms | ### Developer Guide | Document | Description | |----------|-------------| -| [04-your-first-worker.md](04-your-first-worker.md) | Create and deploy a custom worker | -| [05-implementing-jobs.md](05-implementing-jobs.md) | Write jobs with DI, error handling, testing | -| [06-configuration.md](06-configuration.md) | All configuration options for API and Workers | -| [14-built-in-workers.md](14-built-in-workers.md) | Pre-built workers (HTTP Worker, etc.) | -| [20-workflows.md](20-workflows.md) | Build multi-step job pipelines with DAG-based orchestration | -| [21-reporter-worker.md](21-reporter-worker.md) | Automated metric report generation worker | +| [Your First Worker](04-your-first-worker.md) | Create and deploy a custom worker | +| [Implementing Jobs](05-implementing-jobs.md) | Write jobs with DI, error handling, testing | +| [Configuration](06-configuration.md) | All configuration options for API and Workers | +| [Testing](23-testing.md) | Unit test jobs locally without RabbitMQ, Redis, or a live environment | ### Operations Guide | Document | Description | |----------|-------------| -| [07-deployment.md](07-deployment.md) | Production deployment with Docker and Kubernetes | -| [08-reliability.md](08-reliability.md) | Retry, DLQ, zombie detection, idempotency | -| [09-scaling.md](09-scaling.md) | Horizontal scaling strategies | -| [10-monitoring.md](10-monitoring.md) | Health checks, metrics, logging, alerting | -| [11-maintenance.md](11-maintenance.md) | Database cleanup and retention policies | -| [20-enterprise-features.md](20-enterprise-features.md) | User, role, permission management, activity tracking, auditing, and metric reports | +| [Deployment](07-deployment.md) | Production deployment with Docker and Kubernetes | +| [Reliability](08-reliability.md) | Retry, DLQ, zombie detection, idempotency | +| [Scaling](09-scaling.md) | Horizontal scaling strategies | +| [Monitoring](10-monitoring.md) | Health checks, metrics, logging, alerting | +| [Maintenance](11-maintenance.md) | Database cleanup and retention policies | +| [Built-in Workers](./built-in-workers/14-http-worker.md) | Pre-built workers (HTTP Worker, SQL Worker, Email Worker, Maintenance Worker) | +| [Workflows](20-workflows.md) | Build multi-step job pipelines with DAG-based orchestration | +| [Api Keys](24-api-keys.md) | Credentials for CI pipelines, scripts and MCP clients | +| [MCP Server](25-mcp-server.md) | Connect Claude Code, Cursor or Copilot and ask about your jobs | + ## Quick Links -- **First time→** Start with [Introduction](01-introduction.md) -- **Want to run it→** Jump to [Quick Start](02-quick-start.md) -- **Building a worker→** See [Your First Worker](04-your-first-worker.md) -- **Using built-in workers→** See [Built-in Workers](14-built-in-workers.md) -- **Going to production→** Read [Deployment](07-deployment.md) +- **First time →** Start with [Introduction](01-introduction.md) +- **Want to run it →** Jump to [Quick Start](02-quick-start.md) +- **Building a worker →** See [Your First Worker](04-your-first-worker.md) +- **Going to production →** Read [Deployment](07-deployment.md) +- **Security considerations →** Read [Security](12-security.md) +- **Milvaion UI Features →** See [UI Screenshots](13-dashboard-screenshots.md) +- **Built-in Workers →** See [Built-in Workers](./built-in-workers/14-http-worker.md) +- **Workflows →** See [Workflows](20-workflows.md) +- **Using it from an AI assistant →** See [MCP Server](25-mcp-server.md) +- **CI or script access →** See [Api Keys](24-api-keys.md) ## Reading Order @@ -60,7 +70,7 @@ For new users, we recommend reading in this order: 3. Core Concepts → Understand the architecture 4. Your First Worker → Build something real 5. Configuration → Reference as needed -6. (Optional) Reliability, Scaling, Monitoring for production +6. (Optional) Reliability, Scaling, Monitoring, Security for production, Built-in workers ``` ## Version @@ -73,4 +83,4 @@ This documentation covers **Milvaion 1.0.0** with: ## Feedback -Found an issue or want to suggest improvements? Open an issue on GitHub. +Found an issue or want to suggest improvements? Open an issue on [GitHub](https://github.com/Milvasoft/milvaion). diff --git a/docs/portaldocs/01-introduction.md b/docs/portaldocs/01-introduction.md index 5682c80..9a84cf4 100644 --- a/docs/portaldocs/01-introduction.md +++ b/docs/portaldocs/01-introduction.md @@ -1,17 +1,29 @@ --- id: introduction title: Introduction -sidebar_position: 1 +sidebar_position: 10 description: Overview of Milvaion, its purpose, and core philosophy. --- +

+ Milvaion +

+ # Introduction Milvaion is a **distributed job scheduling system** built on .NET 10. It separates the *scheduler* (API that decides when jobs run) from the *workers* (processes that execute jobs), connected via Redis and RabbitMQ. +:::tip Already using Hangfire or Quartz.NET? + +You do not have to migrate. Milvaion plugs into your existing scheduler in **two lines of code** and gives you a unified dashboard, execution history, metrics and alerting across every service — without touching your job code. + +**→ [Hangfire & Quartz.NET Integration](./18-external-scheduler.md)** + +::: + ## What Problem Does Milvaion Solve? -Most job schedulers run jobs **inside the same process** as the scheduling logic. This works fine until: +Most job schedulers run jobs **inside the same process** as the scheduling logic by default. This works fine until: - A long-running job blocks other jobs from executing - A crashing job takes down the entire scheduler @@ -43,16 +55,17 @@ Milvaion solves these problems by **completely separating scheduling from execut | **Multi-tenant systems** | Route jobs to specific worker groups | | **Jobs needing different resources** | GPU workers, high-memory workers, etc. | | **Compliance/audit requirements** | Full execution history with logs stored in PostgreSQL | +| **Multi-step pipelines with branching** | DAG-based [Workflows](./20-workflows.md) with conditions, merges and data mappings | +| **Existing Hangfire/Quartz.NET estates** | [Integrate without migrating](./18-external-scheduler.md) — monitor every scheduler in one dashboard | ### Not a Good Fit ❌ | Scenario | Better Alternative | |----------|-------------------| -| **Simple in-process background tasks** | Use `BackgroundService` or Hangfire | +| **Simple in-process background tasks** | Use `BackgroundService` or Hangfire — you can still [add Milvaion monitoring](./18-external-scheduler.md) on top | | **Real-time event processing** | Use dedicated event streaming (Kafka, Azure Event Hubs) | | **Sub-second job scheduling** | Milvaion polls every 1 second minimum | | **Single-server deployments** | Overhead of Redis + RabbitMQ not justified | -| **Workflow orchestration with branching** | Use Temporal, Elsa, or Azure Durable Functions | ## Core Concepts @@ -67,7 +80,7 @@ Milvaion solves these problems by **completely separating scheduling from execut ## How It Works -1. **Worker Auto Disvovery** service will auto discover your worker and containing worker jobs. +1. **Worker Auto Discovery** service will auto discover your worker and containing worker jobs. 2. **You create a job according to worker job** via REST API or Dashboard (e.g., "Send daily report at 9 AM") 3. **Scheduler stores it** in PostgreSQL and adds to Redis ZSET with next run time 4. **Dispatcher** checks Redis for due jobs @@ -97,6 +110,7 @@ Milvaion solves these problems by **completely separating scheduling from execut - Technical Logs -> Logs are sent to Seq. - **Worker health monitoring** via heartbeats - **OpenTelemetry support** for metrics and tracing +- **Alerting** - configurable notifications for job failures and system events via multiple channels (Google Chat, Microsoft Teams, Slack, Email and in-app) ### Developer Experience - **Simple `IAsyncJob` interface** - implement one method @@ -104,6 +118,18 @@ Milvaion solves these problems by **completely separating scheduling from execut - **Auto-discovery** - jobs registered automatically - **Cancellation support** - graceful shutdown +### AI Integration +- **[MCP server](./25-mcp-server.md)** - connect Claude Code, Cursor or GitHub Copilot and ask about your jobs in plain language +- **40+ tools** - reading, triggering, pausing, editing and deleting, each behind its own permission +- **Scoped by api key** - a read-only key lets an assistant investigate everything and change nothing +- **No model provider keys** - Milvaion is the data source; the model runs in the user's own editor + +### Enterprise Management +- **Role-based access control** - define roles with specific permissions +- **User management** - create and manage users with assigned roles +- **Permission granularity** - control access at job, worker, and dashboard level +- **[Api keys](./24-api-keys.md)** - non-interactive credentials for CI pipelines, scripts and MCP clients + ## Comparison with Alternatives | Feature | Milvaion | Hangfire | Quartz.NET | @@ -115,10 +141,21 @@ Milvaion solves these problems by **completely separating scheduling from execut | **Real-time Dashboard** | Built-in | Built-in | None | | **Log Streaming** | Real-time via RabbitMQ | Console plugin | None | | **Offline Resilience** | SQLite fallback | None | None | +| **Workflow Engine (DAG)** | Built-in, visual builder | Continuations only | None | +| **MCP server for AI assistants** | Built-in, 40+ tools | None | None | +| **Monitors the others** | ✅ Hangfire + Quartz.NET | N/A | N/A | | **Best For** | Distributed systems | Simple .NET apps | Embedded scheduling | +:::note Not an either/or + +The last row is the important one. Milvaion is not only a Hangfire or Quartz.NET replacement — it also **monitors them**. If you already run either, start with [the integration](./18-external-scheduler.md) and decide about migration later. + +::: + ## Next Steps +- **[Hangfire & Quartz.NET Integration](./18-external-scheduler.md)** - Already have a scheduler? Start here +- **[MCP Server](./25-mcp-server.md)** - Ask your scheduler questions from Claude Code, Cursor or Copilot - **[Quick Start](02-quick-start.md)** - Run Milvaion locally in 5 minutes - **[Core Concepts](03-core-concepts.md)** - Understand the architecture - **[Your First Worker](04-your-first-worker.md)** - Build and deploy a custom worker diff --git a/docs/portaldocs/06-configuration.md b/docs/portaldocs/06-configuration.md index a0da7fa..1eb3bac 100644 --- a/docs/portaldocs/06-configuration.md +++ b/docs/portaldocs/06-configuration.md @@ -54,7 +54,9 @@ This page documents all configuration options for Milvaion API and Workers. "AutomaticRecoveryEnabled": true, "NetworkRecoveryInterval": 10, "QueueDepthWarningThreshold": 100, - "QueueDepthCriticalThreshold": 500 + "QueueDepthCriticalThreshold": 500, + "ManagementEnabled": false, + "ManagementPort": 15672 }, "JobDispatcher": { "Enabled": true, @@ -141,8 +143,10 @@ Open telemetry configurations. Set null or empty via environment variables if yo | `Heartbeat` | `/` | Heartbeat interval in seconds (0 = disabled). | | `AutomaticRecoveryEnabled` | `/` | Automatic connection recovery enabled. | | `NetworkRecoveryInterval` | `/` | Network recovery interval in seconds. | -| `QueueDepthWarningThreshold` | `/` | Queue depth warning threshold. | -| `QueueDepthCriticalThreshold` | `/` | Queue depth critical threshold. | +| `QueueDepthWarningThreshold` | `5000` | Message count that triggers a Warning health status. | +| `QueueDepthCriticalThreshold` | `10000` | Message count that triggers a Critical health status. | +| `ManagementEnabled` | `false` | Enable RabbitMQ Management HTTP API integration for richer queue stats and dynamic queue discovery. | +| `ManagementPort` | `15672` | RabbitMQ Management plugin HTTP port. Used when `ManagementEnabled` is `true`. | ### Job Dispatcher Configuration @@ -208,6 +212,57 @@ Open telemetry configurations. Set null or empty via environment variables if yo | `ConsecutiveFailureThreshold` | `5` | Number of consecutive failures before a job is automatically disabled. Individual jobs can override this with their own AutoDisableThreshold setting. Default: 5 consecutive failures | | `FailureWindowMinutes` | `60` | Time window in minutes for counting consecutive failures. Failures older than this window don't count towards the threshold. This prevents jobs from being disabled due to old historical failures. Default: 60 minutes (1 hour) | +### BasePath Configuration + +Milvaion API supports hosting under a configurable sub-path (e.g. `/milvaion`) so that both the UI and the backend API can be scoped to a URL prefix. This is useful when deploying behind a reverse proxy alongside other services. + +| Setting | Default | Description | +|---------|---------|-------------| +| `BasePath` | `""` | URL prefix the application is mounted under. Leave empty to host at the root. Example: `/milvaion` | + +When `BasePath` is set: + +- The REST API is available at `{BasePath}/api/v1/...` (e.g. `/milvaion/api/v1/jobs`) +- The SignalR hub is available at `{BasePath}/hubs/jobs` +- The Prometheus metrics endpoint is available at `{BasePath}/metrics` +- The Scalar/OpenAPI documentation is available at `{BasePath}/scalar/v1` +- The SPA (UI) is served at `{BasePath}` and all sub-routes fall back to the SPA index + +When `BasePath` is empty or not set, the application is hosted at the root (`/`). + +#### Example Configuration + +```json +{ + "MilvaionConfig": { + "BasePath": "/milvaion" + } +} +``` + +#### Environment Variable Override + +```bash +MilvaionConfig__BasePath=/milvaion +``` + +#### Docker / docker-compose + +```yaml +services: + milvaion-api: + image: milvasoft/milvaion-api:latest # pull from Docker Hub, no rebuild needed + environment: + - MilvaionConfig__BasePath=/milvaion +``` + +To revert to root hosting, remove the override or set it to empty: + +```yaml +environment: + - MilvaionConfig__BasePath= +``` + --- ## Worker Configuration diff --git a/docs/portaldocs/10-monitoring.md b/docs/portaldocs/10-monitoring.md index db3053e..c30a003 100644 --- a/docs/portaldocs/10-monitoring.md +++ b/docs/portaldocs/10-monitoring.md @@ -741,6 +741,82 @@ docker exec milvaion-rabbitmq rabbitmqctl list_connections --- +### Queue Depth Monitoring (`IQueueDepthMonitor`) + +Milvaion includes a built-in `IQueueDepthMonitor` service that tracks queue depths and health status. When the **RabbitMQ Management HTTP API** is enabled, it provides richer statistics — including unacknowledged message counts — and automatically discovers **dynamic consumer queues** (e.g. `scheduled_jobs_queue.SampleWorker`) that are created at runtime when workers bind to the exchange. + +#### Configuration + +Enable Management API integration in `appsettings.json`: + +```json +{ + "MilvaionConfig": { + "RabbitMQ": { + "ManagementEnabled": true, + "ManagementPort": 15672 + } + } +} +``` + +| Setting | Default | Description | +|---------|---------|-------------| +| `ManagementEnabled` | `false` | Enable RabbitMQ Management HTTP API integration | +| `ManagementPort` | `15672` | Management plugin HTTP port | +| `QueueDepthWarningThreshold` | `5000` | Message count that triggers a Warning health status | +| `QueueDepthCriticalThreshold` | `10000` | Message count that triggers a Critical health status | + +> **Note:** The Management API uses the same `Username` and `Password` credentials configured under `MilvaionConfig:RabbitMQ`. + +#### How it Works + +| Scenario | Behavior | +|----------|----------| +| `ManagementEnabled: true` | Uses `GET /api/queues/{vhost}/{queue}` for individual queue stats | +| `ManagementEnabled: true` | Uses `GET /api/queues/{vhost}` to **discover all queues**, including dynamic routing-key queues | +| `ManagementEnabled: false` or API unavailable | Falls back to AMQP passive queue declare (core queues only, `MessagesUnacknowledged` will be `0`) | + +#### Queue Health Statuses + +| Status | Condition | +|--------|-----------| +| `Healthy` | Message count below warning threshold | +| `Warning` | Message count ≥ `QueueDepthWarningThreshold` | +| `Critical` | Message count ≥ `QueueDepthCriticalThreshold` | +| `Unavailable` | Queue or broker unreachable | + +#### API Endpoints + +Retrieve queue stats via the Admin API: + +```bash +# All queues (includes dynamic consumer queues when Management API is enabled) +curl http://localhost:5000/api/v1/admin/queues + +# Single queue depth +curl http://localhost:5000/api/v1/admin/queues/{queueName} +``` + +**Example response for a single queue:** + +```json +{ + "queueName": "scheduled_jobs_queue", + "messageCount": 42, + "consumerCount": 3, + "messagesReady": 40, + "messagesUnacknowledged": 2, + "healthStatus": "Healthy", + "healthMessage": "Queue operating normally", + "timestamp": "2026-01-14T18:10:00.000Z" +} +``` + +> **Dynamic queues:** Without Management API enabled, only the 8 core queues (`jobs`, `worker_logs`, `worker_heartbeat`, `worker_registration`, `status_updates`, `failed_occurrences`, `external_job_registration`, `external_job_occurrence`) are monitored. With it enabled, all queues in the vhost — including per-worker routing-key queues like `scheduled_jobs_queue.SampleWorker` — are returned automatically. + +--- + ## Redis Monitoring ### Key Metrics diff --git a/docs/portaldocs/19-alerting.md b/docs/portaldocs/19-alerting.md index 0658a59..074c21e 100644 --- a/docs/portaldocs/19-alerting.md +++ b/docs/portaldocs/19-alerting.md @@ -273,7 +273,7 @@ Configure alerting using environment variables in your `docker-compose.yml`: ```yaml services: milvaion-api: - image: milvasoft/milvaion:latest + image: milvasoft/milvaion-api:latest environment: # Base URL for action links - MilvaionConfig__Alerting__MilvaionAppUrl=https://milvaion.example.com @@ -380,7 +380,7 @@ spec: spec: containers: - name: milvaion-api - image: milvasoft/milvaion:latest + image: milvasoft/milvaion-api:latest envFrom: - configMapRef: name: milvaion-alerting-config diff --git a/docs/portaldocs/23-local-job-testing.md b/docs/portaldocs/23-local-job-testing.md new file mode 100644 index 0000000..6e9700d --- /dev/null +++ b/docs/portaldocs/23-local-job-testing.md @@ -0,0 +1,600 @@ +--- +id: local-job-testing +title: Local Job Testing +sidebar_position: 23 +description: Test your job implementations locally without RabbitMQ, Redis, or a running Milvaion environment. +--- + + +# Local Job Testing + +This guide explains how to unit-test your job implementations locally. The `Milvasoft.Milvaion.Sdk.Worker` package lets you execute any job and inspect its result without a running RabbitMQ, Redis, or database. + +## Why Test Locally? + +When you develop a new job in your worker, the typical deployment cycle looks like this: + +``` +Code → Build → Docker image → Deploy to Dev → Activate job → Trigger manually → Check logs +``` + +This cycle is slow and requires a live environment for every iteration. Local testing short-circuits this by running the exact same job execution path directly in a test process: + +| | Local Test | Live Environment | +|---|---|---| +| **RabbitMQ** | Not required | Required | +| **Redis** | Not required | Required | +| **Database** | Not required | Required | +| **Milvaion API** | Not required | Required | +| **Feedback speed** | Milliseconds | Minutes | +| **Debugger** | Full support | Not available | +| **Result inspection** | Programmatic | Dashboard only | + +> **Tip**: Local tests do not replace live testing in a Dev environment. Use them to iterate on business logic fast, then validate end-to-end in Dev before promoting. + +--- + +## Template — Pre-configured Test Project + +If you created your worker from the **Milvaion Worker Template**, a test project is already included. You don't need to create or configure anything. + +The template generates the following structure out of the box: + +``` +MyWorker/ +├── MyWorker.csproj ← your worker +├── MyJobs.cs +├── Program.cs +└── MyWorker.Tests/ + ├── MyWorker.Tests.csproj ← test project (pre-configured) + └── SampleJobTests.cs ← ready-to-run example tests +``` + +The test project already has: +- `xUnit`, `FluentAssertions`, and `Microsoft.NET.Test.Sdk` referenced +- A `ProjectReference` pointing to your worker +- Sample tests for every job type included in the template + +To verify everything works right after scaffolding, just run: + +```bash +dotnet test MyWorker.Tests +``` + +All sample tests should pass immediately with no additional setup. + +> If you used the template, skip to [JobTestRunner API](#jobtestrunner-api) and start writing tests for your own jobs. + +--- + +## Setup + +### 1. Create a Test Project + +Add an xUnit test project alongside your worker: + +```bash +dotnet new xunit -n MyWorker.Tests +cd MyWorker.Tests +``` + +### 2. Add References + +Add the testing SDK and a reference to your worker project: + +```bash +dotnet add reference ../MyWorker/MyWorker.csproj +dotnet add package Milvasoft.Milvaion.Sdk.Worker +dotnet add package FluentAssertions +``` + +Your `.csproj` should look like: + +```xml + + + + net10.0 + true + disable + + + + + + + + + + + + + + + +``` + +--- + +## JobTestRunner API + +`JobTestRunner` is a fluent builder that wraps `JobExecutor` — the same component that runs your jobs in production. + +### Builder Methods + +| Method | Description | Default | +|--------|-------------|---------| +| `For(IJobBase job)` | Sets the job instance to run | — | +| `WithJobData(T data)` | Serializes `data` as JSON and passes it as job data | `{}` | +| `WithJobData(string json)` | Sets raw JSON string as job data | `{}` | +| `WithTimeout(int seconds)` | Sets execution timeout; `0` disables timeout | `30` | +| `WithWorkerId(string id)` | Sets the worker ID in the execution context | `"local-test"` | +| `WithCancellationToken(CancellationToken)` | Passes a cancellation token | `None` | +| `WithLoggerFactory(ILoggerFactory)` | Captures logs through a custom logger | Console | + +All methods return the builder, so they can be chained freely. + +### Return Value: `JobExecutionResult` + +`RunAsync()` returns a `JobExecutionResult` record with the following fields: + +| Field | Type | Description | +|-------|------|-------------| +| `Status` | `JobOccurrenceStatus` | `Completed`, `Failed`, `Cancelled`, `TimedOut` | +| `DurationMs` | `long` | Wall-clock time of the job in milliseconds | +| `Exception` | `string` | Exception message and inner exception chain (if any) | +| `Result` | `string` | Serialized return value for `IJobWithResult` jobs | +| `Logs` | `List` | All log entries written via `context.Log*()` | +| `IsPermanentFailure` | `bool` | `true` if a `PermanentJobException` was thrown | +| `CorrelationId` | `Guid` | Correlation ID of this test run | + +--- + +## Basic Usage + +### Simplest Test + +```csharp +[Fact] +public async Task MyJob_ShouldComplete_WhenRunNormally() +{ + var result = await JobTestRunner + .For(new MyJob()) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Completed); + result.Exception.Should().BeNull(); +} +``` + +### With Typed Job Data + +When your job reads typed data via `context.GetData()`, pass it with `WithJobData()`: + +```csharp +[Fact] +public async Task SendInvoiceJob_ShouldComplete_WhenDataIsValid() +{ + var jobData = new InvoiceJobData + { + CustomerId = 42, + InvoiceId = 1001, + Currency = "USD" + }; + + var result = await JobTestRunner + .For(new SendInvoiceJob()) + .WithJobData(jobData) + .WithTimeout(60) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Completed); + result.Exception.Should().BeNull(); + result.DurationMs.Should().BeGreaterThan(0); +} +``` + +### With Raw JSON + +When you want full control over the raw payload: + +```csharp +[Fact] +public async Task MyJob_ShouldHandleMissingFields_Gracefully() +{ + var result = await JobTestRunner + .For(new MyJob()) + .WithJobData("""{"customerId": 99}""") + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Completed); +} +``` + +--- + +## Testing Failure Scenarios + +### Exception Handling + +```csharp +[Fact] +public async Task ProcessOrderJob_ShouldFail_WhenOrderNotFound() +{ + var result = await JobTestRunner + .For(new ProcessOrderJob()) + .WithJobData(new OrderJobData { OrderId = -1 }) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Failed); + result.Exception.Should().Contain("Order not found"); + result.IsPermanentFailure.Should().BeFalse(); +} +``` + +### Permanent Failures + +If your job throws `PermanentJobException`, the result marks the failure as permanent (no retry): + +```csharp +// In your job +throw new PermanentJobException("Invalid invoice data — will not retry."); +``` + +```csharp +// In your test +[Fact] +public async Task ProcessOrderJob_ShouldFailPermanently_WhenDataIsInvalid() +{ + var result = await JobTestRunner + .For(new ProcessOrderJob()) + .WithJobData(new OrderJobData { OrderId = 0 }) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Failed); + result.IsPermanentFailure.Should().BeTrue(); +} +``` + +--- + +## Testing Timeouts + +Use `WithTimeout()` to verify your job respects execution time limits: + +```csharp +[Fact] +public async Task LongRunningJob_ShouldTimeOut_WhenExceedsLimit() +{ + var result = await JobTestRunner + .For(new DataSyncJob()) + .WithTimeout(1) // 1 second — much shorter than the job's actual runtime + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.TimedOut); + result.Exception.Should().Contain("timeout"); +} +``` + +> **Note**: The timeout in tests uses the same `CancellationTokenSource` mechanism as production. If your job passes `context.CancellationToken` to all async operations, it will be cancelled correctly. + +--- + +## Testing Cancellation + +Test that your job stops cleanly when cancelled: + +```csharp +[Fact] +public async Task MyJob_ShouldCancel_WhenTokenCancelled() +{ + using var cts = new CancellationTokenSource(); + await cts.CancelAsync(); + + var result = await JobTestRunner + .For(new MyJob()) + .WithCancellationToken(cts.Token) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Cancelled); +} + +[Fact] +public async Task MyJob_ShouldCancelMidExecution_WhenTokenCancelledAfterDelay() +{ + using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(500)); + + var result = await JobTestRunner + .For(new MyJob()) + .WithCancellationToken(cts.Token) + .WithTimeout(0) // Disable runner timeout; only use the external token + .RunAsync(); + + result.Status.Should().BeOneOf(JobOccurrenceStatus.Cancelled, JobOccurrenceStatus.Completed); +} +``` + +--- + +## Testing Jobs with Return Values + +For jobs implementing `IAsyncJobWithResult` or `IJobWithResult`, the serialized result is available in `result.Result`: + +```csharp +[Fact] +public async Task GenerateReportJob_ShouldReturnFilePath_WhenSucceeds() +{ + var result = await JobTestRunner + .For(new GenerateReportJob()) + .WithJobData(new ReportJobData { ReportType = "monthly", Month = 6 }) + .WithTimeout(120) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Completed); + result.Result.Should().NotBeNullOrEmpty(); + result.Result.Should().Contain("monthly"); +} +``` + +--- + +## Testing Jobs with Dependency Injection + +When your job has constructor-injected services, build the job manually using a `ServiceCollection`: + +```csharp +[Fact] +public async Task SendInvoiceJob_ShouldSendEmail_WhenEmailServiceAvailable() +{ + // Arrange — wire up services + var services = new ServiceCollection(); + services.AddLogging(b => b.AddConsole()); + services.AddScoped(); // or a stub/mock + services.AddScoped(); + var sp = services.BuildServiceProvider(); + + // Resolve the job from DI (same as production) + var job = sp.GetRequiredService(); + + // Act + var result = await JobTestRunner + .For(job) + .WithJobData(new InvoiceJobData { InvoiceId = 1001 }) + .RunAsync(); + + // Assert + result.Status.Should().Be(JobOccurrenceStatus.Completed); +} +``` + +### Using Mocks + +For unit tests where you want to isolate the job from real services, use Moq: + +```csharp +[Fact] +public async Task SendInvoiceJob_ShouldCallEmailService_ExactlyOnce() +{ + // Arrange + var emailServiceMock = new Mock(); + emailServiceMock + .Setup(x => x.SendAsync(It.IsAny(), It.IsAny(), It.IsAny())) + .Returns(Task.CompletedTask); + + var job = new SendInvoiceJob(emailServiceMock.Object); + + // Act + var result = await JobTestRunner + .For(job) + .WithJobData(new InvoiceJobData { InvoiceId = 1001, RecipientEmail = "client@example.com" }) + .RunAsync(); + + // Assert + result.Status.Should().Be(JobOccurrenceStatus.Completed); + emailServiceMock.Verify( + x => x.SendAsync("client@example.com", It.IsAny(), It.IsAny()), + Times.Once); +} +``` + +--- + +## Capturing Logs in Tests + +By default `JobTestRunner` writes to the console. To capture log output in xUnit test output, supply a custom `ILoggerFactory`: + +```csharp +using Microsoft.Extensions.Logging; +using Xunit.Abstractions; + +public class MyJobTests(ITestOutputHelper output) +{ + private ILoggerFactory CreateLoggerFactory() => + LoggerFactory.Create(b => + { + b.SetMinimumLevel(LogLevel.Debug); + b.AddConsole(); + // Optional: pipe to xUnit output via a custom provider + }); + + [Fact] + public async Task MyJob_ShouldLogStartAndCompletion() + { + var result = await JobTestRunner + .For(new MyJob()) + .WithLoggerFactory(CreateLoggerFactory()) + .RunAsync(); + + // Logs written via context.Log*() are available in result.Logs + result.Logs.Should().Contain(l => l.Message.Contains("started")); + result.Logs.Should().Contain(l => l.Message.Contains("completed")); + } +} +``` + +> **Note**: `result.Logs` contains every entry written by `context.LogInformation()`, `context.LogWarning()`, and `context.LogError()` — the same entries that appear in the Milvaion dashboard occurrence detail view. + +--- + +## Recommended Test Structure + +Organize your test file to cover all meaningful scenarios for each job: + +``` +MyWorker.Tests/ +├── MyWorker.Tests.csproj +└── Jobs/ + ├── ProcessOrderJobTests.cs + ├── SendInvoiceJobTests.cs + └── DataSyncJobTests.cs +``` + +A typical test class per job: + +```csharp +public class ProcessOrderJobTests +{ + // ── Happy path ────────────────────────────────────────── + [Fact] + public async Task ShouldComplete_WhenOrderIsValid() { ... } + + // ── Job data edge cases ───────────────────────────────── + [Fact] + public async Task ShouldComplete_WhenJobDataIsNull() { ... } + + [Fact] + public async Task ShouldFail_WhenOrderIdIsNegative() { ... } + + // ── Resilience ────────────────────────────────────────── + [Fact] + public async Task ShouldFailPermanently_WhenOrderIsDuplicate() { ... } + + [Fact] + public async Task ShouldTimeOut_WhenExceedsTimeLimit() { ... } + + [Fact] + public async Task ShouldCancel_WhenTokenCancelled() { ... } +} +``` + +--- + +## Complete Example + +The `SampleWorker.Tests` project in the repository demonstrates the full pattern across multiple job types: + +```csharp +public class SampleJobTests +{ + private ILoggerFactory CreateLoggerFactory() => + LoggerFactory.Create(b => b.AddConsole().SetMinimumLevel(LogLevel.Debug)); + + [Fact] + public async Task TestJob_ShouldComplete_WhenRunNormally() + { + var result = await JobTestRunner + .For(new TestJob()) + .WithLoggerFactory(CreateLoggerFactory()) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Completed); + result.Exception.Should().BeNull(); + result.DurationMs.Should().BeGreaterThan(0); + } + + [Fact] + public async Task SampleSendEmailJob_ShouldComplete_WhenValidDataProvided() + { + var jobData = new EmailJobData + { + To = "test@example.com", + Subject = "Local test email", + Body = "Hello from local test!" + }; + + var result = await JobTestRunner + .For(new SampleSendEmailJob()) + .WithJobData(jobData) + .WithTimeout(120) + .WithLoggerFactory(CreateLoggerFactory()) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Completed); + } + + [Fact] + public async Task AlwaysFailingJob_ShouldFail_WithExceptionMessage() + { + var result = await JobTestRunner + .For(new AlwaysFailingJob()) + .WithLoggerFactory(CreateLoggerFactory()) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Failed); + result.Exception.Should().Contain("always fails"); + } + + [Fact] + public async Task HaveResultJob_ShouldReturnSerializedResult() + { + var result = await JobTestRunner + .For(new HaveResultJob()) + .WithLoggerFactory(CreateLoggerFactory()) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Completed); + result.Result.Should().Contain("Test Product"); + } + + [Fact] + public async Task TestJob_ShouldTimeOut_WhenTimeoutIsVeryShort() + { + var result = await JobTestRunner + .For(new TestJob()) + .WithTimeout(1) + .WithLoggerFactory(CreateLoggerFactory()) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.TimedOut); + } + + [Fact] + public async Task TestJob_ShouldBeCancelled_WhenTokenCancelled() + { + using var cts = new CancellationTokenSource(); + await cts.CancelAsync(); + + var result = await JobTestRunner + .For(new TestJob()) + .WithCancellationToken(cts.Token) + .WithLoggerFactory(CreateLoggerFactory()) + .RunAsync(); + + result.Status.Should().Be(JobOccurrenceStatus.Cancelled); + } +} +``` + +--- + +## What's Not Tested + +Local tests use `JobExecutor` directly and therefore do not cover: + +| Concern | Where to Test | +|---------|--------------| +| RabbitMQ message routing | Integration tests / Dev environment | +| Retry + DLQ behavior | Integration tests / Dev environment | +| Concurrent execution limits | Dev environment | +| Heartbeat / zombie detection | Dev environment | +| Dashboard visibility | Dev environment | +| Redis cancellation channel | Integration tests | + +--- + +## What's Next? + +- **[Implementing Jobs](05-implementing-jobs.md)** - Advanced job patterns: DI, error handling, long-running jobs +- **[Configuration](06-configuration.md)** - Timeout and retry configuration for production +- **[Reliability](08-reliability.md)** - Retry, DLQ, and permanent failure handling +- **[Monitoring](10-monitoring.md)** - Viewing job logs and occurrence details in the dashboard diff --git a/docs/portaldocs/24-api-keys.md b/docs/portaldocs/24-api-keys.md new file mode 100644 index 0000000..4a8f6cf --- /dev/null +++ b/docs/portaldocs/24-api-keys.md @@ -0,0 +1,120 @@ +--- +id: api-keys +title: "Api Keys" +sidebar_label: "Api Keys" +sidebar_position: 121 +description: "Create and manage api keys so CI pipelines, scripts and MCP clients can call the Milvaion API without an interactive login." +keywords: [milvaion api key, api authentication, service account, ci integration] +--- + +# Api Keys + +An api key is a non-interactive credential. It lets something that cannot log in through a browser — a CI pipeline, a deployment script, an MCP client — call the Milvaion API on its own. + +Keys carry the same permissions as users, drawn from the same catalog. A key is therefore never "admin by default": it can do exactly what you granted it and nothing else. + +## Creating a Key + +Go to **User Management → Api Keys → Create Api Key**. + +| Field | Notes | +|-------|-------| +| **Name** | How you will recognise it later. Name it after the thing that will hold it: `CI pipeline`, `Claude Code - Bugra`. | +| **Description** | Optional. Useful when somebody finds the key in six months and wonders what breaks if they revoke it. | +| **Expires** | 30 days, 90 days, 1 year, or never. Fixed at creation — see [Rotating a Key](#rotating-a-key). | +| **Permissions** | The same permission catalog used by roles. Grant the minimum. | + +:::danger The key is shown once + +Milvaion does not store the key, only a signed record of it. The create dialog is the only place it ever appears. If you lose it, revoke it and create another — there is no way to recover it. + +::: + +## Using a Key + +Send it in the `X-ApiKey` header: + +```bash +curl -X PATCH http://localhost:5000/api/v1/jobs \ + -H "Content-Type: application/json" \ + -H "X-ApiKey: $MILVAION_API_KEY" \ + -d '{"pageNumber": 1, "rowCount": 20}' +``` + +Every endpoint that accepts an interactive session also accepts an api key. The permission requirements are identical — a key without `ScheduledJobManagement.List` gets the same 403 a user without it would. + +The one exception is the api key endpoints themselves. Those require an interactive session, so a leaked key cannot mint replacements for itself and survive being revoked. + +## Choosing Permissions + +Grant the narrowest set that does the job. Two common shapes: + +**Read-only monitoring** — a dashboard, an alerting script, an [MCP client](./25-mcp-server.md) you want to be safe by construction: + +- `ScheduledJobManagement.List`, `ScheduledJobManagement.Detail` +- `FailedOccurrenceManagement.List` +- `WorkerManagement.List` +- `WorkflowManagement.List` + +**CI pipeline that triggers a job after deploy** — the above plus: + +- `ScheduledJobManagement.Trigger` + +Avoid granting `App.SuperAdmin` to a key. It bypasses every other check, which defeats the point of scoping the key at all. + +## Revoking and Deleting + +**Revoke** stops the key working immediately while keeping the record, so the audit trail still explains what the key was and who created it. This is what you want in almost every case, including a suspected leak. + +**Delete** removes the record entirely. Reserve it for housekeeping — keys created by mistake, or long-dead integrations. + +Revocation takes effect at once across every API replica. Authentication caches key lookups, but revoking invalidates that cache explicitly rather than waiting for it to expire. + +## Rotating a Key + +Expiry is fixed at creation, and the key material cannot be regenerated for an existing record. To rotate: + +1. Create a new key with the same permissions. +2. Deploy it wherever the old one is used. +3. Revoke the old key. + +The **Last Used** column tells you whether step 2 actually landed everywhere. If the old key is still being used, something you forgot is still holding it. + +## Rotating the Signing Secret + +All keys are signed with `MilvaionConfig.ApiKey.Secret`. Rotating that secret invalidates **every** key at once, so it is a deliberate, disruptive operation — do it if you believe the secret itself leaked. + +```json +{ + "MilvaionConfig": { + "ApiKey": { + "Secret": "your-new-secret", + "Version": 2 + } + } +} +``` + +Increment `Version` at the same time. Keys issued under an older version are then rejected with a clear message about a retired signing secret, instead of failing signature validation with no explanation. After rotating, re-create every key that is still in use. + +:::warning Keep the secret out of source control + +Anyone holding `ApiKey.Secret` can mint valid keys for any permission set. Supply it through an environment variable or a secret store in production: + +```bash +MilvaionConfig__ApiKey__Secret=... +``` + +::: + +## Auditing + +Key creation, update, revocation and deletion are recorded as user activities, visible under **User Management → Activity Logs**. + +Each key also tracks **Last Used**, written at most once every few minutes to avoid a database write per request. Treat it as "used recently", not as a precise access log. + +## Next Steps + +- **[MCP Server](./25-mcp-server.md)** — point an AI coding assistant at Milvaion using a read-only key +- **[Enterprise Features](./22-enterprise-features.md)** — roles, permissions and activity tracking +- **[Security](./12-security.md)** — hardening guidance for the whole deployment diff --git a/docs/portaldocs/25-mcp-server.md b/docs/portaldocs/25-mcp-server.md new file mode 100644 index 0000000..db4626e --- /dev/null +++ b/docs/portaldocs/25-mcp-server.md @@ -0,0 +1,395 @@ +--- +id: mcp-server +title: "MCP Server" +sidebar_label: "MCP Server" +sidebar_position: 122 +description: "Connect Claude Code, Cursor or GitHub Copilot to Milvaion over the Model Context Protocol and ask about jobs, failures and workers in plain language." +keywords: [milvaion mcp, model context protocol, claude code, cursor, copilot, ai job scheduler] +--- + +# MCP Server + +Milvaion exposes a [Model Context Protocol](https://modelcontextprotocol.io/) server at `/mcp`. Point an AI coding assistant at it and you can ask things like *"which jobs failed last night and why?"* instead of clicking through the dashboard. + +## Which Way Round Does This Work? + +This is the part people usually get backwards, so it is worth being explicit. + +```text +Your machine Your Milvaion server +┌────────────────────────┐ ┌──────────────────┐ +│ Claude Code / Cursor / │ │ Milvaion API │ +│ Copilot │ ──MCP──▶ │ /mcp │ +│ │ │ │ +│ • the model runs here │ ◀──JSON── │ • jobs │ +│ • your subscription │ │ • logs │ +│ • your tokens │ │ • metrics │ +└────────────────────────┘ └──────────────────┘ +``` + +**Milvaion never calls a language model.** It does not store an OpenAI, Anthropic or Google key, and it does not consume tokens. It is the data source; the model lives entirely in your editor, on your subscription. + +The only credential involved is a **Milvaion api key**, which your editor uses to authenticate to Milvaion. + +## Setup + +### 1. Create a read-only api key + +Follow [Api Keys](./24-api-keys.md) and grant only: + +- `ScheduledJobManagement.List`, `ScheduledJobManagement.Detail` +- `FailedOccurrenceManagement.List` +- `WorkerManagement.List` +- `WorkflowManagement.List` + +With this set the assistant can investigate anything and change nothing. Add `ScheduledJobManagement.Trigger` or `WorkflowManagement.Trigger` later if you want it to be able to run jobs. + +### 2. Register the server in your client + +Milvaion is a **remote HTTP** MCP server, so any client that supports streamable HTTP transport can connect. What differs between clients is the config file location and, annoyingly, the key names. + +| Client | Config location | Top-level key | URL field | +|--------|-----------------|---------------|-----------| +| Claude Code | `.mcp.json` or `claude mcp add` | `mcpServers` | `url` | +| Cursor | `.cursor/mcp.json` | `mcpServers` | `url` | +| VS Code / Copilot | `.vscode/mcp.json` | **`servers`** | `url` | +| Windsurf | `~/.codeium/windsurf/mcp_config.json` | `mcpServers` | **`serverUrl`** | +| Gemini CLI | `~/.gemini/settings.json` | `mcpServers` | `httpUrl` | +| Claude Desktop | Connectors UI | — | — | +| ChatGPT | Connectors UI | — | — | + +Getting `servers` and `mcpServers` the wrong way round, or `url` instead of `serverUrl`, is the single most common reason a client silently lists no tools. + +--- + +#### Claude Code + +No file editing needed: + +```bash +claude mcp add --transport http milvaion \ + https://milvaion.yourcompany.com/mcp \ + --header "X-ApiKey: $MILVAION_API_KEY" +``` + +Add `--scope project` to write it to `.mcp.json` and share it with the team. Verify with `claude mcp list`, or type `/mcp` inside a session to see the tool list. + +#### Cursor + +`.cursor/mcp.json` in the project, or the global equivalent from **Settings → MCP**: + +```json +{ + "mcpServers": { + "milvaion": { + "type": "http", + "url": "https://milvaion.yourcompany.com/mcp", + "headers": { + "X-ApiKey": "your-api-key" + } + } + } +} +``` + +#### VS Code / GitHub Copilot + +`.vscode/mcp.json`. Note the top-level key is `servers`, and `type` is required: + +```json +{ + "inputs": [ + { + "id": "milvaion-key", + "type": "promptString", + "description": "Milvaion api key", + "password": true + } + ], + "servers": { + "milvaion": { + "type": "http", + "url": "https://milvaion.yourcompany.com/mcp", + "headers": { + "X-ApiKey": "${input:milvaion-key}" + } + } + } +} +``` + +The `inputs` block makes VS Code prompt for the key at runtime instead of storing it in the file. + +:::warning Agent mode is required + +MCP tools do not appear in Copilot's default Ask mode. Switch the chat to **Agent** mode or the server will look connected while nothing can call it. + +::: + +#### Windsurf + +`~/.codeium/windsurf/mcp_config.json`. Windsurf uses `serverUrl`, not `url`: + +```json +{ + "mcpServers": { + "milvaion": { + "serverUrl": "https://milvaion.yourcompany.com/mcp", + "headers": { + "X-ApiKey": "your-api-key" + } + } + } +} +``` + +#### Gemini CLI + +`~/.gemini/settings.json` for all projects, or `.gemini/settings.json` for one. Gemini CLI expands `${VAR}` from your environment, so the key never has to sit in the file: + +```json +{ + "mcpServers": { + "milvaion": { + "httpUrl": "https://milvaion.yourcompany.com/mcp", + "headers": { + "X-ApiKey": "${MILVAION_API_KEY}" + } + } + } +} +``` + +Restart the CLI after editing. `includeTools` and `excludeTools` let you narrow the surface further on the client side — though scoping the api key is the stronger control, since it is enforced server side. + +#### Claude Desktop + +Claude Desktop's `claude_desktop_config.json` validates **stdio servers only**. Putting a `url` in it does nothing — the entry is ignored, which is why the tools never show up. + +Two options: + +1. **Connectors UI** — Settings → Connectors → Add custom connector, and paste the `/mcp` URL. This path requires **HTTPS**, so a `localhost` instance will not work. +2. **`mcp-remote` bridge** — run a local stdio process that forwards to your HTTP endpoint: + +```json +{ + "mcpServers": { + "milvaion": { + "command": "npx", + "args": [ + "-y", "mcp-remote", + "http://localhost:5000/mcp", + "--header", "X-ApiKey:${MILVAION_API_KEY}" + ], + "env": { "MILVAION_API_KEY": "your-api-key" } + } + } +} +``` + +For local development, Claude Code is far less friction than either of these. + +#### ChatGPT + +Settings → Connectors → **Advanced → Developer mode**, then add a connector pointing at your `/mcp` URL. + +:::warning ChatGPT limitations worth knowing before you try + +- **HTTPS only, remote only.** No stdio, and no `localhost` — expose the endpoint through a tunnel or a real domain first. +- **Write tools need a Business, Enterprise or Edu workspace.** On individual Plus and Pro plans, custom connectors are restricted to read and fetch operations, so `trigger_job` and anything else that changes state will not be callable. +- **The connector must be enabled per conversation.** Turning it on once in settings is not enough; most "it stopped working" reports are this. + +::: + +For a read-only Milvaion key the Plus/Pro restriction does not matter much, since the fifteen reading tools are the ones you would grant anyway. + +#### Google AI Studio + +AI Studio does not currently expose custom MCP connectors in the same way. For a Google-based workflow, use **Gemini CLI** as documented above. + +--- + +:::tip Keep the key out of the repository + +Every client above supports either environment variable expansion or a runtime prompt. Use it. A config file with a live key in it will eventually be committed, and a key in git history means creating a new one and revoking the old. + +::: + +### 3. Ask something + +> Which jobs failed last night? + +> Job `daily-invoice-export` has been failing since Tuesday. Read the logs and tell me what changed. + +> Is there a worker alive that can run `SendReportJob`? + +## Available Tools + +Every tool is gated by the permission in the right hand column. A key that lacks a permission cannot call the tool, so the tool surface an assistant actually sees is decided entirely by how you scope the key. + +### Reading + +| Tool | What it does | Permission | +|------|--------------|------------| +| `get_overview` | Dashboard statistics: counts, throughput, worker health | `ScheduledJobManagement.List` | +| `list_jobs` | Scheduled jobs, with free text search | `ScheduledJobManagement.List` | +| `get_job` | One job in full: schedule, worker, job data, retry and timeout settings | `ScheduledJobManagement.Detail` | +| `list_tags` | Distinct tags in use across jobs | `ScheduledJobManagement.List` | +| `list_occurrences` | Executions with status, duration and result | `ScheduledJobManagement.List` | +| `get_occurrence` | One execution with its logs and exception detail | `ScheduledJobManagement.Detail` | +| `list_failures` | Jobs that exhausted their retries and reached the dead letter queue | `FailedOccurrenceManagement.List` | +| `get_failure` | One dead letter record with exception and resolution notes | `FailedOccurrenceManagement.Detail` | +| `list_workers` | Workers with heartbeat status and executable job types | `WorkerManagement.List` | +| `get_worker` | One worker in full | `WorkerManagement.Detail` | +| `list_workflows` | Workflows | `WorkflowManagement.List` | +| `get_workflow` | One workflow with its step graph | `WorkflowManagement.Detail` | +| `list_workflow_runs` | Workflow runs and their status | `WorkflowManagement.List` | +| `get_workflow_run` | One run with per step outcomes | `WorkflowManagement.Detail` | +| `list_activity_logs` | Who changed what, and when | `ActivityLogManagement.List` | +| `list_reports` | Metric reports produced by the reporter worker | `ScheduledJobManagement.List` | +| `get_latest_report` | Newest report of a metric type, with its data | `ScheduledJobManagement.Detail` | +| `get_report` | One report by id, for comparing against an older one | `ScheduledJobManagement.Detail` | +| `get_system_health` | Health of the scheduler, database, Redis and RabbitMQ | `SystemAdministration.List` | +| `get_queue_stats` | RabbitMQ queue depths and consumer counts | `SystemAdministration.List` | +| `get_queue_info` | Depth detail for one queue | `SystemAdministration.List` | +| `get_job_statistics` | Aggregate counters across jobs and executions | `SystemAdministration.List` | +| `get_database_statistics` | Size, index efficiency, cache hit ratio, bloat | `SystemAdministration.List` | +| `get_configuration` | Effective scheduler configuration | `SystemAdministration.List` | +| `list_notifications` | Alerts Milvaion itself raised | `InternalNotificationManagement.List` | +| `list_permissions` | The permission catalog, for explaining what to grant | `PermissionManagement.List` | + +### Running and pausing + +| Tool | What it does | Permission | +|------|--------------|------------| +| `trigger_job` | Runs a job immediately, outside its schedule | `ScheduledJobManagement.Trigger` | +| `cancel_occurrence` | Cancels a running execution | `ScheduledJobManagement.Trigger` | +| `set_job_active` | Pauses or resumes a job without deleting it | `ScheduledJobManagement.Update` | +| `trigger_workflow` | Starts a workflow run immediately | `WorkflowManagement.Trigger` | +| `cancel_workflow_run` | Cancels a run in progress | `WorkflowManagement.Trigger` | + +### Editing + +| Tool | What it does | Permission | +|------|--------------|------------| +| `create_job` | Creates a scheduled job | `ScheduledJobManagement.Create` | +| `update_job` | Changes name, description, tags, cron, job data or timeout | `ScheduledJobManagement.Update` | +| `resolve_failures` | Marks dead letter records resolved with a note | `FailedOccurrenceManagement.Update` | + +### Deleting + +| Tool | What it does | Permission | +|------|--------------|------------| +| `delete_job` | Deletes a job and its history | `ScheduledJobManagement.Delete` | +| `delete_occurrences` | Deletes execution records | `ScheduledJobManagement.Delete` | +| `delete_failures` | Deletes dead letter records | `FailedOccurrenceManagement.Delete` | +| `delete_worker` | Removes a worker registration | `WorkerManagement.Delete` | +| `delete_workflow` | Deletes a workflow and its run history | `WorkflowManagement.Delete` | + +### Filtering + +The list tools filter rather than make the assistant page through everything: + +| Tool | Filters | +|------|---------| +| `list_jobs` | `tag`, `workerId`, `isActive`, free text | +| `list_occurrences` | `jobId`, `status`, `workerId`, `since`, `until`, free text | +| `list_failures` | `jobId`, `resolved`, `since`, `until`, free text | +| `list_activity_logs` | `since`, `until` | +| `list_workflow_runs` | `workflowId` | + +All times are UTC. `since` on its own is the common case — "what failed since midnight" needs one bound, not two. + +`get_occurrence` returns the tail of the log rather than all of it, defaulting to the last 100 lines with a `logLines` parameter to raise it. A job that logs in a loop can otherwise fill an assistant's entire context in one call. + +### Prompts + +Three prompt templates ship with the server, selectable in clients that support them: + +| Prompt | What it does | +|--------|--------------| +| `diagnose_job` | Walks a failing job in order: failures, pattern, logs, worker health, then recent config changes | +| `overnight_review` | Reviews a recent window and groups failures by cause rather than listing them | +| `explain_workflow` | Describes a workflow's steps, branching and data flow in plain language | + +These matter more than they look. Left to itself a model tends to start with whichever tool it read first; the prompts put a working order in front of it. + +### Not exposed + +Users, roles, permissions, api keys, dispatcher control, configuration and content management have no tools and are not reachable over MCP, regardless of what the key is granted. Workflow creation and editing are also absent: authoring a directed graph with conditions and data mappings belongs in the visual builder, not in a tool call. + +:::tip Scope the key, not the tool list + +You do not choose which tools to expose — you choose what the key can do. Granting only `List` and `Detail` permissions leaves an assistant with the fifteen reading tools and nothing else, which is the right default for most people. + +::: + +## Security Model + +**Authentication happens at the endpoint.** `/mcp` requires a valid credential; an anonymous request never reaches a tool. + +**Authorization happens per tool.** Each tool checks its own permission before doing any work. A key without `ScheduledJobManagement.Trigger` calling `trigger_job` gets an error naming the missing permission — the assistant is told exactly what it lacks, so it stops retrying and can tell you what to grant. + +**Side effects are attributed.** Anything triggered, cancelled or resolved through MCP is recorded with the key's name, so the history distinguishes it from a human acting in the dashboard. + +**Concurrency policies are never bypassed.** `trigger_job` always dispatches with force disabled. Overriding a job's concurrency policy stays a deliberate human action in the dashboard. + +**Fields cannot be cleared by accident.** `update_job` only changes the arguments it is given; an omitted argument is left alone rather than blanked. The trade-off is that clearing a field on purpose has to be done in the dashboard, which is the safer way round. + +**Tools declare what they do.** Every tool carries the MCP annotations clients use to decide whether to prompt: reading tools are marked read-only, `delete_job` and the other four deletions are marked destructive, and the `set_*` and `update_*` tools are marked idempotent. A client that offers "always allow" for a tool can then treat `list_jobs` and `delete_job` differently, which without these it cannot. + +:::warning Treat write permissions as production access + +`Trigger` lets an assistant cause real work to run. `Update` lets it change when your jobs run. `Delete` is irreversible. The tool descriptions tell the model to confirm before doing any of this, and to prefer `set_job_active` over `delete_job` — but those are prompts, not guarantees. + +Grant `Create`, `Update` and `Delete` only where you would be comfortable with the same person having dashboard access. For everything else, a read-only key is the right answer. + +::: + +## Verifying It Works + +`/mcp` speaks JSON-RPC over HTTP, so you can exercise it with `curl` before involving a client. + +List the tools: + +```bash +curl -X POST http://localhost:5000/mcp \ + -H "Content-Type: application/json" \ + -H "Accept: application/json, text/event-stream" \ + -H "X-ApiKey: $MILVAION_API_KEY" \ + -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' +``` + +Call one: + +```bash +curl -X POST http://localhost:5000/mcp \ + -H "Content-Type: application/json" \ + -H "Accept: application/json, text/event-stream" \ + -H "X-ApiKey: $MILVAION_API_KEY" \ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_overview","arguments":{}}}' +``` + +Without the header the same request returns 401, which is the quickest confirmation that the endpoint is protected. + +## Troubleshooting + +| Symptom | Cause | +|---------|-------| +| **401 Unauthorized** | Missing or malformed `X-ApiKey` header, or the key was revoked or expired. | +| **Api key was issued with a retired signing secret** | `ApiKey.Version` was incremented after the key was created. Create a new key. | +| **"does not have the '...' permission"** | Working as intended. Grant that permission to the key, or use a different key. | +| **404 on /mcp** | The API predates MCP support, or a reverse proxy is not forwarding the path. | +| **Client connects but lists no tools** | Usually a proxy stripping the `Accept: text/event-stream` header. | + +## Deployment Notes + +The server runs in **stateless** mode, so any API replica can serve any request and no sticky sessions are needed behind a load balancer. Server-initiated features that depend on a persistent session — sampling, elicitation, roots — are therefore not available. + +If Milvaion is behind a reverse proxy, make sure `/mcp` is forwarded and that `Content-Type` and `Accept` headers survive the hop. + +## Next Steps + +- **[Api Keys](./24-api-keys.md)** — creating and scoping the credential this needs +- **[Monitoring](./10-monitoring.md)** — the metrics behind `get_overview` +- **[Workflows](./20-workflows.md)** — what `list_workflows` is showing you diff --git a/dotnet-tools.json b/dotnet-tools.json new file mode 100644 index 0000000..b0e38ab --- /dev/null +++ b/dotnet-tools.json @@ -0,0 +1,5 @@ +{ + "version": 1, + "isRoot": true, + "tools": {} +} \ No newline at end of file diff --git a/for multi-arch support b/for multi-arch support deleted file mode 100644 index 1250959..0000000 --- a/for multi-arch support +++ /dev/null @@ -1,314 +0,0 @@ - - SSUUMMMMAARRYY OOFF LLEESSSS CCOOMMMMAANNDDSS - - Commands marked with * may be preceded by a number, _N. - Notes in parentheses indicate the behavior if _N is given. - A key preceded by a caret indicates the Ctrl key; thus ^K is ctrl-K. - - h H Display this help. - q :q Q :Q ZZ Exit. - --------------------------------------------------------------------------- - - MMOOVVIINNGG - - e ^E j ^N CR * Forward one line (or _N lines). - y ^Y k ^K ^P * Backward one line (or _N lines). - f ^F ^V SPACE * Forward one window (or _N lines). - b ^B ESC-v * Backward one window (or _N lines). - z * Forward one window (and set window to _N). - w * Backward one window (and set window to _N). - ESC-SPACE * Forward one window, but don't stop at end-of-file. - d ^D * Forward one half-window (and set half-window to _N). - u ^U * Backward one half-window (and set half-window to _N). - ESC-) RightArrow * Right one half screen width (or _N positions). - ESC-( LeftArrow * Left one half screen width (or _N positions). - ESC-} ^RightArrow Right to last column displayed. - ESC-{ ^LeftArrow Left to first column. - F Forward forever; like "tail -f". - ESC-F Like F but stop when search pattern is found. - r ^R ^L Repaint screen. - R Repaint screen, discarding buffered input. - --------------------------------------------------- - Default "window" is the screen height. - Default "half-window" is half of the screen height. - --------------------------------------------------------------------------- - - SSEEAARRCCHHIINNGG - - /_p_a_t_t_e_r_n * Search forward for (_N-th) matching line. - ?_p_a_t_t_e_r_n * Search backward for (_N-th) matching line. - n * Repeat previous search (for _N-th occurrence). - N * Repeat previous search in reverse direction. - ESC-n * Repeat previous search, spanning files. - ESC-N * Repeat previous search, reverse dir. & spanning files. - ^O^N ^On * Search forward for (_N-th) OSC8 hyperlink. - ^O^P ^Op * Search backward for (_N-th) OSC8 hyperlink. - ^O^L ^Ol Jump to the currently selected OSC8 hyperlink. - ESC-u Undo (toggle) search highlighting. - ESC-U Clear search highlighting. - &_p_a_t_t_e_r_n * Display only matching lines. - --------------------------------------------------- - A search pattern may begin with one or more of: - ^N or ! Search for NON-matching lines. - ^E or * Search multiple files (pass thru END OF FILE). - ^F or @ Start search at FIRST file (for /) or last file (for ?). - ^K Highlight matches, but don't move (KEEP position). - ^R Don't use REGULAR EXPRESSIONS. - ^S _n Search for match in _n-th parenthesized subpattern. - ^W WRAP search if no match found. - ^L Enter next character literally into pattern. - --------------------------------------------------------------------------- - - JJUUMMPPIINNGG - - g < ESC-< * Go to first line in file (or line _N). - G > ESC-> * Go to last line in file (or line _N). - p % * Go to beginning of file (or _N percent into file). - t * Go to the (_N-th) next tag. - T * Go to the (_N-th) previous tag. - { ( [ * Find close bracket } ) ]. - } ) ] * Find open bracket { ( [. - ESC-^F _<_c_1_> _<_c_2_> * Find close bracket _<_c_2_>. - ESC-^B _<_c_1_> _<_c_2_> * Find open bracket _<_c_1_>. - --------------------------------------------------- - Each "find close bracket" command goes forward to the close bracket - matching the (_N-th) open bracket in the top line. - Each "find open bracket" command goes backward to the open bracket - matching the (_N-th) close bracket in the bottom line. - - m_<_l_e_t_t_e_r_> Mark the current top line with . - M_<_l_e_t_t_e_r_> Mark the current bottom line with . - '_<_l_e_t_t_e_r_> Go to a previously marked position. - '' Go to the previous position. - ^X^X Same as '. - ESC-m_<_l_e_t_t_e_r_> Clear a mark. - --------------------------------------------------- - A mark is any upper-case or lower-case letter. - Certain marks are predefined: - ^ means beginning of the file - $ means end of the file - --------------------------------------------------------------------------- - - CCHHAANNGGIINNGG FFIILLEESS - - :e [_f_i_l_e] Examine a new file. - ^X^V Same as :e. - :n * Examine the (_N-th) next file from the command line. - :p * Examine the (_N-th) previous file from the command line. - :x * Examine the first (or _N-th) file from the command line. - ^O^O Open the currently selected OSC8 hyperlink. - :d Delete the current file from the command line list. - = ^G :f Print current file name. - --------------------------------------------------------------------------- - - MMIISSCCEELLLLAANNEEOOUUSS CCOOMMMMAANNDDSS - - -_<_f_l_a_g_> Toggle a command line option [see OPTIONS below]. - --_<_n_a_m_e_> Toggle a command line option, by name. - __<_f_l_a_g_> Display the setting of a command line option. - ___<_n_a_m_e_> Display the setting of an option, by name. - +_c_m_d Execute the less cmd each time a new file is examined. - - !_c_o_m_m_a_n_d Execute the shell command with $SHELL. - #_c_o_m_m_a_n_d Execute the shell command, expanded like a prompt. - |XX_c_o_m_m_a_n_d Pipe file between current pos & mark XX to shell command. - s _f_i_l_e Save input to a file. - v Edit the current file with $VISUAL or $EDITOR. - V Print version number of "less". - --------------------------------------------------------------------------- - - OOPPTTIIOONNSS - - Most options may be changed either on the command line, - or from within less by using the - or -- command. - Options may be given in one of two forms: either a single - character preceded by a -, or a name preceded by --. - - -? ........ --help - Display help (from command line). - -a ........ --search-skip-screen - Search skips current screen. - -A ........ --SEARCH-SKIP-SCREEN - Search starts just after target line. - -b [_N] .... --buffers=[_N] - Number of buffers. - -B ........ --auto-buffers - Don't automatically allocate buffers for pipes. - -c ........ --clear-screen - Repaint by clearing rather than scrolling. - -d ........ --dumb - Dumb terminal. - -D xx_c_o_l_o_r . --color=xx_c_o_l_o_r - Set screen colors. - -e -E .... --quit-at-eof --QUIT-AT-EOF - Quit at end of file. - -f ........ --force - Force open non-regular files. - -F ........ --quit-if-one-screen - Quit if entire file fits on first screen. - -g ........ --hilite-search - Highlight only last match for searches. - -G ........ --HILITE-SEARCH - Don't highlight any matches for searches. - -h [_N] .... --max-back-scroll=[_N] - Backward scroll limit. - -i ........ --ignore-case - Ignore case in searches that do not contain uppercase. - -I ........ --IGNORE-CASE - Ignore case in all searches. - -j [_N] .... --jump-target=[_N] - Screen position of target lines. - -J ........ --status-column - Display a status column at left edge of screen. - -k _f_i_l_e ... --lesskey-file=_f_i_l_e - Use a compiled lesskey file. - -K ........ --quit-on-intr - Exit less in response to ctrl-C. - -L ........ --no-lessopen - Ignore the LESSOPEN environment variable. - -m -M .... --long-prompt --LONG-PROMPT - Set prompt style. - -n ......... --line-numbers - Suppress line numbers in prompts and messages. - -N ......... --LINE-NUMBERS - Display line number at start of each line. - -o [_f_i_l_e] .. --log-file=[_f_i_l_e] - Copy to log file (standard input only). - -O [_f_i_l_e] .. --LOG-FILE=[_f_i_l_e] - Copy to log file (unconditionally overwrite). - -p _p_a_t_t_e_r_n . --pattern=[_p_a_t_t_e_r_n] - Start at pattern (from command line). - -P [_p_r_o_m_p_t] --prompt=[_p_r_o_m_p_t] - Define new prompt. - -q -Q .... --quiet --QUIET --silent --SILENT - Quiet the terminal bell. - -r -R .... --raw-control-chars --RAW-CONTROL-CHARS - Output "raw" control characters. - -s ........ --squeeze-blank-lines - Squeeze multiple blank lines. - -S ........ --chop-long-lines - Chop (truncate) long lines rather than wrapping. - -t _t_a_g .... --tag=[_t_a_g] - Find a tag. - -T [_t_a_g_s_f_i_l_e] --tag-file=[_t_a_g_s_f_i_l_e] - Use an alternate tags file. - -u -U .... --underline-special --UNDERLINE-SPECIAL - Change handling of backspaces, tabs and carriage returns. - -V ........ --version - Display the version number of "less". - -w ........ --hilite-unread - Highlight first new line after forward-screen. - -W ........ --HILITE-UNREAD - Highlight first new line after any forward movement. - -x [_N[,...]] --tabs=[_N[,...]] - Set tab stops. - -X ........ --no-init - Don't use termcap init/deinit strings. - -y [_N] .... --max-forw-scroll=[_N] - Forward scroll limit. - -z [_N] .... --window=[_N] - Set size of window. - -" [_c[_c]] . --quotes=[_c[_c]] - Set shell quote characters. - -~ ........ --tilde - Don't display tildes after end of file. - -# [_N] .... --shift=[_N] - Set horizontal scroll amount (0 = one half screen width). - - --exit-follow-on-close - Exit F command on a pipe when writer closes pipe. - --file-size - Automatically determine the size of the input file. - --follow-name - The F command changes files if the input file is renamed. - --header=[_L[,_C[,_N]]] - Use _L lines (starting at line _N) and _C columns as headers. - --incsearch - Search file as each pattern character is typed in. - --intr=[_C] - Use _C instead of ^X to interrupt a read. - --lesskey-context=_t_e_x_t - Use lesskey source file contents. - --lesskey-src=_f_i_l_e - Use a lesskey source file. - --line-num-width=[_N] - Set the width of the -N line number field to _N characters. - --match-shift=[_N] - Show at least _N characters to the left of a search match. - --modelines=[_N] - Read _N lines from the input file and look for vim modelines. - --mouse - Enable mouse input. - --no-keypad - Don't send termcap keypad init/deinit strings. - --no-histdups - Remove duplicates from command history. - --no-number-headers - Don't give line numbers to header lines. - --no-search-header-lines - Searches do not include header lines. - --no-search-header-columns - Searches do not include header columns. - --no-search-headers - Searches do not include header lines or columns. - --no-vbell - Disable the terminal's visual bell. - --redraw-on-quit - Redraw final screen when quitting. - --rscroll=[_C] - Set the character used to mark truncated lines. - --save-marks - Retain marks across invocations of less. - --search-options=[EFKNRW-] - Set default options for every search. - --show-preproc-errors - Display a message if preprocessor exits with an error status. - --proc-backspace - Process backspaces for bold/underline. - --PROC-BACKSPACE - Treat backspaces as control characters. - --proc-return - Delete carriage returns before newline. - --PROC-RETURN - Treat carriage returns as control characters. - --proc-tab - Expand tabs to spaces. - --PROC-TAB - Treat tabs as control characters. - --status-col-width=[_N] - Set the width of the -J status column to _N characters. - --status-line - Highlight or color the entire line containing a mark. - --use-backslash - Subsequent options use backslash as escape char. - --use-color - Enables colored text. - --wheel-lines=[_N] - Each click of the mouse wheel moves _N lines. - --wordwrap - Wrap lines at spaces. - - - --------------------------------------------------------------------------- - - LLIINNEE EEDDIITTIINNGG - - These keys can be used to edit text being entered - on the "command line" at the bottom of the screen. - - RightArrow ..................... ESC-l ... Move cursor right one character. - LeftArrow ...................... ESC-h ... Move cursor left one character. - ctrl-RightArrow ESC-RightArrow ESC-w ... Move cursor right one word. - ctrl-LeftArrow ESC-LeftArrow ESC-b ... Move cursor left one word. - HOME ........................... ESC-0 ... Move cursor to start of line. - END ............................ ESC-$ ... Move cursor to end of line. - BACKSPACE ................................ Delete char to left of cursor. - DELETE ......................... ESC-x ... Delete char under cursor. - ctrl-BACKSPACE ESC-BACKSPACE ........... Delete word to left of cursor. - ctrl-DELETE .... ESC-DELETE .... ESC-X ... Delete word under cursor. - ctrl-U ......... ESC (MS-DOS only) ....... Delete entire line. - UpArrow ........................ ESC-k ... Retrieve previous command line. - DownArrow ...................... ESC-j ... Retrieve next command line. - TAB ...................................... Complete filename & cycle. - SHIFT-TAB ...................... ESC-TAB Complete filename & reverse cycle. - ctrl-L ................................... Complete filename, list all. diff --git a/src/Milvaion.Api/AppStartup/ApplicationBuilderExtensions.cs b/src/Milvaion.Api/AppStartup/ApplicationBuilderExtensions.cs index 666eb88..9d701de 100644 --- a/src/Milvaion.Api/AppStartup/ApplicationBuilderExtensions.cs +++ b/src/Milvaion.Api/AppStartup/ApplicationBuilderExtensions.cs @@ -1,9 +1,6 @@ using Microsoft.AspNetCore.Authentication.JwtBearer; using Microsoft.AspNetCore.Localization; -using Milvaion.Application.Utils.Constants; using Milvaion.Application.Utils.Extensions; -using Milvaion.Application.Utils.Models.Options; -using Milvaion.Application.Utils.PermissionManager; using Milvaion.Domain.Enums; using Milvasoft.Core.MultiLanguage.Manager; using Milvasoft.Identity.Concrete.Options; diff --git a/src/Milvaion.Api/AppStartup/McpExtensions.cs b/src/Milvaion.Api/AppStartup/McpExtensions.cs new file mode 100644 index 0000000..766a3a8 --- /dev/null +++ b/src/Milvaion.Api/AppStartup/McpExtensions.cs @@ -0,0 +1,73 @@ +using Milvaion.Api.Mcp; + +namespace Milvaion.Api.AppStartup; + +/// +/// Registration and mapping for the Milvaion MCP server. +/// +public static class McpExtensions +{ + /// + /// Route the MCP endpoint is served from. + /// + public const string McpRoute = "/mcp"; + + /// + /// Adds the MCP server and its tools. + /// + public static IServiceCollection AddMilvaionMcp(this IServiceCollection services) + { + services.AddHttpContextAccessor(); + services.AddScoped(); + + services.AddMcpServer(options => + { + options.ServerInfo = new() + { + Name = "milvaion", + Version = typeof(McpExtensions).Assembly.GetName().Version?.ToString() ?? "1.0.0" + }; + + options.ServerInstructions = + "Milvaion is a distributed job scheduler. Jobs are definitions with a schedule; occurrences are " + + "individual executions of those jobs. When investigating a problem, start with list_failures " + + "rather than list_occurrences - it is a much smaller set of jobs that exhausted their retries. " + + "Use get_occurrence to read the logs and exception for a specific execution. " + + "trigger_job and trigger_workflow cause real work to run in the user's environment; ask before " + + "calling them. " + + "Tools whose id parameter is a list - delete_failures, resolve_failures, delete_occurrences - are " + + "bulk operations: collect every id first and make one call, never a call per record."; + }) + .WithHttpTransport(options => + { + // Stateless keeps any API replica able to serve any request, which matters because Milvaion is + // routinely run behind a load balancer with several API instances. It also rules out + // server-to-client requests such as sampling, which none of these tools need. + options.Stateless = true; + }) + // The assembly must be passed explicitly. The parameterless overload scans Assembly.GetEntryAssembly(), + // which is this API when it runs normally but the test host when it runs under WebApplicationFactory - + // so tool discovery either finds nothing or fails outright, and the failure surfaces as the host never + // being built. + .WithToolsFromAssembly(typeof(McpExtensions).Assembly) + .WithPromptsFromAssembly(typeof(McpExtensions).Assembly); + + return services; + } + + /// + /// Maps the MCP endpoint. + /// + /// + /// Authorization is required at the endpoint, so an anonymous caller never reaches a tool. The default policy + /// accepts both the login token and an api key, which means a developer can point an MCP client at this + /// endpoint with nothing more than a key created in the dashboard. Per-permission checks happen inside the + /// tools themselves - see . + /// + public static WebApplication MapMilvaionMcp(this WebApplication app) + { + app.MapMcp(McpRoute).RequireAuthorization(); + + return app; + } +} diff --git a/src/Milvaion.Api/AppStartup/MissingResxKeyFinder.cs b/src/Milvaion.Api/AppStartup/MissingResxKeyFinder.cs index dda7c72..4f885a4 100644 --- a/src/Milvaion.Api/AppStartup/MissingResxKeyFinder.cs +++ b/src/Milvaion.Api/AppStartup/MissingResxKeyFinder.cs @@ -1,8 +1,5 @@ using Bogus.DataSets; using Milvaion.Application; -using Milvaion.Application.Utils.Constants; -using Milvaion.Application.Utils.PermissionManager; -using Milvaion.Domain; using Milvaion.Domain.Enums; using Milvasoft.Core.Helpers; using System.Reflection; @@ -19,18 +16,45 @@ public static partial class MissingResxKeyFinder /// /// Finds missing keys in .resx files. /// + /// + /// A development-time convenience, called before the host is built. It must never throw: anything escaping + /// here kills the entry point before an IHost exists, which surfaces as the unhelpful + /// "The entry point exited without ever building an IHost" and takes the whole application - or the whole + /// integration test suite - down with it. + /// + /// The paths are resolved from the current working directory, which is the API project only when it is run + /// directly. Under a test host it points somewhere else entirely, so the folders legitimately do not exist. + /// + /// public static void FindAndPrintToConsole() { #if !DEBUG return; #endif - string projectFolderPath = Directory.GetParent(Directory.GetCurrentDirectory()).FullName; + string projectFolderPath = Directory.GetParent(Directory.GetCurrentDirectory())?.FullName; string resxFolderProjectPath = Directory.GetCurrentDirectory(); string resxFolderPath = Path.Combine(resxFolderProjectPath, "LocalizationResources", "Resources"); - if (string.IsNullOrWhiteSpace(projectFolderPath) || string.IsNullOrWhiteSpace(resxFolderPath)) + // Existence, not just non-emptiness: the reader throws DirectoryNotFoundException on a missing folder. + if (string.IsNullOrWhiteSpace(projectFolderPath) || !Directory.Exists(projectFolderPath) || !Directory.Exists(resxFolderPath)) return; + try + { + RunCheck(projectFolderPath, resxFolderPath); + } + catch (Exception ex) + { + // Report and carry on. A broken resx check is never a reason to stop the application starting. + Console.ForegroundColor = ConsoleColor.Yellow; + Console.WriteLine($"Resx key check skipped: {ex.Message}"); + Console.ResetColor(); + } + } + + private static void RunCheck(string projectFolderPath, string resxFolderPath) + { + var nameofReferences = FindNameofReferencesInLocalizer(projectFolderPath); HashSet keysInNameOfReferences = []; diff --git a/src/Milvaion.Api/AppStartup/Program.cs b/src/Milvaion.Api/AppStartup/Program.cs index dcc09cb..03d86c8 100644 --- a/src/Milvaion.Api/AppStartup/Program.cs +++ b/src/Milvaion.Api/AppStartup/Program.cs @@ -1,13 +1,11 @@ -using Milvaion.Api; +using Microsoft.Extensions.Options; +using Milvaion.Api; using Milvaion.Api.AppStartup; using Milvaion.Api.Hubs; using Milvaion.Api.Middlewares; using Milvaion.Api.Migrations; using Milvaion.Application; -using Milvaion.Application.Utils.Constants; using Milvaion.Application.Utils.LinkedWithFormatters; -using Milvaion.Application.Utils.Models.Options; -using Milvaion.Domain; using Milvaion.Infrastructure; using Milvaion.Infrastructure.Services.RabbitMQ; using Milvasoft.Components.Rest; @@ -17,6 +15,8 @@ try { + MissingResxKeyFinder.FindAndPrintToConsole(); + var builder = WebApplication.CreateBuilder(new WebApplicationOptions { WebRootPath = GlobalConstant.WWWRoot @@ -31,9 +31,9 @@ // Add services to the container. var services = builder.Services; - var fineConfig = builder.Configuration.GetSection(nameof(MilvaionConfig)).Get(); + var config = builder.Configuration.GetSection(nameof(MilvaionConfig)).Get(); - services.AddSingleton(fineConfig); + services.AddSingleton(config); services.AddControllers().AddApplicationPart(PresentationAssembly.Assembly); @@ -45,6 +45,8 @@ services.AddAuthorization(builder.Configuration); + services.AddMilvaionMcp(); + services.AddHttpContextAccessor(); services.AddMultiLanguageSupport(builder.Configuration); @@ -82,6 +84,19 @@ // Configure the HTTP request pipeline. var app = builder.Build(); + // PathBase support: when BasePath is configured (e.g. "/milvaion"), all middleware and endpoints (controllers, hubs, static files, SPA fallback) are automatically scoped to it. + // UseRouting() MUST be called explicitly right after UsePathBase so that route matching happens on the already-stripped path, not the full original path. + if (!string.IsNullOrWhiteSpace(config?.BasePath)) + { + app.UsePathBase(config.BasePath); + app.UseRouting(); + } + + var effectiveBasePath = string.IsNullOrWhiteSpace(config?.BasePath) ? "/" : config.BasePath; + + if (app.Logger.IsEnabled(LogLevel.Information)) + app.Logger.LogInformation("Milvaion starting on base path: {BasePath}", effectiveBasePath); + app.UseCorsFromConfiguration(builder.Configuration); #region Configure @@ -92,7 +107,7 @@ // Prometheus metrics endpoint - /api/metrics app.UsePrometheusMetrics(builder.Configuration); - // Serve static files (wwwroot) - Use Microsoft.AspNetCore.Builder extension directly + // Serve static files (wwwroot) StaticFileExtensions.UseStaticFiles(app, new StaticFileOptions { OnPrepareResponse = ctx => @@ -118,20 +133,39 @@ // Map SignalR hub app.MapHub("/hubs/jobs"); + // Must come before MapFallbackToFile, otherwise the SPA fallback would swallow /mcp. + app.MapMilvaionMcp(); + app.UseScalarWithOpenApi(); // SPA Fallback - Serve React app for all non-API routes // Must be LAST (after MapControllers, MapHub, etc.) app.MapFallbackToFile("index.html"); + // Runtime config endpoint — serves base path and other boot-time settings to the SPA. + // Must be reachable BEFORE authentication so the browser can load it as a plain diff --git a/src/MilvaionUI/src/App.jsx b/src/MilvaionUI/src/App.jsx index 83acdfa..5e62dd4 100644 --- a/src/MilvaionUI/src/App.jsx +++ b/src/MilvaionUI/src/App.jsx @@ -17,6 +17,7 @@ import FailedOccurrenceList from './pages/FailedOccurrences/FailedOccurrenceList import FailedOccurrenceDetail from './pages/FailedOccurrences/FailedOccurrenceDetail' import UserList from './pages/UserManagement/UserList' import RoleList from './pages/UserManagement/RoleList' +import ApiKeyList from './pages/UserManagement/ApiKeyList' import ActivityLogList from './pages/UserManagement/ActivityLogList' import Profile from './pages/Profile/Profile' import WorkflowList from './pages/Workflows/WorkflowList' @@ -37,9 +38,10 @@ import WorkflowStepBottleneckReport from './pages/Reports/WorkflowStepBottleneck import WorkflowDurationTrendReport from './pages/Reports/WorkflowDurationTrendReport' function App() { + const basename = (window.__MILVAION_CONFIG__?.basePath ?? import.meta.env.VITE_BASE_PATH ?? '/') || '/' return ( - + } /> @@ -65,6 +67,7 @@ function App() { } /> } /> } /> + } /> } /> } /> } /> diff --git a/src/MilvaionUI/src/components/AuditInfoCard.css b/src/MilvaionUI/src/components/AuditInfoCard.css index 120c0a9..5dead49 100644 --- a/src/MilvaionUI/src/components/AuditInfoCard.css +++ b/src/MilvaionUI/src/components/AuditInfoCard.css @@ -23,7 +23,7 @@ } .audit-info-btn:hover { - color: var(--accent-color); + color: var(--accent-text); border-color: var(--accent-color); background-color: var(--bg-hover); } diff --git a/src/MilvaionUI/src/components/DatabaseStatistics/DatabaseStatistics.css b/src/MilvaionUI/src/components/DatabaseStatistics/DatabaseStatistics.css index 86f04ab..b00f403 100644 --- a/src/MilvaionUI/src/components/DatabaseStatistics/DatabaseStatistics.css +++ b/src/MilvaionUI/src/components/DatabaseStatistics/DatabaseStatistics.css @@ -37,9 +37,9 @@ } .db-stats-card .refresh-btn-small:hover { - background: rgba(99, 102, 241, 0.1); - border-color: #6366f1; - color: #6366f1; + background: var(--accent-glow); + border-color: var(--accent-color); + color: var(--accent-text); } .db-stats-card .card-content { @@ -62,9 +62,9 @@ .db-stat-summary { margin-bottom: 2rem; padding: 1.5rem; - background: rgba(99, 102, 241, 0.1); + background: var(--accent-glow); border-radius: 8px; - border: 1px solid rgba(99, 102, 241, 0.2); + border: 1px solid rgba(var(--accent-color-rgb), 0.2); } .stat-item-large { @@ -74,7 +74,7 @@ } .stat-icon-large { - color: #6366f1; + color: var(--accent-text); } .stat-value-large { @@ -148,7 +148,7 @@ .progress-fill { height: 100%; - background: linear-gradient(90deg, #6366f1 0%, #747bff 100%); + background: var(--accent-color); border-radius: 3px; transition: width 0.3s ease; } @@ -387,8 +387,8 @@ align-items: flex-start; gap: 0.75rem; padding: 0.75rem; - background: rgba(99, 102, 241, 0.1); - border: 1px solid rgba(99, 102, 241, 0.2); + background: var(--accent-glow); + border: 1px solid rgba(var(--accent-color-rgb), 0.2); border-radius: 6px; font-size: 0.875rem; color: var(--text-secondary); diff --git a/src/MilvaionUI/src/components/JsonEditor.css b/src/MilvaionUI/src/components/JsonEditor.css index 8d79578..2bc8d06 100644 --- a/src/MilvaionUI/src/components/JsonEditor.css +++ b/src/MilvaionUI/src/components/JsonEditor.css @@ -30,8 +30,8 @@ } .json-editor-container:focus-within { - border-color: #6366f1; - box-shadow: 0 0 0 3px rgba(99, 102, 241, 0.1); + border-color: var(--accent-color); + box-shadow: 0 0 0 3px var(--accent-glow); } .json-editor.has-error .json-editor-container:focus-within { @@ -76,7 +76,7 @@ .toolbar-btn:hover:not(:disabled) { background: var(--bg-hover); border-color: var(--accent-color); - color: var(--accent-color); + color: var(--accent-text); } .toolbar-btn:disabled { diff --git a/src/MilvaionUI/src/components/JsonViewer.css b/src/MilvaionUI/src/components/JsonViewer.css index c744708..2569ec1 100644 --- a/src/MilvaionUI/src/components/JsonViewer.css +++ b/src/MilvaionUI/src/components/JsonViewer.css @@ -30,7 +30,7 @@ } .json-toggle:hover { - color: var(--accent-color); + color: var(--accent-text); } .json-copy-btn { diff --git a/src/MilvaionUI/src/components/Layout.css b/src/MilvaionUI/src/components/Layout.css index 62159c4..270354d 100644 --- a/src/MilvaionUI/src/components/Layout.css +++ b/src/MilvaionUI/src/components/Layout.css @@ -151,8 +151,8 @@ } .nav-menu li.active a { - background-color: rgba(99, 102, 241, 0.1); - color: var(--accent-color); + background-color: var(--accent-glow); + color: var(--accent-text); } /* Collapsible Nav Group */ @@ -243,8 +243,8 @@ } .nav-submenu li.active a { - background-color: rgba(99, 102, 241, 0.1); - color: var(--accent-color); + background-color: var(--accent-glow); + color: var(--accent-text); } /* Sidebar Footer - User Menu */ @@ -541,7 +541,7 @@ z-index: 999; height: 44px; border-radius: 8px; - background-color: #6366f1; + background-color: var(--accent-color); border: none; color: white; display: flex; @@ -553,7 +553,7 @@ } .mobile-menu-toggle:hover { - background-color: #4f46e5; + background-color: var(--accent-hover); transform: scale(1.05); } diff --git a/src/MilvaionUI/src/components/Layout.jsx b/src/MilvaionUI/src/components/Layout.jsx index 45694c3..9d8e18a 100644 --- a/src/MilvaionUI/src/components/Layout.jsx +++ b/src/MilvaionUI/src/components/Layout.jsx @@ -248,6 +248,12 @@ const user = authService.getCurrentUser() Roles +
  • + + + Api Keys + +
  • @@ -294,6 +300,11 @@ const user = authService.getCurrentUser()
  • +
  • + + + +
  • diff --git a/src/MilvaionUI/src/components/LoadingSpinner.css b/src/MilvaionUI/src/components/LoadingSpinner.css index 45efed8..7906ca2 100644 --- a/src/MilvaionUI/src/components/LoadingSpinner.css +++ b/src/MilvaionUI/src/components/LoadingSpinner.css @@ -8,8 +8,8 @@ } .spinner { - border: 4px solid rgba(99, 102, 241, 0.1); - border-top: 4px solid #6366f1; + border: 4px solid var(--accent-glow); + border-top: 4px solid var(--accent-color); border-radius: 50%; width: 50px; height: 50px; @@ -23,6 +23,6 @@ .loading-spinner p { margin-top: 1rem; - color: #999; + color: var(--text-muted); font-size: 1.1rem; } diff --git a/src/MilvaionUI/src/components/Modal.css b/src/MilvaionUI/src/components/Modal.css index 4c8796b..b991b0b 100644 --- a/src/MilvaionUI/src/components/Modal.css +++ b/src/MilvaionUI/src/components/Modal.css @@ -204,7 +204,7 @@ } .modal-confirm { - border-left: 4px solid #6366f1; + border-left: 4px solid var(--accent-color); } @media (max-width: 768px) { diff --git a/src/MilvaionUI/src/components/NotificationPanel.css b/src/MilvaionUI/src/components/NotificationPanel.css index 67effa0..1ef764c 100644 --- a/src/MilvaionUI/src/components/NotificationPanel.css +++ b/src/MilvaionUI/src/components/NotificationPanel.css @@ -204,7 +204,7 @@ .notification-icon.info { background: color-mix(in srgb, var(--accent-color) 15%, transparent); - color: var(--accent-color); + color: var(--accent-text); } .notification-icon.warning { @@ -272,7 +272,7 @@ .notification-action-btn:hover { background: var(--bg-tertiary); - color: var(--accent-color); + color: var(--accent-text); } .notification-action-btn.danger:hover { @@ -296,7 +296,7 @@ .notification-load-more:hover { border-color: var(--accent-color); - color: var(--accent-color); + color: var(--accent-text); background: color-mix(in srgb, var(--accent-color) 5%, transparent); } diff --git a/src/MilvaionUI/src/components/OccurrenceTable.css b/src/MilvaionUI/src/components/OccurrenceTable.css index 87bafca..dbba110 100644 --- a/src/MilvaionUI/src/components/OccurrenceTable.css +++ b/src/MilvaionUI/src/components/OccurrenceTable.css @@ -45,14 +45,14 @@ } .status-chip:hover { - border-color: #6366f1; + border-color: var(--accent-color); transform: translateY(-1px); } .status-chip.active { - background-color: rgba(99, 102, 241, 0.14); - border-color: #6366f1; - color: #6366f1; + background-color: var(--accent-glow); + border-color: var(--accent-color); + color: var(--accent-text); } .status-chip.pending.active { @@ -145,7 +145,7 @@ cursor: pointer; width: 18px; height: 18px; - accent-color: #6366f1; + accent-color: var(--accent-color); } .occurrence-table td.checkbox-column input[type="checkbox"]:disabled { @@ -178,7 +178,7 @@ .job-link, .occurrence-link { - color: #6366f1; + color: var(--accent-text); text-decoration: none; font-weight: 500; } @@ -190,8 +190,8 @@ .worker-badge { padding: 0.25rem 0.75rem; - background-color: rgba(99, 102, 241, 0.14); - border: 1px solid rgba(99, 102, 241, 0.25); + background-color: var(--accent-glow); + border: 1px solid rgba(var(--accent-color-rgb), 0.25); border-radius: 12px; font-size: 0.85rem; font-family: monospace; @@ -313,13 +313,13 @@ display: inline-flex; align-items: center; justify-content: center; + gap: 0.25rem; min-width: 36px; - max-width: 20px; } .pagination .btn:hover:not(:disabled) { border-color: var(--accent-color); - color: var(--accent-color); + color: var(--accent-text); background: var(--bg-hover); } @@ -329,16 +329,22 @@ } .pagination .btn.btn-primary { - background: rgba(99, 102, 241, 0.14); - color: #6366f1; - border-color: #6366f1; + background: var(--accent-glow); + color: var(--accent-text); + border-color: var(--accent-color); } .pagination .btn-sm { - padding: 0.375rem 0.625rem; + padding: 0.375rem 0.75rem; font-size: 0.8125rem; } +/* Cursor pagination shows Previous / Next with a label, so the 36px min-width meant for numbered page buttons + squashes them. Scoped to these two buttons rather than widening every .btn-sm in the pagination bar. */ +.pagination .btn-cursor { + min-width: 80px; +} + .page-ellipsis { color: var(--text-muted); font-size: 0.875rem; diff --git a/src/MilvaionUI/src/components/OccurrenceTable.jsx b/src/MilvaionUI/src/components/OccurrenceTable.jsx index a383a8a..5319082 100644 --- a/src/MilvaionUI/src/components/OccurrenceTable.jsx +++ b/src/MilvaionUI/src/components/OccurrenceTable.jsx @@ -1,5 +1,5 @@ import { useState, useEffect } from 'react' -import { Link } from 'react-router-dom' +import { Link, useNavigate } from 'react-router-dom' import Icon from './Icon' import { formatDateTime, formatDuration } from '../utils/dateUtils' import './OccurrenceTable.css' @@ -15,10 +15,16 @@ function OccurrenceTable({ onPageChange, onPageSizeChange, onBulkDelete, - showJobName = false + showJobName = false, + useCursorPagination = false, + hasNextPage = false, + hasPreviousPage = false, + onNextPage, + onPreviousPage, }) { const [currentTime, setCurrentTime] = useState(Date.now()) const [selectedOccurrences, setSelectedOccurrences] = useState([]) + const navigate = useNavigate() // Update current time every second for running occurrences useEffect(() => { @@ -95,6 +101,51 @@ function OccurrenceTable({ const totalPages = Math.ceil(totalCount / pageSize) const renderPagination = () => { + if (useCursorPagination) { + return ( +
    +
    + + + Page {currentPage} + {totalCount > 0 ? ` (${totalCount} total)` : ''} + + +
    +
    + + +
    +
    + ) + } + if (totalPages <= 1 && totalCount <= pageSize) return null const maxVisiblePages = 5 @@ -324,7 +375,7 @@ function OccurrenceTable({ window.location.href = `/occurrences/${occurrence.id}`} + onClick={() => navigate(`/occurrences/${occurrence.id}`)} style={{ cursor: 'pointer' }} > e.stopPropagation()}> diff --git a/src/MilvaionUI/src/components/ServiceMemoryStats/ServiceMemoryStats.css b/src/MilvaionUI/src/components/ServiceMemoryStats/ServiceMemoryStats.css index 0dae2e2..955e1ec 100644 --- a/src/MilvaionUI/src/components/ServiceMemoryStats/ServiceMemoryStats.css +++ b/src/MilvaionUI/src/components/ServiceMemoryStats/ServiceMemoryStats.css @@ -31,7 +31,7 @@ .service-memory-stats .refresh-btn-small:hover { background: var(--bg-hover); border-color: var(--accent-color); - color: var(--accent-color); + color: var(--accent-text); } /* Overview Stats */ @@ -74,7 +74,7 @@ line-height: 1.2; } -.memory-stat-value.primary { color: #6366f1; } +.memory-stat-value.primary { color: var(--accent-text); } .memory-stat-value.success { color: #10b981; } .memory-stat-value.warning { color: #f59e0b; } .memory-stat-value.error { color: #ef4444; } @@ -105,7 +105,7 @@ gap: 0.5rem; display: inline-block; width: 4px; height: 16px; - background: linear-gradient(135deg, #6366f1 0%, #4f46e5 100%); + background: var(--accent-color); border-radius: 2px; } @@ -276,8 +276,8 @@ gap: 0.5rem; .gc-stats { margin-top: 1rem; padding: 0.75rem; - background: rgba(99, 102, 241, 0.04); - border: 1px solid rgba(99, 102, 241, 0.1); + background: var(--accent-glow); + border: 1px solid rgba(var(--accent-color-rgb), 0.1); border-radius: 6px; } @@ -303,7 +303,7 @@ gap: 0.5rem; .gc-stat-value { font-size: 1.1rem; font-weight: 600; - color: #6366f1; + color: var(--accent-text); } .gc-stat-label { @@ -340,8 +340,8 @@ gap: 0.5rem; .memory-stats-loading .spinner { width: 24px; height: 24px; - border: 3px solid #333; - border-top-color: #6366f1; + border: 3px solid var(--border-color); + border-top-color: var(--accent-text); border-radius: 50%; animation: spin 1s linear infinite; } @@ -407,7 +407,7 @@ gap: 0.5rem; } .service-card-header { - background: rgba(99, 102, 241, 0.02); + background: var(--accent-glow); border-bottom-color: #ddd; } @@ -429,8 +429,8 @@ gap: 0.5rem; } .gc-stats { - background: rgba(99, 102, 241, 0.03); - border-color: rgba(99, 102, 241, 0.1); + background: var(--accent-glow); + border-color: rgba(var(--accent-color-rgb), 0.1); } .section-title { @@ -443,7 +443,7 @@ gap: 0.5rem; } .service-memory-stats .refresh-btn-small:hover { - background: rgba(99, 102, 241, 0.04); + background: var(--accent-glow); } .memory-bar-container { diff --git a/src/MilvaionUI/src/components/TriggerWorkflowModal.css b/src/MilvaionUI/src/components/TriggerWorkflowModal.css new file mode 100644 index 0000000..bc7f983 --- /dev/null +++ b/src/MilvaionUI/src/components/TriggerWorkflowModal.css @@ -0,0 +1,260 @@ +/* ── TriggerWorkflowModal ─────────────────────────────────── */ + +.twm-fetch-loading { + display: flex; + align-items: center; + gap: 8px; + justify-content: center; + padding: 2rem; + color: var(--text-muted); +} + +.twm-modal { + width: 660px; + max-width: 95vw; + max-height: 88vh; + display: flex; + flex-direction: column; +} + +.twm-body { + overflow-y: auto; + flex: 1; + display: flex; + flex-direction: column; + gap: 1.25rem; + padding: 1.25rem 1.5rem; +} + +.twm-field { + display: flex; + flex-direction: column; + gap: 0.4rem; +} + +.twm-label { + display: flex; + align-items: center; + gap: 0.35rem; + font-size: 0.78rem; + font-weight: 600; + color: var(--text-secondary); + text-transform: uppercase; + letter-spacing: 0.05em; +} + +.twm-hint { + font-weight: 400; + text-transform: none; + letter-spacing: 0; + color: var(--text-muted); + margin-left: 0.25rem; +} + +/* Steps */ +.twm-steps { + display: flex; + flex-direction: column; + gap: 1rem; +} + +.twm-steps-label { + margin-bottom: 0.25rem; +} + +.twm-step { + background: var(--bg-secondary); + border: 1px solid var(--border-color); + border-radius: 8px; + padding: 0.875rem 1rem; + display: flex; + flex-direction: column; + gap: 0.6rem; +} + +.twm-step-header { + display: flex; + align-items: center; + gap: 0.5rem; + flex-wrap: wrap; +} + +.twm-step-order { + font-size: 0.72rem; + font-weight: 600; + color: var(--text-muted); + background: var(--bg-tertiary); + border-radius: 4px; + padding: 1px 6px; +} + +.twm-step-name { + font-size: 0.9rem; +} + +.twm-step-job { + display: inline-flex; + align-items: center; + gap: 0.25rem; + font-size: 0.76rem; + color: var(--text-muted); + background: var(--bg-tertiary); + border-radius: 4px; + padding: 2px 7px; + margin-left: auto; +} + +/* Schema */ +.twm-schema { + background: var(--bg-tertiary); + border: 1px solid var(--border-color); + border-radius: 6px; + padding: 0.625rem 0.75rem; + display: flex; + flex-direction: column; + gap: 0.5rem; +} + +.twm-schema-error { + font-size: 0.78rem; + color: var(--color-failed); +} + +.twm-schema-header { + display: flex; + align-items: center; + gap: 0.35rem; + font-size: 0.76rem; + font-weight: 600; + color: var(--text-secondary); +} + +.twm-schema-gen-btn { + display: inline-flex; + align-items: center; + gap: 0.25rem; + margin-left: auto; + font-size: 0.75rem; + padding: 2px 8px; + border-radius: 4px; + border: 1px solid var(--border-color); + background: var(--bg-secondary); + color: var(--text-secondary); + cursor: pointer; + transition: background 0.15s, color 0.15s; +} + +.twm-schema-gen-btn:hover { + background: var(--accent-color); + color: #fff; + border-color: var(--accent-color); +} + +.twm-schema-props { + display: flex; + flex-direction: column; + gap: 0.35rem; +} + +.twm-schema-prop { + display: flex; + align-items: baseline; + gap: 0.4rem; + flex-wrap: wrap; + font-size: 0.78rem; +} + +.twm-prop-name { + font-weight: 600; + color: var(--text-primary); + font-family: 'Fira Code', monospace; +} + +.twm-prop-type { + border-radius: 3px; + padding: 0 5px; + font-size: 0.7rem; + font-weight: 600; + text-transform: uppercase; +} + +.twm-type-string { background: #1e6a4a22; color: #2ecc71; } +.twm-type-number, +.twm-type-integer { background: #1a4a9022; color: #5b9cf6; } +.twm-type-boolean { background: #6a1a6a22; color: #c678dd; } +.twm-type-array, +.twm-type-object { background: #6a4a1a22; color: #e5c07b; } + +.twm-prop-required { + font-size: 0.68rem; + font-weight: 700; + color: var(--color-failed); + text-transform: uppercase; + letter-spacing: 0.04em; +} + +.twm-prop-enum { + font-size: 0.72rem; + color: var(--text-muted); + font-style: italic; +} + +.twm-prop-desc { + font-size: 0.72rem; + color: var(--text-muted); + flex-basis: 100%; + padding-left: 0.25rem; +} + +.twm-schema-toggle { + background: none; + border: none; + color: var(--accent-text); + font-size: 0.74rem; + cursor: pointer; + padding: 0; + text-align: left; + width: fit-content; +} + +.twm-schema-raw { + font-size: 0.74rem; + background: var(--bg-primary); + border: 1px solid var(--border-color); + border-radius: 4px; + padding: 0.5rem 0.75rem; + overflow-x: auto; + white-space: pre; + color: var(--text-secondary); +} + +/* Textarea */ +.twm-textarea { + width: 100%; + font-family: 'Fira Code', 'Cascadia Code', monospace; + font-size: 0.82rem; + line-height: 1.5; + resize: vertical; + min-height: 70px; + padding: 0.5rem 0.75rem; + border-radius: 6px; + border: 1px solid var(--border-color); + background: var(--bg-primary); + color: var(--text-primary); + transition: border-color 0.15s; + box-sizing: border-box; +} + +.twm-textarea:focus { + outline: none; + border-color: var(--accent-color); +} + +.twm-textarea--error { + border-color: var(--color-failed); +} + +.twm-error { + font-size: 0.75rem; + color: var(--color-failed); +} diff --git a/src/MilvaionUI/src/components/TriggerWorkflowModal.jsx b/src/MilvaionUI/src/components/TriggerWorkflowModal.jsx new file mode 100644 index 0000000..7c7e5a1 --- /dev/null +++ b/src/MilvaionUI/src/components/TriggerWorkflowModal.jsx @@ -0,0 +1,276 @@ +import { useState, useEffect } from 'react' +import PropTypes from 'prop-types' +import Icon from './Icon' +import workerService from '../services/workerService' +import workflowService from '../services/workflowService' +import './TriggerWorkflowModal.css' + +// ── helpers ────────────────────────────────────────────────────────────────── + +function generateExampleFromSchema(schema) { + if (typeof schema === 'string') { + try { schema = JSON.parse(schema) } catch { return {} } + } + if (!schema?.properties) return {} + const example = {} + for (const [key, prop] of Object.entries(schema.properties)) { + if (prop.default !== undefined) { example[key] = prop.default; continue } + if (prop.enum?.length) { example[key] = prop.enum[0]; continue } + switch (prop.type) { + case 'string': + example[key] = prop.format === 'date-time' ? new Date().toISOString() + : prop.format === 'date' ? new Date().toISOString().split('T')[0] + : prop.format === 'email' ? 'example@email.com' + : prop.format === 'uri' ? 'https://example.com' + : `<${key}>` + break + case 'integer': case 'number': example[key] = 0; break + case 'boolean': example[key] = false; break + case 'array': example[key] = []; break + case 'object': example[key] = {}; break + default: example[key] = null + } + } + return example +} + +function SchemaViewer({ schema, onGenerate }) { + const [expanded, setExpanded] = useState(false) + let parsed = schema + if (typeof schema === 'string') { + try { parsed = JSON.parse(schema) } catch { return
    Invalid schema
    } + } + if (!parsed?.properties) return null + const required = parsed.required || [] + return ( +
    +
    + + Expected Schema + +
    +
    + {Object.entries(parsed.properties).map(([key, prop]) => ( +
    + {key} + {prop.type} + {required.includes(key) && required} + {prop.enum && enum: {prop.enum.join(' | ')}} + {prop.description && {prop.description}} +
    + ))} +
    + + {expanded &&
    {JSON.stringify(parsed, null, 2)}
    } +
    + ) +} + +SchemaViewer.propTypes = { + schema: PropTypes.oneOfType([PropTypes.string, PropTypes.object]).isRequired, + onGenerate: PropTypes.func.isRequired, +} + +// ── main component ──────────────────────────────────────────────────────────── + +/** + * Modal for triggering a workflow run with optional per-step job data. + * + * Props: + * workflowId – workflow ID to trigger + * onClose – called when modal should close (optionally with error message) + * onSuccess – called with (runId) after successful trigger + */ +export default function TriggerWorkflowModal({ workflowId, workflow: workflowProp, onClose, onSuccess }) { + const [reason, setReason] = useState('') + const [stepData, setStepData] = useState({}) + const [stepErrors, setStepErrors] = useState({}) + const [loading, setLoading] = useState(false) + const [fetchLoading, setFetchLoading] = useState(!workflowProp) + const [workers, setWorkers] = useState([]) + const [workflow, setWorkflow] = useState(workflowProp ?? null) + + const taskSteps = (workflow?.steps || []) + .filter(s => s.nodeType === 0 || s.nodeType == null) + .sort((a, b) => a.order - b.order) + + useEffect(() => { + const id = workflowProp?.id ?? workflowId + if (!id) return + + const fetchWorkers = workerService.getAll().then(r => setWorkers(r?.data || [])).catch(() => {}) + + if (workflowProp) { + fetchWorkers + return + } + + setFetchLoading(true) + Promise.all([ + workflowService.getById(id), + workerService.getAll(), + ]).then(([wfRes, workerRes]) => { + setWorkflow(wfRes?.data ?? null) + setWorkers(workerRes?.data || []) + }).catch(() => {}).finally(() => setFetchLoading(false)) + // eslint-disable-next-line react-hooks/exhaustive-deps + }, []) + + const getSchema = (step) => { + if (!step.workerId || !step.jobNameInWorker) return null + const worker = workers.find(w => w.workerId === step.workerId) + return worker?.jobDataDefinitions?.[step.jobNameInWorker] ?? null + } + + const handleDataChange = (stepId, value) => { + setStepData(prev => ({ ...prev, [stepId]: value })) + if (!value.trim()) { + setStepErrors(prev => { const n = { ...prev }; delete n[stepId]; return n }) + return + } + try { + JSON.parse(value) + setStepErrors(prev => { const n = { ...prev }; delete n[stepId]; return n }) + } catch { + setStepErrors(prev => ({ ...prev, [stepId]: 'Invalid JSON' })) + } + } + + const handleGenerateExample = (step) => { + const schema = getSchema(step) + if (!schema) return + const example = generateExampleFromSchema(schema) + handleDataChange(step.id, JSON.stringify(example, null, 2)) + } + + const hasErrors = Object.keys(stepErrors).length > 0 + + const handleSubmit = async () => { + if (hasErrors) return + const stepJobData = {} + Object.entries(stepData).forEach(([id, val]) => { + if (val.trim()) stepJobData[id] = val.trim() + }) + setLoading(true) + try { + const result = await workflowService.trigger(workflow.id, reason || 'Manual trigger', stepJobData) + if (result?.isSuccess) { + onSuccess(result.data.id) + } else { + onClose(result?.message || 'Failed to trigger workflow') + } + } catch { + onClose('Failed to trigger workflow') + } finally { + setLoading(false) + } + } + + return ( +
    !loading && onClose()}> +
    e.stopPropagation()}> + + {/* Header */} +
    +

    Run Workflow

    + +
    + + {/* Body */} +
    + {fetchLoading ? ( +
    Loading...
    + ) : (<> +
    + + setReason(e.target.value)} + disabled={loading} + /> +
    + + {/* Per-step data */} + {taskSteps.length > 0 && ( +
    + + + {taskSteps.map(step => { + const schema = getSchema(step) + return ( +
    +
    + #{step.order} + {step.stepName} + {step.jobDisplayName && ( + + + {step.jobDisplayName} + + )} +
    + + {schema && ( + handleGenerateExample(step)} + /> + )} + +