Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions docs/site/src/content/docs/grains/journaling/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,20 @@ ms.topic: how-to

# Configure Orleans Journaling

Configure a default journal storage provider on every silo that can activate a <xref:Orleans.Journaling.DurableGrain>. Additional named providers give journal consumers independent storage namespaces alongside default grain journaling. Provider registration also adds the core Journaling services, durable-state keyed services, JSON format, and Orleans binary reader.
Configure a default journal storage provider on every silo that hosts grains or activation-scoped features using durable state. Additional named providers give journal consumers independent storage namespaces alongside default grain journaling. Provider registration also adds the core Journaling services, durable-state keyed services, JSON format, and Orleans binary reader.

The Journaling packages are pre-release alpha packages and their APIs carry diagnostic `ORLEANSEXP005`.

## Configure grain-scoped composition

The standard <xref:Orleans.Journaling.HostingExtensions.AddJournalStorage*> registration supplies one scoped <xref:Orleans.Journaling.IJournaledStateManager> per activation. The manager enrolls itself in the grain lifecycle when constructed with the activation's <xref:Orleans.Runtime.IGrainContext>. Inject that manager and keyed durable states into an ordinary <xref:Orleans.Grain>, an application-owned grain base, or an activation-scoped feature. Recovery then completes before <xref:Orleans.Grain.OnActivateAsync*> and requests.

Storage registration makes these services available across the silo. Per-grain journal reads and writes are triggered by activations which resolve the manager directly or through a durable-state dependency. Unrelated grain activations retain their existing persistence behavior.

<xref:Orleans.Journaling.DurableGrain> provides optional protected helpers for the same scoped manager. Grain-scoped managers are enrolled before resolution returns; custom managers establish this in their constructor or registration factory. Reusable features compose their own lifecycle work through a [shared activation setup action](../grain-lifecycle.md#shared-activation-setup). Explicit <xref:Orleans.Journaling.JournalId> factories and managers constructed directly from storage retain caller-owned initialization and disposal unless the caller deliberately assigns lifecycle ownership.

See [Use durable state](durable-state.md) for source-backed constructor injection and feature setup examples, and [Activation and recovery](runtime-behavior.md#activation-and-recovery) for lifecycle ordering.

## Choose a storage provider

| Provider | Package | Storage model | Compaction trigger |
Expand Down Expand Up @@ -131,6 +141,6 @@ The default minimum is seven days. Removal is persisted by a compaction after th

## Development storage

<xref:Orleans.Journaling.HostingExtensions.AddJournalStorage*> registers core services and resolves an <xref:Orleans.Journaling.IJournalStorageProvider>. Runtime tests and disposable development hosts can use <xref:Orleans.Journaling.HostingExtensions.AddVolatileJournalStorage*> with a provider name. Its contents live in process memory, so use persistent emulator storage to validate restart recovery and provider migration.
<xref:Orleans.Journaling.HostingExtensions.AddJournalStorage*> registers core services which use an <xref:Orleans.Journaling.IJournalStorageProvider>. Runtime tests and disposable development hosts can use <xref:Orleans.Journaling.HostingExtensions.AddVolatileJournalStorage*> with a provider name. Its contents live in process memory, so use persistent emulator storage to validate restart recovery and provider migration.

Use the same durable provider category in staging that production uses so recovery, compaction, concurrency, and backup procedures receive realistic validation.
30 changes: 25 additions & 5 deletions docs/site/src/content/docs/grains/journaling/durable-state.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,23 @@ ms.topic: how-to

# Use durable state

Install the pre-release [`Microsoft.Orleans.Journaling`](https://www.nuget.org/packages/Microsoft.Orleans.Journaling) package in the silo project. Install a [journal storage provider](configuration.md#choose-a-storage-provider) and configure it before activating a <xref:Orleans.Journaling.DurableGrain>.
Install the pre-release [`Microsoft.Orleans.Journaling`](https://www.nuget.org/packages/Microsoft.Orleans.Journaling) package in the silo project. Install a [journal storage provider](configuration.md#choose-a-storage-provider) and configure it on the silos which host grains using durable state.

All Journaling APIs are experimental and carry diagnostic `ORLEANSEXP005`.

## Define a durable grain

Inject durable states with <xref:Microsoft.Extensions.DependencyInjection.FromKeyedServicesAttribute>. The service key becomes the state's stable name in the grain journal:
Compose an ordinary <xref:Orleans.Grain> with an injected <xref:Orleans.Journaling.IJournaledStateManager> and durable states. Inject durable states with <xref:Microsoft.Extensions.DependencyInjection.FromKeyedServicesAttribute>. The service key becomes the state's stable name in the grain journal:

:::code language="csharp" source="./snippets/journaling/JournalingBasics.cs" id="durable_shopping_cart":::
:::code language="csharp" source="../../snippets/compiled/Grains/JournalingSnippets.cs" id="composed_shopping_cart":::

The dictionary mutation is immediately visible to the current activation. Awaiting <xref:Orleans.Journaling.DurableGrain.WriteStateAsync*> establishes the durability point for every pending durable-state mutation on that grain.
The standard manager enrolls itself in the grain lifecycle when constructed with the activation's <xref:Orleans.Runtime.IGrainContext>. Durable states register during construction, and recovery finishes before <xref:Orleans.Grain.OnActivateAsync*> and request processing. The same composition works with an application-owned grain base class.

The dictionary mutation is immediately visible to the current activation. Awaiting <xref:Orleans.Journaling.IJournaledStateManager.WriteStateAsync*> establishes the durability point for every pending durable-state mutation on that grain. Accept a <xref:System.Threading.CancellationToken> on grain operations and flow it through state-manager calls so cancellation follows the caller's operation lifetime.

<xref:Orleans.Journaling.DurableGrain> remains a convenience base exposing its protected <xref:Orleans.Journaling.DurableGrain.StateManager>, <xref:Orleans.Journaling.DurableGrain.GetOrCreateState*>, and <xref:Orleans.Journaling.DurableGrain.WriteStateAsync*> members. Choose that base when those helpers fit the application; constructor-injected composition gives existing grain hierarchies the same standard recovery behavior.

The composition example is compiled against repository source so it exercises constructor-owned lifecycle enrollment.

## Select a state type

Expand All @@ -31,7 +37,7 @@ The dictionary mutation is immediately visible to the current activation. Awaiti
| <xref:Orleans.Journaling.IDurableTaskCompletionSource`1> | Durable task completion | Complete, fault, or cancel |
| <xref:Orleans.Runtime.IPersistentState`1> | Record-style state | Set or clear a versioned state value |

All named states in one grain share the grain's journal and participate in the same write. This makes a single <xref:Orleans.Journaling.DurableGrain.WriteStateAsync*> the atomic storage boundary for their pending changes. Coordination with another grain or an external service requires an application protocol such as idempotency, an inbox, or an outbox.
All named states in one grain share the grain's journal and participate in the same write. This makes a single <xref:Orleans.Journaling.IJournaledStateManager.WriteStateAsync*> the atomic storage boundary for their pending changes. Coordination with another grain or an external service requires an application protocol such as idempotency, an inbox, or an outbox.

An <xref:Orleans.Journaling.IDurableTaskCompletionSource`1> changes status in memory when `TrySetResult`, `TrySetException`, or `TrySetCanceled` succeeds. Its `Task` completes after a write acknowledges that status or recovery replays it, allowing waiters to observe a durable completion.

Expand All @@ -47,6 +53,20 @@ The keyed service name identifies a durable state across activations and deploym

Registering two states with the same name fails activation. Registering a state after activation setup also fails because recovery has already assigned journal stream identities.

## Compose an activation-scoped feature

A reusable feature can own durable state independently of the grain's constructor and base class. Select the grain implementation once per grain type using <xref:Orleans.Metadata.GrainClassMap> in an <xref:Orleans.Runtime.IConfigureGrainTypeComponents> implementation, then register a shared setup action. This example selects the existing shopping-cart interface and records an activation count:

:::code language="csharp" source="../../snippets/compiled/Grains/JournalingSnippets.cs" id="journaled_feature":::

Register the feature as scoped and the configurator as singleton:

:::code language="csharp" source="../../snippets/compiled/Grains/JournalingSnippets.cs" id="journaled_feature_registration":::

The action resolves the feature from <xref:Orleans.Runtime.IGrainContext.ActivationServices> and enrolls its lifecycle participant. Resolving the feature's dependencies also constructs the standard state manager, which enrolls itself in the grain lifecycle. The feature uses a stage after <xref:Orleans.Runtime.GrainLifecycleStage.SetupState> and before <xref:Orleans.Runtime.GrainLifecycleStage.Activate> so it increments recovered state before application activation begins.

Keep one enrollment owner per participant. Setup actions are shared across concurrent activations; resolve activation-specific data from the supplied context and keep shared callbacks stateless. See [Shared activation setup](../grain-lifecycle.md#shared-activation-setup) and [Journaling activation and recovery](runtime-behavior.md#activation-and-recovery) for ordering and failure behavior.

## Use journal-backed persistent state

Journaling registers keyed <xref:Orleans.Runtime.IPersistentState`1> services. Its familiar `State`, `WriteStateAsync`, and `ClearStateAsync` members write through the same journal manager as the durable collections. `ReadStateAsync` completes from the already-recovered in-memory state because activation setup replayed the grain journal.
Expand Down
15 changes: 9 additions & 6 deletions docs/site/src/content/docs/grains/journaling/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ Orleans Journaling is an experimental persistence model that records mutations t

## Programming model

A journaling grain derives from <xref:Orleans.Journaling.DurableGrain> and receives named durable states through keyed dependency injection. Orleans currently provides:
A journaling grain composes <xref:Orleans.Journaling.IJournaledStateManager> with named durable states received through keyed dependency injection. The standard manager enrolls itself in the lifecycle during grain-bound construction, so constructor-injected state recovers before application activation and requests. This works with an ordinary <xref:Orleans.Grain>, an application-owned grain base, or a reusable activation-scoped feature. <xref:Orleans.Journaling.DurableGrain> remains an optional convenience base with protected state-manager helpers.

Orleans currently provides:

- <xref:Orleans.Journaling.IDurableValue`1>
- <xref:Orleans.Journaling.IDurableDictionary`2>
Expand All @@ -24,7 +26,7 @@ A journaling grain derives from <xref:Orleans.Journaling.DurableGrain> and recei
- <xref:Orleans.Journaling.IDurableTaskCompletionSource`1>
- <xref:Orleans.Runtime.IPersistentState`1> backed by the grain's journal

Mutations update the activation's in-memory state and add encoded operations to its pending journal buffer. Await <xref:Orleans.Journaling.DurableGrain.WriteStateAsync*> at the application durability point. The returned task completes after the storage provider acknowledges the append or snapshot replacement.
Mutations update the activation's in-memory state and add encoded operations to its pending journal buffer. Await <xref:Orleans.Journaling.IJournaledStateManager.WriteStateAsync*> at the application durability point. The returned task completes after the storage provider acknowledges the append or snapshot replacement.

Prepare fallible work in operation-local data and stage mutations once they are safe to commit. Interleaved
calls share the pending journal. A failed journal operation fences the manager and requests grain deactivation;
Expand All @@ -34,13 +36,14 @@ Each named state has a stable stream identity within the grain journal. Keep tho

## Journal lifecycle

1. During activation setup, Orleans reads the journal in order and replays each state stream.
1. Resolving the grain-scoped manager enrolls it in the lifecycle. Constructor injection and shared setup register states before lifecycle startup.
1. During the setup-state stage, Orleans reads the journal in order and replays each state stream before application activation and requests.
1. Grain code synchronously mutates durable values and collections during a grain turn.
1. <xref:Orleans.Journaling.DurableGrain.WriteStateAsync*> gathers pending operations for the grain and submits one atomic journal append or replacement to storage.
1. <xref:Orleans.Journaling.IJournaledStateManager.WriteStateAsync*> gathers pending operations for the grain and submits one atomic journal append or replacement to storage.
1. The storage provider can request compaction when its configured size or row threshold is reached. The next write creates a snapshot of the current durable states and atomically replaces the journal.
1. A later activation replays the latest snapshot and subsequent operations to restore the same durable state.

For the detailed guarantees, see [Runtime behavior and consistency](runtime-behavior.md).
Registering a provider makes the services available; activations which resolve the manager trigger per-grain journal I/O. For the detailed guarantees and caller-owned manager lifetimes, see [Runtime behavior and consistency](runtime-behavior.md).

## Journal formats

Expand All @@ -56,7 +59,7 @@ Orleans offers two separate journal-oriented programming models:

| Model | Application state model | Persistence coordination |
| --- | --- | --- |
| Orleans Journaling | Mutable durable values and collections on <xref:Orleans.Journaling.DurableGrain> | One per-grain journal managed by `Microsoft.Orleans.Journaling` |
| Orleans Journaling | Mutable durable values and collections composed with a grain or feature | One per-grain journal managed by `Microsoft.Orleans.Journaling` |
| [Orleans Event Sourcing](../event-sourcing/index.md) | Application-defined events applied to `JournaledGrain<TState, TEvent>` | Log-consistency providers confirm, persist, and synchronize events |

Choose Journaling when evaluating operation-based persistence for built-in mutable state structures. Choose Event Sourcing when domain events, event history, and the supported log-consistency programming model are application requirements.
Expand Down
Loading
Loading