Skip to content

deployment: rewrite ConfigMap configuration doc for native config loading #14844

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
nimrod-starkware wants to merge 1 commit into
base: nimrod/jsonnet/common_dir
Choose a base branch
from
Draft

deployment: rewrite ConfigMap configuration doc for native config loading #14844

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
281 changes: 70 additions & 211 deletions deployments/sequencer/docs/CONFIGMAP_CONFIGURATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,269 +1,128 @@
# ConfigMap Configuration Guide

This document describes all available configuration options for the ConfigMap construct.
This document describes the `config` (ConfigMap) options for the sequencer cdk8s constructs.

## Basic Configuration
## Overview

```yaml
config:
configPaths:
- "crates/apollo_deployments/resources/app_configs/base_layer_config.json"
- "crates/apollo_deployments/resources/app_configs/sequencer_config.json"
```
Each node service's ConfigMap holds a **single, fully-resolved `SequencerNodeConfig`** as nested JSON. The
config is generated by evaluating the service's leaf overlay `node.jsonnet` through the jsonnet `build()`
evaluator — you do **not** assemble it from a list of files. The resulting JSON is stored under the
ConfigMap data key `config` and mounted as a file for the node to load.

The node uses the **native** config loader, which takes exactly **two** `--config_file` arguments:

## Advanced Configuration
1. the **base** config — this ConfigMap (a complete, resolved nested config), and
2. a **secrets** overlay — provided by a Secret / ExternalSecret (see
[SECRET_CONFIGURATION.md](./SECRET_CONFIGURATION.md) and
[EXTERNAL_SECRET_CONFIGURATION.md](./EXTERNAL_SECRET_CONFIGURATION.md)).

Both arguments are added automatically by the pod builder.

## Basic Configuration

`config` is **required** for every node service. With no options set, the ConfigMap is generated from the
service's leaf overlay `node.jsonnet` and mounted at the default path:

```yaml
config:
configPaths:
- "crates/apollo_deployments/resources/app_configs/base_layer_config.json"
- "crates/apollo_deployments/resources/app_configs/sequencer_config.json"
- "crates/apollo_deployments/resources/app_configs/monitoring_config.json"
- "crates/apollo_deployments/resources/app_configs/logging_config.json"
# mountPath: /config/sequencer/presets/ # Optional: Override default mount path
config: {}
```

## Configuration Options

### `config` (object)
- **Required**: Yes
- **Description**: Configuration for ConfigMap creation and mounting

#### `configPaths` (array of strings)
- **Required**: Yes
- **Description**: List of JSON configuration file paths to load and merge into the ConfigMap
- **Example**:
```yaml
config:
configPaths:
- "crates/apollo_deployments/resources/app_configs/base_layer_config.json"
- "crates/apollo_deployments/resources/app_configs/sequencer_config.json"
```
- **Description**: Enables ConfigMap generation and mounting. The content is the nested
`SequencerNodeConfig` produced by jsonnet `build()` for the service's overlay.

#### `mountPath` (string, optional)
- **Default**: `"/config/sequencer/presets/"`
- **Description**: Path where the config will be mounted in the container
- **Default**: `/config/sequencer/presets/`
- **Description**: Directory the ConfigMap is mounted at. The config file lands at `<mountPath>config`.

#### `readOnly` (bool, optional)
- **Default**: `true`
- **Description**: Whether the ConfigMap volume mount is read-only.

#### `sequencerConfig` (object, optional)
- **Description**: Override values applied on top of the generated config, for small per-deployment tweaks
without editing the overlay. Keys identify the config field to override; values are the replacement.
- **Example**:
```yaml
config:
configPaths:
- "crates/apollo_deployments/resources/app_configs/base_layer_config.json"
mountPath: "/custom/config/path"
sequencerConfig:
base_layer_config_starknet_contract_address: "<STARKNET_CONTRACT_ADDRESS>"
```

## File Path Resolution

The `configPaths` are resolved relative to the project root directory. The paths should be:

1. **Relative to project root**: All paths are relative to the main project directory
2. **JSON files only**: Only JSON files are supported for configuration
3. **Merged in order**: Files are loaded and merged in the order specified
4. **Error handling**: Missing files will cause deployment to fail

## Example Configuration Files
## Where the config comes from

### base_layer_config.json
```json
{
"logging": {
"level": "info",
"format": "json"
},
"database": {
"host": "localhost",
"port": 5432
}
}
```
The ConfigMap content is produced by evaluating:

### sequencer_config.json
```json
{
"sequencer": {
"batch_size": 1000,
"timeout": 30
},
"database": {
"name": "sequencer_db"
}
}
```

### monitoring_config.json
```json
{
"monitoring": {
"enabled": true,
"port": 9090,
"path": "/metrics"
}
}
deployments/sequencer/configs/jsonnet/overlays/<dotted-overlay>/node.jsonnet
```

## Merged Configuration

The above configuration files would be merged into a single JSON object:

```json
{
"logging": {
"level": "info",
"format": "json"
},
"database": {
"host": "localhost",
"port": 5432,
"name": "sequencer_db"
},
"sequencer": {
"batch_size": 1000,
"timeout": 30
},
"monitoring": {
"enabled": true,
"port": 9090,
"path": "/metrics"
}
}
```
`node.jsonnet` imports `lib/build.libsonnet` and emits the full per-service `SequencerNodeConfig`. There is
no `configPaths` list and no `resources/app_configs/*.json` files — those belonged to the retired preset
config path and no longer exist.

## Generated Kubernetes Resource

The configuration above generates a ConfigMap resource like this:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: sequencer-node-config
namespace: default
name: sequencer-<service>-config
labels:
app: sequencer
service: sequencer-node
service: <service>
data:
config.json: |
# The whole nested SequencerNodeConfig under a single "config" key (not "config.json").
config: |
{
"logging": {
"level": "info",
"format": "json"
},
"database": {
"host": "localhost",
"port": 5432,
"name": "sequencer_db"
},
"sequencer": {
"batch_size": 1000,
"timeout": 30
},
"monitoring": {
"enabled": true,
"port": 9090,
"path": "/metrics"
}
"components": { "batcher": { "execution_mode": "Enabled" }, ... },
"batcher_config": { ... },
"consensus_manager_config": { ... },
"http_server_config": { ... },
...
}
```

## Mounting in Pods

The ConfigMap is **automatically mounted** in pods when the `config` section is provided. The mount path defaults to `/config/sequencer/presets/` but can be customized:
The data key is `config` (not `config.json`); it is mounted as a file named `config`.

```yaml
config:
configPaths:
- "crates/apollo_deployments/resources/app_configs/base_layer_config.json"
mountPath: "/custom/config/path" # Optional: defaults to "/config/sequencer/presets/"
```
## Mounting in Pods

The ConfigMap is mounted as a directory at the specified path. The generated volume mount looks like:
The ConfigMap is mounted automatically when `config` is set:

```yaml
volumeMounts:
- name: sequencer-node-config
mountPath: /config/sequencer/presets/ # or custom path
- name: sequencer-<service>-config
mountPath: /config/sequencer/presets/ # or custom mountPath
readOnly: true
```

## Automatic Container Arguments

When a ConfigMap is configured, the container **automatically receives** the `--config_file` argument pointing to the mount path:
The pod builder always adds the base config as the first `--config_file`, and — when a Secret or
ExternalSecret is configured — the secrets file as the second `--config_file`:

```yaml
args:
- --config_file
- /config/sequencer/presets/ # or custom mountPath if specified
- /config/sequencer/presets/config # base: this ConfigMap (or <mountPath>config)
- --config_file
- /etc/secrets/secrets.json # secrets: ExternalSecret (or /etc/secrets/secret.json for a Secret)
```

This argument is always added first, before any additional arguments you may provide in the `args` section of `node.yaml`.

## Environment Variable Injection

You can also inject specific values as environment variables:

```yaml
env:
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: sequencer-node-config
key: logging.level
- name: DB_HOST
valueFrom:
configMapKeyRef:
name: sequencer-node-config
key: database.host
```
The native loader requires **exactly two** `--config_file` arguments (base + secrets). A service that
mounts a ConfigMap but no Secret/ExternalSecret emits only one argument and fails to boot — always pair the
ConfigMap with a secrets source (an empty secrets file is acceptable).

## Best Practices

1. **File Organization**: Organize configuration files by feature or environment
2. **Naming Convention**: Use descriptive names for configuration files
3. **Validation**: Validate JSON files before deployment
4. **Version Control**: Keep configuration files in version control
5. **Environment Separation**: Use different config files for different environments
6. **Sensitive Data**: Never put sensitive data in ConfigMaps (use Secrets instead)
7. **Documentation**: Document the structure of your configuration files

## Common Use Cases

### Environment-Specific Configuration

```yaml
# Development
config:
configPaths:
- "crates/apollo_deployments/resources/app_configs/base_layer_config.json"
- "crates/apollo_deployments/resources/app_configs/sequencer_config.json"
- "crates/apollo_deployments/resources/app_configs/dev_config.json"

# Production
config:
configPaths:
- "crates/apollo_deployments/resources/app_configs/base_layer_config.json"
- "crates/apollo_deployments/resources/app_configs/sequencer_config.json"
- "crates/apollo_deployments/resources/app_configs/prod_config.json"
```

### Feature-Based Configuration

```yaml
config:
configPaths:
- "crates/apollo_deployments/resources/app_configs/base_layer_config.json"
- "crates/apollo_deployments/resources/app_configs/sequencer_config.json"
- "crates/apollo_deployments/resources/app_configs/monitoring_config.json"
- "crates/apollo_deployments/resources/app_configs/logging_config.json"
- "crates/apollo_deployments/resources/app_configs/caching_config.json"
```

### Layered Configuration

```yaml
config:
configPaths:
- "crates/apollo_deployments/resources/app_configs/base_layer_config.json" # Base configuration
- "crates/apollo_deployments/resources/app_configs/sequencer_config.json" # Service-specific
- "crates/apollo_deployments/resources/app_configs/environment_config.json" # Environment-specific
- "crates/apollo_deployments/resources/app_configs/override_config.json" # Overrides
mountPath: "/config/sequencer/presets/" # Optional: defaults to "/config/sequencer/presets/"
```
1. **jsonnet is the source of truth**: change config through the overlay `node.jsonnet` / `chain_params`,
not by hand-editing the generated JSON.
2. **Keep secrets out of the ConfigMap**: sensitive values belong in a Secret / ExternalSecret, loaded as
the second `--config_file`.
3. **Use `sequencerConfig` sparingly**: for small per-deployment overrides only; prefer overlay changes for
anything structural.
4. **Always provide a secrets source**: the native loader needs two config files, so a ConfigMap alone will
not boot the node.
Loading