Skip to content

Commit 9e6e832

Browse files
kevincodex1openclaudeai
kevincodex1
and
openclaudeai
authored
Infra reorg (#27)
* refactor: move fly.toml into infra/fly/ Deployment configs now live under infra/, organized per target. Dockerfiles and docker-compose.yml intentionally stay at the repo root (shared by CI, build scripts, and the macOS app). Deploy with: fly deploy -c infra/fly/fly.toml Co-Authored-By: OpenClaude <openclaude@gitlawb.com> * feat(infra): Terraform for single-EC2 AWS deployment infra/aws/ provisions a production-leaning single-node deployment: - t4g.small (AL2023 arm64) running the published ghcr.io/gitlawb/node image + postgres:16 via docker compose - separate encrypted EBS data volume (prevent_destroy) with daily DLM snapshots; survives instance replacement - postgres password via random_password -> SSM SecureString, fetched at boot by instance profile (never in user-data); optional secrets (operator key, Pinata JWT, S3 secret) follow the same path - SSM Session Manager access (no SSH by default), IMDSv2 required, metrics port closed unless explicitly opened - SSM command document for image upgrades (user-data runs once) Verified: terraform fmt/init/validate clean; rendered compose passes docker compose config; rendered user-data passes bash -n. Co-Authored-By: OpenClaude <openclaude@gitlawb.com> * fix(infra): address CodeRabbit review feedback - api_url output now honors the public_url override (local.public_url) - disable xtrace around SSM secret fetch + .env write so secret values never reach the bootstrap log - add retry/timeout flags to the docker compose download - validate snapshot_retain_count (1-1000) at plan time - add language tags to fenced tree blocks (MD040) Co-Authored-By: OpenClaude <openclaude@gitlawb.com> * fix(infra): address PR #27 review feedback - security group now derives its VPC from the selected subnet, so subnet_id overrides into non-default VPCs work - s3_access_key_id moves to SSM SecureString (same path as the secret key) instead of being embedded in user-data and state - ignore user_data drift on the instance: it only runs at first boot, so re-rendering caused pointless stop/starts; README documents the -replace workflow for config changes - DLM snapshot targeting is stack-specific (Snapshot + Name tags), not any Snapshot=true volume in the account - comment in fly.toml that it targets the shared test instance - user-data compose up gains --remove-orphans (consistent with the upgrade SSM document) - node service gets an HTTP /health healthcheck in compose Co-Authored-By: OpenClaude <openclaude@gitlawb.com> * feat(infra): optional customer-managed KMS key for SSM secrets Add ssm_kms_key_id (default null = AWS-managed aws/ssm key). When set, all five SecureString parameters are encrypted with the CMK and the instance role gains kms:Decrypt scoped to that key — without it the boot-time secret fetch would fail. Co-Authored-By: OpenClaude <openclaude@gitlawb.com> * fix(infra): scope CMK kms:Decrypt to this stack's SSM parameters The grant on the customer-managed key was unconditioned, allowing the instance role to decrypt any ciphertext under the same CMK. Restrict it with kms:ViaService = ssm.<region>.amazonaws.com and kms:EncryptionContext:PARAMETER_ARN limited to the stack's parameter ARNs. Co-Authored-By: OpenClaude <openclaude@gitlawb.com> --------- Co-authored-by: OpenClaude <openclaude@gitlawb.com>
1 parent 09a3397 commit 9e6e832

13 files changed

Lines changed: 1147 additions & 2 deletions

‎docs/RUN-A-NODE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Step-by-step guide to staking $GITLAWB, registering your node on-chain, and earn
88

99
- A wallet with at least **10,000 $GITLAWB** (minimum stake) plus a small amount of ETH on Base for gas
1010
- Docker or Rust 1.91+ (for running the node process)
11-
- A public HTTP URL (your-host.com) — can be a VPS, Fly.io app, or anything reachable
11+
- A public HTTP URL (your-host.com) — can be a VPS, Fly.io app, or anything reachable. A Fly.io config is provided at `infra/fly/fly.toml` (deploy from the repo root with `fly deploy -c infra/fly/fly.toml`)
1212

1313
---
1414

‎infra/README.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# infra/
2+
3+
Deployment configuration, organized by target — one subdirectory per platform.
4+
5+
```text
6+
infra/
7+
├── fly/
8+
│ └── fly.toml # Fly.io app config (gitlawb-node-test)
9+
└── aws/ # Terraform: single EC2 + Docker (see aws/README.md)
10+
```
11+
12+
## Deploying to Fly.io
13+
14+
Run from the **repo root** so the Docker build context includes `crates/`,
15+
`Cargo.toml`, and `bootstrap-peers.json`:
16+
17+
```sh
18+
fly deploy -c infra/fly/fly.toml
19+
```
20+
21+
The `dockerfile` path inside `fly.toml` is resolved relative to the config
22+
file, so it points to `../../Dockerfile`.
23+
24+
## Deploying to AWS
25+
26+
See [`aws/README.md`](aws/README.md) — Terraform for a single EC2 instance
27+
running the published `ghcr.io/gitlawb/node` image with Docker compose.
28+
29+
## What intentionally stays at the repo root
30+
31+
- `Dockerfile` / `Dockerfile.bins` — shared by the release CI workflow
32+
(`.github/workflows/release.yml`), `scripts/build-bins.sh`, and Fly builds.
33+
- `docker-compose.yml` — local dev stack; bundled into the macOS app by
34+
`scripts/build-macos-app.sh` and used for repo detection by the app.
35+
36+
Future targets should follow the same per-platform layout.

‎infra/aws/.gitignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Terraform state and workdir — never commit (state contains secrets)
2+
*.tfstate
3+
*.tfstate.*
4+
.terraform/
5+
crash.log
6+
crash.*.log
7+
8+
# Local variable files may contain secrets; keep the example only
9+
*.tfvars
10+
*.tfvars.json
11+
!terraform.tfvars.example
12+
13+
# Note: .terraform.lock.hcl IS committed (reproducible provider versions)

‎infra/aws/.terraform.lock.hcl‎

Lines changed: 46 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎infra/aws/README.md‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# gitlawb node on AWS (Terraform)
2+
3+
Single EC2 instance running the published node image + Postgres via Docker
4+
compose, with a persistent encrypted EBS volume, Elastic IP, SSM access, and
5+
daily snapshots.
6+
7+
```text
8+
Elastic IP ──► EC2 t4g.small (Amazon Linux 2023, arm64)
9+
7545/tcp docker compose:
10+
7546/udp ├─ node (ghcr.io/gitlawb/node, pulled — not built)
11+
└─ postgres:16-alpine
12+
EBS gp3 volume mounted at /mnt/data
13+
├─ node/ → container /data (repos + identity key)
14+
└─ postgres/ → postgres data dir
15+
```
16+
17+
## Prerequisites
18+
19+
- Terraform ≥ 1.6
20+
- AWS credentials configured (`aws sts get-caller-identity` works)
21+
- AWS CLI + [Session Manager plugin](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html) (for shell access)
22+
- A default VPC in the target region (or pass `subnet_id`)
23+
24+
## Quick start
25+
26+
```sh
27+
cd infra/aws
28+
cp terraform.tfvars.example terraform.tfvars # edit: public_url at minimum
29+
terraform init
30+
terraform plan
31+
terraform apply # ⚠ creates billable resources (~$25/mo: EC2 + EBS + snapshots)
32+
```
33+
34+
After apply (~3-5 min for first boot to pull images and start):
35+
36+
```sh
37+
curl "$(terraform output -raw api_url)/health"
38+
```
39+
40+
## ⚠ First boot: back up the identity key
41+
42+
The node generates `/data/keys/identity.pem` on first start — it defines the
43+
node's DID. **Losing it permanently changes the node's identity.** Back it up
44+
immediately:
45+
46+
```sh
47+
$(terraform output -raw ssm_session_command)
48+
# in the session:
49+
sudo cat /mnt/data/node/keys/identity.pem
50+
```
51+
52+
Store the key somewhere safe (password manager / offline). The volume's
53+
`prevent_destroy` guard and daily DLM snapshots protect against accidents, but
54+
are not a substitute for an offline backup.
55+
56+
## Shell access
57+
58+
SSM Session Manager — no SSH port, no keys to manage:
59+
60+
```sh
61+
$(terraform output -raw ssm_session_command)
62+
```
63+
64+
Bootstrap log: `/var/log/gitlawb-bootstrap.log`. Stack lives in `/opt/gitlawb`
65+
(`docker compose ps`, `docker compose logs node`).
66+
67+
SSH is off by default; set `ssh_ingress_cidr` + `ssh_key_name` if you need it.
68+
69+
## Upgrading the node
70+
71+
User-data only runs at first boot, so upgrades go through SSM:
72+
73+
```sh
74+
$(terraform output -raw upgrade_command)
75+
```
76+
77+
This runs `docker compose pull && docker compose up -d` on the instance.
78+
79+
- With `image_tag = "latest"` (default) that picks up the newest release.
80+
- With a **pinned tag**, first edit the tag in `/opt/gitlawb/compose.yaml` on
81+
the instance (via SSM session), then run the upgrade command — and keep
82+
`image_tag` in terraform.tfvars in sync so a future instance replacement
83+
boots the same version.
84+
85+
Replace the instance itself (OS/AMI/instance-type changes) with
86+
`terraform apply -replace=aws_instance.node` — the data volume reattaches and
87+
`/data` (including the identity key) survives.
88+
89+
## Changing configuration
90+
91+
User-data only runs at first boot, and the instance ignores `user_data` drift
92+
(`ignore_changes`), so editing terraform.tfvars values that feed the bootstrap
93+
(`bootstrap_peers`, `public_url`, integrations, `image_tag`) does **not**
94+
affect a running instance on `terraform apply`. To roll out such changes,
95+
either edit `/opt/gitlawb/.env` on the instance (SSM session, then
96+
`docker compose up -d`), or replace the instance:
97+
98+
```sh
99+
terraform apply -replace=aws_instance.node
100+
```
101+
102+
The data volume reattaches; repos, postgres data, and the identity key survive.
103+
104+
## Remote state (optional)
105+
106+
Local state is the default. To move state to S3: create a versioned bucket,
107+
uncomment the `backend "s3"` block in `versions.tf`, then:
108+
109+
```sh
110+
terraform init -migrate-state
111+
```
112+
113+
## Teardown
114+
115+
`terraform destroy` will **fail on the data volume by design**
116+
(`prevent_destroy`). To tear everything down:
117+
118+
1. Back up the identity key (above) and take a final snapshot if you may return.
119+
2. Remove the `prevent_destroy` line from `aws_ebs_volume.data` in `main.tf`.
120+
3. `terraform destroy`.
121+
122+
Note: DLM snapshots created by the policy are not deleted by destroy — clean
123+
them up in the EC2 console if unwanted. The Elastic IP is released on destroy.
124+
125+
## Security notes
126+
127+
- Postgres password: generated by Terraform, stored as an SSM SecureString,
128+
fetched at boot via the instance profile — never in user-data or state-free
129+
files on disk (only in `/opt/gitlawb/.env`, mode 600). It IS in Terraform
130+
state — treat state as sensitive (another reason for the S3 backend).
131+
- Sensitive optional vars (`operator_private_key`, `pinata_jwt`,
132+
`s3_access_key_id`, `s3_secret_access_key`) follow the same SSM path.
133+
- SSM secrets use the AWS-managed `aws/ssm` key by default; set
134+
`ssm_kms_key_id` to encrypt with a customer-managed KMS key instead (the
135+
instance role is granted `kms:Decrypt` on that key automatically).
136+
- IMDSv2 is required; metrics port is closed unless `metrics_ingress_cidr` is set.
137+
- The node serves plain HTTP on 7545. For TLS, put a DNS name + proxy
138+
(ALB/CloudFront/Caddy) in front and set `public_url` accordingly.

‎infra/aws/compose.yaml.tftpl‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Rendered by Terraform (compose.yaml.tftpl) and written to /opt/gitlawb/compose.yaml
2+
# by user-data. Adapted from the repo-root docker-compose.yml: pulls the published
3+
# image instead of building, and binds data dirs to the dedicated EBS volume.
4+
# `$${VAR}` entries are resolved by docker compose from /opt/gitlawb/.env at runtime.
5+
6+
services:
7+
postgres:
8+
image: postgres:16-alpine
9+
environment:
10+
POSTGRES_DB: ${pg_db}
11+
POSTGRES_USER: ${pg_user}
12+
POSTGRES_PASSWORD: $${POSTGRES_PASSWORD}
13+
volumes:
14+
- /mnt/data/postgres:/var/lib/postgresql/data
15+
healthcheck:
16+
test: ["CMD-SHELL", "pg_isready -U ${pg_user}"]
17+
interval: 10s
18+
timeout: 5s
19+
retries: 5
20+
restart: unless-stopped
21+
22+
node:
23+
image: ${image_repo}:${image_tag}
24+
depends_on:
25+
postgres:
26+
condition: service_healthy
27+
ports:
28+
- "${gitlawb_port}:${gitlawb_port}" # HTTP API + git smart-HTTP
29+
- "${p2p_port}:${p2p_port}/udp" # libp2p QUIC
30+
%{ if expose_metrics ~}
31+
- "${metrics_port}:${metrics_port}" # Prometheus /metrics
32+
%{ endif ~}
33+
volumes:
34+
- /mnt/data/node:/data
35+
environment:
36+
DATABASE_URL: postgresql://${pg_user}:$${POSTGRES_PASSWORD}@postgres:5432/${pg_db}
37+
GITLAWB_HOST: 0.0.0.0
38+
GITLAWB_PORT: "${gitlawb_port}"
39+
GITLAWB_P2P_PORT: "${p2p_port}"
40+
GITLAWB_REPOS_DIR: /data/repos
41+
GITLAWB_KEY: /data/keys/identity.pem
42+
GITLAWB_PUBLIC_URL: $${GITLAWB_PUBLIC_URL}
43+
GITLAWB_BOOTSTRAP_PEERS: $${GITLAWB_BOOTSTRAP_PEERS}
44+
GITLAWB_AUTO_SYNC: $${GITLAWB_AUTO_SYNC}
45+
GITLAWB_MAX_PACK_BYTES: $${GITLAWB_MAX_PACK_BYTES}
46+
# On-chain PoS (optional — empty unless set in terraform.tfvars)
47+
GITLAWB_CHAIN_RPC_URL: $${GITLAWB_CHAIN_RPC_URL}
48+
GITLAWB_CONTRACT_NODE_STAKING: $${GITLAWB_CONTRACT_NODE_STAKING}
49+
GITLAWB_OPERATOR_PRIVATE_KEY: $${GITLAWB_OPERATOR_PRIVATE_KEY}
50+
# IPFS pinning (optional)
51+
GITLAWB_PINATA_JWT: $${GITLAWB_PINATA_JWT}
52+
# Shared S3-compatible pack storage (optional)
53+
GITLAWB_TIGRIS_BUCKET: $${GITLAWB_TIGRIS_BUCKET}
54+
AWS_ACCESS_KEY_ID: $${S3_ACCESS_KEY_ID}
55+
AWS_SECRET_ACCESS_KEY: $${S3_SECRET_ACCESS_KEY}
56+
AWS_ENDPOINT_URL_S3: $${S3_ENDPOINT_URL}
57+
healthcheck:
58+
test: ["CMD-SHELL", "curl -sf http://localhost:${gitlawb_port}/health || exit 1"]
59+
interval: 30s
60+
timeout: 5s
61+
retries: 3
62+
start_period: 30s
63+
restart: unless-stopped

0 commit comments

Comments
 (0)