Repository navigation
Expand file tree
/
Copy path.env.example
More file actions
273 lines (236 loc) · 11.6 KB
/
Copy path.env.example
File metadata and controls
273 lines (236 loc) · 11.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
# Copy to .env and work down. Ordered by how much attention each section needs:
# the top will not run without you, the bottom has defaults you can ignore for
# years. Sections 1 and 2 are the ones worth reading before a first deploy.
#
# 1. Required nothing works until these are right
# 2. Stability defaults that keep the service alive under load
# 3. Security review before this is reachable from the internet
# 4. Optional features off, or inert, until you turn them on
# 5. Tuning rarely touched
# ===========================================================================
# 1. REQUIRED
# ===========================================================================
# The one value you must change. At least 32 characters and not a placeholder,
# or the service refuses to boot. Generate one with: openssl rand -hex 32
TOKEN=REPLACE_ME
# App
APP_NAME=cdn
APP_PORT=9090
# Used to build the links returned by upload responses, so callers store what
# this says. Getting it wrong hands them URLs that point somewhere else.
APP_URL=https://cdn.example.com
# MinIO
#
# Pin the server image on any deployment holding real data. Blank means latest,
# which is fine for a fresh install and not fine for an existing one: MinIO
# upgrades forward across the on-disk format but never backward, so an
# accidental jump is not something you undo by changing the tag back.
#
# MINIO_VERSION=RELEASE.2022-10-24T18-35-07Z
MINIO_VERSION=
# Image repository without the tag. MinIO no longer publishes community images
# anonymously, so set this to the name the image already has on the host (for
# example minio/minio) instead of relying on a pull. Blank means quay.io/minio/minio.
MINIO_IMAGE=
MINIO_ENDPOINT=cdn-minio:9000
MINIO_ROOT_USER=your-minio-user
MINIO_ROOT_PASSWORD=your-minio-password
# Set true when MinIO is served over TLS
MINIO_USE_SSL=false
# Redis. Credentials and database go inside the URL: redis://:pass@host:6379/0
REDIS_URL=redis://cdn-redis:6379
# ===========================================================================
# 2. STABILITY
#
# These are not micro-tuning. Left at ImageMagick's own defaults, three replicas
# on an 8-core host grew to 2.5-2.9 GB each and were OOM-killed several times a
# day. Change them only with a measurement in hand.
# ===========================================================================
# The single most important number here for memory. ImageMagick otherwise starts
# one OpenMP thread per core, and because its buffers come from malloc rather
# than the Go heap, glibc keeps a per-thread arena that it never returns to the
# OS: RSS only climbs. At 2 threads the workload above holds ~200 MB per replica.
# Raise it only alongside MALLOC_ARENA_MAX in the dockerfile.
IMAGICK_THREAD_LIMIT=2
# Concurrent decodes on the read path. OPTIMIZE_MAX_CONCURRENT covers uploads
# only, which left GET unbounded: one gallery page could start twenty full
# decodes at once. A request that finds every slot busy waits, and only if the
# wait runs out is the original served unresized (marked no-store so no cache
# keeps it under the resized URL).
RESIZE_MAX_CONCURRENT=4
RESIZE_QUEUE_TIMEOUT_SEC=10
# Per-container memory ceiling, read by docker-compose.yml. Without it the
# kernel's OOM killer works host-wide and picks the largest process, which is as
# likely to be MinIO, and every object it serves, as the replica that misbehaved.
API_MEM_LIMIT=2500m
API_MEMSWAP_LIMIT=3g
# Go runtime. GOMEMLIMIT is a soft ceiling that makes the collector work harder
# as it is approached; it bounds the Go heap only, not ImageMagick's C
# allocations, so keep it comfortably under API_MEM_LIMIT.
GOMEMLIMIT=1200MiB
GOGC=50
# ImageMagick resource limits (decompression bombs, oversized resizes)
IMAGICK_MEMORY_LIMIT_MB=512
IMAGICK_MAP_LIMIT_MB=1024
IMAGICK_AREA_LIMIT_MP=256
IMAGICK_DISK_LIMIT_MB=2048
IMAGICK_TIME_LIMIT_SEC=60
IMAGICK_WIDTH_LIMIT=16384
IMAGICK_HEIGHT_LIMIT=16384
# Maximum width/height accepted for on-the-fly resize (larger requests are clamped)
MAX_RESIZE_DIMENSION=4096
# ===========================================================================
# 3. SECURITY
# ===========================================================================
# Which interface MinIO and Redis are published on, read by docker-compose.yml.
#
# Leave these on loopback. Nothing in the stack needs the ports published: the
# API replicas reach both over the compose network. Publishing them puts an S3
# endpoint holding every object, an admin console, and a password-less Redis on
# the public internet of any host with a public IP, and a host firewall does not
# cover it, because Docker writes its own iptables rules and bypasses ufw.
#
# Verify from OUTSIDE the host: nc -zv <public-ip> 9000 9001 6379
MINIO_BIND=127.0.0.1
REDIS_BIND=127.0.0.1
# Upload validation. Two gates, neither of them client-controlled: the extension
# allowlist and the file's own leading bytes. Setting this false disables both at
# once, so .php and .html become uploadable. Turn it off only where every caller
# is trusted; if a legitimate format is being rejected, the fix is to add its
# extension, not to disable the gate. See docs/api.md.
VALIDATE_FILE=true
MAX_FILE_SIZE=104857600
# Rate limiting. Duration is in MINUTES.
RATE_LIMIT=100
UPLOAD_RATE_LIMIT=50
RATE_LIMIT_DURATION=1
# SSRF guard for /upload-url. True only for self-hosted setups that deliberately
# fetch from internal hosts.
UPLOAD_URL_ALLOW_PRIVATE=false
# Request hardening
MAX_BATCH_FILES=100
READ_BUFFER_SIZE_KB=1024
# ===========================================================================
# 4. OPTIONAL FEATURES
# ===========================================================================
# Bucket-scoped tokens (see config/tokens.template.json). Each entry authorizes
# writes to a single bucket via "Authorization: Bearer <bucket>:<token>". When
# the file is absent, TOKEN above is the only accepted credential. Read once at
# boot; restart after editing it.
TOKENS_FILE=config/tokens.json
# AWS
#
# Leave these blank to run on MinIO alone. The archive below switches itself off
# when they are empty, silently and by design: a deployment without AWS is a
# supported deployment, not a misconfigured one. The placeholder values shipped
# here count as empty, so copying this file does not produce an archive that
# reports itself enabled and then fails every upload.
AWS_ACCESS_KEY_ID=your-aws-key
AWS_SECRET_ACCESS_KEY=your-aws-secret
AWS_SESSION_TOKEN=
AWS_REGION=eu-central-1
AWS_BUCKET=
# Cold-storage archive
#
# When AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION are all set, every
# upload is mirrored to S3 in the Glacier Instant Retrieval class, and objects
# MinIO no longer has are served from there at the same URL. Instant Retrieval is
# what makes that possible: it answers a normal GET in milliseconds, where the
# older Glacier classes need a restore request and hours of waiting.
#
# Set false to keep AWS credentials configured for other purposes without
# archiving to them.
ARCHIVE_ENABLED=true
# Blank means one S3 bucket per MinIO bucket, sharing its name and keys. Setting
# a name puts everything in that one bucket under a "<minio-bucket>/" prefix.
#
# Setting it is RECOMMENDED. S3 bucket names are unique across every AWS account
# on earth, while MinIO's only have to be unique locally, so ordinary names like
# "media" or "uploads" cannot be recreated in S3 and bucket parity fails for them
# partway through a backfill. A single bucket also means a new MinIO bucket needs
# no AWS work: with parity, uploads auto-create the MinIO bucket but not the S3
# one, and that bucket then quietly never gets archived.
#
# Bucket parity is the better fit when you have a few distinctively-named buckets
# and want per-bucket IAM or replication rules in AWS.
#
# Choose before the first backfill and do not change it afterwards: the mapping is
# resolved fresh on every request, so switching strands everything already
# archived at keys nothing will look for again. See docs/archive.md.
ARCHIVE_BUCKET=
# Which local buckets are archived. Blank means all of them, which is the
# default because a deployment that turns the archive on usually wants its data
# covered, and an opt-in list would leave new buckets silently unprotected until
# somebody remembered to add them.
#
# Comma-separated to narrow it: ARCHIVE_ONLY_BUCKETS=photos,documents
#
# Note the name is not ARCHIVE_BUCKETS. That would be one letter from
# ARCHIVE_BUCKET above, which sets the destination, and a typo between the two
# would configure something entirely different without looking wrong.
#
# This gates writing, never reading. An object already archived is still served
# from the archive even if its bucket is later removed from this list, so
# narrowing the scope cannot break URLs. It does mean anything not yet archived
# in a removed bucket stays local forever, and retention will skip that bucket
# entirely rather than warn about every object in it.
ARCHIVE_ONLY_BUCKETS=
# Retention: free local disk by deleting objects the archive already holds.
#
# Off by default, and reporting-only the first time it is switched on, because
# the one thing this job does is delete files. An object is removed from MinIO
# only after the archive has confirmed a copy of the same size; anything
# unverified is kept and logged. Nothing is ever deleted from the archive.
#
# Before enabling on an existing deployment, check that older objects really were
# archived: uploads made before this feature existed have no copy, and the job
# will (correctly) refuse to delete them. Run `backfill` first.
#
# RETENTION_DAYS is compared against the object's stored timestamp. On a CDN that
# objects were migrated into, that timestamp is the migration date rather than the
# content's age, and no window can separate a five-year-old document from
# yesterday's photo. Drive archiving from POST /archive instead where that is the
# case. Years, in days: 730 = 2, 1095 = 3, 1825 = 5.
RETENTION_ENABLED=false
RETENTION_DRY_RUN=true
RETENTION_DAYS=365
RETENTION_INTERVAL_HOURS=24
# Blank means every bucket; comma-separated to limit the sweep.
RETENTION_BUCKETS=
# Each eligible object costs one HeadObject against the archive, so a sweep of a
# few million objects is unfinishable one at a time.
RETENTION_MAX_CONCURRENT=8
# Upload optimization, opt-in per request via optimize=true. Defaults are
# visually lossless.
OPTIMIZE_MAX_DIMENSION=2560
OPTIMIZE_JPEG_QUALITY=85
OPTIMIZE_WEBP_QUALITY=85
OPTIMIZE_PNG_QUALITY=95
OPTIMIZE_MAX_CONCURRENT=4
# Feature flags
DISABLE_DELETE=false
DISABLE_UPLOAD=false
DISABLE_GET=false
# ===========================================================================
# 5. TUNING
# ===========================================================================
# Worker pool
WORKER_POOL_SIZE=5
WORKER_QUEUE_SIZE=10
WORKER_MAX_RETRIES=3
WORKER_RETRY_DELAY_MS=1000
WORKER_BATCH_SIZE=10
WORKER_FLUSH_TIMEOUT_MS=5000
WORKER_MAX_CONCURRENT=5
# Version of the mc client container (docker compose --profile tools up -d mc).
# Pin this if you run an older MinIO server: newer mc is fine for ls/du/cp, but
# mc admin commands are sensitive to the server version.
MC_VERSION=latest
# ===========================================================================
# 6. HOST MONITORING (optional, docs/deployment.md "Host monitoring")
# ===========================================================================
# Uncomment to start the hostwatch container with the stack. It mails when disk,
# inode or memory usage on the host crosses a threshold. Its settings live in
# hostwatch.env (copy hostwatch.env.example), not here, so that the SMTP
# password never reaches the API replicas, which also read this file.
#COMPOSE_PROFILES=hostwatch