feat(configuration): stacked environment configuration overlays

[eaguad1337]

Summary

Adds backwards-compatible stacked / environment configuration: a base config plus thin per-environment overlays that only declare the values they change. This is an alternative to pushing every per-environment customization through .env variables.

• Configuration.load(overlays=None) — an optional list of overlay locations, merged in order on top of the base config.
• An overlay at {config.location}/environment/{APP_ENV} is applied automatically when that directory exists (a no-op otherwise), so the feature works out of the box.
• Overlays are partial: dict values are deep-merged (override wins, untouched keys preserved) while lists and scalars are replaced. A new pure deep_merge helper implements this and never mutates its inputs.

Relationship to #49

This supersedes #49, which proposed the same capability but could not be merged:

• CI was red on Python 3.10–3.13: its new test built a fresh Application() inside a plain unittest.TestCase. Because Container.objects is a class variable, that clobbered shared global state and cascaded ~15 failures across unrelated suites (RouteMiddlewareNotFound: 'web').
• It flipped the base loader to raise_exception=False, silently swallowing broken config modules for every user.
• Its list merge de-duplicated and reordered entries (surprising for e.g. providers/middleware).
• It had no framework wiring (an unintegrated primitive), shipped a fixture with a SyntaxError, and asserted brittle magic key-counts.

How this PR addresses each point

• No global-state pollution. The overlay tests reuse the shared wsgi application via Masonite's TestCase and only swap the bound config.location (restored in tearDown); a throwaway local Configuration is asserted on, so the globally bound config is never touched.
• No silent swallowing. The base load keeps raise_exception=True, and overlays use it too — a broken config/overlay module fails loudly. Missing overlay modules are naturally skipped, so partial overlays still work.
• Predictable list behavior. Lists are replaced, not merged/reordered.
• Wired in, tested, documented. The automatic environment overlay is applied inside load(), so every call site gets it for free. New unit tests cover deep_merge; integration-style tests cover explicit overlays, partial overlays, list replacement, automatic environment application, and the no-op case.

Backwards compatibility

Fully backwards compatible — no major version bump:

• All existing callers invoke load() with no arguments → unchanged behavior.
• With no config/environment/ folder and no explicit overlays, load() behaves byte-for-byte as before.
• merge_with semantics and the stored config representation are unchanged.

Tests

Full non-integration suite green locally (732 passed, 1 skipped), including a check that the new tests don't pollute ordering for the previously-affected suites (exceptions, http requests, response/url helpers).

DB_CONFIG_PATH=tests/integrations/config/database python -m pytest tests -m "not integrations"

Docs

The configuration page gains a "Stacked Configuration" section (environment-overlay convention, merge rules, explicit overlays), updated in the documentation repository.

[circulon]

@eaguad1337
WOW! nice work based on my clunky implementation ;)

issue 1:

• Predictable list behavior. Lists are replaced, not merged/reordered.
  Thinking about this, list overlays should:

• be appended to not replaced
• • eg adding more providers in an environment or from another location will replace the previously loaded providers
• if an item is already in the existing list it and in the overlay list should be skipped/ignored, thus preserving ordering
• • if the item is a string this should a case-insensitive search

issue 2:

• An overlay at {config.location}/environment/{APP_ENV} is applied automatically when that directory exists (a no-op otherwise), so the feature works out of the box.
  this is a tricky one as not all environments have the same naming convention.
  example:

• masonite uses these names in APP_ENV
• • development
• • local
• • production
• AWS uses these names
• • dev
• • staging
• • prod
• When developing locally via LocalStack I also use
• • local
    There is obviously several naming issues here (overlaps, missing and different)
    So using an auto find config folder based on environment name can become problematic deplending on the deployment mechanism used and how it names its deploy stages.

It just makes more sense to have the overlays as an entirely "opt in" mechanism.
This provides maximum flexibility and removes any possible issues due to differences in naming.

What do you think?

[circulon]

see my omment about replacing lists

[circulon]

See my comment about environment naming problems.

[eaguad1337]

Thanks for the review @circulon! Quick thoughts on each:

1. Lists: append instead of replace
Fair point for providers/middleware. The catch with append-only is you can add but never remove or reorder an entry from an overlay. So how about making it a choice: replace by default, append opt-in? The deep_merge list branch already leaves room for it.

I'd skip the case-insensitive match for de-dup, it'd treat MyProvider and myprovider as the same, which feels bug-prone. Exact match is safer.

2. Auto-discovery by env name
There's actually no hardcoded convention, it just uses whatever APP_ENV is set to as the folder name (prod -> environment/prod, local -> environment/local, etc.), and it's a no-op if the folder doesn't exist. So the naming mismatches don't collide.

But I agree the implicit loading can be surprising. The opt-in path you want already exists via load(overlays=[...]), the auto overlay just sits on top as a zero-config default.

Happy to gate it behind a flag or just document it better instead of dropping it.

[circulon]

@eaguad1337

1. Lists: append instead of replace Fair point for providers/middleware. The catch with append-only is you can add but never remove or reorder an entry from an overlay. So how about making it a choice: replace by default, append opt-in? The deep_merge list branch already leaves room for it.

Yeah i see yout point and having a choice sounds optimal with replace as the default.
so we could then do something like:

configuration = Configuration(self.application)
configuration.load([f"config.environment.core"])
configuration.load(["config.environment.prod", "config.environment.deployed"], replace=False)

which would:

• load any config in the config.location
• load core config overriding (replacing) any existing keys with the ones from core
• load prod config adding/updating any existing keys with the ones from prod
• load deployed adding/updating any existing keys with the ones from prod.
  maximum flexibility and easy to use and document

One thing I found in my considerations of this merging scenario was the "merge_deep" package
which does exactly what we are talking about and provides a Strategy option for reploace or additive.
https://mergedeep.readthedocs.io/en/latest/
using this package would mean not having to maintain our own "deep_merge" method at all
WDYT about using this instead?

I'd skip the case-insensitive match for de-dup, it'd treat MyProvider and myprovider as the same, which feels bug-prone. Exact match is safer.

Yeah true ignore this

2. Auto-discovery by env name There's actually no hardcoded convention, it just uses whatever APP_ENV is set to as the folder name (prod -> environment/prod, local -> environment/local, etc.), and it's a no-op if the folder doesn't exist. So the naming mismatches don't collide.

Aaah ok I see.
In our case we hardcode the APP_ENV into the application config in application.py as a per environment value .and use the config a a source of truth.
This is then not the same as the envirnoment var APP_ENV which I think is where the confusion lies.
Also the is_dev() and is_prodiction() are specifically linked to the environment var only

But I agree the implicit loading can be surprising. The opt-in path you want already exists via load(overlays=[...]), the auto overlay just sits on top as a zero-config default.
Happy to gate it behind a flag or just document it better instead of dropping it.

Yeah I see what your saying and if the env var APP_ENV is not set nothing gets done.

Sating this feature as an "opt-out" on the Config() class would work along with an internal flag denoting that the auto-load has been attempted.
Then multiple calls to .load() would not do the auto-load again.

Documenting with examples and the fact that this auto load functionality is specifically linked to the env var is the way to go I think

One thing I would change is to put the "environment" auto load BEFORE any overlays are added as these are opt-in and would currently be changed/replaced by the config in the auto-loaded environment folder

[circulon]

@eaguad1337
any movement on this please?