feat(qc,#15534): overlay Fear & Greed sur le regime markovien — verdict NO BEATS consolide (DEEP/qc)

MyIA.AI.Notebooks/QuantConnect/projects/Markov-Regime-Detection/README.en.md

@@ -1,27 +1,47 @@
 # Markov-Regime-Detection
 
 **Asset class:** US Equities/ETF (SPY, TLT, GLD)
-**Cloud project ID:** None (local only)
+**Cloud project ID:** `36387308`
 
 ## Description
 
-Markov regime detection using statsmodels MarkovRegression. Identifies 2-regime (bull/bear) states on SPY returns.
+Markov regime detection using statsmodels MarkovRegression. Identifies 2-regime (bull/bear) states on SPY returns, rotates monthly between SPY (calm regime) and TLT (turbulent regime), with a constant 10% GLD sleeve.
 
 **Consolidated from ML-HMM-Regime** (near-identical copy with same class name, same k_regimes=2, same allocation logic).
 
+## Fear & Greed overlay (v1.2, #15534)
+
+The project consolidates QuantConnect research article [#19465](https://www.quantconnect.com/research/19465/filtering-trades-with-the-fear-and-greed-index/) as an **optional overlay**, gated by the `use_feargreed` parameter (default `0` = v1.1 behavior strictly unchanged).
+
+**Primary source**: Huang, Jiang, Tu & Zhou (2015), "Investor Sentiment Aligned: A Powerful Predictor of Stock Returns", *Review of Financial Studies* 28(3), 791-837 — aligned investor sentiment as a powerful return predictor. The QuantConnect article remains the operational entry point; this reference is its primary academic source.
+
+**Mechanism**: the same tool as the main regime (`MarkovRegression`, `k_regimes=2`) is fit on the trailing Fear & Greed history, and SPY exposure is **halved** when the SPY regime asks for risk but the index regime sits in its greedy state (the remainder stays in cash — never TLT, which would blend two regime signals). The greedy state is identified by its **higher fitted mean**, never by a hardcoded regime number, so a regime renumbering across fits cannot silently flip the filter.
+
+**Dataset**: `FearGreedIndex`, exposed by the `AlgorithmImports` wildcard (no explicit import), ticker `"FG"`. Availability probed on QC Cloud 2026-09-11: >= 2500 daily rows delivered through Dec 2025, coverage from July 2014, not gated.
+
+**Seeds**: the article's seed sweep (30-50) has no object here — the article drew random trades, whereas this strategy is a deterministic monthly rotation whose `MarkovRegression` fit depends on no seed. Robustness is therefore tested by **market sub-periods** instead.
+
 ## How to Run
 
+**QC Cloud:** project `36387308`. Parameters: `use_feargreed` (0/1), `start_year`/`end_year`, `lookback_years` (default 3).
 **Lean CLI:** `lean backtest "MyIA.AI.Notebooks/QuantConnect/projects/Markov-Regime-Detection"`
-**QC Cloud:** Not yet deployed. Copy files to a new QC Cloud project to run.
 
 ## Backtest Metrics
 
-| Metric | Value |
-|--------|-------|
-| Sharpe Ratio | 0.408 |
-| Model | MarkovRegression |
-| Regimes | 2 (bull/bear) |
+Measured 2026-09-11 on QC Cloud, Interactive Brokers fees kept in both arms (unlike the article, which removed them via `ConstantFeeModel(0)`), monthly rebalance. Arm A is the baseline (`use_feargreed=0`), arm B the variant (`use_feargreed=1`); each window is an independent run (not a slice of the full run), since the 3-year warm-up and the regime state are not carried across windows.
+
+| Window | Arm | Sharpe | CAGR | MaxDD | Net profit | Net profit ($) | PSR | Orders |
+|---|---|---:|---:|---:|---:|---:|---:|---:|
+| 2015-2026 | A — baseline | 0.290 | 7.182% | 23.700% | 114.572% | 74,574.09 | 0.197% | 103 |
+| 2015-2026 | B — variant | 0.019 | 3.318% | 26.300% | 43.243% | 23,093.34 | 0.004% | 105 |
+| IS 2015-2020 | A — baseline | 0.229 | 4.735% | 16.700% | 26.038% | 14,496.06 | 1.859% | 44 |
+| IS 2015-2020 | B — variant | 0.051 | 2.653% | 14.500% | 13.996% | 7,532.51 | 0.526% | 45 |
+| OOS 2021-2026 | A — baseline | 0.297 | 8.991% | 23.700% | 53.825% | 26,626.13 | 2.596% | 41 |
+| OOS 2021-2026 | B — variant | -0.123 | 3.278% | 22.500% | 17.510% | 2,344.36 | 0.121% | 42 |
+
+**Verdict: NO BEATS in all three windows.** The overlay degrades Sharpe (to negative out-of-sample), CAGR and net profit everywhere. The one metric where B does better is IS drawdown (14.500% against 16.700%), a mechanical consequence of halved exposure rather than a better signal. The order count is nearly identical within each window (103/105, 44/45, 41/42), the expected behavior of an overlay that only changes the **exposure weight** and never the decision frequency — turnover is not exposed by the reading tool (qc-mcp-lite maps six statistics), so the order count is the declared proxy.
 
 ## Files
 
-- main.py - Strategy (v1.1, Markov regime, consolidated from ML-HMM-Regime)
+- `main.py` - Strategy (v1.2, Markov regime + optional Fear & Greed overlay)
+- `README.md` - French version

MyIA.AI.Notebooks/QuantConnect/projects/Markov-Regime-Detection/README.md

@@ -1,27 +1,47 @@
 # Markov-Regime-Detection
 
 **Classe d'actifs :** Actions/ETF américains (SPY, TLT, GLD)
-**ID projet Cloud :** Aucun (local uniquement)
+**ID projet Cloud :** `36387308`
 
 ## Description
 
-Détection de régime markovien avec `MarkovRegression` de statsmodels. Identifie 2 régimes (haussier/baissier) sur les rendements de SPY.
+Détection de régime markovien avec `MarkovRegression` de statsmodels. Identifie 2 régimes (haussier/baissier) sur les rendements de SPY, arbitre mensuellement entre SPY (régime calme) et TLT (régime agité), avec une couche GLD constante de 10 %.
 
 **Consolidé depuis ML-HMM-Regime** (copie quasi-identique avec même nom de classe, même `k_regimes=2`, même logique d'allocation).
 
+## Overlay Fear & Greed (v1.2, #15534)
+
+Le projet consolide l'article de recherche QuantConnect [#19465](https://www.quantconnect.com/research/19465/filtering-trades-with-the-fear-and-greed-index/) sous forme d'un **overlay optionnel**, activé par le paramètre `use_feargreed` (défaut `0` = comportement v1.1 strictement inchangé).
+
+**Source primaire** : Huang, Jiang, Tu & Zhou (2015), « Investor Sentiment Aligned: A Powerful Predictor of Stock Returns », *Review of Financial Studies* 28(3), 791-837 — l'indice de sentiment aligné comme prédicteur puissant des rendements. L'article QuantConnect reste le point d'entrée opérationnel, cette référence est la source académique primaire.
+
+**Mécanisme** : le même outil que le régime principal (`MarkovRegression`, `k_regimes=2`) est ajusté sur l'historique glissant de l'indice Fear & Greed, et l'exposition SPY est **réduite de moitié** quand le régime SPY demande du risque mais que le régime de l'indice est dans son état « greedy » (le reste demeure en cash — jamais en TLT, qui mélangerait deux signaux de régime). Le régime « greedy » est identifié par sa **moyenne ajustée** plus élevée, jamais par un numéro de régime codé en dur, pour qu'un renumérotage entre ajustements ne puisse pas inverser silencieusement le filtre.
+
+**Dataset** : `FearGreedIndex`, exposé par le wildcard `AlgorithmImports` (aucun import explicite), ticker `"FG"`. Sonde de disponibilité mesurée sur QC Cloud le 2026-09-11 : ≥ 2500 lignes quotidiennes livrées jusqu'à déc. 2025, couverture depuis juillet 2014, non gated.
+
+**Seeds** : le balayage de seeds de l'article (30-50) n'a pas d'objet ici — l'article tirait des trades aléatoires, alors que cette stratégie est un arbitrage mensuel déterministe dont l'ajustement `MarkovRegression` ne dépend d'aucune graine. La robustesse est donc testée par **sous-périodes de marché** à la place.
+
 ## Comment lancer
 
+**QC Cloud :** projet `36387308` (public). Paramètres : `use_feargreed` (0/1), `start_year`/`end_year`, `lookback_years` (défaut 3).
 **Lean CLI :** `lean backtest "MyIA.AI.Notebooks/QuantConnect/projects/Markov-Regime-Detection"`
-**QC Cloud :** Pas encore déployé. Copier les fichiers dans un nouveau projet QC Cloud pour lancer.
 
 ## Métriques de backtest
 
-| Métrique | Valeur |
-|----------|--------|
-| Sharpe Ratio | 0.408 |
-| Modèle | MarkovRegression |
-| Régimes | 2 (haussier/baissier) |
+Mesurées le 2026-09-11 sur QC Cloud, frais Interactive Brokers conservés dans les deux bras (contrairement à l'article, qui les supprimait par `ConstantFeeModel(0)`), réajustement mensuel. Le bras A est la baseline (`use_feargreed=0`), le bras B la variante (`use_feargreed=1`) ; chaque fenêtre est un run indépendant (pas une tranche du run complet), le réchauffement de 3 ans et l'état de régime n'étant pas reportés d'une fenêtre à l'autre.
+
+| Fenêtre | Bras | Sharpe | CAGR | MaxDD | Profit net | Profit net ($) | PSR | Ordres |
+|---|---|---:|---:|---:|---:|---:|---:|---:|
+| 2015-2026 | A — baseline | 0.290 | 7.182 % | 23.700 % | 114.572 % | 74 574.09 | 0.197 % | 103 |
+| 2015-2026 | B — variante | 0.019 | 3.318 % | 26.300 % | 43.243 % | 23 093.34 | 0.004 % | 105 |
+| IS 2015-2020 | A — baseline | 0.229 | 4.735 % | 16.700 % | 26.038 % | 14 496.06 | 1.859 % | 44 |
+| IS 2015-2020 | B — variante | 0.051 | 2.653 % | 14.500 % | 13.996 % | 7 532.51 | 0.526 % | 45 |
+| OOS 2021-2026 | A — baseline | 0.297 | 8.991 % | 23.700 % | 53.825 % | 26 626.13 | 2.596 % | 41 |
+| OOS 2021-2026 | B — variante | −0.123 | 3.278 % | 22.500 % | 17.510 % | 2 344.36 | 0.121 % | 42 |
+
+**Verdict : NO BEATS sur les trois fenêtres.** L'overlay dégrade le Sharpe (jusqu'à le rendre négatif en OOS), le CAGR et le profit net partout. La seule métrique où B fait mieux est le MaxDD en IS (14.500 % contre 16.700 %), conséquence mécanique d'une exposition réduite de moitié et non d'un meilleur signal. Le nombre d'ordres est quasi identique dans chaque fenêtre (103/105, 44/45, 41/42), ce qui est le comportement attendu d'un overlay qui ne modifie que le **poids** d'exposition et jamais la fréquence de décision — le turnover n'étant pas exposé par l'outil de lecture (qc-mcp-lite mappe six statistiques), le nombre d'ordres en est le proxy déclaré.
 
 ## Fichiers
 
-- `main.py` - Stratégie (v1.1, régime markovien, consolidé depuis ML-HMM-Regime)
+- `main.py` - Stratégie (v1.2, régime markovien + overlay Fear & Greed optionnel)
+- `README.en.md` - Version anglaise

MyIA.AI.Notebooks/QuantConnect/projects/Markov-Regime-Detection/main.py

@@ -21,6 +21,19 @@ class MarkovRegimeDetection(QCAlgorithm):
     - Causal forward-filter: explicit look-ahead guard in regime detection
     - Extended end date to 2026
 
+    Version 1.2 (#15534, consolidating research article #19465):
+    - Optional Fear & Greed exposure overlay, gated by the parameter
+      use_feargreed (default 0 = baseline behavior strictly unchanged)
+    - The overlay fits the same MarkovRegression tool on the trailing
+      FearGreedIndex history (k_regimes=2, the article's setting) and
+      halves the SPY allocation when the SPY regime asks for SPY but the
+      index regime sits in its greedy state (entries gated in greed, the
+      article's filter rule, adapted to monthly rotation)
+    - The greedy regime is identified by its fitted mean index level,
+      never by a hardcoded regime number
+    - start_year/end_year parameters (defaults preserve 2015-2026) enable
+      IS/OOS and sub-period runs without code forks
+
     Version 1.0 - Binary Regime Switching:
     - Binary allocation: SPY in low volatility regime, TLT in high volatility
     - Monthly rebalance schedule
@@ -41,8 +54,8 @@ class MarkovRegimeDetection(QCAlgorithm):
     """
 
     def initialize(self):
-        self.set_start_date(2015, 1, 1)
-        self.set_end_date(2026, 1, 1)
+        self.set_start_date(int(self.get_parameter('start_year', 2015)), 1, 1)
+        self.set_end_date(int(self.get_parameter('end_year', 2026)), 1, 1)
         self.set_cash(100_000)
         self.set_brokerage_model(BrokerageName.INTERACTIVE_BROKERS_BROKERAGE, AccountType.MARGIN)
 
@@ -57,6 +70,17 @@ def initialize(self):
         self._lookback_period = timedelta(
             self.get_parameter('lookback_years', 3) * 365
         )
+
+        # Fear & Greed overlay (#15534 / article #19465). Default OFF: the
+        # baseline never subscribes to the dataset and its behavior is
+        # byte-for-byte the v1.1 logic.
+        self._use_fg = self.get_parameter('use_feargreed', 0) == 1
+        self._fg_symbol = None
+        if self._use_fg:
+            # FearGreedIndex comes with the AlgorithmImports wildcard; the
+            # custom ticker is 'FG' (official dataset docs), coverage from
+            # July 2014, daily.
+            self._fg_symbol = self.add_data(FearGreedIndex, "FG").symbol
         self._gld_weight = 0.10
         self._equity_weight = 0.80  # Max allocation to SPY or TLT
         self._confirmation_threshold = 0.55
@@ -88,7 +112,10 @@ def initialize(self):
         # Track previous regime to avoid unnecessary rebalancing
         self._previous_regime = None
 
-        self.log("MarkovRegimeDetection v1.1 initialized: anti-micro-rebalancing, extended to 2026")
+        self.log(
+            f"MarkovRegimeDetection v1.2 initialized: feargreed_overlay="
+            f"{'ON' if self._use_fg else 'OFF'}"
+        )
 
     def _update_event_handler(self, indicator, indicator_data_point):
         """Update trailing returns series."""
@@ -103,6 +130,33 @@ def _update_event_handler(self, indicator, indicator_data_point):
             t - self._daily_returns.index <= self._lookback_period
         ]
 
+    def _fg_greedy(self):
+        """True when the trailing Fear & Greed regime is its greedy state.
+
+        Mirrors the #19465 filter: MarkovRegression(k_regimes=2) on the
+        trailing index history (the article's exact setting), entries gated
+        in the greedy regime. The greedy state is the one with the higher
+        fitted mean index level, so a regime renumbering across fits cannot
+        silently flip the filter.
+        """
+        if not self._use_fg or self._fg_symbol is None:
+            return False
+        df = self.history(self._fg_symbol, self._lookback_period + timedelta(7))
+        if df is None or df.empty:
+            return False
+        if isinstance(df.index, pd.MultiIndex):
+            series = df.reset_index(level=0, drop=True)["value"]
+        else:
+            series = df["value"]
+        series = series.dropna()
+        if len(series) < 100:
+            return False
+        fitted = MarkovRegression(series, k_regimes=2).fit()
+        regime = int(fitted.smoothed_marginal_probabilities.values.argmax(axis=1)[-1])
+        means = [float(fitted.params[f"const[{i}]"]) for i in range(2)]
+        greedy_regime = 0 if means[0] >= means[1] else 1
+        return regime == greedy_regime
+
     def _trade(self):
         """
         Detect current regime and rebalance portfolio using binary allocation.
@@ -146,6 +200,13 @@ def _trade(self):
                     regime_name = "LOW_VOL"
                     spy_weight = self._equity_weight
                     tlt_weight = 0.0
+                    if self._fg_greedy():
+                        # #15534 overlay: the SPY regime asks for risk but
+                        # the Fear & Greed regime sits in greed -- halve the
+                        # exposure, remainder stays in cash (never TLT: that
+                        # would blend the two regime signals).
+                        regime_name = "LOW_VOL/FG_GREEDY"
+                        spy_weight = self._equity_weight * 0.5
                 else:
                     # High volatility -> bearish -> TLT
                     regime_name = "HIGH_VOL"