Clarify Forge scoring confidence intervals

pages/build/forge/competitions.mdx

@@ -2,8 +2,8 @@
 title: Allora Forge Competitions
 description: How Allora Forge competitions work — compete on live topics, build a testnet track record, and graduate to mainnet where top performers earn ALLO rewards.
 persona: ML builder
-verified_against: forge.allora.network (live site + public API, 2026-07-30), allora-forge-builder-kit README (main @ 8ef3200)
-last_reviewed: 2026-07-30
+verified_against: forge.allora.network (live site + public API, 2026-07-30), allora-forge-builder-kit README (main @ 8ef3200), Allora Research promotion/relegation post (2026-08-06)
+last_reviewed: 2026-08-12
 ---
 
 import { Callout } from 'nextra/components'
@@ -28,9 +28,49 @@ The cycle repeats every epoch, so a competition is not a one-shot submission: yo
 
 ## Scoring
 
-Scoring happens on-chain. Each epoch, the network compares every submitted inference against the topic's ground truth and ranks workers by loss.
+Scoring happens on-chain. Each epoch, the network compares every submitted inference against the topic's ground truth and ranks workers by loss. Forge then summarizes that history into promotion-readiness metrics so you can see whether a worker has enough evidence of skill to move from testnet to mainnet.
 
-The [Forge Builder Kit](https://github.com/allora-network/allora-forge-builder-kit) mirrors this methodology off-chain: its `PerformanceEvaluator` grades your model against Allora's scoring methodology *before* you deploy. It scores seven metrics — directional accuracy (plus its confidence-interval lower bound and p-value), Pearson correlation (plus its p-value), weighted-RMSE improvement, and CZAR improvement — each with a pass/fail threshold, and maps the composite to a letter grade from A+ to F. A higher grade means better generalization and a higher expected score on the network.
+Promotion is evaluated **per worker, per topic**. A worker can be eligible on one topic and ineligible on another because the horizon, participation history, ground truth lag, and submitted inferences are topic-specific.
+
+The key dashboard detail is that some eligibility rows use a **confidence interval (CI)**, not only the point estimate shown as "your value." When a metric is displayed as:
+
+`your value (lower CI, upper CI)`
+
+the top-level value is the worker's point estimate for that metric. The values in parentheses are the confidence interval around that estimate. For promotion checks that say the **lower CI** must exceed a threshold, the relevant number is the **first value inside the parentheses**, not the top-level point estimate. For example, if directional accuracy appears as `72.3% (44.7%, 100%)`, the lower CI is `44.7%`; this would not pass a `> 50%` lower-bound threshold even though the point estimate is above 50%.
+
+Allora uses confidence intervals because workers may have different numbers of submissions. More submissions generally give more evidence and a tighter interval; fewer submissions leave more uncertainty. Where adjacent epochs share overlapping ground-truth windows, Allora corrects the interval calculation using an effective sample size so repeated, correlated observations do not overstate confidence.
+
+For mainnet promotion, a worker must pass every promotion metric on the topic:
+
+- **Effective sample size** — at least 20 raw effective observations
+- **Directional accuracy** — one-sided 95% lower CI greater than 50%
+- **Pearson correlation** — two-sided 95% lower CI greater than 0
+- **WRMSE improvement** — adaptive lower CI greater than 0%
+- **WCZAR improvement** — adaptive lower CI greater than 0%
+- **Log aspect ratio** — confidence interval overlaps `[-0.5, +0.5]`, meaning forecast variation is not clearly too small or too large
+- **Participation** — strictly greater than 90%
+
+Long-horizon topics accumulate independent evidence more slowly, so the promotion system applies a forecast-horizon adjustment to directional accuracy, WRMSE improvement, and WCZAR improvement. This relaxes the effective confidence requirement for long horizons after the minimum effective-sample-size gate has already been met. Pearson correlation and log aspect ratio do not use this horizon adjustment.
+
+The [Forge Builder Kit](https://github.com/allora-network/allora-forge-builder-kit) mirrors this methodology off-chain: its `PerformanceEvaluator` grades your model against Allora's scoring methodology *before* you deploy, including directional accuracy, Pearson correlation, weighted-RMSE improvement, CZAR improvement, and related confidence checks. A higher grade means better generalization and a higher expected score on the network.
+
+For the full policy, including relegation criteria, see the Allora Research forum post on [inference worker promotion and relegation](https://research.allora.network/t/inference-worker-promotion-and-relegation/157).
+
+### Log-return topics
+
+For log-return topics, the worker already submits the predicted log return, so no price conversion is needed before evaluation. The submitted prediction and the realized ground truth are compared in log-return space, and the promotion metrics above are calculated directly from that series.
+
+This makes log-return topics directly comparable with price topics after price forecasts have been converted into log returns. The zero baseline represents no change, and positive WRMSE or WCZAR improvement means the worker is improving over that baseline.
+
+### Volatility topics
+
+Volatility topics are evaluated on the change in volatility, not the raw volatility level. Ground truth is calculated from one-minute log-price returns over trailing windows:
+
+- The base volatility covers the window ending at forecast time.
+- The target volatility covers the following horizon beginning at forecast time.
+- The evaluated change is the log ratio of target volatility to base volatility.
+
+Volatility naturally tends to mean-revert, which can inflate directional accuracy if it is not accounted for. Allora therefore estimates a causal trailing mean-reversion baseline from information available at forecast time and subtracts that expectation from both the worker prediction and the actual log-volatility change before calculating the evaluation metrics. Workers are evaluated on skill beyond that volatility baseline.
 
 ## From testnet to mainnet
 

public/llms-full.txt

@@ -3264,9 +3264,49 @@ The cycle repeats every epoch, so a competition is not a one-shot submission: yo
 
 ## Scoring
 
-Scoring happens on-chain. Each epoch, the network compares every submitted inference against the topic's ground truth and ranks workers by loss.
+Scoring happens on-chain. Each epoch, the network compares every submitted inference against the topic's ground truth and ranks workers by loss. Forge then summarizes that history into promotion-readiness metrics so you can see whether a worker has enough evidence of skill to move from testnet to mainnet.
 
-The [Forge Builder Kit](https://github.com/allora-network/allora-forge-builder-kit) mirrors this methodology off-chain: its `PerformanceEvaluator` grades your model against Allora's scoring methodology *before* you deploy. It scores seven metrics — directional accuracy (plus its confidence-interval lower bound and p-value), Pearson correlation (plus its p-value), weighted-RMSE improvement, and CZAR improvement — each with a pass/fail threshold, and maps the composite to a letter grade from A+ to F. A higher grade means better generalization and a higher expected score on the network.
+Promotion is evaluated **per worker, per topic**. A worker can be eligible on one topic and ineligible on another because the horizon, participation history, ground truth lag, and submitted inferences are topic-specific.
+
+The key dashboard detail is that some eligibility rows use a **confidence interval (CI)**, not only the point estimate shown as "your value." When a metric is displayed as:
+
+`your value (lower CI, upper CI)`
+
+the top-level value is the worker's point estimate for that metric. The values in parentheses are the confidence interval around that estimate. For promotion checks that say the **lower CI** must exceed a threshold, the relevant number is the **first value inside the parentheses**, not the top-level point estimate. For example, if directional accuracy appears as `72.3% (44.7%, 100%)`, the lower CI is `44.7%`; this would not pass a `> 50%` lower-bound threshold even though the point estimate is above 50%.
+
+Allora uses confidence intervals because workers may have different numbers of submissions. More submissions generally give more evidence and a tighter interval; fewer submissions leave more uncertainty. Where adjacent epochs share overlapping ground-truth windows, Allora corrects the interval calculation using an effective sample size so repeated, correlated observations do not overstate confidence.
+
+For mainnet promotion, a worker must pass every promotion metric on the topic:
+
+- **Effective sample size** — at least 20 raw effective observations
+- **Directional accuracy** — one-sided 95% lower CI greater than 50%
+- **Pearson correlation** — two-sided 95% lower CI greater than 0
+- **WRMSE improvement** — adaptive lower CI greater than 0%
+- **WCZAR improvement** — adaptive lower CI greater than 0%
+- **Log aspect ratio** — confidence interval overlaps `[-0.5, +0.5]`, meaning forecast variation is not clearly too small or too large
+- **Participation** — strictly greater than 90%
+
+Long-horizon topics accumulate independent evidence more slowly, so the promotion system applies a forecast-horizon adjustment to directional accuracy, WRMSE improvement, and WCZAR improvement. This relaxes the effective confidence requirement for long horizons after the minimum effective-sample-size gate has already been met. Pearson correlation and log aspect ratio do not use this horizon adjustment.
+
+The [Forge Builder Kit](https://github.com/allora-network/allora-forge-builder-kit) mirrors this methodology off-chain: its `PerformanceEvaluator` grades your model against Allora's scoring methodology *before* you deploy, including directional accuracy, Pearson correlation, weighted-RMSE improvement, CZAR improvement, and related confidence checks. A higher grade means better generalization and a higher expected score on the network.
+
+For the full policy, including relegation criteria, see the Allora Research forum post on [inference worker promotion and relegation](https://research.allora.network/t/inference-worker-promotion-and-relegation/157).
+
+### Log-return topics
+
+For log-return topics, the worker already submits the predicted log return, so no price conversion is needed before evaluation. The submitted prediction and the realized ground truth are compared in log-return space, and the promotion metrics above are calculated directly from that series.
+
+This makes log-return topics directly comparable with price topics after price forecasts have been converted into log returns. The zero baseline represents no change, and positive WRMSE or WCZAR improvement means the worker is improving over that baseline.
+
+### Volatility topics
+
+Volatility topics are evaluated on the change in volatility, not the raw volatility level. Ground truth is calculated from one-minute log-price returns over trailing windows:
+
+- The base volatility covers the window ending at forecast time.
+- The target volatility covers the following horizon beginning at forecast time.
+- The evaluated change is the log ratio of target volatility to base volatility.
+
+Volatility naturally tends to mean-revert, which can inflate directional accuracy if it is not accounted for. Allora therefore estimates a causal trailing mean-reversion baseline from information available at forecast time and subtracts that expectation from both the worker prediction and the actual log-volatility change before calculating the evaluation metrics. Workers are evaluated on skill beyond that volatility baseline.
 
 ## From testnet to mainnet
 

public/raw/build/forge/competitions.md

@@ -2,8 +2,8 @@
 title: Allora Forge Competitions
 description: How Allora Forge competitions work — compete on live topics, build a testnet track record, and graduate to mainnet where top performers earn ALLO rewards.
 persona: ML builder
-verified_against: forge.allora.network (live site + public API, 2026-07-30), allora-forge-builder-kit README (main @ 8ef3200)
-last_reviewed: 2026-07-30
+verified_against: forge.allora.network (live site + public API, 2026-07-30), allora-forge-builder-kit README (main @ 8ef3200), Allora Research promotion/relegation post (2026-08-06)
+last_reviewed: 2026-08-12
 ---
 
 # Allora Forge Competitions
@@ -25,9 +25,49 @@ The cycle repeats every epoch, so a competition is not a one-shot submission: yo
 
 ## Scoring
 
-Scoring happens on-chain. Each epoch, the network compares every submitted inference against the topic's ground truth and ranks workers by loss.
+Scoring happens on-chain. Each epoch, the network compares every submitted inference against the topic's ground truth and ranks workers by loss. Forge then summarizes that history into promotion-readiness metrics so you can see whether a worker has enough evidence of skill to move from testnet to mainnet.
 
-The [Forge Builder Kit](https://github.com/allora-network/allora-forge-builder-kit) mirrors this methodology off-chain: its `PerformanceEvaluator` grades your model against Allora's scoring methodology *before* you deploy. It scores seven metrics — directional accuracy (plus its confidence-interval lower bound and p-value), Pearson correlation (plus its p-value), weighted-RMSE improvement, and CZAR improvement — each with a pass/fail threshold, and maps the composite to a letter grade from A+ to F. A higher grade means better generalization and a higher expected score on the network.
+Promotion is evaluated **per worker, per topic**. A worker can be eligible on one topic and ineligible on another because the horizon, participation history, ground truth lag, and submitted inferences are topic-specific.
+
+The key dashboard detail is that some eligibility rows use a **confidence interval (CI)**, not only the point estimate shown as "your value." When a metric is displayed as:
+
+`your value (lower CI, upper CI)`
+
+the top-level value is the worker's point estimate for that metric. The values in parentheses are the confidence interval around that estimate. For promotion checks that say the **lower CI** must exceed a threshold, the relevant number is the **first value inside the parentheses**, not the top-level point estimate. For example, if directional accuracy appears as `72.3% (44.7%, 100%)`, the lower CI is `44.7%`; this would not pass a `> 50%` lower-bound threshold even though the point estimate is above 50%.
+
+Allora uses confidence intervals because workers may have different numbers of submissions. More submissions generally give more evidence and a tighter interval; fewer submissions leave more uncertainty. Where adjacent epochs share overlapping ground-truth windows, Allora corrects the interval calculation using an effective sample size so repeated, correlated observations do not overstate confidence.
+
+For mainnet promotion, a worker must pass every promotion metric on the topic:
+
+- **Effective sample size** — at least 20 raw effective observations
+- **Directional accuracy** — one-sided 95% lower CI greater than 50%
+- **Pearson correlation** — two-sided 95% lower CI greater than 0
+- **WRMSE improvement** — adaptive lower CI greater than 0%
+- **WCZAR improvement** — adaptive lower CI greater than 0%
+- **Log aspect ratio** — confidence interval overlaps `[-0.5, +0.5]`, meaning forecast variation is not clearly too small or too large
+- **Participation** — strictly greater than 90%
+
+Long-horizon topics accumulate independent evidence more slowly, so the promotion system applies a forecast-horizon adjustment to directional accuracy, WRMSE improvement, and WCZAR improvement. This relaxes the effective confidence requirement for long horizons after the minimum effective-sample-size gate has already been met. Pearson correlation and log aspect ratio do not use this horizon adjustment.
+
+The [Forge Builder Kit](https://github.com/allora-network/allora-forge-builder-kit) mirrors this methodology off-chain: its `PerformanceEvaluator` grades your model against Allora's scoring methodology *before* you deploy, including directional accuracy, Pearson correlation, weighted-RMSE improvement, CZAR improvement, and related confidence checks. A higher grade means better generalization and a higher expected score on the network.
+
+For the full policy, including relegation criteria, see the Allora Research forum post on [inference worker promotion and relegation](https://research.allora.network/t/inference-worker-promotion-and-relegation/157).
+
+### Log-return topics
+
+For log-return topics, the worker already submits the predicted log return, so no price conversion is needed before evaluation. The submitted prediction and the realized ground truth are compared in log-return space, and the promotion metrics above are calculated directly from that series.
+
+This makes log-return topics directly comparable with price topics after price forecasts have been converted into log returns. The zero baseline represents no change, and positive WRMSE or WCZAR improvement means the worker is improving over that baseline.
+
+### Volatility topics
+
+Volatility topics are evaluated on the change in volatility, not the raw volatility level. Ground truth is calculated from one-minute log-price returns over trailing windows:
+
+- The base volatility covers the window ending at forecast time.
+- The target volatility covers the following horizon beginning at forecast time.
+- The evaluated change is the log ratio of target volatility to base volatility.
+
+Volatility naturally tends to mean-revert, which can inflate directional accuracy if it is not accounted for. Allora therefore estimates a causal trailing mean-reversion baseline from information available at forecast time and subtracts that expectation from both the worker prediction and the actual log-volatility change before calculating the evaluation metrics. Workers are evaluated on skill beyond that volatility baseline.
 
 ## From testnet to mainnet