Merge pull request #99 from CodingThrust/feat/expr-overhead-system

Replace Polynomial overhead system with Expr AST

Author: isPANN

.claude/CLAUDE.md

@@ -67,9 +67,7 @@ Problem (core trait — all problems must implement)
 ├── fn dims(&self) -> Vec<usize>       // config space: [2, 2, 2] for 3 binary variables
 ├── fn evaluate(&self, config) -> Metric
 ├── fn variant() -> Vec<(&str, &str)>  // e.g., [("graph","SimpleGraph"), ("weight","i32")]
-├── fn num_variables(&self) -> usize   // default: dims().len()
-├── fn problem_size_names() -> &[&str] // static field names for size metrics
-└── fn problem_size_values(&self) -> Vec<usize>  // instance-level size values
+└── fn num_variables(&self) -> usize   // default: dims().len()
 
 OptimizationProblem : Problem<Metric = SolutionSize<Self::Value>> (extension for optimization)
 │
@@ -98,6 +96,20 @@ enum Direction { Maximize, Minimize }
 - Weight management via inherent methods (`weights()`, `set_weights()`, `is_weighted()`), not traits
 - `NumericSize` supertrait bundles common numeric bounds (`Clone + Default + PartialOrd + Num + Zero + Bounded + AddAssign + 'static`)
 
+### Overhead System
+Reduction overhead is expressed using `Expr` AST (in `src/expr.rs`) with the `#[reduction]` macro:
+```rust
+#[reduction(overhead = {
+    num_vertices = "num_vertices + num_clauses",
+    num_edges = "3 * num_clauses",
+})]
+impl ReduceTo<Target> for Source { ... }
+```
+- Expression strings are parsed at compile time by a Pratt parser in the proc macro crate
+- Each problem type provides inherent getter methods (e.g., `num_vertices()`, `num_edges()`) that the overhead expressions reference
+- `ReductionOverhead` stores `Vec<(&'static str, Expr)>` — field name to symbolic expression mappings
+- Expressions support: constants, variables, `+`, `*`, `^`, `exp()`, `log()`, `sqrt()`
+
 ### Problem Names
 Problem types use explicit optimization prefixes:
 - `MaximumIndependentSet`, `MaximumClique`, `MaximumMatching`, `MaximumSetPacking`

docs/plans/2026-02-25-overhead-system-design.md

@@ -0,0 +1,138 @@
+# Overhead System Redesign
+
+**Issue:** #61 — Introduce overhead system
+**Date:** 2026-02-25
+**Approach:** Macro-first dual emission
+
+## Summary
+
+Replace the current `Polynomial`-based overhead system with a general `Expr` AST, compile-time macro-parsed expression strings, and per-problem inherent getters. The proc macro emits both compiled Rust code (for evaluation + compiler validation) and symbolic `Expr` AST literals (for composition + export).
+
+## Motivation
+
+Three pain points with the current system:
+1. **Ergonomics** — `problem_size_names()`/`problem_size_values()` parallel arrays are awkward; `poly!` macro is verbose
+2. **Correctness** — variable name mismatches between overhead expressions and problem size fields are caught only at runtime
+3. **Simplification** — `Polynomial` only supports sums of monomials; general math (exp, log) requires a new representation anyway
+
+## Design
+
+### 1. Expression AST (`Expr`)
+
+Replaces `Polynomial` and `Monomial` with a general math expression tree.
+
+```rust
+// src/expr.rs (replaces src/polynomial.rs)
+
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
+pub enum Expr {
+    Const(f64),
+    Var(&'static str),
+    Add(Box<Expr>, Box<Expr>),
+    Mul(Box<Expr>, Box<Expr>),
+    Pow(Box<Expr>, Box<Expr>),
+    Exp(Box<Expr>),
+    Log(Box<Expr>),
+    Sqrt(Box<Expr>),
+}
+```
+
+Key operations:
+- `eval(&self, vars: &ProblemSize) -> f64`
+- `substitute(&self, mapping: &HashMap<&str, &Expr>) -> Expr`
+- `variables(&self) -> HashSet<&'static str>`
+- `is_polynomial(&self) -> bool`
+- `degree(&self) -> Option<u32>`
+- `Display` for human-readable formulas
+- `simplify(&self) -> Expr` — minimal constant folding
+
+### 2. Problem Getters
+
+Remove `problem_size_names()` and `problem_size_values()` from the `Problem` trait. Each problem type implements inherent getter methods instead.
+
+```rust
+// Before: trait methods returning parallel arrays
+impl Problem for MaximumIndependentSet<SimpleGraph, i32> {
+    fn problem_size_names() -> &'static [&'static str] { &["num_vertices", "num_edges"] }
+    fn problem_size_values(&self) -> Vec<usize> {
+        vec![self.graph().num_vertices(), self.graph().num_edges()]
+    }
+}
+
+// After: inherent methods — natural, compiler-checked, IDE-friendly
+impl<G: Graph, W: WeightElement> MaximumIndependentSet<G, W> {
+    pub fn num_vertices(&self) -> usize { self.graph().num_vertices() }
+    pub fn num_edges(&self) -> usize { self.graph().num_edges() }
+}
+```
+
+### 3. Proc Macro — Dual Emission
+
+The `#[reduction]` macro parses expression strings at compile time and emits two outputs.
+
+User-facing syntax:
+```rust
+#[reduction(overhead = {
+    num_vars = "num_vertices",
+    num_constraints = "num_edges + num_vertices^2",
+})]
+impl ReduceTo<QUBO<f64>> for MaximumIndependentSet<SimpleGraph, i32> { ... }
+```
+
+Macro emits:
+1. **Compiled evaluation function** — `src.num_vertices()`, `src.num_edges()` calls. Compiler catches missing getters.
+2. **Symbolic Expr AST** — `Expr::Add(...)` construction for composition/export.
+
+Expression grammar (Pratt parser, ~200 LOC in proc macro crate):
+```
+expr     = term (('+' | '-') term)*
+term     = factor (('*' | '/') factor)*
+factor   = base ('^' factor)?
+base     = NUMBER | IDENT | func_call | '(' expr ')'
+func_call = ('exp' | 'log' | 'sqrt') '(' expr ')'
+```
+
+### 4. Updated `ReductionOverhead` and `ReductionEntry`
+
+```rust
+pub struct ReductionOverhead {
+    pub output_size: Vec<(&'static str, Expr)>,  // Expr replaces Polynomial
+}
+
+pub struct ReductionEntry {
+    // ...existing fields...
+    pub overhead_fn: fn() -> ReductionOverhead,       // symbolic (composition/export)
+    pub overhead_eval_fn: fn(&dyn Any) -> ProblemSize, // compiled (evaluation)
+    // REMOVED: source_size_names_fn, target_size_names_fn
+}
+```
+
+`PathCostFn` uses the symbolic `ReductionOverhead` (via `Expr::eval`) since it operates on type-erased `ProblemSize` during graph traversal.
+
+### 5. Export Pipeline
+
+JSON format gains both structured AST and display string:
+```json
+{
+  "overhead": [{
+    "field": "num_vars",
+    "expr": {"Pow": [{"Var": "num_vertices"}, {"Const": 2.0}]},
+    "formula": "num_vertices^2"
+  }]
+}
+```
+
+The paper reads `formula` strings — no Typst code changes needed.
+
+## Migration Strategy
+
+| Phase | Description | Files | Risk |
+|-------|-------------|-------|------|
+| 1 | Add `Expr` type alongside `Polynomial` | 2-3 new | Low (additive) |
+| 2 | Update proc macro with Pratt parser, support new syntax | 1 file | Medium |
+| 3 | Add inherent getters to all problem types | ~15 model files | Low (additive) |
+| 4 | Migrate all reductions to new syntax | ~20 rule files | Low (mechanical) |
+| 5 | Remove deprecated APIs (`problem_size_*`, `Polynomial`, `poly!`) | ~10 files | Medium (breaking) |
+| 6 | Update documentation and regenerate exports | 3-4 files | Low |
+
+Phases 1-3 are purely additive. Phase 4 is bulk migration. Phase 5 is cleanup.

docs/src/design.md

@@ -37,8 +37,6 @@ trait Problem: Clone {
     fn evaluate(&self, config: &[usize]) -> Self::Metric;
     fn variant() -> Vec<(&'static str, &'static str)>; // e.g., [("graph", "SimpleGraph"), ("weight", "i32")]
     fn num_variables(&self) -> usize;      // default: dims().len()
-    fn problem_size_names() -> &'static [&'static str]; // e.g., ["num_vertices", "num_edges"]
-    fn problem_size_values(&self) -> Vec<usize>;       // e.g., [10, 15] for a specific instance
 }
 
 trait OptimizationProblem: Problem<Metric = SolutionSize<Self::Value>> {
@@ -49,7 +47,7 @@ trait OptimizationProblem: Problem<Metric = SolutionSize<Self::Value>> {
 trait SatisfactionProblem: Problem<Metric = bool> {}  // marker trait
 ```
 
-- **`Problem`** — the base trait. Every problem declares a `NAME` (e.g., `"MaximumIndependentSet"`). The solver explores the configuration space defined by `dims()` and scores each configuration with `evaluate()`. For example, a 4-vertex MIS has `dims() = [2, 2, 2, 2]` (each vertex is selected or not); `evaluate(&[1, 0, 1, 0])` returns `Valid(2)` if vertices 0 and 2 form an independent set, or `Invalid` if they share an edge. `problem_size_names()` and `problem_size_values()` expose the instance's structural dimensions (e.g., `num_vertices`, `num_edges`) as a `ProblemSize` — used by the reduction graph to evaluate overhead polynomials along a path.
+- **`Problem`** — the base trait. Every problem declares a `NAME` (e.g., `"MaximumIndependentSet"`). The solver explores the configuration space defined by `dims()` and scores each configuration with `evaluate()`. For example, a 4-vertex MIS has `dims() = [2, 2, 2, 2]` (each vertex is selected or not); `evaluate(&[1, 0, 1, 0])` returns `Valid(2)` if vertices 0 and 2 form an independent set, or `Invalid` if they share an edge. Each problem also provides inherent getter methods (e.g., `num_vertices()`, `num_edges()`) used by reduction overhead expressions.
 - **`OptimizationProblem`** — extends `Problem` with a comparable `Value` type and a `direction()` (`Maximize` or `Minimize`).
 - **`SatisfactionProblem`** — constrains `Metric = bool`: `true` if all constraints are satisfied, `false` otherwise.
 

problemreductions-cli/src/commands/graph.rs

@@ -143,7 +143,7 @@ pub fn show(problem: &str, out: &OutputConfig) -> Result<()> {
             "\n{}\n",
             crate::output::fmt_section(&format!("Size fields ({}):", size_fields.len()))
         ));
-        for f in size_fields {
+        for f in &size_fields {
             text.push_str(&format!("  {f}\n"));
         }
     }

problemreductions-cli/src/commands/inspect.rs

@@ -33,16 +33,10 @@ fn inspect_problem(pj: &ProblemJson, out: &OutputConfig) -> Result<()> {
 
     let mut text = format!("Type: {}{}\n", name, variant_str);
 
-    // Size info
-    let size_names = problem.problem_size_names_dyn();
-    let size_values = problem.problem_size_values_dyn();
-    if !size_names.is_empty() {
-        let sizes: Vec<String> = size_names
-            .iter()
-            .zip(size_values.iter())
-            .map(|(n, v)| format!("{} {}", v, n))
-            .collect();
-        text.push_str(&format!("Size: {}\n", sizes.join(", ")));
+    // Size fields from the reduction graph
+    let size_fields = graph.size_field_names(name);
+    if !size_fields.is_empty() {
+        text.push_str(&format!("Size fields: {}\n", size_fields.join(", ")));
     }
     text.push_str(&format!("Variables: {}\n", problem.num_variables_dyn()));
 
@@ -60,9 +54,7 @@ fn inspect_problem(pj: &ProblemJson, out: &OutputConfig) -> Result<()> {
         "kind": "problem",
         "type": name,
         "variant": variant,
-        "size": size_names.iter().zip(size_values.iter())
-            .map(|(n, v)| serde_json::json!({"field": n, "value": v}))
-            .collect::<Vec<_>>(),
+        "size_fields": size_fields,
         "num_variables": problem.num_variables_dyn(),
         "solvers": ["ilp", "brute-force"],
         "reduces_to": targets,

problemreductions-cli/src/dispatch.rs

@@ -39,8 +39,6 @@ pub trait DynProblem: Any {
     fn dims_dyn(&self) -> Vec<usize>;
     fn problem_name(&self) -> &'static str;
     fn variant_map(&self) -> BTreeMap<String, String>;
-    fn problem_size_names_dyn(&self) -> &'static [&'static str];
-    fn problem_size_values_dyn(&self) -> Vec<usize>;
     fn num_variables_dyn(&self) -> usize;
 }
 
@@ -70,12 +68,6 @@ where
             .map(|(k, v)| (k.to_string(), v.to_string()))
             .collect()
     }
-    fn problem_size_names_dyn(&self) -> &'static [&'static str] {
-        T::problem_size_names()
-    }
-    fn problem_size_values_dyn(&self) -> Vec<usize> {
-        self.problem_size_values()
-    }
     fn num_variables_dyn(&self) -> usize {
         self.num_variables()
     }

problemreductions-cli/src/mcp/prompts.rs

@@ -25,9 +25,7 @@ pub fn list_prompts() -> Vec<Prompt> {
             Some(vec![PromptArgument {
                 name: "description".into(),
                 title: None,
-                description: Some(
-                    "Free-text description of your real-world problem".into(),
-                ),
+                description: Some("Free-text description of your real-world problem".into()),
                 required: Some(true),
             }]),
         ),
@@ -114,9 +112,7 @@ pub fn list_prompts() -> Vec<Prompt> {
         ),
         Prompt::new(
             "overview",
-            Some(
-                "Explore the full landscape of NP-hard problems and reductions in the graph",
-            ),
+            Some("Explore the full landscape of NP-hard problems and reductions in the graph"),
             None,
         ),
     ]
@@ -266,10 +262,7 @@ pub fn get_prompt(
                 .unwrap_or("QUBO");
 
             Some(GetPromptResult {
-                description: Some(format!(
-                    "Find reduction path from {} to {}",
-                    source, target
-                )),
+                description: Some(format!("Find reduction path from {} to {}", source, target)),
                 messages: vec![PromptMessage::new_text(
                     PromptMessageRole::User,
                     format!(

problemreductions-cli/src/mcp/tools.rs

@@ -177,7 +177,7 @@ impl McpServer {
         let mut json = serde_json::json!({
             "name": spec.name,
             "variants": variants,
-            "size_fields": size_fields,
+            "size_fields": &size_fields,
             "reduces_to": outgoing.iter().map(|e| {
                 serde_json::json!({
                     "source": {"name": e.source_name, "variant": e.source_variant},
@@ -605,8 +605,7 @@ impl McpServer {
         let variant = problem.variant_map();
         let graph = ReductionGraph::new();
 
-        let size_names = problem.problem_size_names_dyn();
-        let size_values = problem.problem_size_values_dyn();
+        let size_fields = graph.size_field_names(name);
 
         let outgoing = graph.outgoing_reductions(name);
         let mut targets: Vec<String> = outgoing.iter().map(|e| e.target_name.to_string()).collect();
@@ -617,9 +616,7 @@ impl McpServer {
             "kind": "problem",
             "type": name,
             "variant": variant,
-            "size": size_names.iter().zip(size_values.iter())
-                .map(|(n, v)| serde_json::json!({"field": n, "value": v}))
-                .collect::<Vec<_>>(),
+            "size_fields": size_fields,
             "num_variables": problem.num_variables_dyn(),
             "solvers": ["ilp", "brute-force"],
             "reduces_to": targets,

problemreductions-cli/tests/cli_tests.rs

@@ -1937,7 +1937,10 @@ fn test_inspect_problem() {
         stdout.contains("Type: MaximumIndependentSet"),
         "expected 'Type: MaximumIndependentSet', got: {stdout}"
     );
-    assert!(stdout.contains("Size:"), "expected 'Size:', got: {stdout}");
+    assert!(
+        stdout.contains("Size fields:"),
+        "expected 'Size fields:', got: {stdout}"
+    );
     assert!(
         stdout.contains("Variables:"),
         "expected 'Variables:', got: {stdout}"
@@ -2093,7 +2096,22 @@ fn test_inspect_json_output() {
     let json: serde_json::Value = serde_json::from_str(&content).unwrap();
     assert_eq!(json["kind"], "problem");
     assert_eq!(json["type"], "MaximumIndependentSet");
-    assert!(json["size"].is_array());
+    let size_fields: Vec<&str> = json["size_fields"]
+        .as_array()
+        .expect("size_fields should be an array")
+        .iter()
+        .map(|v| v.as_str().unwrap())
+        .collect();
+    assert!(
+        size_fields.contains(&"num_vertices"),
+        "MIS size_fields should contain num_vertices, got: {:?}",
+        size_fields
+    );
+    assert!(
+        size_fields.contains(&"num_edges"),
+        "MIS size_fields should contain num_edges, got: {:?}",
+        size_fields
+    );
     assert!(json["solvers"].is_array());
     assert!(json["reduces_to"].is_array());