[pull] master from industrial-optimization-group:master - #154
Open
pull[bot] wants to merge 279 commits into
Open
[pull] master from industrial-optimization-group:master#154pull[bot] wants to merge 279 commits into
pull[bot] wants to merge 279 commits into
Conversation
- Introduce rules in pyproject.toml to ignore certain warning in notebooks, such as use of print statements. - Fixed some legitmate linter warning in the xlemoo notebook.
* Include mpi-sppy dependencies * Updated how scenarios are handled * Purged references to scenario_keys * Added anticipation info for scenario variables * Added the scenarios to the API * Updated the scenario models and added probabilities * Added some stochastic optimization * Started working on solving stochastic problems. Does not work yet. * Expected value calculation works now * Added robust optimization * Added weighted scenarios aggregation * Removed mpi-sppy so that the automated tests can work * Update uv.lock as well * Changed the order of evaluating scalarizations and constraints Now scalarization symbols can be used in constraints, which helps aggregating them over multiple scenarios. * Cleaning up the tests * Add matrix random access to gurobipy parsing * Added an initial version of a summer cabin electricity system optimization problem * Use function expressions instead of symbols when constructing new expressions * Fixed multi valued constraints for gurobipy Co-authored-by: Copilot <copilot@github.com> * Cleaned stuff up and started documentation * Expanded the vector handling of the json parser Co-authored-by: Copilot <copilot@github.com> * Fixed a bug with multiplying a gurobipy variable with a numpy array * Cleaned up summer cabin problem and added tests Co-authored-by: Copilot <copilot@github.com> * Updated documentation Co-authored-by: Copilot <copilot@github.com> * Removed some test notebooks * Updated the source for mpi-sppy to point to github * Remove unnecessary file * Fix some bugs with Pyomo implementation and test the summer cabin problem with multiple solvers Co-authored-by: Copilot <copilot@github.com> * Remove the big optimization problems from github tests Co-authored-by: Copilot <copilot@github.com> * Simplified extract and exclude a little * Changed min-max robust to worst case robust to support maximized objectives * Fixed a bug with gurobipy parsing Co-authored-by: Copilot <copilot@github.com> * Updated add_expected_asf to validate the scenario model Co-authored-by: Copilot <copilot@github.com> * Protect the scenario probabilities from being tampered * Removed dead code * Check that scenario tree is a tree --------- Co-authored-by: Copilot <copilot@github.com>
- Move the elitist top-N-by-target-column selector out of selection.py into desdeo/emo/operators/scalar_selection.py and rename it to ElitistSelection. The class is no longer ASF-specific: it ranks the combined (decision_variables, outputs) tuple by any single output column (target_column) and keeps the top winner_size rows. Reads its fitness from outputs by default; an explicit fitness= argument still overrides. Empty interested_topics so it doesn't subscribe to anything it can't use; verbosity-graded state still emits STATE, SELECTED_VERBOSE_OUTPUTS, and SELECTED_FITNESS for archivers and other listeners. - Drop ASFSelector from selection.py and ASFSelectorOptions from options/selection.py; remove the matching dispatch entries and re-exports from desdeo/emo/__init__.py and desdeo/emo/options/__init__.py. - Add ElitistSelectionOptions (winner_size, target_column) to options/scalar_selection.py and wire it into scalar_selector_constructor. - TemplateXLEMOOOptions.selection is now ScalarSelectionOptions (overrides the inherited multiobjective SelectorOptions). _xlemoo_preference_handler validates the new name; emo_constructor takes the XLEMOO branch through scalar_selector_constructor and isinstance-checks ElitistSelection. - template_xlemoo's selection arg is BaseScalarSelector. Darwinian step vstacks parents and offspring before calling selection.do((combined_decvars, combined_outputs)); post-learning step does the same with the operator's returned population. - LearningModeOperator now annotates ElitistSelection, reads winner_size instead of population_size, and combines H-group + instantiated before calling its selector. - xlemoo_options() builds ElitistSelectionOptions(winner_size=50, target_column="asf"). - Tests: rename test_asf_selector.py to test_elitist_selection.py and rewrite against the new tuple-based do(). Update test_learning_mode, test_template_xlemoo, and test_rules to construct ElitistSelection and run inline Darwinian loops where they previously called template1. Notebook builder switched the override from population_size to winner_size; notebook re-executes deterministically end-to-end.
- LearningModeOperator now subclasses Subscriber so it participates in emo_constructor's pub/sub consistency check. It still subscribes to nothing and publishes nothing — the public surface remains the do() return value — but the wiring is in place for future hooks. - Default seed=0 to match every other operator. h_split and l_split are now plain fractions in (0, 1] (the dual fraction/count interpretation and the _split_indices helper are gone). - LearningModeOperator no longer takes an evaluator and no longer evaluates or selects anything internally. do() returns just the instantiated decision-variable DataFrame, or None when the archive is empty / both H/L groups are empty / SkopeRules returns no rules. The template handles evaluation and selection, so any BaseScalarSelector works and there is no double-evaluation through the archive. - template_xlemoo is now a single while-not-terminator loop with a gen_in_cycle counter modulo (n_darwin_per_cycle + n_learning_per_cycle). No stop flag, no nested for-loops, no break cascades. Each iteration is one generation; the phase counter selects Darwinian vs Learning. The Learning branch evaluates the operator's returned candidates once and feeds them through the EA's own scalar selector. If the operator returns None, the current population is kept unchanged. - emo_constructor builds and registers the LearningModeOperator before the consistency check, so an inconsistent template (e.g. a selector with topics no one publishes) fails fast with a clear message. - Tests updated for the new contract: test_learning_mode_returns_decision_dataframe, test_learning_mode_returns_expected_row_count, test_learning_mode_returns_none_on_empty_archive added. _split_indices test removed. test_template_xlemoo and test_rules construct the operator with publisher=... instead of evaluator=... and add it to the subscriber list. Notebook re-executes deterministically end-to-end.
* Shorten names and add descriptions * Update cvxpy and have scenarios example use it instead of gurobi in the hopes that documentation compiles
test: implement GDM-SCORE-bands test for multiple users in Playwright
* Load the preran notebook from github into the documentation
* Inline the scenario example and add a link to download the notebook * Remove dead code * Fixed some other notebooks while at it
Changed execute option to false for mkdocs-jupyter to troubleshoot doc build issues.
(cherry picked from commit 4cb51780a4089d4dc97a6bcec2779624557face7)
… main block in algorithms.py
EMOEvaluator.__init__ stored a lambda that constructed a fresh SimulatorEvaluator on every evaluate() call. For a problem with surrogate objectives, constructing one runs joblib.load for each surrogate, so every generation re-read the models from disk. Measured on a 2000-evaluation NSGA-III run against three 124 MB surrogates: 9.47s before, 2.99s after. The gain scales with model size, so it is small for toy surrogates and large for real ones. Removes the TODO that flagged this.
_rvea_selection leaves the running best index at -1 when a reference vector has no associated solution, and wrote selection[select] unguarded. That sets selection[-1] = True, silently promoting the last individual in the array once per empty vector. Empty reference vectors are common and become commoner as the number of objectives grows: on DTLZ2 with 36 vectors at 8 objectives, RVEA realises only 21-36 survivors, so most generations hit this path. The constrained twin _rvea_selection_constrained already had the correct guard.
ReferenceVectorOptions accepted creation_type="s_energy" but the selector raised NotImplementedError. The simplex lattice can only realise the binomial counts C(H + m - 1, m - 1), so it cannot honour an arbitrary number_of_vectors: at eight objectives the reachable counts jump 36, 120, 330, and the default number_of_vectors=100 silently rounds down to 36. Since a decomposition-based algorithm's population size is its reference vector count, that rounding chose the population size rather than the caller, and it did so differently per objective count -- 91 / 70 / 36 at m = 3 / 5 / 8 for NSGA-III and RVEA, while NSGA-II and IBEA held 100. create_s_energy delegates to pymoo, already a dependency, rather than reimplementing the optimizer, and caches on (m, n, seed) since the design is deterministic and costs 0.5-3 s to build. Every axis direction is guaranteed present. A decomposition-based algorithm only searches towards directions it holds a vector for, so a set missing an axis never targets that objective's extreme. The current optimizer happens to pin the simplex vertices; _ensure_axis_vectors makes that DESDEO's guarantee instead of an upstream implementation detail, substituting the nearest vector so the requested count is preserved. RVEA is deliberately excluded from the equal-population-size test: it keeps at most one survivor per reference vector, so unassociated vectors leave it slightly short of the nominal count by design.
SimulatedBinaryCrossover had a single crossover probability, applied per decision variable. The literature reports two levels: a per-pair p_c (1.0 in the RVEA and NSGA-III papers, 0.9 in NSGA-II) together with Deb and Agrawal's per-variable 0.5. Transcribing p_c = 1.0 into the per-variable slot crossed every variable instead of half of them, and the standard configuration could not be expressed at all -- folding p_c into the per-variable rate reproduces the marginal rate but destroys the within-pair correlation, since a failed pair must copy all of its variables together. Adds pair_xover_probability, changes the per-variable default from 1.0 to 0.5, and updates the algorithm factories to put p_c in the new parameter. xover_distribution stays at 30; it measured best at both rates. Also aligns two factory settings with their source papers: nsga2_options and ibea_options now use mutation_probability=None (1/n_variables) rather than a hardcoded 0.01, which was only equivalent for a 100-variable problem, and nsga2_options uses ClipRepair, as Deb's NSGA-II reference code clamps offspring onto the variable bounds.
Also moves the axis-guarantee paragraph above the Args block, where it belongs.
A constant objective column makes f_maxs[m] - f_mins[m] zero. Numba raises ZeroDivisionError on that rather than returning inf like numpy, so the run dies outright. Before reaching the division, the loop also marks the first and last entry of an arbitrary argsort as boundary points, which awards an infinite crowding distance to two arbitrary solutions. Reachable whenever an objective collapses across the population; constrained runs hit it readily because they converge onto a much smaller feasible region.
Both raised or warned on any problem with constraints, which shut the constrained real-world test problems out of every algorithm except RVEA and NSGA-III. NSGA-II gets the constrained domination of Deb, Pratap, Agarwal & Meyarivan (2002): a feasible solution beats an infeasible one, two infeasible solutions are compared by total violation, and two feasible ones by ordinary Pareto dominance. Since that relation never compares a feasible solution with an infeasible one on objectives, the two groups are ranked separately and concatenated rather than a new dominance relation being introduced. Crowding distance, the partial-front trim and the fitness are untouched, which is where NSGA-II confines its constraint handling. IBEA gets feasibility-first, mirroring the block NSGA3Selector already has: the infeasible solutions are dropped while the feasible ones can fill the population, and otherwise every feasible solution survives and the remaining places go to the least infeasible. The binary indicator compares objective vectors and has no reading of an infeasible one, so it is left alone; the published fitness places every infeasible solution below every feasible one so the mating tournament agrees. On car_side_impact at 6000 evaluations, NSGA-II ends with 16 of 100 solutions feasible before this change and 100 of 100 after.
Everything downstream of SELECTED_FITNESS reads it that way: the mating tournament takes an argmax, and roulette-wheel selection treats it as a weight, which also requires it to be positive. NSGA-II builds the opposite quantity -- front rank plus a reversed crowding distance, lower is better -- and published it unflipped, so the binary tournament mated the loser of every pair. Reflecting it about the worst attainable value flips the direction, preserves the ordering exactly, and keeps every value strictly positive: a front's fitness lies in (n_fronts - rank - 1, n_fronts - rank], since the reversed crowding distance never reaches 1. The first-iteration branch, which publishes the bare front rank, gets the same treatment. dtlz2 (m=3, n=12), 20000 evaluations, median IGD+ over 5 seeds: 0.0437 before, 0.0393 after.
TournamentSelection's seed doubled as the switch between deterministic and stochastic selection: leaving it unset chose the deterministic rule and also built the generator from OS entropy. Since TournamentSelectionOptions carries no seed, every options-built algorithm with a mating tournament took that path, so template.seed reached every operator except the mating step and NSGA-II and IBEA runs could not be reproduced. RVEA and NSGA-III were unaffected, having no mating tournament. seed now only seeds and defaults to 0; the new stochastic flag chooses the rule, and RouletteWheelSelectionOptions sets it. All four algorithms now reproduce exactly from template.seed alone. Callers that passed a seed to obtain roulette-wheel behaviour need stochastic=True as well. The one in-tree caller, the deprecated ibea() helper, is left as is: with IBEA's non-positive fitness, drawing proportionally to it favoured the worst individual, so a seeded deterministic tournament is both the fix and what ibea_options() already does.
The constrained selector runs the same per-vector loop as the unconstrained one but keeps two running indices: the best feasible solution, and the least infeasible one as a fallback. Only the first was guarded against staying at -1. A reference vector with no associated solution at all leaves both at -1, so the fallback ran selection[-1] = True and promoted the last individual in the array, once per empty vector. This is the same defect as 540b005, in the branch that commit did not cover, and constrained runs meet it constantly because feasibility shrinks the region the population occupies.
…ensorconstants and scenarios (#578) * Bugfix: Tensor constants were not correctly updated in different scenarios * Ignore ruff complaints about imports in a test * Make solution descriptions work with tensors and large expressions Description parts could not reach tensor results and were too slow to render for expressions over many symbols: - Expose tensor solver results element by element (x_1, x_2, ... and x_1_2 for nested tensors) instead of collapsing a tensor to its first element, so a part can aggregate over one, e.g. the maximum over a schedule. - Make the problem's constants available to descriptions, so a part can refer to p_1 rather than having its value written into the expression. - Substitute values into the MathJSON tree before parsing it, rather than parsing symbolically and then substituting. Rendering a part that aggregates over a few hundred symbols went from ~52 s to ~10 ms. - Extract validate_description_parts() so callers other than the update_metadata endpoint can check a template, and have it reject parts referring to symbols the problem does not have. Such a part previously failed only when a description was requested. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Small fix, so that npm is found on Windows as well. * Update the guide on using uv with conda * Added documentation for solving problems using the webui on your local machine
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )