Solve / compat internals (compat)¶
This page documents compat internals
solve(), run_experiment(), and the preset catalog (sezgi.presets.*,
on the Presets page) are the spec-file-driven internals
every built-in class's .run() is itself implemented on top of. They
remain fully working, public, and documented here — but the
class-first surface (Algorithm,
built-in classes) is the primary, recommended API for
everyday use. Reach for this page when you need a raw AlgorithmSpec
(e.g. TOML spec files, run_experiment, or R interop).
solve ¶
solve(spec, problem, master_seed=0, run_id=0, log_dir=None, algo_name=None)
Run an algorithm spec against a problem.
spec: dict (JSON-compatible algorithm spec) or a JSON string.
problem: a native Problem handle (sezgi.bbob(...), ...) OR a
sezgi.Problem subclass instance -- both routed through
sezgi.as_native_problem (final-review fix 5), so a native handle
passes through unchanged and a Problem subclass is converted the
same way Algorithm.run/the builtin wrapper classes already do;
anything else raises as_native_problem's own friendly TypeError
instead of pyo3's raw conversion error.
Returns a dict including best_x: the best EVALUATED point (paired with
best_f). For most algorithms this always lies within problem's
declared bounds. It is not guaranteed to for every algorithm: some (e.g.
HHO) charge raw, pre-boundary-repair trial points against the budget
before boundary repair, and such a point can become the reported best if
it happens to be the run's own minimum.
run_experiment ¶
run_experiment(spec_toml, journal=None, parallel=True, threads=None, log_dir=None)
Run an [ExperimentSpec] (parsed from TOML) and return its run
records as a list of dicts with keys: algo, fid, dim, instance, seed,
budget, best_f, f_opt, gap, evals_used, wall_secs.
spec_toml: TOML string of an ExperimentSpec.
journal: optional path to a JSONL checkpoint file; if given, runs via
the checkpoint/resume executor (already-completed runs, by key,
are skipped and loaded from the journal instead of re-executed).
parallel: if True, runs via rayon in parallel; if False, runs
sequentially. Applies both with and without a journal (a journaled
parallel run checkpoints each record incrementally as it
completes). Both modes produce bit-identical results.
threads: thread pool size for the parallel executor; None uses rayon's
global pool.
log_dir: optional directory to also write an IOH-profiler-format log
tree to. Only runs this call actually EXECUTES are logged -- a run
resumed from an existing journal was executed in a prior process
and is never re-logged, so the on-disk IOH tree does not grow on
resume. Read back with read_ioh_records/ecdf/coco_export.
read_ioh_records ¶
read_ioh_records(log_root, budgets)
Reconstruct run records from an on-disk IOH archive at log_root
(as written by run_experiment(..., log_dir=...) or
solve(..., log_dir=...)), one record per (run, budget) pair.
log_root: root directory of the IOH archive.
budgets: list of evaluation budgets to reconstruct a best-so-far value
at (see sezgi_bench::ioh_records's doc comment for the exact
best_f/evals_used semantics, and the curtailed-view-vs-
independent-run distinction for a budget smaller than a run's
logged budget).
Returns the SAME record-dict shape run_experiment returns, so
results_matrix/per_budget_packages accept it unchanged.
ecdf ¶
ecdf(log_root, targets=None, per_algo=True)
ECDF (anytime performance) curve(s) over an on-disk IOH archive.
log_root: root directory of the IOH archive.
targets: precision targets; None uses the COCO-convention 51-value
default target set (10^(2 - 0.2*k) for k = 0..=50).
per_algo: if True (default), returns a list of (algo, curve) pairs,
one per distinct algorithm in the archive, in first-appearance
order; if False, returns a single pooled curve over every scenario.
Grouping is by algo only, not (algo, suite): a mixed-suite tree
(BBOB and CEC 2022 runs for the same algo) pools both suites' runs
into that one algo's curve, so read a per-suite tree or filter
records by suite first for a curve that is suite-specific.
Each curve is a dict {"evals": [...], "proportion": [...]}: evals
ascending, proportion in [0, 1] and monotonically nondecreasing.
coco_export ¶
coco_export(log_root, out_dir)
Export an on-disk IOH archive at log_root as a COCO/BBOB "old
format" archive rooted at out_dir, so it can be post-processed with
cocopp. Returns the list of written file paths (as strings), sorted
for determinism.
BBOB-only: COCO's "old format" IS the BBOB archive format and has no
CEC counterpart, so this raises ValueError if log_root holds any
non-BBOB scenario (naming the offending suite), rather than silently
merging it into a bbob-labeled archive. A mixed BBOB+CEC tree must be
filtered to its BBOB records before exporting.
results_matrix ¶
results_matrix(records, budget, aggregate='mean')
Build a sezgi.stats.paper_package-shaped results matrix for one
budget from run_experiment's record dicts.
records: list of dicts as returned by run_experiment (or any list of
dicts with the same fields: algo, fid, dim, instance, seed, budget,
best_f, f_opt, evals_used).
budget: only records with this budget are used.
aggregate: how to combine a (problem, algorithm) cell's per-seed gaps
(best_f - f_opt) into one number: "mean" or "median".
Returns (algo_names, problem_labels, matrix): matrix[i][j] is the
aggregated gap of algo_names[j] on problem_labels[i]. Problem labels
are f{fid}d{dim}i{instance}; both lists are ordered by first
appearance in records. Raises ValueError if a (problem, algorithm)
pair present for one algorithm/problem is missing for another at this
budget (an incomplete experiment) -- see
sezgi_bench::reporting::results_matrix.
per_budget_packages ¶
per_budget_packages(records, rope=0.0, samples=20000, seed=1, aggregate='mean')
Build one sezgi.stats.paper_package PER DISTINCT BUDGET present in
records, in ascending budget order.
Piotrowski et al. (2025) show algorithm rankings on benchmark comparisons can flip depending on which evaluation budget is examined, so this makes multi-budget reporting the default rather than a single, arbitrarily-chosen budget's report: compare algorithms per budget, never pooled across budgets.
records: list of dicts as returned by run_experiment.
rope, samples, seed: forwarded to the Bayesian signed-rank test inside
each budget's paper_package (same as sezgi.stats.paper_package).
aggregate: "mean" or "median" -- see results_matrix.
Returns a list of [budget, package_dict] pairs; each package_dict
has exactly the shape sezgi.stats.paper_package returns.