Statistics¶
The full statistical-comparison suite (sezgi.stats.*): Friedman +
Nemenyi CD, Wilcoxon signed-rank with Holm correction, Cliff's delta,
Bayesian signed-rank, Plackett-Luce, and the one-call paper_package
that runs all of them over a results matrix.
stats.friedman
builtin
¶
stats_friedman(results)
sezgi.stats.friedman(results) -- the Friedman rank test for comparing
3+ algorithms across multiple problems (a non-parametric alternative to
repeated-measures ANOVA). results: a 2D array-like, rows = problems,
columns = algorithms (same shape results_matrix produces). Returns a
dict: statistic (the chi-square statistic), p_value, mean_ranks
(list of floats, one per algorithm/column -- lower rank is better).
stats.wilcoxon
builtin
¶
stats_wilcoxon(a, b)
sezgi.stats.wilcoxon(a, b) -- the Wilcoxon signed-rank test for two
PAIRED samples (a[i]/b[i] from the same problem/seed). Returns a
dict: w_statistic, z, p_value, n_effective (pairs after dropping
zero-difference ties), method ("exact" when n_effective <= 25 with
no tied |difference| ranks, "normal_approx" otherwise).
stats.cliffs_delta
builtin
¶
stats_cliffs_delta(a, b)
sezgi.stats.cliffs_delta(a, b) -- Cliff's delta, a non-parametric
effect size for two INDEPENDENT samples (unlike wilcoxon, a/b need
not be paired or equal length): (greater - less) / (len(a) * len(b)),
where greater/less count pairs (ai, bj) with ai > bj / ai < bj.
Range [-1, 1]; 0 = no stochastic dominance either way. Pass the
result to stats.cliffs_magnitude for a qualitative label.
stats.cliffs_magnitude
builtin
¶
stats_cliffs_magnitude(delta)
sezgi.stats.cliffs_magnitude(delta) -- Romano et al.'s qualitative
bucketing of a Cliff's delta value (as returned by stats.cliffs_delta)
into "negligible" (|delta| < 0.147), "small" (< 0.33),
"medium" (< 0.474), or "large" (otherwise) -- thresholds applied to
|delta|, so the sign (direction) is discarded.
stats.plackett_luce
builtin
¶
stats_plackett_luce(rankings)
sezgi.stats.plackett_luce(rankings) -- fits a Plackett-Luce model
(Minorization-Maximization) to a set of full rankings over the same k
items. rankings: a list of lists, each a permutation of 0..k (item
indices, best-to-worst; every ranking must cover all k items). Returns
a dict: worths (list of k non-negative floats, one per item,
normalized to sum to 1.0 -- a higher worth means the item tends to rank
better), p_best (list of k floats, the PL-model probability each
item is ranked first -- numerically identical to worths under Luce's
choice axiom, kept as a distinct field for API clarity), iterations
(MM iterations to convergence, capped at 10000).
stats.bayesian_signed_rank ¶
_bayesian_signed_rank(a, b, rope=0.0, samples=20000, seed=1)
stats.bayesian_signed_rank(a, b, rope=0.0, samples=20000, seed=1)
-> dict
A Bayesian alternative to stats.wilcoxon for two PAIRED samples, with a Region Of Practical Equivalence (rope: |a[i] - b[i]| <= rope counts as "practically equal"). samples draws from the posterior (Monte Carlo); seed makes the draw reproducible. Returns a dict (minimize convention, lower is better, values sum to 1.0): p_left (P(a practically better than b)), p_rope (P(practically equivalent)), p_right (P(b practically better than a)).
stats.bayesian_plackett_luce ¶
_bayesian_plackett_luce(rankings, samples=2000, burn_in=500, seed=1)
stats.bayesian_plackett_luce(rankings, samples=2000, burn_in=500,
seed=1) -> dict
A Bayesian (posterior-sampling) counterpart to stats.plackett_luce: same rankings input (a list of lists, each a permutation of 0..k item indices, best-to-worst), but returns a posterior distribution over worths instead of a single point estimate. burn_in draws are discarded before samples are recorded; seed makes the draw reproducible. Returns a dict: mean_worths (posterior mean per item, sums to 1.0), ci_low/ ci_high (per-item 95% credible interval), p_best (per-item posterior probability of the largest worth), samples (the recorded draw count).
stats.paper_package ¶
_paper_package(algo_names, problem_names, results, rope=0.0, samples=20000, seed=1)
stats.paper_package(algo_names, problem_names, results, rope=0.0,
samples=20000, seed=1) -> dict
Runs the FULL statistical comparison suite (stats.friedman + Nemenyi CD + pairwise stats.wilcoxon with Holm correction + pairwise stats.cliffs_delta + pairwise stats.bayesian_signed_rank + stats.plackett_luce) over one results matrix in a single call, ready to drop into a paper. results: rows = problems (matching problem_names, one row each), columns = algorithms (matching algo_names). rope/samples/seed are forwarded to the Bayesian sub-tests. Returns a dict with keys friedman, nemenyi_cd, pairwise_wilcoxon_holm (list of [i, j, p_value] rows, Holm-corrected), cliffs (list of [i, j, delta] rows), bayes (list of [i, j, {p_left, p_rope, p_right}] rows), plackett_luce, latex_summary, latex_tests -- see each individual stats.* function's own docstring for its sub-result's shape.