API reference¶
The full public surface, autogenerated from docstrings.
Top-level¶
polar-high — Python library for building indexed linear and mixed-integer programs in polars.
Problem ¶
LP container. Generic — no flextool-specific knowledge.
Construct an empty LP container.
Pure polar-high is a generic LP kernel; scaling decisions are
left to the caller. See :mod:polar_high.autoscale for the
opt-in autoscaler (Layer 1 detect + Layer 3 recommendation)
that callers (e.g. FlexTool) use to drive
user_bound_scale / user_objective_scale automatically.
dense_axes — the explicit client contract for the block-COO
LHS arm. When the client (e.g. FlexTool) declares the dense
trailing axes once here (e.g. Problem(dense_axes=("d", "t"))),
it makes a binding PROMISE about every frame it passes that
contains those columns:
the frame is globally lexicographically sorted by
``(other_dims_in_declared_order..., *dense_axes)`` — i.e. the
declared dense axes are the trailing sort keys, in the given
order, and the leading dims form a sorted prefix.
This lets block-COO slice the dense suffix of each Var with NO
re-sort (a re-sort would cost more than the multiply itself).
polar-high VERIFIES this promise cheaply (a single-pass monotonic
scan — see :func:_verify_dense_sorted) on every Var that the
block-COO arm classifies + fires on, and RAISES a clear
ValueError naming the Var if the client breaks it. Frames
that do not contain the dense axes (e.g. an investment Var
("p", "d") when dense_axes=("d", "t")) simply do not fire
block-COO and are unaffected. None (default) leaves the
block-COO arm dormant — it only fires once dense axes are declared
(here or via :meth:declare_dense_axes).
declare_dense_axes ¶
Declare the dense trailing axes for block-COO (see init contract).
Equivalent to passing dense_axes= to the constructor; provided so
callers that receive an already-constructed Problem (e.g. FlexTool's
build_flextool step, which builds the Problem first and populates
it afterwards) can declare them. Pass None to clear.
set_solver_options ¶
Store HiGHS options to be applied in solve(). Pass None
to clear. Keys are HiGHS canonical option names (presolve,
solver, parallel, time_limit etc); values must be
already coerced to the type HiGHS expects (str/int/float/bool).
Unknown keys are tolerated (a warning is emitted at solve time).
set_solver_option ¶
Set a single HiGHS option, leaving the rest untouched.
Convenience for callers that want to add one knob without
re-passing the whole dict (e.g. user_bound_scale set by
the autoscaler). Equivalent to a dict merge plus
:meth:set_solver_options.
get_solver_option ¶
Return the caller-set value of name (or None if unset).
Reads self._solver_options only — does NOT consult HiGHS
(the option may not have been pushed to a live Highs
instance yet). Returns None for unset options so callers
can use if get_solver_option(...) is not None to test
explicit setting.
add_cstr ¶
add_cstr(name: str, *, over: DataFrame | None = None, sense: str, lhs_terms: dict[str, Var | Expr | Param | int | float], rhs_terms: dict[str, Var | Expr | Param | int | float] | None = None) -> None
Add a constraint of the form Σ lhs_terms sense Σ rhs_terms.
Each term entry is either
- a
VarorExpr— variable contribution, or - a
Param,intorfloat— constant contribution.
The engine sorts variables and constants out per side, builds
(lhs_var − rhs_var) sense (rhs_const − lhs_const), and adds
the row to highspy at solve time. Labels (the dict keys) are
used in row names and diagnostics.
cstr_names ¶
All constraint family names currently registered, in declaration order. Useful for emission audits and debugging.
cstrs_named ¶
Return constraint metadata records matching name.
An exact-name match returns the single record; otherwise a prefix
match returns every record whose name starts with name + "_"
(so passing "minimum_uptime" returns both
minimum_uptime_linear and minimum_uptime_integer).
Each :class:CstrRecord carries:
* name: full registered name of the constraint family;
* over: the polars DataFrame of axis tuples (len(over)
is the row count);
* proto: the underlying _CstrProto (expr, sense,
rhs) for advanced introspection.
set_named_basis ¶
Record a warm-start basis to inject at the next solve (spec §4.1).
This is the in-process (non-save_memory) primitive: it only
RECORDS INTENT — it stores the carrier on self._warm_basis and
the policy on self._warm_basis_policy. It does NOT build or
inject anything. The solve-time hook (streaming path, just before
h.run()) constructs the sized HiGHS basis via
:func:polar_high._warm_basis.build_highs_basis and calls
h.setBasis.
policy="exact" transfers 1:1 by name and requires the carrier's
fingerprint to match this model's; a mismatch under "exact" is
expected to be caught by the hook, which then falls back safely to
a cold solve. policy="alien" maps shared names and lets HiGHS
repair the remainder.
basis_name_fingerprint ¶
Structural fingerprint of THIS model's LP name-set (spec §4.5).
A cheap, side-effect-free accessor the orchestrator uses as the
cache key for a captured :class:NamedBasis: it is the
:func:polar_high._warm_basis.basis_fingerprint of this model's
rendered column + constraint-row names, so a basis captured off one
build and re-injected into an identically-shaped build key on the
same string. It mirrors what :meth:write_mps stashes as
_last_mps_fingerprint (same basis_fingerprint(col_names,
m.row_names) inputs) but is callable without writing an MPS.
Uses the canonical matrix's names (m = self.canonicalise());
canonicalise is memoized, and LP names are Layer-2-invariant, so
the value is valid pre- or post-scaling and stable across repeated
calls. Returns None when the problem has been released or has no
columns (no meaningful key).
seed_primal ¶
seed_primal(values: dict[str, float], *, frame: Literal['user', 'scaled'], missing: str = 'skip') -> None
Record a partial primal seed to inject at the next solve.
Phase 3 Step 3a primitive. Like :meth:set_named_basis this only
RECORDS INTENT — it stores the seed on self._primal_seed and the
frame on self._primal_seed_frame. It does NOT build or inject
anything: the solve-time hook (streaming path, just before
h.run(), immediately after the basis block) resolves each name to
a column index, converts + sanitises the value, and calls
h.setSolution.
values maps column name (rendered exactly as
Solution.col_names — "name[d0,d1,...]" for indexed vars, the
bare name for scalars) to a numeric value. A partial seed is
legal and expected: names present in the model are seeded, names
absent are silently skipped (the subset behaviour, missing="skip").
frame is REQUIRED and has NO default (a silent default frame is a
corruption vector). frame="scaled" uses each value as-is in the
solver's post-Layer-2 scaled space. frame="user" converts with
the model's own Layer-2 side-vector
(x_scaled = x_user / _layer2_col_factor[col_id]; identity when
Layer 2 was not applied), so callers can seed in natural units.
Feasibility-safety: a primal seed is only a starting point for the solver — it never binds the optimum. A non-optimal (or even infeasible-looking) seed cannot change the objective HiGHS returns, with presolve on or off.
MUTUALLY EXCLUSIVE with a warm basis. setSolution and
setBasis clobber each other on the same handle (last writer wins;
setSolution invalidates a previously-set basis). For an LP a
basis dominates a partial seed, so the hook enforces the rule "if a
basis is set, it wins": when self._warm_basis is also set the seed
is skipped.
In-process (streaming) ONLY — there is no subprocess / MPS channel
for a primal seed; a save_memory solve ignores the seed.
A seed over integer columns is a MIP-start. (Noted for a future UC5; not implemented or specialised here — integer columns are seeded the same way and HiGHS treats them as a MIP-start.)
cstr_row_count ¶
Total LP-row count across all constraint families matching
name (exact or prefix; see :meth:cstrs_named). Returns 0
when no families match — letting callers distinguish "absent"
from "empty" without exception handling. A scalar constraint
(over=None) counts as one row.
add_obj_constant ¶
Accumulate a constant into the objective offset. HiGHS adds
this to the reported getObjectiveValue() after solve, so it
shows up in Solution.obj even though no decision variable
carries it. Used for pure-Param objective terms like the §8.1
existing-entity fixed cost.
solve ¶
solve(*, options: dict | None = None, keep_solver: bool = False, streaming: bool = True, save_memory: bool = False, tmp_dir: str | PathLike | None = None) -> Solution
Solve the LP and return a :class:Solution.
Parameters¶
options
Per-call HiGHS options dict (overrides set_solver_options).
keep_solver
When True, the live HiGHS instance is kept on the returned
:class:Solution so callers can inspect it post-solve (e.g.
sol.highs.writeModel("model.mps")). Default False —
the C-side LP storage is released as soon as primal/dual/
objective have been extracted.
streaming
When True (default), columns are added once via addCols
and each constraint family is emitted to HiGHS via addRows
immediately after its COO triples are built; the family's local
arrays then go out of scope before the next family is processed.
This caps peak memory at one family's COO + the running HiGHS
LP. When False, the entire model is assembled into a single
:class:highspy.HighsLp and loaded via passModel —
numerically identical results either way; False is mostly
useful for benchmarking the legacy path.
save_memory
Single one-shot knob that trades wall time for peak RSS. When
True, two things happen right before HiGHS.run():
1. polar-high's polars/numpy LP source-of-truth (constraint
family ``Expr.terms`` lists, ``_CstrProto.rhs`` Param
frames, objective ``_Term.lazy`` plans, the caller-side
``col_names`` / ``row_names`` lists) is dropped. Only the
per-:class:`Var` ``col_id`` frames survive — they are
needed by :meth:`Solution.value` to map column indices
back to user-space dim tuples.
2. The HiGHS instance is round-tripped through disk: the LP
is written to a temp MPS file, the original ``Highs()`` is
cleared and discarded, ``malloc_trim(0)`` is called (best
effort, Linux only), a fresh ``Highs()`` is created, the
same solver options are re-applied, and the MPS file is
read back. This resets the HiGHS allocator's high-water
mark — the C++ side accumulates ~5 GB of slack from the
incremental ``addRows`` loading path that ``readModel``
avoids by sizing once up front.
Cost: ~+90 s of MPS file I/O at N=3000 dense. Benefit: peak
RSS drops by ~5 GB on the same problem. Intended for one-
shot single-solve benchmarks where warm-start / re-solve
isn't needed. After the call returns, the :class:`Problem`
is in a "released" state and any further :meth:`solve` call
raises ``RuntimeError`` (the polar-side source AND the
original HiGHS instance with its basis have both been
discarded, so neither a fresh re-solve nor a WarmProblem-
style update is possible). Honoured only by the streaming
path; on ``streaming=False`` a warning is emitted and the
flag is ignored. Default ``False``.
tmp_dir
Directory for the temporary MPS file written by the
save_memory=True round-trip. None (default) uses
the system temp dir ($TMPDIR / /tmp). Set when the
caller wants the spill file on a specific volume (e.g. the
same filesystem as its workspace, or a per-job scratch
directory) instead of the system default. Ignored when
save_memory=False.
build_only ¶
Build the LP into HiGHS, write to mps_path, release everything.
For callers that want to drive the actual solve out-of-process — typically a subprocess HiGHS reading the MPS file in a clean address space. After this call:
mps_pathexists and contains the LP in MPS format.- polar-side LP source-of-truth has been dropped
(:meth:
_release_python_lp_inputs). - The live :class:
highspy.Highsinstance has been torn down and the glibc allocator trimmed (Linux best-effort). self._releasedis True — further calls to :meth:solvewill raise.
What stays alive: self._vars and each :class:Var.frame,
so the caller can later construct a :class:Solution from
externally-produced col_value / row_dual arrays via
Solution(..., vars=dict(self._vars), ...).
Honoured only on the streaming path; the non-streaming
(passModel) path is not supported here. Raises
RuntimeError if called on an already-released Problem.
Parameters¶
mps_path
Where to write the MPS file. Caller owns the file —
polar-high will not delete it.
options
HiGHS solver options to apply during the build (mainly
presolve / solver / simplex_scale_strategy —
options that affect what writeModel serialises).
None uses :attr:_solver_options.
canonicalise ¶
Build (or return cached) the canonical CSC matrix + metadata.
Idempotent: returns the cached _matrix unless
_canonical_dirty is set (which add_var / add_cstr
flip). Side vectors (_layer2_col_factor /
_layer2_row_factor) are baked into the returned arrays at
build time per orchestrator decision D8.
write_mps ¶
write_mps(path: str | PathLike, *, free_format: bool = True, column_order_strict: bool = True, emit_names: bool = True, release: bool = False, name: str = 'POLAR_HIGH') -> None
Stream the LP/MIP to path as a free-format MPS file.
This is an in-house writer that consumes the polar-side LP
source-of-truth (_vars, _cstrs, _obj_terms) directly
and emits MPS without ever instantiating :class:highspy.Highs
or building an :class:LpView. The motivation is peak-memory:
HiGHS' own writeModel serialiser materialises ~20× more
transient state than the in-process LP itself, which OOMs on
very large LPs (~10 M rows × 5 M cols × 20 M nonzeros). This
writer targets ~2-3 GB peak on that same LP.
Parameters¶
path
Output MPS file path. Caller owns the file; this writer
does not delete on failure.
free_format
Reserved for future use. Currently always free-format MPS
(the modern default — HiGHS, Gurobi, CPLEX, Xpress all
accept it). Fixed 8-char-column MPS is not yet implemented.
column_order_strict
When True (default), all nonzeros for a single column
are emitted contiguously in the COLUMNS section, sorted by
(col_id, row_id). This is the spec-compliant ordering
every MPS reader accepts. False is currently not
implemented — see :ref:Open follow-ups in the module docs.
emit_names
When True (default), row and column names come from the
variable family name + dim tuple and the constraint family
name + dim tuple (matching the format used by
:meth:Problem.solve for Solution.col_names /
Solution.row_names). When False, generic
C0000001 / R0000001 names are emitted instead —
saves ~30-50% file size at the cost of losing the
dim-tuple → column mapping in the MPS itself. Callers
using emit_names=False must use index-based solution
readback (column 0 in the MPS is column 0 in the polars
Var.frame ordering, etc.). The objective row name is
always cost regardless — it's a reserved unique name.
release
When True, calls :meth:_release_python_lp_inputs
after the write completes. Mirrors the
solve(save_memory=True) semantics — drops the polars
LazyFrames, constraint family list, and rhs Param frames
while keeping self._vars (so solutions read back from an
external solver can still be mapped to user-space dim
tuples via the surviving Var.frame["col_id"] columns).
After release the :class:Problem is no longer solvable —
:meth:solve will raise.
name
MPS NAME header; default "POLAR_HIGH".
Raises¶
RuntimeError
If called on a :class:Problem whose source has already
been released (by an earlier solve(save_memory=True),
build_only(...), or write_mps(..., release=True)).
ValueError
If any coefficient is NaN or infinite (silent filtering
would hide model bugs). The error message identifies at
least one offending (col_id, row_id) pair.
OSError
File I/O failures (passed through from open()).
Notes¶
- The MPS format has no portable encoding for an objective
constant offset. If
self._obj_offset != 0.0this method emits a :class:UserWarningand writes the LP without the offset. Solutions from any downstream solver will then be off byself._obj_offset— the caller must add it back manually if needed. - Integer columns are bracketed with
MARKER 'INTORG'/MARKER 'INTEND'in the COLUMNS section, per MPS convention. - The writer is single-pass per family and uses one global polars sort to put the COLUMNS section into column-major order. Streaming-engine fallback to disk activates automatically for triple counts that don't fit in RAM.
- Stashes
self._last_mps_fingerprint— a structural basis-fingerprint (:func:polar_high._warm_basis.basis_fingerprint) over the emitted column/constraint-row name-set — computed before anyrelease.Noneunlessemit_namesis True (generic names are useless as a cross-run key). Consumed by thesave_memorysubprocess warm-start arm to key a basis-cache file.
Var ¶
Var(name: str, dims: tuple[str, ...], frame: DataFrame, lower: float = 0.0, upper: float = float('inf'), integer: bool = False)
A variable family. frame carries columns *dims, col_id.
Var.frame stays an eager polars DataFrame — it's small (one row
per LP column), produced once in :meth:Problem.add_var, and
consumed by both flextool integration (v.frame["col_id"].unique())
and Problem.solve (col_id → bound/name lookups). Algebra ops on
Var lazify on the fly so the resulting _Term is lazy.
Param ¶
Param(dims: tuple[str, ...], frame: DataFrame | LazyFrame, name: str | None = None, _sources: list[tuple[Param, int]] | None = None, _value_scalar: float = 1.0)
A parameter table. frame carries columns *dims, value.
Stored internally as a polars.LazyFrame so that chained algebra
ops (Param * Param, Param + Param etc.) defer materialization
until a consumer reads .frame or the engine collects in
Problem.solve. The .frame property caches the eager
DataFrame on first read — flextool reads .frame.rename(...)
repeatedly off the same Param so we want that to be cheap.
name (optional) is a logical Param identifier (e.g. "p_inflow").
It is opt-in metadata used by :class:WarmProblem's Param-tracked
auto-update (declare_mutable / update_param). When unset,
Params are anonymous and carry no tracking overhead.
_sources records constituent named Params for composite results
of Param * Param / Param / Param. Each entry is
(param_name, dims_tuple, direction) where direction is +1 if the
Param contributes to the numerator and -1 if to the denominator.
Anonymous-only chains have _sources is None.
Expr ¶
A sum of terms (decision-variable contributions).
The terms can have different open-dim sets — they're concatenated,
not broadcast. Broadcasting happens once, at constraint emission,
via a join to the constraint's over= row index.
WarmProblem ¶
Warm-update wrapper around a :class:Problem.
Build a :class:Problem as usual (add_var, add_cstr,
set_objective). Wrap with :class:WarmProblem, then alternate
update_* calls and solve() calls — the LP is built ONCE and
only the changed coefficients / RHS values are pushed to HiGHS
between solves.
Typical rolling-horizon usage::
wp = WarmProblem(p)
sol_0 = wp.solve()
for r in range(1, n_rolls):
wp.update_rhs("balance", demand_param_for_roll[r])
wp.update_obj_coef("v_flow", cost_param_for_roll[r])
sol_r = wp.solve()
The update_* calls are O(rows_or_cols_in_family); the solve()
benefits from HiGHS's hot-start (basis is preserved across calls).
problem
property
¶
The underlying :class:Problem.
Useful for diagnostics that need to inspect the un-built LP.
For coefficient ranges, prefer
:attr:Solution.streamed_lp_ranges on a returned
:class:Solution (populated during the streaming solve at zero
extra cost).
set_named_basis ¶
Record a warm-start basis to inject on the fresh build (spec §4.1).
RECORDS INTENT only — stores the carrier on self._warm_basis and
the policy on self._warm_basis_policy; it does NOT build or
inject anything here. The actual h.setBasis fires exactly once,
in :meth:_initial_build, immediately before the first solve's
h.run(). This is the in-process default-path equivalent of the
streaming basis hook in :meth:Problem._solve_streaming.
policy="exact" transfers 1:1 by name and requires the carrier's
fingerprint to match this model's; a mismatch under "exact" is
detected in _initial_build and safely falls back to a cold solve.
policy="alien" maps shared names and lets HiGHS repair the rest.
The inject fires only on the fresh build — _initial_build runs
once per WarmProblem lifetime (solve calls it only while
self._h is None), so a subsequent height-match reuse re-run never
re-injects.
update_rhs ¶
Replace the RHS of constraint family cstr_name with values
drawn from new_param.
new_param may be a :class:Param whose dims match a subset
of the constraint's over= axis (broadcasting to the rest), a
scalar (broadcast to all rows), or a numpy array (positional —
length must equal the family's row count, in the order of the
original over frame).
update_obj_coef ¶
Replace the objective coefficient on every column of var_name.
Assumes the objective contribution from var_name is exactly
coef[*dims] * var[*dims] for some coef Param; this method
OVERWRITES that coefficient via h.changeColsCost. If the
objective also has contributions from this variable through more
complex algebra (e.g. var * unitsize * slope), the update is
still valid as long as new_param carries the full product —
the caller is responsible for collapsing multi-Param products
ahead of the call.
This DOES NOT touch the cost coefficients of other variables.
update_obj_coef_array ¶
Array-form of :meth:update_obj_coef.
dim_tuples is a list of dim-value tuples (one per cell) for
variable var_name; each tuple must have one entry per dim
in the var's declared signature. values is a same-length
numpy array of new objective coefficients.
The columns are resolved positionally: values[k] becomes the
new objective coefficient on the column whose dim-tuple is
dim_tuples[k]. Vectorised — a single changeColsCost
call regardless of cell count.
fix_cols ¶
Fix the listed columns of var_name to the given values.
For each (dim_tuple, value) pair, sets both the column's
lower and upper bound to value (so the LP has no choice
but to set the column at that level). Used by the Lagrangian
primal-recovery step ("fix-and-resolve"). Vectorised — single
changeColsBounds call.
fix_col_ids ¶
Fix columns BY RAW COL ID: set lower = upper = value for each
(col_id, value) pair.
The col-id-level counterpart of :meth:fix_cols — the same
vectorised changeColsBounds(lo=hi=value) write, but skipping
the per-call dim-tuple → col-id join for callers that already
hold the raw ids (resolve once via :meth:col_id_of_var, or use
ids returned by :meth:add_recourse_col). Typical consumer: a
cutting-plane driver temporarily pinning coupling columns.
Bounds are written verbatim into the live model — any caller-side
scaling must be applied to values beforehand.
get_col_bounds ¶
Read the current (lower, upper) bounds of the given raw col
ids from the live model.
Returns two float64 arrays aligned POSITIONALLY with col_ids
(any order, repeats allowed — HiGHS' getCols itself requires
an ordered duplicate-free index set, so the query goes through
the unique sorted set and gathers back to caller order).
Save/restore companion of :meth:set_col_bounds for temporary
pins made with :meth:fix_col_ids::
lo, hi = wp.get_col_bounds(ids) # save
wp.fix_col_ids(ids, values) # temporary pin
... wp.solve() ...
wp.set_col_bounds(ids, lo, hi) # restore
set_col_bounds ¶
Write (lower, upper) bounds onto the given raw col ids.
Restore companion of :meth:get_col_bounds (see there for the
save/pin/restore pattern). Vectorised — single
changeColsBounds call; ±inf entries round-trip unchanged
(HiGHS' bound sentinel IS the IEEE infinity).
update_coef ¶
Update a single (row, col) coefficient in the constraint
matrix. Use :meth:row_id_of_cstr / :meth:col_id_of_var to
resolve indices semantically.
add_cut_row ¶
Append a >= constraint row to the live (already-built) LP and
return the new row's id.
Appends Σ coefs[k] · x[col_ids[k]] >= lower to the live HiGHS
model via addRow. This is a POST-build mutation of the warm
_h handle (the same class of live edit as :meth:update_rhs /
:meth:update_coef); it deliberately bypasses the build-time
Problem.add_cstr DSL lock, which only guards the fixed-size
Layer-2 autoscale side vectors and is irrelevant once the model is
built. The caller is responsible for keeping the master autoscale
OFF (or for pre-scaling coefs) so the appended row lives on the
built columns' scale — there is no auto-scaling for appended rows.
col_ids may mix existing column ids (resolve via
:meth:col_id_of_var) and ids of columns previously appended with
:meth:add_recourse_col. A subsequent :meth:solve warm
re-optimises the grown model; the appended row's dual is then
readable on the returned :class:Solution by row_dual[row_id]
(read BY ROW ID, not via :meth:Solution.constraint_dual, which
only knows named build-time rows).
Bumps the cached _n_rows (the zero-fill fallback size in
:meth:solve) and appends a generated name to _row_names (kept
index-aligned so name-indexed reads stay correct). Returns the
row_id HiGHS assigned (== the pre-append getNumRow()).
add_recourse_col ¶
Append a single free/bounded column (e.g. a Benders recourse
η) to the live LP and return its col id.
Symmetric with :meth:add_cut_row: a POST-build addCol on the
warm _h handle, bumping the cached _n_cols (zero-fill
fallback size) and _col_names. ±np.inf bounds are mapped to
the HiGHS kHighsInf sentinel. The returned id can be referenced
from a later :meth:add_cut_row; its value is read on the
:class:Solution by col_value[col_id] (appended columns are not
in any Problem._vars frame, so :meth:Solution.value cannot see
them — read by id).
compact_cuts ¶
compact_cuts(solution: Solution, *, policy: str = 'slack', trial_col_values=None, tol_rel: float = 1e-07, verify: bool = True) -> dict
Prune retained cut rows to bound the active master LP.
A cutting-plane master (the Benders master) grows one >= cut row
per iteration via :meth:add_cut_row; left unpruned its warm
re-solve time grows super-linearly. Two selection policies are
available, both LB-safe (they only drop cuts that do not support the
certified lower bound) and both belted by the same verify-restore
rollback:
policy="slack" (default):
Drop the cut rows that are strictly slack (positive primal slack,
hence zero dual by complementary slackness) at solution,
keeping the binding ones. Classification is purely by PRIMAL
slack of the stored cut against solution.col_value: for a cut
Σ coefs·x >= lower the slack is Σ coefs·x - lower >= 0; a
cut with slack <= tol is BINDING (kept), one with
slack > tol is strictly slack (dropped). tol is a per-cut
relative tolerance scaled by the magnitude of lower and the
terms.
policy="dominance" (de Matos–Philpott–Finardi / Guigues
Limited-Memory Level-1):
At a DEGENERATE optimum almost every cut can be primal-binding
(slack ≈ 0), so slack-deletion prunes nothing even though many
cuts are redundant TIES at their recourse group's max value.
Dominance selection instead groups the retained cuts by the
recourse variable they bound (the single member of
col_ids ∩ _recourse_cols, registered by
:meth:add_recourse_col) and, over a WINDOW of recent master
trial points trial_col_values (a list of col_value arrays,
most-recent last; defaults to [solution.col_value]), keeps for
each group at each trial point the OLDEST cut that achieves the
group's max value there (within a relative tolerance). Every cut
not a keeper at any trial point is DROPPED — it is dominated (or a
redundant tie) at every trial point, so the outer approximation of
the recourse function at those points, and the LB it certifies,
is unchanged. A cut whose col_ids contains no recourse column
forms its own singleton group and is always kept (generic
non-Benders use degrades safely to "keep all").
When verify (default), after deleting rows the master is re-solved
and its objective compared to the incumbent solution.obj. A drift
larger than :data:_COMPACT_VERIFY_TOL (relative) means a dropped cut
was in fact supporting the optimum (a degenerate numerical edge) — the
deletion is ROLLED BACK by re-appending every dropped cut verbatim and
re-solving, and the returned dict reports restored=True with
dropped=0. On a well-behaved master the belt never fires.
Returns {"kept": int, "dropped": int, "restored": bool} — cut
counts AFTER the operation (kept == number of retained cut rows
that remain, dropped == number removed; on a restore kept is
the pre-compaction count and dropped is 0).
declare_mutable ¶
Declare a set of :class:Param names whose values should be
tracked into LP cells, so :meth:update_param can later push
new values into the live HiGHS instance via changeCoeff.
MUST be called BEFORE the first :meth:solve. Tracking is
opt-in: Params not declared here pay no bookkeeping cost.
Pass the same names that the Params carry on their .name
field — typically the FlexData attribute name ("p_inflow",
"p_penalty_up" etc.).
set_output_flag ¶
Enable or disable HiGHS' native solve log for this problem.
enabled=False mutes the per-solve HiGHS output (the
output_flag HiGHS option) for THIS WarmProblem across all of
its cold / warm / retry solves — callers that drive many parallel
sub-solves (Benders regions, Lagrangian subproblems) use this to
keep stdout clean and emit their own concise progress log instead.
May be called either before the first :meth:solve (the flag is
applied when the HiGHS handle is built) or after it (applied to the
live handle immediately). The preference persists on the handle,
so a single call suffices for the whole lifetime.
update_param ¶
Replace the values of a tracked Param. Every LP cell whose
coefficient was originally a function of param_name is
re-computed from the new Param's values and pushed via
h.changeCoeff.
new_param must be either a scalar (broadcast to all tracked
cells) or a :class:Param whose dim signature matches the
signature recorded for that Param at build time.
Raises if param_name was not in :meth:declare_mutable's
list (silent corruption is worse than a hard error).
col_id_of_var ¶
Return the col_id(s) for a variable.
dims=None returns every col_id in the variable's family
(numpy array, ordered by the var's declaration order).
dims as a tuple of dim values returns the single col_id for
that one cell (a python int). dims as a dict
{dim_name: value} is a partial filter — returns a numpy
array of the matching col_ids.
row_id_of_cstr ¶
Return the row_id(s) for a constraint family. Mirrors
:meth:col_id_of_var.
solve ¶
Solve the LP. First call builds the LP from scratch (same
pipeline as :meth:Problem.solve); subsequent calls just run
HiGHS again on the (possibly updated) live model.
options is honoured on the FIRST solve only — subsequent
solves use the same HiGHS instance. To change options on a
rebuilt LP, drop the WarmProblem and create a new one.
retry_on_unknown (default False → byte-identical to the
legacy path for every existing caller) enables a warm-restart
retry: the solver runs WARM first (retaining the basis so dual
simplex hot-starts after rows appended via :meth:add_cut_row);
only if the warm run does NOT return a certified kOptimal —
i.e. a stale-basis transient (kUnknown / kSolveError) or a
spurious kUnbounded / kInfeasible / primal-infeasible miss — do we
drop the basis with clearSolver() and re-run ONCE (the proven
cold fallback) to re-certify the true status. Used by the Benders
master, which appends a cut each iteration; on a well-scaled
master the warm path stays kOptimal and the fallback never
fires.
solve_with_fallback ¶
Solve; on a non-optimal primary, retry ONCE under
fallback_options and restore the prior option values.
Runs :meth:solve first. If it certifies kOptimal the
solution is returned as (solution, "primary") and the options
are never touched. Otherwise each fallback_options entry is
applied to the live HiGHS handle via setOptionValue
(solve(options=...) is honoured on the FIRST solve only — a
live-handle write is the only way to change options on a built
model), the retained solver state is dropped with
clearSolver() (so options like solver / presolve take
effect from cold, and no stale basis biases the retry), the model
is re-run once, and the prior option values are RESTORED — a
later plain :meth:solve sees the original configuration.
Returns (solution, "fallback") with the retry's solution,
optimal or not: the caller decides what a still-failing fallback
means.
Raises ValueError on an unknown option name or a rejected
option value; any options already changed are restored first.
Solution ¶
Solution(*, optimal: bool, obj: float, col_value: ndarray, row_dual: ndarray, col_names: list[str], row_names: list[str], vars: dict[str, Var], col_dual: ndarray | None = None, highs: Highs | None = None, streamed_lp_ranges: dict | None = None)
Read-only view of the solved LP. Look up variable values by
name; values come back as a polars frame (*dims, value).
max_primal_infeasibility
property
¶
Largest primal-constraint violation in the returned solution.
A solver returns a vertex that sits within its feasibility tolerance
of the active constraints, so a hand-rolled feasibility re-check on the
solution (f <= cap, balance rows, …) must use a tolerance at least
this large — a hard-coded magic constant either masks real violations
or trips on the normal solver slack. HiGHS enforces feasibility on the
INTERNALLY-SCALED problem, so the unscaled slack reported here can
exceed :attr:primal_feasibility_tolerance.
0.0 when no live solver is attached (a synthesised Solution).
primal_feasibility_tolerance
property
¶
The solver's primal feasibility tolerance for this solve (the
nominal, scaled-problem tolerance; see
:attr:max_primal_infeasibility for the achieved unscaled slack).
0.0 when no live solver is attached (a synthesised Solution).
solve_diagnostics ¶
Snapshot the solve's HiGHS status + feasibility/optimality counters.
Returns a :class:SolveDiagnostics read from the live solver, or
None when no queryable HiGHS handle is attached — i.e. a
synthesised :class:Solution (highs is None) OR a read-only
shim that reconstructs a solution from a solver .sol file
(the subprocess / commercial path) and does not implement
getInfo / getModelStatus. None means "cannot
diagnose", which callers must treat as un-verifiable — never as
an implicit pass.
Follows the highs is None → neutral guard of
:attr:max_primal_infeasibility, but extends it to the shim case
via hasattr so no AttributeError can leak from the
subprocess path.
value_wide ¶
value_wide(var_name: str, time_dims: tuple[str, ...] = ('d', 't'), solve_name: str | None = None) -> pl.DataFrame
Wide-form, flextool-compatible: time dims become rows, the remaining dims are encoded as a tuple-stringified column header.
For a 2-d variable like vq_state_up(n, d, t):
long : rows = (n, d, t, value)
wide : rows = (d, t) + one column per n (header = "west").
For a 5-d variable like v_flow(p, source, sink, d, t):
wide : rows = (d, t) + one column per (p, source, sink),
header = "('coal_plant', 'coal_market', 'west')"
to match flextool's MultiIndex parquet round-trip.
If solve_name is given, prepend a constant solve column
for fuller flextool-output compatibility.
constraint_dual ¶
Per-row dual values for a named constraint. Returns a frame
(over_dims..., dual) if the constraint had over= rows,
else a single-row scalar frame (dual,).
get_named_basis ¶
Extract the optimal basis as a name-keyed carrier (spec §4.1, §4.5).
Zips the live HiGHS basis (getBasis().col_status /
row_status) against this solution's rendered col_names /
row_names, storing each status as its integer enum value so
the carrier is portable across processes.
Synthetic row_<i> names are EXCLUDED from row_status — they
are positional, not stable across models (§4.5) — and from the
fingerprint, consistent with what the set-side computes.
Requires the solve to have retained the live HiGHS handle
(keep_solver=True, or a :class:WarmProblem solve); raises
:class:RuntimeError otherwise. Raises :class:ValueError if a
column name — or a non-synthetic row name — repeats (a duplicate
would silently overwrite its status in the dict).
CstrRecord ¶
Read-only metadata for a registered constraint family.
Returned by :meth:Problem.cstrs_named for emission-introspection
tests. proto carries the LHS Expr, sense and rhs
structures; most callers only need over (whose height is the
row count) and name.
Sum ¶
Aggregate an Expr. over lists the dims to sum out; the
remaining dims become the term's open dims. where is an index
frame that pre-filters the term frames (inner join on shared
columns) before the group-by-sum.
Sum(expr) with over=None collapses every open dim — useful
for a scalar (objective term, single-row constraint).
Any deferred :func:Where filters recorded on each term
(t.where_frames) are baked into t.lazy before the
aggregation — Sum collapses dims so the filter could not be
applied at the leaf level downstream. The result clears
var_source / where_frames (today's behaviour); the
per-term where kwarg is applied on top, unchanged.
Where ¶
Inner-join an Expr against frame. Two effects in one op:
- Filter — rows of the term whose shared-column values don't
appear in
frameare dropped (e.g.Where(v_flow, wind_only)keeps only the wind rows). - Map — any columns of
framethat the term doesn't already carry become new open dims of the resulting term (e.g.Where(v_flow, flow_to_n)whereflow_to_nhas columns(p, source, sink, n)addsnso the term can be bound to a constraint indexed by(n, t)).
The pure-filter case (no extras) does not bake the join into
t.lazy — it records frame into _Term.where_frames so
the LHS prune-down (_build_lhs_pruned_plan) can apply it at the
leaf-rebuild step (where the row count is bounded by the smaller of
Var.frame and the row_index key set). var_source / Param
chain / coef_scalar are preserved.
The map-effect case (extras non-empty) ALSO defers: frame is
recorded into _Term.where_map_frames as (frame_lf,
frozenset(extras)), the term's dims are extended with the extras,
and var_source / Param chain / coef_scalar / where_frames
are preserved. The deferred inner-join (which produces the extras
dim columns) is baked at leaf-rebuild time (or at Sum / Lag /
consumer fallback) via :func:_apply_where_map_frames. Param
multiplies AFTER the map-effect Where bake first iff their dims
overlap any pending extras (correctness); otherwise the deferral
propagates through.
Set POLAR_HIGH_DISABLE_WHERE_PUSHDOWN=1 to recover today's
behaviour (eager join + clear metadata) verbatim for BOTH branches.
Lag ¶
Return an Expr that, for each (carry_dims, time_dim) in
lag_frame, references var at (carry_dims, lag_col).
Used for shifting variables in time, e.g. for storage state-change:
v_state[n, d, t] - v_state[n, d, t_prev]
= v_state - Lag(v_state, dtttdt, "t", "t_prev_within_timeset")
lag_frame carries the (d, t, t_prev) lookup; carry_dims are
the columns shared between var and lag_frame other than the
time dim itself (typically d).
prewarm_global_scheduler ¶
Initialize HiGHS' process-global task scheduler ONCE, single-threaded,
so subsequent concurrent solves on distinct Highs instances need not
each call resetGlobalScheduler. Best-effort; returns False if any
highspy step fails (the caller then falls back to a sequential path).
Once this returns True the global scheduler is pinned to threads and
a subsequent run() with NO threads option inherits that pool — so
concurrent solves need not (and must not) pass threads per instance,
which would re-trigger resetGlobalScheduler and is unsafe to run
concurrently.
resolve_worker_count ¶
Clamp a requested worker count to [1, n_items].
workers is None auto-resolves to min(n_items, cpu_count - 1) (at
least 1) — leave one core for the main thread / OS. A non-positive request
means sequential (1).
solve_indexed_parallel ¶
solve_indexed_parallel(warmproblems: Sequence[WarmProblem], fn: Callable[[int], T], *, workers: int | None = None) -> list[T]
Run fn(i) for every index i of warmproblems deterministically,
optionally across a thread pool, and return the results in index order.
fn(i) is the caller's per-subproblem work: typically it pins the i-th
:class:WarmProblem's coupling columns, calls wp.solve(), and extracts
a domain result (objective + dual slopes). Because each WarmProblem owns
its own HiGHS handle, fn(i) for distinct i touch disjoint state and
are safe to run concurrently — provided every WarmProblem is already built
(the cold first build calls the process-global resetGlobalScheduler and
must run sequentially up front). This helper enforces that precondition.
Parameters¶
warmproblems
The per-index :class:WarmProblems. Each MUST already be built
(solve() called at least once); a ValueError is raised
otherwise. Only used for the built-precondition check + count — the
actual solving is delegated to fn.
fn
fn(i) -> result for index i. May run on a worker thread; must
confine its mutations to the i-th WarmProblem's HiGHS handle.
workers
Effective worker count (see :func:resolve_worker_count). None
auto-resolves to min(n, cpu_count - 1). <= 1 runs a fully
sequential path on the calling thread (no pool, no scheduler prewarm) —
byte-identical to a plain for loop.
Returns¶
list
[fn(0), fn(1), ..., fn(n-1)] — always in index order, independent of
thread timing.
Modules¶
polar_high.engine ¶
Generic polars-backed LP kernel.
Three primitives — Var, Param, Sum — and one container
(Problem). Knows nothing about flextool, energy systems, or any
specific model. A constraint is built as either:
-
an
Exprproduced by overloaded operators (v <= cap,Sum(...) >= rhs,lhs.eq(rhs)), passed positionally toProblem.add_cstr; or -
a labelled
termsdict, summed across all entries, with an explicitsenseandrhs. Use this when a constraint is naturally a sum of named contributions (storage transitions, sink flow, source flow, slack — like flextool's nodeBalance_eq).
A variable is a polars frame (*dims, col_id) — one LP column per
row. A parameter is a polars frame (*dims, value). Var * Param
joins on shared dims and emits an Expr term (*union_dims, col_id,
coef). Sum(expr, over=…) group-by-sums one or more dims; the
remaining dims become the constraint's row dims when the term is bound
to over= at add_cstr time.
Param ¶
Param(dims: tuple[str, ...], frame: DataFrame | LazyFrame, name: str | None = None, _sources: list[tuple[Param, int]] | None = None, _value_scalar: float = 1.0)
A parameter table. frame carries columns *dims, value.
Stored internally as a polars.LazyFrame so that chained algebra
ops (Param * Param, Param + Param etc.) defer materialization
until a consumer reads .frame or the engine collects in
Problem.solve. The .frame property caches the eager
DataFrame on first read — flextool reads .frame.rename(...)
repeatedly off the same Param so we want that to be cheap.
name (optional) is a logical Param identifier (e.g. "p_inflow").
It is opt-in metadata used by :class:WarmProblem's Param-tracked
auto-update (declare_mutable / update_param). When unset,
Params are anonymous and carry no tracking overhead.
_sources records constituent named Params for composite results
of Param * Param / Param / Param. Each entry is
(param_name, dims_tuple, direction) where direction is +1 if the
Param contributes to the numerator and -1 if to the denominator.
Anonymous-only chains have _sources is None.
Var ¶
Var(name: str, dims: tuple[str, ...], frame: DataFrame, lower: float = 0.0, upper: float = float('inf'), integer: bool = False)
A variable family. frame carries columns *dims, col_id.
Var.frame stays an eager polars DataFrame — it's small (one row
per LP column), produced once in :meth:Problem.add_var, and
consumed by both flextool integration (v.frame["col_id"].unique())
and Problem.solve (col_id → bound/name lookups). Algebra ops on
Var lazify on the fly so the resulting _Term is lazy.
SumBlockMeta
dataclass
¶
SumBlockMeta(var_source: Var, param_sources: tuple[tuple[Param, int], ...], coef_scalar: float, where_frames: tuple[LazyFrame, ...] | None, where_map_frames: tuple[tuple[LazyFrame, frozenset[str]], ...] | None, reduce_dims: tuple[str, ...], keep: tuple[str, ...])
Inert reconstruction recipe captured at Sum-time (Phase C-2).
When a block-eligible term (var_source present, non-empty
param_sources list, non-empty over) is reduced by
:func:Sum, the aggregation clears var_source and survivor-
filters param_sources on the returned term, discarding the
information needed to rebuild the pre-Sum row_index → Var → P1 →
P2 … chain and reduce it in-block. :class:SumBlockMeta snapshots
that pre-Sum state so a future block-COO classifier (Phase C-3) can
evaluate the Sum-wrapped chain by rebuilding from leaves and reducing
within the block.
All fields are captured from the PRE-Sum term, before any clearing or survivor-filtering:
var_source— the originating :class:Var.param_sources— the FULL pre-Sumlist[(Param, direction)], NOT survivor-filtered: block-COO needs every factor, including Params whose dims are summed out (e.g.p_unitsizeoverp).coef_scalar— the cumulative constant scalar folded into coef.where_frames— the pre-Sum deferred pure-filter frames.where_map_frames— the pre-Sum deferred map-effect frames.reduce_dims— the dims summed out (the Sum'sover).keep— the post-Sum open dims (the returned term's dims).
This class is currently INERT: nothing reads it (confirmed by grep).
It is set ONLY at the point of reduction in :func:Sum, never
propagated through later :class:Expr ops or a nested / re-reduced
:func:Sum (those set it to None), so a stale recipe can never
attach to an already-reduced term.
Expr ¶
A sum of terms (decision-variable contributions).
The terms can have different open-dim sets — they're concatenated,
not broadcast. Broadcasting happens once, at constraint emission,
via a join to the constraint's over= row index.
CstrRecord ¶
Read-only metadata for a registered constraint family.
Returned by :meth:Problem.cstrs_named for emission-introspection
tests. proto carries the LHS Expr, sense and rhs
structures; most callers only need over (whose height is the
row count) and name.
Problem ¶
LP container. Generic — no flextool-specific knowledge.
Construct an empty LP container.
Pure polar-high is a generic LP kernel; scaling decisions are
left to the caller. See :mod:polar_high.autoscale for the
opt-in autoscaler (Layer 1 detect + Layer 3 recommendation)
that callers (e.g. FlexTool) use to drive
user_bound_scale / user_objective_scale automatically.
dense_axes — the explicit client contract for the block-COO
LHS arm. When the client (e.g. FlexTool) declares the dense
trailing axes once here (e.g. Problem(dense_axes=("d", "t"))),
it makes a binding PROMISE about every frame it passes that
contains those columns:
the frame is globally lexicographically sorted by
``(other_dims_in_declared_order..., *dense_axes)`` — i.e. the
declared dense axes are the trailing sort keys, in the given
order, and the leading dims form a sorted prefix.
This lets block-COO slice the dense suffix of each Var with NO
re-sort (a re-sort would cost more than the multiply itself).
polar-high VERIFIES this promise cheaply (a single-pass monotonic
scan — see :func:_verify_dense_sorted) on every Var that the
block-COO arm classifies + fires on, and RAISES a clear
ValueError naming the Var if the client breaks it. Frames
that do not contain the dense axes (e.g. an investment Var
("p", "d") when dense_axes=("d", "t")) simply do not fire
block-COO and are unaffected. None (default) leaves the
block-COO arm dormant — it only fires once dense axes are declared
(here or via :meth:declare_dense_axes).
declare_dense_axes ¶
Declare the dense trailing axes for block-COO (see init contract).
Equivalent to passing dense_axes= to the constructor; provided so
callers that receive an already-constructed Problem (e.g. FlexTool's
build_flextool step, which builds the Problem first and populates
it afterwards) can declare them. Pass None to clear.
set_solver_options ¶
Store HiGHS options to be applied in solve(). Pass None
to clear. Keys are HiGHS canonical option names (presolve,
solver, parallel, time_limit etc); values must be
already coerced to the type HiGHS expects (str/int/float/bool).
Unknown keys are tolerated (a warning is emitted at solve time).
set_solver_option ¶
Set a single HiGHS option, leaving the rest untouched.
Convenience for callers that want to add one knob without
re-passing the whole dict (e.g. user_bound_scale set by
the autoscaler). Equivalent to a dict merge plus
:meth:set_solver_options.
get_solver_option ¶
Return the caller-set value of name (or None if unset).
Reads self._solver_options only — does NOT consult HiGHS
(the option may not have been pushed to a live Highs
instance yet). Returns None for unset options so callers
can use if get_solver_option(...) is not None to test
explicit setting.
add_cstr ¶
add_cstr(name: str, *, over: DataFrame | None = None, sense: str, lhs_terms: dict[str, Var | Expr | Param | int | float], rhs_terms: dict[str, Var | Expr | Param | int | float] | None = None) -> None
Add a constraint of the form Σ lhs_terms sense Σ rhs_terms.
Each term entry is either
- a
VarorExpr— variable contribution, or - a
Param,intorfloat— constant contribution.
The engine sorts variables and constants out per side, builds
(lhs_var − rhs_var) sense (rhs_const − lhs_const), and adds
the row to highspy at solve time. Labels (the dict keys) are
used in row names and diagnostics.
cstr_names ¶
All constraint family names currently registered, in declaration order. Useful for emission audits and debugging.
cstrs_named ¶
Return constraint metadata records matching name.
An exact-name match returns the single record; otherwise a prefix
match returns every record whose name starts with name + "_"
(so passing "minimum_uptime" returns both
minimum_uptime_linear and minimum_uptime_integer).
Each :class:CstrRecord carries:
* name: full registered name of the constraint family;
* over: the polars DataFrame of axis tuples (len(over)
is the row count);
* proto: the underlying _CstrProto (expr, sense,
rhs) for advanced introspection.
set_named_basis ¶
Record a warm-start basis to inject at the next solve (spec §4.1).
This is the in-process (non-save_memory) primitive: it only
RECORDS INTENT — it stores the carrier on self._warm_basis and
the policy on self._warm_basis_policy. It does NOT build or
inject anything. The solve-time hook (streaming path, just before
h.run()) constructs the sized HiGHS basis via
:func:polar_high._warm_basis.build_highs_basis and calls
h.setBasis.
policy="exact" transfers 1:1 by name and requires the carrier's
fingerprint to match this model's; a mismatch under "exact" is
expected to be caught by the hook, which then falls back safely to
a cold solve. policy="alien" maps shared names and lets HiGHS
repair the remainder.
basis_name_fingerprint ¶
Structural fingerprint of THIS model's LP name-set (spec §4.5).
A cheap, side-effect-free accessor the orchestrator uses as the
cache key for a captured :class:NamedBasis: it is the
:func:polar_high._warm_basis.basis_fingerprint of this model's
rendered column + constraint-row names, so a basis captured off one
build and re-injected into an identically-shaped build key on the
same string. It mirrors what :meth:write_mps stashes as
_last_mps_fingerprint (same basis_fingerprint(col_names,
m.row_names) inputs) but is callable without writing an MPS.
Uses the canonical matrix's names (m = self.canonicalise());
canonicalise is memoized, and LP names are Layer-2-invariant, so
the value is valid pre- or post-scaling and stable across repeated
calls. Returns None when the problem has been released or has no
columns (no meaningful key).
seed_primal ¶
seed_primal(values: dict[str, float], *, frame: Literal['user', 'scaled'], missing: str = 'skip') -> None
Record a partial primal seed to inject at the next solve.
Phase 3 Step 3a primitive. Like :meth:set_named_basis this only
RECORDS INTENT — it stores the seed on self._primal_seed and the
frame on self._primal_seed_frame. It does NOT build or inject
anything: the solve-time hook (streaming path, just before
h.run(), immediately after the basis block) resolves each name to
a column index, converts + sanitises the value, and calls
h.setSolution.
values maps column name (rendered exactly as
Solution.col_names — "name[d0,d1,...]" for indexed vars, the
bare name for scalars) to a numeric value. A partial seed is
legal and expected: names present in the model are seeded, names
absent are silently skipped (the subset behaviour, missing="skip").
frame is REQUIRED and has NO default (a silent default frame is a
corruption vector). frame="scaled" uses each value as-is in the
solver's post-Layer-2 scaled space. frame="user" converts with
the model's own Layer-2 side-vector
(x_scaled = x_user / _layer2_col_factor[col_id]; identity when
Layer 2 was not applied), so callers can seed in natural units.
Feasibility-safety: a primal seed is only a starting point for the solver — it never binds the optimum. A non-optimal (or even infeasible-looking) seed cannot change the objective HiGHS returns, with presolve on or off.
MUTUALLY EXCLUSIVE with a warm basis. setSolution and
setBasis clobber each other on the same handle (last writer wins;
setSolution invalidates a previously-set basis). For an LP a
basis dominates a partial seed, so the hook enforces the rule "if a
basis is set, it wins": when self._warm_basis is also set the seed
is skipped.
In-process (streaming) ONLY — there is no subprocess / MPS channel
for a primal seed; a save_memory solve ignores the seed.
A seed over integer columns is a MIP-start. (Noted for a future UC5; not implemented or specialised here — integer columns are seeded the same way and HiGHS treats them as a MIP-start.)
cstr_row_count ¶
Total LP-row count across all constraint families matching
name (exact or prefix; see :meth:cstrs_named). Returns 0
when no families match — letting callers distinguish "absent"
from "empty" without exception handling. A scalar constraint
(over=None) counts as one row.
add_obj_constant ¶
Accumulate a constant into the objective offset. HiGHS adds
this to the reported getObjectiveValue() after solve, so it
shows up in Solution.obj even though no decision variable
carries it. Used for pure-Param objective terms like the §8.1
existing-entity fixed cost.
solve ¶
solve(*, options: dict | None = None, keep_solver: bool = False, streaming: bool = True, save_memory: bool = False, tmp_dir: str | PathLike | None = None) -> Solution
Solve the LP and return a :class:Solution.
Parameters¶
options
Per-call HiGHS options dict (overrides set_solver_options).
keep_solver
When True, the live HiGHS instance is kept on the returned
:class:Solution so callers can inspect it post-solve (e.g.
sol.highs.writeModel("model.mps")). Default False —
the C-side LP storage is released as soon as primal/dual/
objective have been extracted.
streaming
When True (default), columns are added once via addCols
and each constraint family is emitted to HiGHS via addRows
immediately after its COO triples are built; the family's local
arrays then go out of scope before the next family is processed.
This caps peak memory at one family's COO + the running HiGHS
LP. When False, the entire model is assembled into a single
:class:highspy.HighsLp and loaded via passModel —
numerically identical results either way; False is mostly
useful for benchmarking the legacy path.
save_memory
Single one-shot knob that trades wall time for peak RSS. When
True, two things happen right before HiGHS.run():
1. polar-high's polars/numpy LP source-of-truth (constraint
family ``Expr.terms`` lists, ``_CstrProto.rhs`` Param
frames, objective ``_Term.lazy`` plans, the caller-side
``col_names`` / ``row_names`` lists) is dropped. Only the
per-:class:`Var` ``col_id`` frames survive — they are
needed by :meth:`Solution.value` to map column indices
back to user-space dim tuples.
2. The HiGHS instance is round-tripped through disk: the LP
is written to a temp MPS file, the original ``Highs()`` is
cleared and discarded, ``malloc_trim(0)`` is called (best
effort, Linux only), a fresh ``Highs()`` is created, the
same solver options are re-applied, and the MPS file is
read back. This resets the HiGHS allocator's high-water
mark — the C++ side accumulates ~5 GB of slack from the
incremental ``addRows`` loading path that ``readModel``
avoids by sizing once up front.
Cost: ~+90 s of MPS file I/O at N=3000 dense. Benefit: peak
RSS drops by ~5 GB on the same problem. Intended for one-
shot single-solve benchmarks where warm-start / re-solve
isn't needed. After the call returns, the :class:`Problem`
is in a "released" state and any further :meth:`solve` call
raises ``RuntimeError`` (the polar-side source AND the
original HiGHS instance with its basis have both been
discarded, so neither a fresh re-solve nor a WarmProblem-
style update is possible). Honoured only by the streaming
path; on ``streaming=False`` a warning is emitted and the
flag is ignored. Default ``False``.
tmp_dir
Directory for the temporary MPS file written by the
save_memory=True round-trip. None (default) uses
the system temp dir ($TMPDIR / /tmp). Set when the
caller wants the spill file on a specific volume (e.g. the
same filesystem as its workspace, or a per-job scratch
directory) instead of the system default. Ignored when
save_memory=False.
build_only ¶
Build the LP into HiGHS, write to mps_path, release everything.
For callers that want to drive the actual solve out-of-process — typically a subprocess HiGHS reading the MPS file in a clean address space. After this call:
mps_pathexists and contains the LP in MPS format.- polar-side LP source-of-truth has been dropped
(:meth:
_release_python_lp_inputs). - The live :class:
highspy.Highsinstance has been torn down and the glibc allocator trimmed (Linux best-effort). self._releasedis True — further calls to :meth:solvewill raise.
What stays alive: self._vars and each :class:Var.frame,
so the caller can later construct a :class:Solution from
externally-produced col_value / row_dual arrays via
Solution(..., vars=dict(self._vars), ...).
Honoured only on the streaming path; the non-streaming
(passModel) path is not supported here. Raises
RuntimeError if called on an already-released Problem.
Parameters¶
mps_path
Where to write the MPS file. Caller owns the file —
polar-high will not delete it.
options
HiGHS solver options to apply during the build (mainly
presolve / solver / simplex_scale_strategy —
options that affect what writeModel serialises).
None uses :attr:_solver_options.
canonicalise ¶
Build (or return cached) the canonical CSC matrix + metadata.
Idempotent: returns the cached _matrix unless
_canonical_dirty is set (which add_var / add_cstr
flip). Side vectors (_layer2_col_factor /
_layer2_row_factor) are baked into the returned arrays at
build time per orchestrator decision D8.
write_mps ¶
write_mps(path: str | PathLike, *, free_format: bool = True, column_order_strict: bool = True, emit_names: bool = True, release: bool = False, name: str = 'POLAR_HIGH') -> None
Stream the LP/MIP to path as a free-format MPS file.
This is an in-house writer that consumes the polar-side LP
source-of-truth (_vars, _cstrs, _obj_terms) directly
and emits MPS without ever instantiating :class:highspy.Highs
or building an :class:LpView. The motivation is peak-memory:
HiGHS' own writeModel serialiser materialises ~20× more
transient state than the in-process LP itself, which OOMs on
very large LPs (~10 M rows × 5 M cols × 20 M nonzeros). This
writer targets ~2-3 GB peak on that same LP.
Parameters¶
path
Output MPS file path. Caller owns the file; this writer
does not delete on failure.
free_format
Reserved for future use. Currently always free-format MPS
(the modern default — HiGHS, Gurobi, CPLEX, Xpress all
accept it). Fixed 8-char-column MPS is not yet implemented.
column_order_strict
When True (default), all nonzeros for a single column
are emitted contiguously in the COLUMNS section, sorted by
(col_id, row_id). This is the spec-compliant ordering
every MPS reader accepts. False is currently not
implemented — see :ref:Open follow-ups in the module docs.
emit_names
When True (default), row and column names come from the
variable family name + dim tuple and the constraint family
name + dim tuple (matching the format used by
:meth:Problem.solve for Solution.col_names /
Solution.row_names). When False, generic
C0000001 / R0000001 names are emitted instead —
saves ~30-50% file size at the cost of losing the
dim-tuple → column mapping in the MPS itself. Callers
using emit_names=False must use index-based solution
readback (column 0 in the MPS is column 0 in the polars
Var.frame ordering, etc.). The objective row name is
always cost regardless — it's a reserved unique name.
release
When True, calls :meth:_release_python_lp_inputs
after the write completes. Mirrors the
solve(save_memory=True) semantics — drops the polars
LazyFrames, constraint family list, and rhs Param frames
while keeping self._vars (so solutions read back from an
external solver can still be mapped to user-space dim
tuples via the surviving Var.frame["col_id"] columns).
After release the :class:Problem is no longer solvable —
:meth:solve will raise.
name
MPS NAME header; default "POLAR_HIGH".
Raises¶
RuntimeError
If called on a :class:Problem whose source has already
been released (by an earlier solve(save_memory=True),
build_only(...), or write_mps(..., release=True)).
ValueError
If any coefficient is NaN or infinite (silent filtering
would hide model bugs). The error message identifies at
least one offending (col_id, row_id) pair.
OSError
File I/O failures (passed through from open()).
Notes¶
- The MPS format has no portable encoding for an objective
constant offset. If
self._obj_offset != 0.0this method emits a :class:UserWarningand writes the LP without the offset. Solutions from any downstream solver will then be off byself._obj_offset— the caller must add it back manually if needed. - Integer columns are bracketed with
MARKER 'INTORG'/MARKER 'INTEND'in the COLUMNS section, per MPS convention. - The writer is single-pass per family and uses one global polars sort to put the COLUMNS section into column-major order. Streaming-engine fallback to disk activates automatically for triple counts that don't fit in RAM.
- Stashes
self._last_mps_fingerprint— a structural basis-fingerprint (:func:polar_high._warm_basis.basis_fingerprint) over the emitted column/constraint-row name-set — computed before anyrelease.Noneunlessemit_namesis True (generic names are useless as a cross-run key). Consumed by thesave_memorysubprocess warm-start arm to key a basis-cache file.
SolveDiagnostics
dataclass
¶
SolveDiagnostics(model_status: HighsModelStatus, model_status_name: str, primal_feasible: bool, dual_feasible: bool, num_primal_infeasibilities: int, num_dual_infeasibilities: int, max_primal_infeasibility: float, max_relative_primal_infeasibility: float, max_dual_infeasibility: float, primal_dual_objective_error: float, objective_value: float)
Normalised snapshot of a solve's HiGHS status + info counters.
Read once from a live highspy.Highs (a single getInfo() call
plus getModelStatus()), so the numbers are a consistent atomic
view of one solve. Consumers use this to decide whether a solve that
HiGHS could NOT certify as kOptimal (e.g. an interior-point solve
run without crossover, whose post-solve dual objective is slightly
inconsistent) is nonetheless usable in practice — the primal solution
is feasible and the primal--dual objective gap is small.
This is a pure fact carrier: it holds no policy and no thresholds. The accept/reject decision (and any tolerance) lives with the caller.
Solution ¶
Solution(*, optimal: bool, obj: float, col_value: ndarray, row_dual: ndarray, col_names: list[str], row_names: list[str], vars: dict[str, Var], col_dual: ndarray | None = None, highs: Highs | None = None, streamed_lp_ranges: dict | None = None)
Read-only view of the solved LP. Look up variable values by
name; values come back as a polars frame (*dims, value).
max_primal_infeasibility
property
¶
Largest primal-constraint violation in the returned solution.
A solver returns a vertex that sits within its feasibility tolerance
of the active constraints, so a hand-rolled feasibility re-check on the
solution (f <= cap, balance rows, …) must use a tolerance at least
this large — a hard-coded magic constant either masks real violations
or trips on the normal solver slack. HiGHS enforces feasibility on the
INTERNALLY-SCALED problem, so the unscaled slack reported here can
exceed :attr:primal_feasibility_tolerance.
0.0 when no live solver is attached (a synthesised Solution).
primal_feasibility_tolerance
property
¶
The solver's primal feasibility tolerance for this solve (the
nominal, scaled-problem tolerance; see
:attr:max_primal_infeasibility for the achieved unscaled slack).
0.0 when no live solver is attached (a synthesised Solution).
solve_diagnostics ¶
Snapshot the solve's HiGHS status + feasibility/optimality counters.
Returns a :class:SolveDiagnostics read from the live solver, or
None when no queryable HiGHS handle is attached — i.e. a
synthesised :class:Solution (highs is None) OR a read-only
shim that reconstructs a solution from a solver .sol file
(the subprocess / commercial path) and does not implement
getInfo / getModelStatus. None means "cannot
diagnose", which callers must treat as un-verifiable — never as
an implicit pass.
Follows the highs is None → neutral guard of
:attr:max_primal_infeasibility, but extends it to the shim case
via hasattr so no AttributeError can leak from the
subprocess path.
value_wide ¶
value_wide(var_name: str, time_dims: tuple[str, ...] = ('d', 't'), solve_name: str | None = None) -> pl.DataFrame
Wide-form, flextool-compatible: time dims become rows, the remaining dims are encoded as a tuple-stringified column header.
For a 2-d variable like vq_state_up(n, d, t):
long : rows = (n, d, t, value)
wide : rows = (d, t) + one column per n (header = "west").
For a 5-d variable like v_flow(p, source, sink, d, t):
wide : rows = (d, t) + one column per (p, source, sink),
header = "('coal_plant', 'coal_market', 'west')"
to match flextool's MultiIndex parquet round-trip.
If solve_name is given, prepend a constant solve column
for fuller flextool-output compatibility.
constraint_dual ¶
Per-row dual values for a named constraint. Returns a frame
(over_dims..., dual) if the constraint had over= rows,
else a single-row scalar frame (dual,).
get_named_basis ¶
Extract the optimal basis as a name-keyed carrier (spec §4.1, §4.5).
Zips the live HiGHS basis (getBasis().col_status /
row_status) against this solution's rendered col_names /
row_names, storing each status as its integer enum value so
the carrier is portable across processes.
Synthetic row_<i> names are EXCLUDED from row_status — they
are positional, not stable across models (§4.5) — and from the
fingerprint, consistent with what the set-side computes.
Requires the solve to have retained the live HiGHS handle
(keep_solver=True, or a :class:WarmProblem solve); raises
:class:RuntimeError otherwise. Raises :class:ValueError if a
column name — or a non-synthetic row name — repeats (a duplicate
would silently overwrite its status in the dict).
WarmProblem ¶
Warm-update wrapper around a :class:Problem.
Build a :class:Problem as usual (add_var, add_cstr,
set_objective). Wrap with :class:WarmProblem, then alternate
update_* calls and solve() calls — the LP is built ONCE and
only the changed coefficients / RHS values are pushed to HiGHS
between solves.
Typical rolling-horizon usage::
wp = WarmProblem(p)
sol_0 = wp.solve()
for r in range(1, n_rolls):
wp.update_rhs("balance", demand_param_for_roll[r])
wp.update_obj_coef("v_flow", cost_param_for_roll[r])
sol_r = wp.solve()
The update_* calls are O(rows_or_cols_in_family); the solve()
benefits from HiGHS's hot-start (basis is preserved across calls).
problem
property
¶
The underlying :class:Problem.
Useful for diagnostics that need to inspect the un-built LP.
For coefficient ranges, prefer
:attr:Solution.streamed_lp_ranges on a returned
:class:Solution (populated during the streaming solve at zero
extra cost).
set_named_basis ¶
Record a warm-start basis to inject on the fresh build (spec §4.1).
RECORDS INTENT only — stores the carrier on self._warm_basis and
the policy on self._warm_basis_policy; it does NOT build or
inject anything here. The actual h.setBasis fires exactly once,
in :meth:_initial_build, immediately before the first solve's
h.run(). This is the in-process default-path equivalent of the
streaming basis hook in :meth:Problem._solve_streaming.
policy="exact" transfers 1:1 by name and requires the carrier's
fingerprint to match this model's; a mismatch under "exact" is
detected in _initial_build and safely falls back to a cold solve.
policy="alien" maps shared names and lets HiGHS repair the rest.
The inject fires only on the fresh build — _initial_build runs
once per WarmProblem lifetime (solve calls it only while
self._h is None), so a subsequent height-match reuse re-run never
re-injects.
update_rhs ¶
Replace the RHS of constraint family cstr_name with values
drawn from new_param.
new_param may be a :class:Param whose dims match a subset
of the constraint's over= axis (broadcasting to the rest), a
scalar (broadcast to all rows), or a numpy array (positional —
length must equal the family's row count, in the order of the
original over frame).
update_obj_coef ¶
Replace the objective coefficient on every column of var_name.
Assumes the objective contribution from var_name is exactly
coef[*dims] * var[*dims] for some coef Param; this method
OVERWRITES that coefficient via h.changeColsCost. If the
objective also has contributions from this variable through more
complex algebra (e.g. var * unitsize * slope), the update is
still valid as long as new_param carries the full product —
the caller is responsible for collapsing multi-Param products
ahead of the call.
This DOES NOT touch the cost coefficients of other variables.
update_obj_coef_array ¶
Array-form of :meth:update_obj_coef.
dim_tuples is a list of dim-value tuples (one per cell) for
variable var_name; each tuple must have one entry per dim
in the var's declared signature. values is a same-length
numpy array of new objective coefficients.
The columns are resolved positionally: values[k] becomes the
new objective coefficient on the column whose dim-tuple is
dim_tuples[k]. Vectorised — a single changeColsCost
call regardless of cell count.
fix_cols ¶
Fix the listed columns of var_name to the given values.
For each (dim_tuple, value) pair, sets both the column's
lower and upper bound to value (so the LP has no choice
but to set the column at that level). Used by the Lagrangian
primal-recovery step ("fix-and-resolve"). Vectorised — single
changeColsBounds call.
fix_col_ids ¶
Fix columns BY RAW COL ID: set lower = upper = value for each
(col_id, value) pair.
The col-id-level counterpart of :meth:fix_cols — the same
vectorised changeColsBounds(lo=hi=value) write, but skipping
the per-call dim-tuple → col-id join for callers that already
hold the raw ids (resolve once via :meth:col_id_of_var, or use
ids returned by :meth:add_recourse_col). Typical consumer: a
cutting-plane driver temporarily pinning coupling columns.
Bounds are written verbatim into the live model — any caller-side
scaling must be applied to values beforehand.
get_col_bounds ¶
Read the current (lower, upper) bounds of the given raw col
ids from the live model.
Returns two float64 arrays aligned POSITIONALLY with col_ids
(any order, repeats allowed — HiGHS' getCols itself requires
an ordered duplicate-free index set, so the query goes through
the unique sorted set and gathers back to caller order).
Save/restore companion of :meth:set_col_bounds for temporary
pins made with :meth:fix_col_ids::
lo, hi = wp.get_col_bounds(ids) # save
wp.fix_col_ids(ids, values) # temporary pin
... wp.solve() ...
wp.set_col_bounds(ids, lo, hi) # restore
set_col_bounds ¶
Write (lower, upper) bounds onto the given raw col ids.
Restore companion of :meth:get_col_bounds (see there for the
save/pin/restore pattern). Vectorised — single
changeColsBounds call; ±inf entries round-trip unchanged
(HiGHS' bound sentinel IS the IEEE infinity).
update_coef ¶
Update a single (row, col) coefficient in the constraint
matrix. Use :meth:row_id_of_cstr / :meth:col_id_of_var to
resolve indices semantically.
add_cut_row ¶
Append a >= constraint row to the live (already-built) LP and
return the new row's id.
Appends Σ coefs[k] · x[col_ids[k]] >= lower to the live HiGHS
model via addRow. This is a POST-build mutation of the warm
_h handle (the same class of live edit as :meth:update_rhs /
:meth:update_coef); it deliberately bypasses the build-time
Problem.add_cstr DSL lock, which only guards the fixed-size
Layer-2 autoscale side vectors and is irrelevant once the model is
built. The caller is responsible for keeping the master autoscale
OFF (or for pre-scaling coefs) so the appended row lives on the
built columns' scale — there is no auto-scaling for appended rows.
col_ids may mix existing column ids (resolve via
:meth:col_id_of_var) and ids of columns previously appended with
:meth:add_recourse_col. A subsequent :meth:solve warm
re-optimises the grown model; the appended row's dual is then
readable on the returned :class:Solution by row_dual[row_id]
(read BY ROW ID, not via :meth:Solution.constraint_dual, which
only knows named build-time rows).
Bumps the cached _n_rows (the zero-fill fallback size in
:meth:solve) and appends a generated name to _row_names (kept
index-aligned so name-indexed reads stay correct). Returns the
row_id HiGHS assigned (== the pre-append getNumRow()).
add_recourse_col ¶
Append a single free/bounded column (e.g. a Benders recourse
η) to the live LP and return its col id.
Symmetric with :meth:add_cut_row: a POST-build addCol on the
warm _h handle, bumping the cached _n_cols (zero-fill
fallback size) and _col_names. ±np.inf bounds are mapped to
the HiGHS kHighsInf sentinel. The returned id can be referenced
from a later :meth:add_cut_row; its value is read on the
:class:Solution by col_value[col_id] (appended columns are not
in any Problem._vars frame, so :meth:Solution.value cannot see
them — read by id).
compact_cuts ¶
compact_cuts(solution: Solution, *, policy: str = 'slack', trial_col_values=None, tol_rel: float = 1e-07, verify: bool = True) -> dict
Prune retained cut rows to bound the active master LP.
A cutting-plane master (the Benders master) grows one >= cut row
per iteration via :meth:add_cut_row; left unpruned its warm
re-solve time grows super-linearly. Two selection policies are
available, both LB-safe (they only drop cuts that do not support the
certified lower bound) and both belted by the same verify-restore
rollback:
policy="slack" (default):
Drop the cut rows that are strictly slack (positive primal slack,
hence zero dual by complementary slackness) at solution,
keeping the binding ones. Classification is purely by PRIMAL
slack of the stored cut against solution.col_value: for a cut
Σ coefs·x >= lower the slack is Σ coefs·x - lower >= 0; a
cut with slack <= tol is BINDING (kept), one with
slack > tol is strictly slack (dropped). tol is a per-cut
relative tolerance scaled by the magnitude of lower and the
terms.
policy="dominance" (de Matos–Philpott–Finardi / Guigues
Limited-Memory Level-1):
At a DEGENERATE optimum almost every cut can be primal-binding
(slack ≈ 0), so slack-deletion prunes nothing even though many
cuts are redundant TIES at their recourse group's max value.
Dominance selection instead groups the retained cuts by the
recourse variable they bound (the single member of
col_ids ∩ _recourse_cols, registered by
:meth:add_recourse_col) and, over a WINDOW of recent master
trial points trial_col_values (a list of col_value arrays,
most-recent last; defaults to [solution.col_value]), keeps for
each group at each trial point the OLDEST cut that achieves the
group's max value there (within a relative tolerance). Every cut
not a keeper at any trial point is DROPPED — it is dominated (or a
redundant tie) at every trial point, so the outer approximation of
the recourse function at those points, and the LB it certifies,
is unchanged. A cut whose col_ids contains no recourse column
forms its own singleton group and is always kept (generic
non-Benders use degrades safely to "keep all").
When verify (default), after deleting rows the master is re-solved
and its objective compared to the incumbent solution.obj. A drift
larger than :data:_COMPACT_VERIFY_TOL (relative) means a dropped cut
was in fact supporting the optimum (a degenerate numerical edge) — the
deletion is ROLLED BACK by re-appending every dropped cut verbatim and
re-solving, and the returned dict reports restored=True with
dropped=0. On a well-behaved master the belt never fires.
Returns {"kept": int, "dropped": int, "restored": bool} — cut
counts AFTER the operation (kept == number of retained cut rows
that remain, dropped == number removed; on a restore kept is
the pre-compaction count and dropped is 0).
declare_mutable ¶
Declare a set of :class:Param names whose values should be
tracked into LP cells, so :meth:update_param can later push
new values into the live HiGHS instance via changeCoeff.
MUST be called BEFORE the first :meth:solve. Tracking is
opt-in: Params not declared here pay no bookkeeping cost.
Pass the same names that the Params carry on their .name
field — typically the FlexData attribute name ("p_inflow",
"p_penalty_up" etc.).
set_output_flag ¶
Enable or disable HiGHS' native solve log for this problem.
enabled=False mutes the per-solve HiGHS output (the
output_flag HiGHS option) for THIS WarmProblem across all of
its cold / warm / retry solves — callers that drive many parallel
sub-solves (Benders regions, Lagrangian subproblems) use this to
keep stdout clean and emit their own concise progress log instead.
May be called either before the first :meth:solve (the flag is
applied when the HiGHS handle is built) or after it (applied to the
live handle immediately). The preference persists on the handle,
so a single call suffices for the whole lifetime.
update_param ¶
Replace the values of a tracked Param. Every LP cell whose
coefficient was originally a function of param_name is
re-computed from the new Param's values and pushed via
h.changeCoeff.
new_param must be either a scalar (broadcast to all tracked
cells) or a :class:Param whose dim signature matches the
signature recorded for that Param at build time.
Raises if param_name was not in :meth:declare_mutable's
list (silent corruption is worse than a hard error).
col_id_of_var ¶
Return the col_id(s) for a variable.
dims=None returns every col_id in the variable's family
(numpy array, ordered by the var's declaration order).
dims as a tuple of dim values returns the single col_id for
that one cell (a python int). dims as a dict
{dim_name: value} is a partial filter — returns a numpy
array of the matching col_ids.
row_id_of_cstr ¶
Return the row_id(s) for a constraint family. Mirrors
:meth:col_id_of_var.
solve ¶
Solve the LP. First call builds the LP from scratch (same
pipeline as :meth:Problem.solve); subsequent calls just run
HiGHS again on the (possibly updated) live model.
options is honoured on the FIRST solve only — subsequent
solves use the same HiGHS instance. To change options on a
rebuilt LP, drop the WarmProblem and create a new one.
retry_on_unknown (default False → byte-identical to the
legacy path for every existing caller) enables a warm-restart
retry: the solver runs WARM first (retaining the basis so dual
simplex hot-starts after rows appended via :meth:add_cut_row);
only if the warm run does NOT return a certified kOptimal —
i.e. a stale-basis transient (kUnknown / kSolveError) or a
spurious kUnbounded / kInfeasible / primal-infeasible miss — do we
drop the basis with clearSolver() and re-run ONCE (the proven
cold fallback) to re-certify the true status. Used by the Benders
master, which appends a cut each iteration; on a well-scaled
master the warm path stays kOptimal and the fallback never
fires.
solve_with_fallback ¶
Solve; on a non-optimal primary, retry ONCE under
fallback_options and restore the prior option values.
Runs :meth:solve first. If it certifies kOptimal the
solution is returned as (solution, "primary") and the options
are never touched. Otherwise each fallback_options entry is
applied to the live HiGHS handle via setOptionValue
(solve(options=...) is honoured on the FIRST solve only — a
live-handle write is the only way to change options on a built
model), the retained solver state is dropped with
clearSolver() (so options like solver / presolve take
effect from cold, and no stale basis biases the retry), the model
is re-run once, and the prior option values are RESTORED — a
later plain :meth:solve sees the original configuration.
Returns (solution, "fallback") with the retry's solution,
optimal or not: the caller decides what a still-failing fallback
means.
Raises ValueError on an unknown option name or a rejected
option value; any options already changed are restored first.
Lag ¶
Return an Expr that, for each (carry_dims, time_dim) in
lag_frame, references var at (carry_dims, lag_col).
Used for shifting variables in time, e.g. for storage state-change:
v_state[n, d, t] - v_state[n, d, t_prev]
= v_state - Lag(v_state, dtttdt, "t", "t_prev_within_timeset")
lag_frame carries the (d, t, t_prev) lookup; carry_dims are
the columns shared between var and lag_frame other than the
time dim itself (typically d).
Where ¶
Inner-join an Expr against frame. Two effects in one op:
- Filter — rows of the term whose shared-column values don't
appear in
frameare dropped (e.g.Where(v_flow, wind_only)keeps only the wind rows). - Map — any columns of
framethat the term doesn't already carry become new open dims of the resulting term (e.g.Where(v_flow, flow_to_n)whereflow_to_nhas columns(p, source, sink, n)addsnso the term can be bound to a constraint indexed by(n, t)).
The pure-filter case (no extras) does not bake the join into
t.lazy — it records frame into _Term.where_frames so
the LHS prune-down (_build_lhs_pruned_plan) can apply it at the
leaf-rebuild step (where the row count is bounded by the smaller of
Var.frame and the row_index key set). var_source / Param
chain / coef_scalar are preserved.
The map-effect case (extras non-empty) ALSO defers: frame is
recorded into _Term.where_map_frames as (frame_lf,
frozenset(extras)), the term's dims are extended with the extras,
and var_source / Param chain / coef_scalar / where_frames
are preserved. The deferred inner-join (which produces the extras
dim columns) is baked at leaf-rebuild time (or at Sum / Lag /
consumer fallback) via :func:_apply_where_map_frames. Param
multiplies AFTER the map-effect Where bake first iff their dims
overlap any pending extras (correctness); otherwise the deferral
propagates through.
Set POLAR_HIGH_DISABLE_WHERE_PUSHDOWN=1 to recover today's
behaviour (eager join + clear metadata) verbatim for BOTH branches.
Sum ¶
Aggregate an Expr. over lists the dims to sum out; the
remaining dims become the term's open dims. where is an index
frame that pre-filters the term frames (inner join on shared
columns) before the group-by-sum.
Sum(expr) with over=None collapses every open dim — useful
for a scalar (objective term, single-row constraint).
Any deferred :func:Where filters recorded on each term
(t.where_frames) are baked into t.lazy before the
aggregation — Sum collapses dims so the filter could not be
applied at the leaf level downstream. The result clears
var_source / where_frames (today's behaviour); the
per-term where kwarg is applied on top, unchanged.
polar_high.parallel ¶
Deterministic parallel solving of independent :class:WarmProblems.
A small, domain-agnostic utility for cutting-plane / decomposition drivers
(e.g. Benders region recourse, Lagrangian subproblems) that need to solve N
independent subproblems each outer iteration. Each subproblem is its own
:class:~polar_high.engine.WarmProblem (its own HiGHS handle), and HiGHS'
run() releases the GIL, so a thread pool yields a real wall-clock speedup
over the sequential loop.
The thread-safety contract this module encapsulates (so the caller's algorithm code never touches the HiGHS-threading detail):
-
Single-threaded HiGHS scheduler. Every subproblem solve runs HiGHS with a single-threaded scheduler. This is required for (a) determinism — HiGHS is non-deterministic with
threads > 1— and (b) to avoidworkers × coresoversubscription. The process-global HiGHS scheduler is pinned to one thread ONCE up front via :func:prewarm_global_scheduler; the per-subproblemrun()calls then inherit that pinned pool. -
Sequential cold first build. The FIRST
WarmProblem.solve()builds the HiGHS model and, if it sees athreads/paralleloption, calls the process-globalresetGlobalScheduler— unsafe to run concurrently. This module therefore only parallelizes solves over WarmProblems that are ALREADY built (wp._h is not None); it raises if asked to fan out an unbuilt one. Callers must do the cold first build sequentially (or rely on the scheduler pre-pin) before calling :func:solve_indexed_parallel. -
Deterministic per-index collection. Results are collected into per-index slots and returned in index order, so the outcome is identical regardless of thread timing. Worker exceptions are re-raised in index order (the lowest failing index wins, matching the sequential loop).
workers <= 1 keeps a fully sequential path (no pool, no prewarm) that is
byte-identical to a plain for loop — so the parallel and sequential code
paths can be exercised against each other for a determinism gate.
resolve_worker_count ¶
Clamp a requested worker count to [1, n_items].
workers is None auto-resolves to min(n_items, cpu_count - 1) (at
least 1) — leave one core for the main thread / OS. A non-positive request
means sequential (1).
prewarm_global_scheduler ¶
Initialize HiGHS' process-global task scheduler ONCE, single-threaded,
so subsequent concurrent solves on distinct Highs instances need not
each call resetGlobalScheduler. Best-effort; returns False if any
highspy step fails (the caller then falls back to a sequential path).
Once this returns True the global scheduler is pinned to threads and
a subsequent run() with NO threads option inherits that pool — so
concurrent solves need not (and must not) pass threads per instance,
which would re-trigger resetGlobalScheduler and is unsafe to run
concurrently.
solve_indexed_parallel ¶
solve_indexed_parallel(warmproblems: Sequence[WarmProblem], fn: Callable[[int], T], *, workers: int | None = None) -> list[T]
Run fn(i) for every index i of warmproblems deterministically,
optionally across a thread pool, and return the results in index order.
fn(i) is the caller's per-subproblem work: typically it pins the i-th
:class:WarmProblem's coupling columns, calls wp.solve(), and extracts
a domain result (objective + dual slopes). Because each WarmProblem owns
its own HiGHS handle, fn(i) for distinct i touch disjoint state and
are safe to run concurrently — provided every WarmProblem is already built
(the cold first build calls the process-global resetGlobalScheduler and
must run sequentially up front). This helper enforces that precondition.
Parameters¶
warmproblems
The per-index :class:WarmProblems. Each MUST already be built
(solve() called at least once); a ValueError is raised
otherwise. Only used for the built-precondition check + count — the
actual solving is delegated to fn.
fn
fn(i) -> result for index i. May run on a worker thread; must
confine its mutations to the i-th WarmProblem's HiGHS handle.
workers
Effective worker count (see :func:resolve_worker_count). None
auto-resolves to min(n, cpu_count - 1). <= 1 runs a fully
sequential path on the calling thread (no pool, no scheduler prewarm) —
byte-identical to a plain for loop.
Returns¶
list
[fn(0), fn(1), ..., fn(n-1)] — always in index order, independent of
thread timing.