API reference¶
The full public surface, autogenerated from docstrings.
Top-level¶
polar-high — Python library for building indexed linear and mixed-integer programs in polars.
Problem ¶
LP container. Generic — no flextool-specific knowledge.
Construct an empty LP container.
Pure polar-high is a generic LP kernel; scaling decisions are
left to the caller. See :mod:polar_high.autoscale for the
opt-in autoscaler (Layer 1 detect + Layer 3 recommendation)
that callers (e.g. FlexTool) use to drive
user_bound_scale / user_objective_scale automatically.
dense_axes — the explicit client contract for the block-COO
LHS arm. When the client (e.g. FlexTool) declares the dense
trailing axes once here (e.g. Problem(dense_axes=("d", "t"))),
it makes a binding PROMISE about every frame it passes that
contains those columns:
the frame is globally lexicographically sorted by
``(other_dims_in_declared_order..., *dense_axes)`` — i.e. the
declared dense axes are the trailing sort keys, in the given
order, and the leading dims form a sorted prefix.
This lets block-COO slice the dense suffix of each Var with NO
re-sort (a re-sort would cost more than the multiply itself).
polar-high VERIFIES this promise cheaply (a single-pass monotonic
scan — see :func:_verify_dense_sorted) on every Var that the
block-COO arm classifies + fires on, and RAISES a clear
ValueError naming the Var if the client breaks it. Frames
that do not contain the dense axes (e.g. an investment Var
("p", "d") when dense_axes=("d", "t")) simply do not fire
block-COO and are unaffected. None (default) leaves the
block-COO arm dormant — it only fires once dense axes are declared
(here or via :meth:declare_dense_axes).
objective_scale
property
¶
The factor set by :meth:set_objective_scale (1.0 by default).
declare_dense_axes ¶
Declare the dense trailing axes for block-COO (see init contract).
Equivalent to passing dense_axes= to the constructor; provided so
callers that receive an already-constructed Problem (e.g. FlexTool's
build_flextool step, which builds the Problem first and populates
it afterwards) can declare them. Pass None to clear.
set_solver_options ¶
Store HiGHS options to be applied in solve(). Pass None
to clear. Keys are HiGHS canonical option names (presolve,
solver, parallel, time_limit etc); values must be
already coerced to the type HiGHS expects (str/int/float/bool).
Unknown keys are tolerated (a warning is emitted at solve time).
set_solver_option ¶
Set a single HiGHS option, leaving the rest untouched.
Convenience for callers that want to add one knob without
re-passing the whole dict (e.g. user_bound_scale set by
the autoscaler). Equivalent to a dict merge plus
:meth:set_solver_options.
get_solver_option ¶
Return the caller-set value of name (or None if unset).
Reads self._solver_options only — does NOT consult HiGHS
(the option may not have been pushed to a live Highs
instance yet). Returns None for unset options so callers
can use if get_solver_option(...) is not None to test
explicit setting.
add_var ¶
add_var(name: str, dims: tuple[str, ...] | str, index: DataFrame, lower: float | Param = 0.0, upper: float | Param = float('inf'), integer: bool = False) -> Var
Declare a variable family with one column per index row.
lower / upper are either scalars (one bound for every
column — the historical form, emitted byte-identically) or a
:class:Param giving per-element bounds:
- The Param's dims must be a SUBSET of
dims(ValueErrorotherwise). Its rows are aligned to the variable's index rows by joining on the Param's dims, so a lower-dim Param broadcasts over the remaining variable dims. - Variable elements with NO matching Param row get the default
bound (
lower0.0,upper+inf): a sparse bound Param only tightens the elements it has rows for. null/NaNParam values likewise mean "default".+inf/-infare allowed (e.g.lower=-inffrees an element).-
Duplicate Param rows on its dims raise
ValueError. -
Param rows matching no variable row are ignored (sparse tighten) — including rows whose key lies outside the variable's
pl.Enumvocabulary. Null keys never match, so such elements keep the default. AnEnumvsUtf8key dtype mismatch is aligned automatically;Enumkeys with disjoint (non-nested) category sets raiseValueError; other dtype mismatches (e.g.Utf8vsInt64) raise polars' join error. - A
+inflower or-infupper bound raisesValueError(Param element or scalar alike). An element withlower > upperis accepted here and makes the LP infeasible at solve time, exactly as a scalarlower > upperdoes.
Scalars may be any numbers.Real (int / float / numpy
scalar ...) or a 0-d numeric numpy array; bool is rejected.
Param bounds are resolved eagerly here into a per-column float64
array stored on :attr:Var.lower / :attr:Var.upper; column
order and ids are unaffected by the bound form. The values are
NOT tracked afterwards: the names of named bound Params are
recorded on :attr:Var.bound_param_names and
:meth:WarmProblem.declare_mutable / :meth:WarmProblem.update_param
raise for them — change bounds on a built problem with
:meth:WarmProblem.set_col_bounds.
add_cstr ¶
add_cstr(name: str, *, over: DataFrame | None = None, sense: str, lhs_terms: dict[str, Var | Expr | Param | int | float], rhs_terms: dict[str, Var | Expr | Param | int | float] | None = None) -> None
Add a constraint of the form Σ lhs_terms sense Σ rhs_terms.
Each term entry is either
- a
VarorExpr— variable contribution, or - a
Param,intorfloat— constant contribution.
The engine sorts variables and constants out per side, builds
(lhs_var − rhs_var) sense (rhs_const − lhs_const), and adds
the row to highspy at solve time. Labels (the dict keys) are
used in row names and diagnostics.
cstr_names ¶
All constraint family names currently registered, in declaration order. Useful for emission audits and debugging.
cstrs_named ¶
Return constraint metadata records matching name.
An exact-name match returns the single record; otherwise a prefix
match returns every record whose name starts with name + "_"
(so passing "minimum_uptime" returns both
minimum_uptime_linear and minimum_uptime_integer).
Each :class:CstrRecord carries:
* name: full registered name of the constraint family;
* over: the polars DataFrame of axis tuples (len(over)
is the row count);
* proto: the underlying _CstrProto (expr, sense,
rhs) for advanced introspection.
set_named_basis ¶
Record a warm-start basis to inject at the next solve (spec §4.1).
This is the in-process (non-save_memory) primitive: it only
RECORDS INTENT — it stores the carrier on self._warm_basis and
the policy on self._warm_basis_policy. It does NOT build or
inject anything. The solve-time hook (streaming path, just before
h.run()) constructs the sized HiGHS basis via
:func:polar_high._warm_basis.build_highs_basis and calls
h.setBasis.
policy="exact" transfers 1:1 by name and requires the carrier's
fingerprint to match this model's; a mismatch under "exact" is
expected to be caught by the hook, which then falls back safely to
a cold solve. policy="alien" maps shared names and lets HiGHS
repair the remainder.
basis_name_fingerprint ¶
Structural fingerprint of THIS model's LP name-set (spec §4.5).
A cheap, side-effect-free accessor the orchestrator uses as the
cache key for a captured :class:NamedBasis: it is the
:func:polar_high._warm_basis.basis_fingerprint of this model's
rendered column + constraint-row names, so a basis captured off one
build and re-injected into an identically-shaped build key on the
same string. It mirrors what :meth:write_mps stashes as
_last_mps_fingerprint (same basis_fingerprint(col_names,
m.row_names) inputs) but is callable without writing an MPS.
Uses the canonical matrix's names (m = self.canonicalise());
canonicalise is memoized, and LP names are Layer-2-invariant, so
the value is valid pre- or post-scaling and stable across repeated
calls. Returns None when the problem has been released or has no
columns (no meaningful key).
seed_primal ¶
seed_primal(values: dict[str, float], *, frame: Literal['user', 'scaled'], missing: str = 'skip') -> None
Record a partial primal seed to inject at the next solve.
Phase 3 Step 3a primitive. Like :meth:set_named_basis this only
RECORDS INTENT — it stores the seed on self._primal_seed and the
frame on self._primal_seed_frame. It does NOT build or inject
anything: the solve-time hook (streaming path, just before
h.run(), immediately after the basis block) resolves each name to
a column index, converts + sanitises the value, and calls
h.setSolution.
values maps column name (rendered exactly as
Solution.col_names — "name[d0,d1,...]" for indexed vars, the
bare name for scalars) to a numeric value. A partial seed is
legal and expected: names present in the model are seeded, names
absent are silently skipped (the subset behaviour, missing="skip").
frame is REQUIRED and has NO default (a silent default frame is a
corruption vector). (The :class:WarmProblem update_rhs /
update_obj_coef methods do default to frame="user": that is
backward compatible when no Layer-2 side vectors are set, and under
Layer 2 it is the deliberate fix — the values mean what they mean at
build.) frame="scaled" uses each value as-is in the
solver's post-Layer-2 scaled space. frame="user" converts with
the model's own Layer-2 side-vector
(x_scaled = x_user / _layer2_col_factor[col_id]; identity when
Layer 2 was not applied), so callers can seed in natural units.
Feasibility-safety: a primal seed is only a starting point for the solver — it never binds the optimum. A non-optimal (or even infeasible-looking) seed cannot change the objective HiGHS returns, with presolve on or off.
MUTUALLY EXCLUSIVE with a warm basis. setSolution and
setBasis clobber each other on the same handle (last writer wins;
setSolution invalidates a previously-set basis). For an LP a
basis dominates a partial seed, so the hook enforces the rule "if a
basis is set, it wins": when self._warm_basis is also set the seed
is skipped.
In-process (streaming) ONLY — there is no subprocess / MPS channel
for a primal seed; a save_memory solve ignores the seed.
A seed over integer columns is a MIP-start. (Noted for a future UC5; not implemented or specialised here — integer columns are seeded the same way and HiGHS treats them as a MIP-start.)
cstr_row_count ¶
Total LP-row count across all constraint families matching
name (exact or prefix; see :meth:cstrs_named). Returns 0
when no families match — letting callers distinguish "absent"
from "empty" without exception handling. A scalar constraint
(over=None) counts as one row.
add_obj_constant ¶
Accumulate a constant into the objective offset.
The constant is in the same units as the objective given to
:meth:set_objective (scaled by :meth:set_objective_scale like
the rest of the objective). It shows up in Solution.obj even
though no decision variable carries it. polar-high keeps it out of
HiGHS and adds it to the objective HiGHS reports, so HiGHS'
user_objective_scale and its MIP gaps see only the variable part
of the objective (a raw Solution.highs.getObjectiveValue()
excludes it). Used for pure-Param objective terms like an existing
entity's fixed cost.
set_objective_scale ¶
Declare that the objective given to :meth:set_objective and
:meth:add_obj_constant is factor times the objective in user
units, the units the caller thinks in (e.g. 1e-6 for a cost in currency built as
million currency).
Objective-valued solver options (targets, bounds, cutoffs, absolute
gaps; see :mod:polar_high.objective_options) are then read in user
units: polar-high multiplies them by factor (and by
HiGHS' 2**user_objective_scale) before they reach HiGHS, and
:meth:objective_transform gives the same mapping for other
solvers. Solution.objective_in_user_units is Solution.obj /
factor. factor must be finite and > 0; the default is 1.0.
objective_transform ¶
The :class:~polar_high.objective_options.ObjectiveTransform that
maps an objective value in user units to solver's value
for this Problem.
solver: "highs", "gurobi", "cplex", "xpress" or
"copt". io: "direct" (the solver gets the model through
its API, e.g. in-process HiGHS or polar_high.solvers.solve) or
"mps" (the solver reads a file from :meth:write_mps, which
does not carry the objective constant). options: the option
dict the solver gets; for HiGHS its user_objective_scale = N
gives the factor 2**N (None reads the options stored with
:meth:set_solver_options). Every other solver has factor 1,
whatever the options hold.
solve ¶
solve(*, options: dict | None = None, keep_solver: bool = False, streaming: bool = True, save_memory: bool = False, tmp_dir: str | PathLike | None = None, highs_names: str = 'always', pre_run_check: Callable[[dict, Highs], Mapping[str, Any] | None] | None = None) -> Solution
Solve the LP and return a :class:Solution.
Parameters¶
options
Per-call HiGHS options dict (overrides set_solver_options).
Objective-valued options (objective_target,
objective_bound, mip_abs_gap) are read in user units and
in the objective's sense and mapped to HiGHS' (see
:meth:set_objective_scale and
:mod:polar_high.objective_options); the dict is not changed.
keep_solver
When True, the live HiGHS instance is kept on the returned
:class:Solution so callers can inspect it post-solve (e.g.
sol.highs.writeModel("model.mps")). Default False —
the C-side LP storage is released as soon as primal/dual/
objective have been extracted.
streaming
When True (default), columns are added once via addCols
and each constraint family is emitted to HiGHS via addRows
immediately after its COO triples are built; the family's local
arrays then go out of scope before the next family is processed.
This caps peak memory at one family's COO + the running HiGHS
LP. When False, the entire model is assembled into a single
:class:highspy.HighsLp and loaded via passModel —
numerically identical results either way; False is mostly
useful for benchmarking the legacy path.
save_memory
Single one-shot knob that trades wall time for peak RSS. When
True, two things happen right before HiGHS.run():
1. polar-high's polars/numpy LP source-of-truth (constraint
family ``Expr.terms`` lists, ``_CstrProto.rhs`` Param
frames, objective ``_Term.lazy`` plans, the caller-side
``col_names`` / ``row_names`` lists) is dropped. Only the
per-:class:`Var` ``col_id`` frames survive — they are
needed by :meth:`Solution.value` to map column indices
back to user-space dim tuples.
2. The HiGHS instance is round-tripped through disk: the LP
is written to a temp MPS file, the original ``Highs()`` is
cleared and discarded, ``malloc_trim(0)`` is called (best
effort, Linux only), a fresh ``Highs()`` is created, the
same solver options are re-applied, and the MPS file is
read back. This resets the HiGHS allocator's high-water
mark — the C++ side accumulates ~5 GB of slack from the
incremental ``addRows`` loading path that ``readModel``
avoids by sizing once up front.
Cost: ~+90 s of MPS file I/O at N=3000 dense. Benefit: peak
RSS drops by ~5 GB on the same problem. Intended for one-
shot single-solve benchmarks where warm-start / re-solve
isn't needed. After the call returns, the :class:`Problem`
is in a "released" state and any further :meth:`solve` call
raises ``RuntimeError`` (the polar-side source AND the
original HiGHS instance with its basis have both been
discarded, so neither a fresh re-solve nor a WarmProblem-
style update is possible). Honoured only by the streaming
path; on ``streaming=False`` a warning is emitted and the
flag is ignored. Default ``False``.
tmp_dir
Directory for the temporary MPS file written by the
save_memory=True round-trip. None (default) uses
the system temp dir ($TMPDIR / /tmp). Set when the
caller wants the spill file on a specific volume (e.g. the
same filesystem as its workspace, or a per-job scratch
directory) instead of the system default. Ignored when
save_memory=False.
highs_names
When the column / row names are passed to HiGHS (see
:data:HIGHS_NAME_OPTIONS): "always" (default), "auto"
(just before run(), only if a name-writing HiGHS option is
on) or "never". HiGHS does not use names to solve, so the
solution is the same in every mode; leaving them out saves
HiGHS' copy of every name. Solution.col_names /
row_names are unchanged; Solution.highs_names_passed
says whether the kept HiGHS instance has them, and
:meth:Solution.pass_names_to_highs passes them later. A
save_memory solve always passes them: its MPS round trip
through HiGHS carries the names.
pre_run_check
Called once, right before HiGHS.run(), as
pre_run_check(ranges, highs): ranges is the
streamed_lp_ranges dict of the LP HiGHS holds, with the
scaling-check keys added (see :attr:Solution.streamed_lp_ranges:
col_bound_meaningful / row_bound_meaningful, (b_lo*,
b_hi) over the continuous-column bounds and the row bounds below
infinite_bound, the small end leaving out entries whose model-
unit magnitude is below residue_floor = 10 × the primal
feasibility tolerance in the effective options;
bound_rhs_residue (count, largest model-unit magnitude);
bound_rhs_beyond_infinite; residue_floor;
infinite_bound); highs is the live handle. It may set
options on highs (e.g. user_bound_scale), return a mapping
of HiGHS options to set, or raise — then no solve runs and the
exception propagates. The ranges also carry the
matrix_continuous / integer_matrix, cost_continuous /
integer_cost and integer_bound (min, max) pairs (HiGHS
multiplies an integer column's matrix entries and cost by
2**user_bound_scale). The names decision of
highs_names="auto" is taken after the hook, so an option it
sets that writes names is honoured. Honoured by the streaming
path only (ValueError with streaming=False); also taken by
:meth:build_only and by the first :meth:WarmProblem.solve.
build_only ¶
build_only(mps_path: str, *, options: dict | None = None, pre_run_check: Callable[[dict, Highs], Mapping[str, Any] | None] | None = None) -> dict[str, Any]
Build the LP into HiGHS, write to mps_path, release everything.
For callers that want to drive the actual solve out-of-process — typically a subprocess HiGHS reading the MPS file in a clean address space. After this call:
mps_pathexists and contains the LP in MPS format.- polar-side LP source-of-truth has been dropped
(:meth:
_release_python_lp_inputs). - The live :class:
highspy.Highsinstance has been torn down and the glibc allocator trimmed (Linux best-effort). self._releasedis True — further calls to :meth:solvewill raise.
What stays alive: self._vars and each :class:Var.frame,
so the caller can later construct a :class:Solution from
externally-produced col_value / row_dual arrays via
Solution(..., vars=dict(self._vars), ...).
Honoured only on the streaming path; the non-streaming
(passModel) path is not supported here. Raises
RuntimeError if called on an already-released Problem.
Parameters¶
mps_path
Where to write the MPS file. Caller owns the file —
polar-high will not delete it.
options
HiGHS solver options to apply during the build (mainly
presolve / solver / simplex_scale_strategy —
options that affect what writeModel serialises).
None uses :attr:_solver_options.
pre_run_check
As on :meth:solve: called once with the ranges of the built
LP and the HiGHS handle, before the MPS file is written; it may
raise (no file is written). The MPS file carries no options, so
the options the hook returns or sets on the handle are returned
for the caller to pass to its solver.
Returns¶
dict
The HiGHS options pre_run_check returned or set on the
handle (those set on the handle are read back from highspy's
HighsOptions record, so return an option to be sure it is
carried); empty without a hook.
canonicalise ¶
Build (or return cached) the canonical CSC matrix + metadata.
Idempotent: returns the cached _matrix unless
_canonical_dirty is set (which add_var / add_cstr
flip). Side vectors (_layer2_col_factor /
_layer2_row_factor) are baked into the returned arrays at
build time per orchestrator decision D8.
write_mps ¶
write_mps(path: str | PathLike, *, free_format: bool = True, column_order_strict: bool = True, emit_names: bool = True, release: bool = False, name: str = 'POLAR_HIGH', pre_run_check: Callable[[dict, Any], Mapping[str, Any] | None] | None = None) -> dict[str, Any]
Stream the LP/MIP to path as a free-format MPS file.
This is an in-house writer that consumes the polar-side LP
source-of-truth (_vars, _cstrs, _obj_terms) directly
and emits MPS without ever instantiating :class:highspy.Highs
or building an :class:LpView. The motivation is peak-memory:
HiGHS' own writeModel serialiser materialises ~20× more
transient state than the in-process LP itself, which OOMs on
very large LPs (~10 M rows × 5 M cols × 20 M nonzeros). This
writer targets ~2-3 GB peak on that same LP.
Parameters¶
path
Output MPS file path. Caller owns the file; this writer
does not delete on failure.
free_format
Reserved for future use. Currently always free-format MPS
(the modern default — HiGHS, Gurobi, CPLEX, Xpress all
accept it). Fixed 8-char-column MPS is not yet implemented.
column_order_strict
When True (default), all nonzeros for a single column
are emitted contiguously in the COLUMNS section, sorted by
(col_id, row_id). This is the spec-compliant ordering
every MPS reader accepts. False is currently not
implemented — see :ref:Open follow-ups in the module docs.
emit_names
When True (default), row and column names come from the
variable family name + dim tuple and the constraint family
name + dim tuple (matching the format used by
:meth:Problem.solve for Solution.col_names /
Solution.row_names). When False, generic
C0000001 / R0000001 names are emitted instead —
saves ~30-50% file size at the cost of losing the
dim-tuple → column mapping in the MPS itself. Callers
using emit_names=False must use index-based solution
readback (column 0 in the MPS is column 0 in the polars
Var.frame ordering, etc.). The objective row name is
always cost regardless — it's a reserved unique name.
release
When True, calls :meth:_release_python_lp_inputs
after the write completes. Mirrors the
solve(save_memory=True) semantics — drops the polars
LazyFrames, constraint family list, and rhs Param frames
while keeping self._vars (so solutions read back from an
external solver can still be mapped to user-space dim
tuples via the surviving Var.frame["col_id"] columns).
After release the :class:Problem is no longer solvable —
:meth:solve will raise.
name
MPS NAME header; default "POLAR_HIGH".
pre_run_check
As on :meth:solve, for a caller that solves the file in
another process: called once as pre_run_check(ranges, None)
before anything is written, with the ranges of the LP in the
file (the same keys as on :meth:solve, read from the
canonical matrix as a :class:WarmProblem build does). There
is no solver instance (None), so the hook returns the
options it wants; they are returned for the caller to pass to
its solver. An exception from it propagates and no file is
written.
Returns¶
dict
The options pre_run_check returned (empty without one).
Raises¶
RuntimeError
If called on a :class:Problem whose source has already
been released (by an earlier solve(save_memory=True),
build_only(...), or write_mps(..., release=True)).
ValueError
If any coefficient is NaN or infinite (silent filtering
would hide model bugs). The error message identifies at
least one offending (col_id, row_id) pair.
OSError
File I/O failures (passed through from open()).
Notes¶
- The MPS format has no portable encoding for an objective
constant offset. If
self._obj_offset != 0.0this method emits a :class:UserWarningand writes the LP without the offset. Solutions from any downstream solver will then be off byself._obj_offset— the caller must add it back manually if needed. - Integer columns are bracketed with
MARKER 'INTORG'/MARKER 'INTEND'in the COLUMNS section, per MPS convention. - The writer is single-pass per family and uses one global polars sort to put the COLUMNS section into column-major order. Streaming-engine fallback to disk activates automatically for triple counts that don't fit in RAM.
- Stashes
self._last_mps_fingerprint— a structural basis-fingerprint (:func:polar_high._warm_basis.basis_fingerprint) over the model's real column/constraint-row name-set — computed before anyrelease, and the same whetheremit_namesis True or False (the generic names are never hashed). Consumed by thesave_memorysubprocess warm-start arm to key a basis-cache file. - Every column is written in the COLUMNS section, in
col_idorder. A column with no matrix entry and no objective coefficient gets one explicitcost 0entry, so MPS readers keep it in place with its bounds and integrality (a column present only in BOUNDS is moved to the end or dropped by some readers, e.g. HiGHS).
Var ¶
Var(name: str, dims: tuple[str, ...], frame: DataFrame, lower: float = 0.0, upper: float = float('inf'), integer: bool = False)
A variable family. frame carries columns *dims, col_id.
Var.frame stays an eager polars DataFrame — it's small (one row
per LP column), produced once in :meth:Problem.add_var, and
consumed by both flextool integration (v.frame["col_id"].unique())
and Problem.solve (col_id → bound/name lookups). Algebra ops on
Var lazify on the fly so the resulting _Term is lazy.
lower / upper are each EITHER a scalar (one bound shared by
every column of the family — the historical form, kept verbatim so
existing callers emit a byte-identical LP) OR a float64
np.ndarray of length frame.height holding one bound per
column, positionally aligned with frame rows (i.e. with
frame["col_id"]). :meth:Problem.add_var produces the array
form when a :class:Param bound is given. Bound consumers read
them through :meth:col_lower / :meth:col_upper (scatter-ready
values) and rescale them through :meth:scale_bounds.
bound_param_names records the .name of every named
:class:Param passed as a lower / upper bound. Bound Params
are resolved to numbers at :meth:Problem.add_var time and are NOT
tracked afterwards, so :meth:WarmProblem.declare_mutable /
:meth:WarmProblem.update_param refuse these names (change such
bounds with :meth:WarmProblem.set_col_bounds instead).
A Var built directly (not through :meth:Problem.add_var) may map
several frame rows to one col_id — e.g. a variable that keeps one
column over a run of time steps, indexed on every step. Terms built
from it are summed per (row, col) like any duplicate cell;
:attr:col_ids_unique tells the builders that a Sum over its rows may
combine coefficients.
col_ids_unique
property
¶
True when every frame row has its own col_id (always for a
family declared with :meth:Problem.add_var and its renamed
views). Computed once per Var.
has_elementwise_bounds
property
¶
True when lower or upper is a per-column array.
col_lower ¶
Lower bound in scatter-ready form: float for a scalar bound
(identical to the historical float(v.lower)), else the
per-column float64 array aligned with frame rows. Use as
col_lb[v.frame["col_id"].to_numpy()] = v.col_lower().
scale_bounds ¶
Multiply every FINITE bound value by factor in place.
Column-scaling hook (e.g. an autoscale Layer 2 column factor
f maps x = f * x_scaled ⇒ bounds scale by 1/f — the
caller passes the factor it wants applied). Infinite bounds are
left untouched. Scalar bounds stay scalars (float(b) * f,
the exact historical arithmetic); per-column arrays are scaled
element-wise with the same per-entry arithmetic, so each column
is scaled exactly as a scalar bound with that value would be.
Param ¶
Param(dims: tuple[str, ...], frame: DataFrame | LazyFrame, name: str | None = None, _sources: list[tuple[Param, int]] | None = None, _value_scalar: float = 1.0, _op: tuple | None = None)
A parameter table. frame carries columns *dims, value.
Stored internally as a polars.LazyFrame so that chained algebra
ops (Param * Param, Param + Param etc.) defer materialization
until a consumer reads .frame or the engine collects in
Problem.solve. The .frame property caches the eager
DataFrame on first read — flextool reads .frame.rename(...)
repeatedly off the same Param so we want that to be cheap.
name (optional) is a logical Param identifier (e.g. "p_inflow").
It is opt-in metadata used by :class:WarmProblem's Param-tracked
auto-update (declare_mutable / update_param). When unset,
Params are anonymous and carry no tracking overhead.
_sources records constituent named Params for composite results
of Param * Param / Param / Param. Each entry is
(param_name, dims_tuple, direction) where direction is +1 if the
Param contributes to the numerator and -1 if to the denominator.
Anonymous-only chains have _sources is None.
_op records the operation tree of a Param ⊗ Param composite
and of scalar multiples / quotients (see :func:_param_tree); None
for a leaf — a frame-constructed Param or a Param + Param /
Param + float result. :func:_prune_composite_lazy uses it to
rebuild an expanding product with every leaf semi-joined to a consuming
term's keys (same association, same per-cell arithmetic).
Expr ¶
A sum of terms (decision-variable contributions).
The terms can have different open-dim sets — they're concatenated,
not broadcast. Broadcasting happens once, at constraint emission,
via a join to the constraint's over= row index.
WarmProblem ¶
Warm-update wrapper around a :class:Problem.
Build a :class:Problem as usual (add_var, add_cstr,
set_objective). Wrap with :class:WarmProblem, then alternate
update_* calls and solve() calls — the LP is built ONCE and
only the changed coefficients / RHS values are pushed to HiGHS
between solves.
Typical usage (re-solving the same LP with new data)::
wp = WarmProblem(p)
sol_0 = wp.solve()
for r in range(1, n_cases):
# user-space values on the build labels (or positional arrays)
wp.update_rhs("balance", demand_param_for_case[r])
wp.update_obj_coef("v_flow", cost_param_for_case[r])
sol_r = wp.solve()
The update_* calls are O(rows_or_cols_in_family); the solve()
benefits from HiGHS's hot-start (basis is preserved across calls).
Value frames. :meth:update_rhs, :meth:update_obj_coef and
:meth:update_obj_coef_array take values in user space by default
(frame="user"): the Layer-2 side vectors set on the Problem are
applied, exactly as at build, so the update means the same as
rebuilding with the new values. frame="scaled" writes values
as given. :meth:update_param recomputes cells from factors cached
in the scaled space, so it takes user-space Param values too. The
column-bound and cut APIs (:meth:fix_cols, :meth:fix_col_ids,
:meth:set_col_bounds, :meth:add_cut_row) write in the scaled
frame: the caller applies any Layer-2 transform itself.
The live LP keeps the labels of the Problem it was built from (the
Var frames and the HiGHS column / row names). The update_*
calls change values, never labels: a :class:Param is matched to rows
and columns by those build labels.
highs_names chooses when the column / row names go to HiGHS:
"always" (default; right after the build), "auto" (before a
run() at which an option in :data:HIGHS_NAME_OPTIONS is on) or
"never". HiGHS solves the same LP without them; leaving them out
saves its copy of every name. Solution.col_names / row_names
are the same in every mode, and :meth:pass_names_to_highs passes the
names on demand (e.g. before Solution.highs.writeModel).
problem
property
¶
The underlying :class:Problem.
Useful for diagnostics that need to inspect the un-built LP.
For coefficient ranges, prefer
:attr:Solution.streamed_lp_ranges on a returned
:class:Solution (populated during the streaming solve at zero
extra cost).
set_named_basis ¶
Record a warm-start basis to inject on the fresh build (spec §4.1).
RECORDS INTENT only — stores the carrier on self._warm_basis and
the policy on self._warm_basis_policy; it does NOT build or
inject anything here. The actual h.setBasis fires exactly once,
in :meth:_initial_build, immediately before the first solve's
h.run(). This is the in-process default-path equivalent of the
streaming basis hook in :meth:Problem._solve_streaming.
policy="exact" transfers 1:1 by name and requires the carrier's
fingerprint to match this model's; a mismatch under "exact" is
detected in _initial_build and safely falls back to a cold solve.
policy="alien" maps shared names and lets HiGHS repair the rest.
The inject fires only on the fresh build — _initial_build runs
once per WarmProblem lifetime (solve calls it only while
self._h is None), so a subsequent height-match reuse re-run never
re-injects.
update_rhs ¶
update_rhs(cstr_name: str, new_param: Param | float | int | ndarray, *, frame: Literal['user', 'scaled'] = 'user') -> None
Replace the RHS of constraint family cstr_name with values
drawn from new_param.
new_param may be a :class:Param whose dims match a subset
of the constraint's over= axis (broadcasting to the rest; the
values are matched by the over labels), a scalar (broadcast to
all rows), or a numpy array (positional — length must equal the
family's row count, in the order of the original over frame).
frame="user" (default; backward compatible when no Layer-2 side
vectors are set, a deliberate change under Layer 2, where earlier
releases wrote the values unscaled): the values are in user space, as
the RHS terms passed to :meth:Problem.add_cstr; the Layer-2 row
factor of each row is applied, exactly as at build.
frame="scaled": the values are already in the solver's scaled
space and are written as given.
The replace is total: the new vector is the whole RHS of each
row, so any constant the build put into the row (from other RHS
terms or constant LHS terms) must be included by the caller.
The small-value floor (Problem.coef_zero_threshold) is judged
on the user-space magnitude in both frames, as at build. ±inf
one-sided sentinels are preserved. With no side vectors set both
frames push the same arrays as earlier releases.
update_obj_coef ¶
update_obj_coef(var_name: str, new_param: Param | float | int, *, frame: Literal['user', 'scaled'] = 'user') -> None
Replace the objective coefficient on every column of var_name.
Assumes the objective contribution from var_name is exactly
coef[*dims] * var[*dims] for some coef Param; this method
OVERWRITES that coefficient via h.changeColsCost. If the
objective also has contributions from this variable through more
complex algebra (e.g. var * unitsize * slope), the update is
still valid as long as new_param carries the full product —
the caller is responsible for collapsing multi-Param products
ahead of the call.
This DOES NOT touch the cost coefficients of other variables.
frame="user" (default; backward compatible when no Layer-2 side
vectors are set, a deliberate change under Layer 2, where earlier
releases wrote the values unscaled): the coefficients are in user
space, as the objective terms passed to :meth:Problem.set_objective; the
Layer-2 column side vector is applied, exactly as at build.
frame="scaled": the coefficients are already in the solver's
scaled space and are written as given. With no side vectors set
both frames push the same values as earlier releases. Values of a
:class:Param are matched to the columns by the variable's dim
labels; columns without a match get 0.0.
update_obj_coef_array ¶
update_obj_coef_array(var_name: str, dim_tuples: list[tuple], values: ndarray, *, frame: Literal['user', 'scaled'] = 'user') -> None
Array-form of :meth:update_obj_coef.
dim_tuples is a list of dim-value tuples (one per cell) for
variable var_name; each tuple must have one entry per dim
in the var's declared signature. values is a same-length
numpy array of new objective coefficients.
The columns are resolved positionally: values[k] becomes the
new objective coefficient on the column whose dim-tuple is
dim_tuples[k]. Vectorised — a single changeColsCost
call regardless of cell count.
frame as in :meth:update_obj_coef: "user" (default)
applies the Layer-2 column side vector, "scaled" writes the
values as given.
fix_cols ¶
Fix the listed columns of var_name to the given values.
For each (dim_tuple, value) pair, sets both the column's
lower and upper bound to value (so the LP has no choice
but to set the column at that level). Used by the Lagrangian
primal-recovery step ("fix-and-resolve"). Vectorised — single
changeColsBounds call.
Scaled frame: values are written verbatim as column bounds of
the live (possibly Layer-2 scaled) model; any caller-side scaling
must be applied beforehand. Unlike :meth:update_obj_coef, there
is no frame="user" conversion.
fix_col_ids ¶
Fix columns BY RAW COL ID: set lower = upper = value for each
(col_id, value) pair.
The col-id-level counterpart of :meth:fix_cols — the same
vectorised changeColsBounds(lo=hi=value) write, but skipping
the per-call dim-tuple → col-id join for callers that already
hold the raw ids (resolve once via :meth:col_id_of_var, or use
ids returned by :meth:add_recourse_col). Typical consumer: a
cutting-plane driver temporarily pinning coupling columns.
Bounds are written verbatim into the live model — any caller-side
scaling must be applied to values beforehand (scaled frame;
there is no frame="user" conversion here).
get_col_bounds ¶
Read the current (lower, upper) bounds of the given raw col
ids from the live model.
Returns two float64 arrays aligned POSITIONALLY with col_ids
(any order, repeats allowed — HiGHS' getCols itself requires
an ordered duplicate-free index set, so the query goes through
the unique sorted set and gathers back to caller order).
Save/restore companion of :meth:set_col_bounds for temporary
pins made with :meth:fix_col_ids::
lo, hi = wp.get_col_bounds(ids) # save
wp.fix_col_ids(ids, values) # temporary pin
... wp.solve() ...
wp.set_col_bounds(ids, lo, hi) # restore
set_col_bounds ¶
Write (lower, upper) bounds onto the given raw col ids.
Restore companion of :meth:get_col_bounds (see there for the
save/pin/restore pattern). Vectorised — single
changeColsBounds call; ±inf entries round-trip unchanged
(HiGHS' bound sentinel IS the IEEE infinity).
Scaled frame: the bounds are written verbatim into the live model
(as read back by :meth:get_col_bounds); there is no
frame="user" conversion here.
update_coef ¶
Update a single (row, col) coefficient in the constraint
matrix. Use :meth:row_id_of_cstr / :meth:col_id_of_var to
resolve indices semantically.
add_cut_row ¶
Append a >= constraint row to the live (already-built) LP and
return the new row's id.
Appends Σ coefs[k] · x[col_ids[k]] >= lower to the live HiGHS
model via addRow. This is a POST-build mutation of the warm
_h handle (the same class of live edit as :meth:update_coef;
unlike :meth:update_rhs, which defaults to the user frame, the
row is written in the scaled frame); it deliberately bypasses the build-time
Problem.add_cstr DSL lock, which only guards the fixed-size
Layer-2 autoscale side vectors and is irrelevant once the model is
built. The caller is responsible for keeping the master autoscale
OFF (or for pre-scaling coefs) so the appended row lives on the
built columns' scale — there is no auto-scaling for appended rows.
col_ids may mix existing column ids (resolve via
:meth:col_id_of_var) and ids of columns previously appended with
:meth:add_recourse_col. A subsequent :meth:solve warm
re-optimises the grown model; the appended row's dual is then
readable on the returned :class:Solution by row_dual[row_id]
(read BY ROW ID, not via :meth:Solution.constraint_dual, which
only knows named build-time rows).
Bumps the cached _n_rows (the zero-fill fallback size in
:meth:solve) and appends a generated name to _row_names (kept
index-aligned so name-indexed reads stay correct). Returns the
row_id HiGHS assigned (== the pre-append getNumRow()).
add_recourse_col ¶
Append a single free/bounded column (e.g. a Benders recourse
η) to the live LP and return its col id.
Symmetric with :meth:add_cut_row: a POST-build addCol on the
warm _h handle, bumping the cached _n_cols (zero-fill
fallback size) and _col_names. ±np.inf bounds are mapped to
the HiGHS kHighsInf sentinel. The returned id can be referenced
from a later :meth:add_cut_row; its value is read on the
:class:Solution by col_value[col_id] (appended columns are not
in any Problem._vars frame, so :meth:Solution.value cannot see
them — read by id).
compact_cuts ¶
compact_cuts(solution: Solution, *, policy: str = 'slack', trial_col_values=None, tol_rel: float = 1e-07, verify: bool = True) -> dict
Prune retained cut rows to bound the active master LP.
A cutting-plane master (the Benders master) grows one >= cut row
per iteration via :meth:add_cut_row; left unpruned its warm
re-solve time grows super-linearly. Two selection policies are
available, both LB-safe (they only drop cuts that do not support the
certified lower bound) and both belted by the same verify-restore
rollback:
policy="slack" (default):
Drop the cut rows that are strictly slack (positive primal slack,
hence zero dual by complementary slackness) at solution,
keeping the binding ones. Classification is purely by PRIMAL
slack of the stored cut against solution.col_value: for a cut
Σ coefs·x >= lower the slack is Σ coefs·x - lower >= 0; a
cut with slack <= tol is BINDING (kept), one with
slack > tol is strictly slack (dropped). tol is a per-cut
relative tolerance scaled by the magnitude of lower and the
terms.
policy="dominance" (de Matos–Philpott–Finardi / Guigues
Limited-Memory Level-1):
At a DEGENERATE optimum almost every cut can be primal-binding
(slack ≈ 0), so slack-deletion prunes nothing even though many
cuts are redundant TIES at their recourse group's max value.
Dominance selection instead groups the retained cuts by the
recourse variable they bound (the single member of
col_ids ∩ _recourse_cols, registered by
:meth:add_recourse_col) and, over a WINDOW of recent master
trial points trial_col_values (a list of col_value arrays,
most-recent last; defaults to [solution.col_value]), keeps for
each group at each trial point the OLDEST cut that achieves the
group's max value there (within a relative tolerance). Every cut
not a keeper at any trial point is DROPPED — it is dominated (or a
redundant tie) at every trial point, so the outer approximation of
the recourse function at those points, and the LB it certifies,
is unchanged. A cut whose col_ids contains no recourse column
forms its own singleton group and is always kept (generic
non-Benders use degrades safely to "keep all").
When verify (default), after deleting rows the master is re-solved
and its objective compared to the incumbent solution.obj. A drift
larger than :data:_COMPACT_VERIFY_TOL (relative) means a dropped cut
was in fact supporting the optimum (a degenerate numerical edge) — the
deletion is ROLLED BACK by re-appending every dropped cut verbatim and
re-solving, and the returned dict reports restored=True with
dropped=0. On a well-behaved master the belt never fires.
Returns {"kept": int, "dropped": int, "restored": bool} — cut
counts AFTER the operation (kept == number of retained cut rows
that remain, dropped == number removed; on a restore kept is
the pre-compaction count and dropped is 0).
declare_mutable ¶
Declare a set of :class:Param names whose values should be
tracked into LP cells, so :meth:update_param can later push
new values into the live HiGHS instance via changeCoeff.
MUST be called BEFORE the first :meth:solve. Tracking is
opt-in: Params not declared here pay no bookkeeping cost.
Pass the same names that the Params carry on their .name
field — typically the FlexData attribute name ("p_inflow",
"p_penalty_up" etc.).
Raises ValueError for the name of a Param used as an
add_var lower / upper bound: bound Params are
resolved at add_var and not tracked (a later update would
leave the bounds stale) — use :meth:set_col_bounds instead.
set_output_flag ¶
Enable or disable HiGHS' native solve log for this problem.
enabled=False mutes the per-solve HiGHS output (the
output_flag HiGHS option) for THIS WarmProblem across all of
its cold / warm / retry solves — callers that drive many parallel
sub-solves (Benders regions, Lagrangian subproblems) use this to
keep stdout clean and emit their own concise progress log instead.
May be called either before the first :meth:solve (the flag is
applied when the HiGHS handle is built) or after it (applied to the
live handle immediately). The preference persists on the handle,
so a single call suffices for the whole lifetime.
update_param ¶
Replace the values of a tracked Param. Every LP cell whose
coefficient was originally a function of param_name is
re-computed from the new Param's values and pushed via
h.changeCoeff.
new_param must be either a scalar (broadcast to all tracked
cells) or a :class:Param whose dim signature matches the
signature recorded for that Param at build time.
Raises if param_name was not in :meth:declare_mutable's
list (silent corruption is worse than a hard error), or if it
names a Param used as an add_var bound (bound Params are
resolved at add_var and not tracked; use
:meth:set_col_bounds).
Scope, which the caller must respect:
- It updates tracked matrix cells only. Contributions of the
Param to the objective, to row bounds (RHS) or to column bounds
are NOT updated, nor are cells reached through an anonymous
composite Param that carries no link to
param_name. The caller must ensure the Param reaches the LP only through tracked cells; a Param with zero tracked cells is a silent no-op. - Values are matched to the cells by the Param's dim labels
(a join on the labels recorded at build); cells whose labels are
missing from
new_paramget0.0. A Param whose labels shifted (e.g. a rolled time axis) is therefore misplaced, not re-aligned by position. - Values are in user space; the cached factors carry the Layer-2 side vectors, and the small-value floor is judged on the user-space magnitude, as at build.
col_id_of_var ¶
Return the col_id(s) for a variable.
dims=None returns every col_id in the variable's family
(numpy array, ordered by the var's declaration order).
dims as a tuple of dim values returns the single col_id for
that one cell (a python int). dims as a dict
{dim_name: value} is a partial filter — returns a numpy
array of the matching col_ids.
row_id_of_cstr ¶
Return the row_id(s) for a constraint family. Mirrors
:meth:col_id_of_var.
solve ¶
solve(*, options: dict | None = None, retry_on_unknown: bool = False, pre_run_check: Callable[[dict, Highs], Mapping[str, Any] | None] | None = None) -> Solution
Solve the LP. First call builds the LP from scratch (same
pipeline as :meth:Problem.solve); subsequent calls just run
HiGHS again on the (possibly updated) live model.
options is honoured on the FIRST solve only — subsequent
solves use the same HiGHS instance. To change options on a
rebuilt LP, drop the WarmProblem and create a new one.
pre_run_check (see :meth:Problem.solve) is called on the FIRST
solve only, after the build and before the names decision and
run(), with the same ranges keys, built from the canonical
matrix this build holds. A re-solve of the built instance does not
call it: its ranges are not re-derived after updates.
retry_on_unknown (default False → byte-identical to the
legacy path for every existing caller) enables a warm-restart
retry: the solver runs WARM first (retaining the basis so dual
simplex hot-starts after rows appended via :meth:add_cut_row);
only if the warm run does NOT return a certified kOptimal —
i.e. a stale-basis transient (kUnknown / kSolveError) or a
spurious kUnbounded / kInfeasible / primal-infeasible miss — do we
drop the basis with clearSolver() and re-run ONCE (the proven
cold fallback) to re-certify the true status; if that cold run is
still kUnknown, it passes the instance's own LP back to HiGHS and
runs once more. Used by the Benders
master, which appends a cut each iteration; on a well-scaled
master the warm path stays kOptimal and the fallback never
fires.
pass_names_to_highs ¶
Pass the column / row names to the live HiGHS instance now.
Idempotent (a no-op once the names are there, which is always the
case under highs_names="always"). Call it before reading names
back from HiGHS or writing a HiGHS file that should carry them
(writeModel, writeSolution, writeBasis) when the
WarmProblem was built with "auto" or "never". Rows and
columns appended later (:meth:add_cut_row,
:meth:add_recourse_col) then pass their names as they are added.
solve_with_fallback ¶
Solve; on a non-optimal primary, retry ONCE under
fallback_options and restore the prior option values.
Runs :meth:solve first. If it certifies kOptimal the
solution is returned as (solution, "primary") and the options
are never touched. Otherwise each fallback_options entry is
applied to the live HiGHS handle via setOptionValue
(solve(options=...) is honoured on the FIRST solve only — a
live-handle write is the only way to change options on a built
model), the retained solver state is dropped with
clearSolver() (so options like solver / presolve take
effect from cold, and no stale basis biases the retry), the model
is re-run once, and the prior option values are RESTORED — a
later plain :meth:solve sees the original configuration.
Returns (solution, "fallback") with the retry's solution,
optimal or not: the caller decides what a still-failing fallback
means.
Objective-valued fallback options are read in user units (as at
the build), with the user_objective_scale
the retry runs under: the fallback's own value, else the live
handle's.
Raises ValueError on an unknown option name or a rejected
option value; any options already changed are restored first.
Solution ¶
Solution(*, optimal: bool, obj: float, col_value: ndarray, row_dual: ndarray, col_names: list[str], row_names: list[str], vars: dict[str, Var], col_dual: ndarray | None = None, highs: Highs | None = None, streamed_lp_ranges: dict | None = None, highs_names_passed: bool = True, highs_name_passer: Callable[[], None] | None = None, obj_offset: float = 0.0, objective_scale: float = 1.0)
Read-only view of the solved LP. Look up variable values by
name; values come back as a polars frame (*dims, value).
objective_in_user_units
property
¶
obj in user units: obj divided by the
Problem's :meth:~Problem.set_objective_scale factor.
max_primal_infeasibility
property
¶
Largest primal-constraint violation in the returned solution.
A solver returns a vertex that sits within its feasibility tolerance
of the active constraints, so a hand-rolled feasibility re-check on the
solution (f <= cap, balance rows, …) must use a tolerance at least
this large — a hard-coded magic constant either masks real violations
or trips on the normal solver slack. HiGHS enforces feasibility on the
INTERNALLY-SCALED problem, so the unscaled slack reported here can
exceed :attr:primal_feasibility_tolerance.
0.0 when no live solver is attached (a synthesised Solution).
primal_feasibility_tolerance
property
¶
The solver's primal feasibility tolerance for this solve (the
nominal, scaled-problem tolerance; see
:attr:max_primal_infeasibility for the achieved unscaled slack).
0.0 when no live solver is attached (a synthesised Solution).
pass_names_to_highs ¶
Pass the LP names to the kept HiGHS instance, if the solve left
them out (highs_names="auto" / "never"). Idempotent; a
no-op when the names are already there or no instance is kept.
Call before highs.writeModel / writeSolution / writeBasis
when those files should carry the LP names.
solve_diagnostics ¶
Snapshot the solve's HiGHS status + feasibility/optimality counters.
Returns a :class:SolveDiagnostics read from the live solver, or
None when no queryable HiGHS handle is attached — i.e. a
synthesised :class:Solution (highs is None) OR a read-only
shim that reconstructs a solution from a solver .sol file
(the subprocess / commercial path) and does not implement
getInfo / getModelStatus. None means "cannot
diagnose", which callers must treat as un-verifiable — never as
an implicit pass.
Follows the highs is None → neutral guard of
:attr:max_primal_infeasibility, but extends it to the shim case
via hasattr so no AttributeError can leak from the
subprocess path.
The result is cached on the Solution (keyed on a weak reference to the handle, so the cache never keeps the handle alive), so a second call is free; it describes the solve this Solution came from.
value_wide ¶
value_wide(var_name: str, time_dims: tuple[str, ...] = ('d', 't'), solve_name: str | None = None) -> pl.DataFrame
Wide-form, flextool-compatible: time dims become rows, the remaining dims are encoded as a tuple-stringified column header.
For a 2-d variable like vq_state_up(n, d, t):
long : rows = (n, d, t, value)
wide : rows = (d, t) + one column per n (header = "west").
For a 5-d variable like v_flow(p, source, sink, d, t):
wide : rows = (d, t) + one column per (p, source, sink),
header = "('coal_plant', 'coal_market', 'west')"
to match flextool's MultiIndex parquet round-trip.
If solve_name is given, prepend a constant solve column
for fuller flextool-output compatibility.
constraint_dual ¶
Per-row dual values for a named constraint. Returns a frame
(over_dims..., dual) if the constraint had over= rows,
else a single-row scalar frame (dual,).
get_named_basis ¶
Extract the optimal basis as a name-keyed carrier (spec §4.1, §4.5).
Zips the live HiGHS basis (getBasis().col_status /
row_status) against this solution's rendered col_names /
row_names, storing each status as its integer enum value so
the carrier is portable across processes.
Synthetic row_<i> names are EXCLUDED from row_status — they
are positional, not stable across models (§4.5) — and from the
fingerprint, consistent with what the set-side computes.
Requires the solve to have retained the live HiGHS handle
(keep_solver=True, or a :class:WarmProblem solve); raises
:class:RuntimeError otherwise. Raises :class:ValueError if a
column name — or a non-synthetic row name — repeats (a duplicate
would silently overwrite its status in the dict).
CstrRecord ¶
Read-only metadata for a registered constraint family.
Returned by :meth:Problem.cstrs_named for emission-introspection
tests. proto carries the LHS Expr, sense and rhs
structures; most callers only need over (whose height is the
row count) and name.
Sum ¶
Aggregate an Expr. over lists the dims to sum out; the
remaining dims become the term's open dims. where is an index
frame that pre-filters the term frames (inner join on shared
columns) before the group-by-sum.
Sum(expr) with over=None collapses every open dim — useful
for a scalar (objective term, single-row constraint).
Any deferred :func:Where filters recorded on each term
(t.where_frames) are baked into t.lazy before the
aggregation — Sum collapses dims so the filter could not be
applied at the leaf level downstream. The result clears
var_source / where_frames (today's behaviour); the
per-term where kwarg is applied on top, unchanged.
Where ¶
Inner-join an Expr against frame. Two effects in one op:
- Filter — rows of the term whose shared-column values don't
appear in
frameare dropped (e.g.Where(v_flow, wind_only)keeps only the wind rows). - Map — any columns of
framethat the term doesn't already carry become new open dims of the resulting term (e.g.Where(v_flow, flow_to_n)whereflow_to_nhas columns(p, source, sink, n)addsnso the term can be bound to a constraint indexed by(n, t)).
The pure-filter case (no extras) does not bake the join into
t.lazy — it records frame into _Term.where_frames so
the LHS prune-down (_build_lhs_pruned_plan) can apply it at the
leaf-rebuild step (where the row count is bounded by the smaller of
Var.frame and the row_index key set). var_source / Param
chain / coef_scalar are preserved.
The map-effect case (extras non-empty) ALSO defers: frame is
recorded into _Term.where_map_frames as (frame_lf,
frozenset(extras)), the term's dims are extended with the extras,
and var_source / Param chain / coef_scalar / where_frames
are preserved. The deferred inner-join (which produces the extras
dim columns) is baked at leaf-rebuild time (or at Sum / Lag /
consumer fallback) via :func:_apply_where_map_frames. Param
multiplies AFTER the map-effect Where bake first iff their dims
overlap any pending extras (correctness); otherwise the deferral
propagates through.
Set POLAR_HIGH_DISABLE_WHERE_PUSHDOWN=1 to recover today's
behaviour (eager join + clear metadata) verbatim for BOTH branches.
Lag ¶
Return an Expr that, for each (carry_dims, time_dim) in
lag_frame, references var at (carry_dims, lag_col).
Used for shifting variables in time, e.g. for storage state-change:
v_state[n, d, t] - v_state[n, d, t_prev]
= v_state - Lag(v_state, dtttdt, "t", "t_prev_within_timeset")
lag_frame carries the (d, t, t_prev) lookup; carry_dims are
the columns shared between var and lag_frame other than the
time dim itself (typically d).
prewarm_global_scheduler ¶
Initialize HiGHS' process-global task scheduler ONCE, single-threaded,
so subsequent concurrent solves on distinct Highs instances need not
each call resetGlobalScheduler. Best-effort; returns False if any
highspy step fails (the caller then falls back to a sequential path).
Once this returns True the global scheduler is pinned to threads and
a subsequent run() with NO threads option inherits that pool — so
concurrent solves need not (and must not) pass threads per instance,
which would re-trigger resetGlobalScheduler and is unsafe to run
concurrently.
resolve_worker_count ¶
Clamp a requested worker count to [1, n_items].
workers is None auto-resolves to min(n_items, cpu_count - 1) (at
least 1) — leave one core for the main thread / OS. A non-positive request
means sequential (1).
solve_indexed_parallel ¶
solve_indexed_parallel(warmproblems: Sequence[WarmProblem], fn: Callable[[int], T], *, workers: int | None = None) -> list[T]
Run fn(i) for every index i of warmproblems deterministically,
optionally across a thread pool, and return the results in index order.
fn(i) is the caller's per-subproblem work: typically it pins the i-th
:class:WarmProblem's coupling columns, calls wp.solve(), and extracts
a domain result (objective + dual slopes). Because each WarmProblem owns
its own HiGHS handle, fn(i) for distinct i touch disjoint state and
are safe to run concurrently — provided every WarmProblem is already built
(the cold first build calls the process-global resetGlobalScheduler and
must run sequentially up front). This helper enforces that precondition.
Parameters¶
warmproblems
The per-index :class:WarmProblems. Each MUST already be built
(solve() called at least once); a ValueError is raised
otherwise. Only used for the built-precondition check + count — the
actual solving is delegated to fn.
fn
fn(i) -> result for index i. May run on a worker thread; must
confine its mutations to the i-th WarmProblem's HiGHS handle.
workers
Effective worker count (see :func:resolve_worker_count). None
auto-resolves to min(n, cpu_count - 1). <= 1 runs a fully
sequential path on the calling thread (no pool, no scheduler prewarm) —
byte-identical to a plain for loop.
Returns¶
list
[fn(0), fn(1), ..., fn(n-1)] — always in index order, independent of
thread timing.
Modules¶
polar_high.engine ¶
Generic polars-backed LP kernel.
Three primitives — Var, Param, Sum — and one container
(Problem). Knows nothing about flextool, energy systems, or any
specific model. A constraint is built as either:
-
an
Exprproduced by overloaded operators (v <= cap,Sum(...) >= rhs,lhs.eq(rhs)), passed positionally toProblem.add_cstr; or -
a labelled
termsdict, summed across all entries, with an explicitsenseandrhs. Use this when a constraint is naturally a sum of named contributions (storage transitions, sink flow, source flow, slack — like flextool's nodeBalance_eq).
A variable is a polars frame (*dims, col_id) — one LP column per
row. A parameter is a polars frame (*dims, value). Var * Param
joins on shared dims and emits an Expr term (*union_dims, col_id,
coef). Sum(expr, over=…) group-by-sums one or more dims; the
remaining dims become the constraint's row dims when the term is bound
to over= at add_cstr time.
Param ¶
Param(dims: tuple[str, ...], frame: DataFrame | LazyFrame, name: str | None = None, _sources: list[tuple[Param, int]] | None = None, _value_scalar: float = 1.0, _op: tuple | None = None)
A parameter table. frame carries columns *dims, value.
Stored internally as a polars.LazyFrame so that chained algebra
ops (Param * Param, Param + Param etc.) defer materialization
until a consumer reads .frame or the engine collects in
Problem.solve. The .frame property caches the eager
DataFrame on first read — flextool reads .frame.rename(...)
repeatedly off the same Param so we want that to be cheap.
name (optional) is a logical Param identifier (e.g. "p_inflow").
It is opt-in metadata used by :class:WarmProblem's Param-tracked
auto-update (declare_mutable / update_param). When unset,
Params are anonymous and carry no tracking overhead.
_sources records constituent named Params for composite results
of Param * Param / Param / Param. Each entry is
(param_name, dims_tuple, direction) where direction is +1 if the
Param contributes to the numerator and -1 if to the denominator.
Anonymous-only chains have _sources is None.
_op records the operation tree of a Param ⊗ Param composite
and of scalar multiples / quotients (see :func:_param_tree); None
for a leaf — a frame-constructed Param or a Param + Param /
Param + float result. :func:_prune_composite_lazy uses it to
rebuild an expanding product with every leaf semi-joined to a consuming
term's keys (same association, same per-cell arithmetic).
Var ¶
Var(name: str, dims: tuple[str, ...], frame: DataFrame, lower: float = 0.0, upper: float = float('inf'), integer: bool = False)
A variable family. frame carries columns *dims, col_id.
Var.frame stays an eager polars DataFrame — it's small (one row
per LP column), produced once in :meth:Problem.add_var, and
consumed by both flextool integration (v.frame["col_id"].unique())
and Problem.solve (col_id → bound/name lookups). Algebra ops on
Var lazify on the fly so the resulting _Term is lazy.
lower / upper are each EITHER a scalar (one bound shared by
every column of the family — the historical form, kept verbatim so
existing callers emit a byte-identical LP) OR a float64
np.ndarray of length frame.height holding one bound per
column, positionally aligned with frame rows (i.e. with
frame["col_id"]). :meth:Problem.add_var produces the array
form when a :class:Param bound is given. Bound consumers read
them through :meth:col_lower / :meth:col_upper (scatter-ready
values) and rescale them through :meth:scale_bounds.
bound_param_names records the .name of every named
:class:Param passed as a lower / upper bound. Bound Params
are resolved to numbers at :meth:Problem.add_var time and are NOT
tracked afterwards, so :meth:WarmProblem.declare_mutable /
:meth:WarmProblem.update_param refuse these names (change such
bounds with :meth:WarmProblem.set_col_bounds instead).
A Var built directly (not through :meth:Problem.add_var) may map
several frame rows to one col_id — e.g. a variable that keeps one
column over a run of time steps, indexed on every step. Terms built
from it are summed per (row, col) like any duplicate cell;
:attr:col_ids_unique tells the builders that a Sum over its rows may
combine coefficients.
col_ids_unique
property
¶
True when every frame row has its own col_id (always for a
family declared with :meth:Problem.add_var and its renamed
views). Computed once per Var.
has_elementwise_bounds
property
¶
True when lower or upper is a per-column array.
col_lower ¶
Lower bound in scatter-ready form: float for a scalar bound
(identical to the historical float(v.lower)), else the
per-column float64 array aligned with frame rows. Use as
col_lb[v.frame["col_id"].to_numpy()] = v.col_lower().
scale_bounds ¶
Multiply every FINITE bound value by factor in place.
Column-scaling hook (e.g. an autoscale Layer 2 column factor
f maps x = f * x_scaled ⇒ bounds scale by 1/f — the
caller passes the factor it wants applied). Infinite bounds are
left untouched. Scalar bounds stay scalars (float(b) * f,
the exact historical arithmetic); per-column arrays are scaled
element-wise with the same per-entry arithmetic, so each column
is scaled exactly as a scalar bound with that value would be.
SumBlockMeta
dataclass
¶
SumBlockMeta(var_source: Var, param_sources: tuple[tuple[Param, int], ...], coef_scalar: float, where_frames: tuple[LazyFrame, ...] | None, where_map_frames: tuple[tuple[LazyFrame, frozenset[str]], ...] | None, reduce_dims: tuple[str, ...], keep: tuple[str, ...])
Inert reconstruction recipe captured at Sum-time (Phase C-2).
When a block-eligible term (var_source present, non-empty
param_sources list, non-empty over) is reduced by
:func:Sum, the aggregation clears var_source and survivor-
filters param_sources on the returned term, discarding the
information needed to rebuild the pre-Sum row_index → Var → P1 →
P2 … chain and reduce it in-block. :class:SumBlockMeta snapshots
that pre-Sum state so a future block-COO classifier (Phase C-3) can
evaluate the Sum-wrapped chain by rebuilding from leaves and reducing
within the block.
All fields are captured from the PRE-Sum term, before any clearing or survivor-filtering:
var_source— the originating :class:Var.param_sources— the FULL pre-Sumlist[(Param, direction)], NOT survivor-filtered: block-COO needs every factor, including Params whose dims are summed out (e.g.p_unitsizeoverp).coef_scalar— the cumulative constant scalar folded into coef.where_frames— the pre-Sum deferred pure-filter frames.where_map_frames— the pre-Sum deferred map-effect frames.reduce_dims— the dims summed out (the Sum'sover).keep— the post-Sum open dims (the returned term's dims).
This class is currently INERT: nothing reads it (confirmed by grep).
It is set ONLY at the point of reduction in :func:Sum, never
propagated through later :class:Expr ops or a nested / re-reduced
:func:Sum (those set it to None), so a stale recipe can never
attach to an already-reduced term.
Expr ¶
A sum of terms (decision-variable contributions).
The terms can have different open-dim sets — they're concatenated,
not broadcast. Broadcasting happens once, at constraint emission,
via a join to the constraint's over= row index.
CstrRecord ¶
Read-only metadata for a registered constraint family.
Returned by :meth:Problem.cstrs_named for emission-introspection
tests. proto carries the LHS Expr, sense and rhs
structures; most callers only need over (whose height is the
row count) and name.
Problem ¶
LP container. Generic — no flextool-specific knowledge.
Construct an empty LP container.
Pure polar-high is a generic LP kernel; scaling decisions are
left to the caller. See :mod:polar_high.autoscale for the
opt-in autoscaler (Layer 1 detect + Layer 3 recommendation)
that callers (e.g. FlexTool) use to drive
user_bound_scale / user_objective_scale automatically.
dense_axes — the explicit client contract for the block-COO
LHS arm. When the client (e.g. FlexTool) declares the dense
trailing axes once here (e.g. Problem(dense_axes=("d", "t"))),
it makes a binding PROMISE about every frame it passes that
contains those columns:
the frame is globally lexicographically sorted by
``(other_dims_in_declared_order..., *dense_axes)`` — i.e. the
declared dense axes are the trailing sort keys, in the given
order, and the leading dims form a sorted prefix.
This lets block-COO slice the dense suffix of each Var with NO
re-sort (a re-sort would cost more than the multiply itself).
polar-high VERIFIES this promise cheaply (a single-pass monotonic
scan — see :func:_verify_dense_sorted) on every Var that the
block-COO arm classifies + fires on, and RAISES a clear
ValueError naming the Var if the client breaks it. Frames
that do not contain the dense axes (e.g. an investment Var
("p", "d") when dense_axes=("d", "t")) simply do not fire
block-COO and are unaffected. None (default) leaves the
block-COO arm dormant — it only fires once dense axes are declared
(here or via :meth:declare_dense_axes).
objective_scale
property
¶
The factor set by :meth:set_objective_scale (1.0 by default).
declare_dense_axes ¶
Declare the dense trailing axes for block-COO (see init contract).
Equivalent to passing dense_axes= to the constructor; provided so
callers that receive an already-constructed Problem (e.g. FlexTool's
build_flextool step, which builds the Problem first and populates
it afterwards) can declare them. Pass None to clear.
set_solver_options ¶
Store HiGHS options to be applied in solve(). Pass None
to clear. Keys are HiGHS canonical option names (presolve,
solver, parallel, time_limit etc); values must be
already coerced to the type HiGHS expects (str/int/float/bool).
Unknown keys are tolerated (a warning is emitted at solve time).
set_solver_option ¶
Set a single HiGHS option, leaving the rest untouched.
Convenience for callers that want to add one knob without
re-passing the whole dict (e.g. user_bound_scale set by
the autoscaler). Equivalent to a dict merge plus
:meth:set_solver_options.
get_solver_option ¶
Return the caller-set value of name (or None if unset).
Reads self._solver_options only — does NOT consult HiGHS
(the option may not have been pushed to a live Highs
instance yet). Returns None for unset options so callers
can use if get_solver_option(...) is not None to test
explicit setting.
add_var ¶
add_var(name: str, dims: tuple[str, ...] | str, index: DataFrame, lower: float | Param = 0.0, upper: float | Param = float('inf'), integer: bool = False) -> Var
Declare a variable family with one column per index row.
lower / upper are either scalars (one bound for every
column — the historical form, emitted byte-identically) or a
:class:Param giving per-element bounds:
- The Param's dims must be a SUBSET of
dims(ValueErrorotherwise). Its rows are aligned to the variable's index rows by joining on the Param's dims, so a lower-dim Param broadcasts over the remaining variable dims. - Variable elements with NO matching Param row get the default
bound (
lower0.0,upper+inf): a sparse bound Param only tightens the elements it has rows for. null/NaNParam values likewise mean "default".+inf/-infare allowed (e.g.lower=-inffrees an element).-
Duplicate Param rows on its dims raise
ValueError. -
Param rows matching no variable row are ignored (sparse tighten) — including rows whose key lies outside the variable's
pl.Enumvocabulary. Null keys never match, so such elements keep the default. AnEnumvsUtf8key dtype mismatch is aligned automatically;Enumkeys with disjoint (non-nested) category sets raiseValueError; other dtype mismatches (e.g.Utf8vsInt64) raise polars' join error. - A
+inflower or-infupper bound raisesValueError(Param element or scalar alike). An element withlower > upperis accepted here and makes the LP infeasible at solve time, exactly as a scalarlower > upperdoes.
Scalars may be any numbers.Real (int / float / numpy
scalar ...) or a 0-d numeric numpy array; bool is rejected.
Param bounds are resolved eagerly here into a per-column float64
array stored on :attr:Var.lower / :attr:Var.upper; column
order and ids are unaffected by the bound form. The values are
NOT tracked afterwards: the names of named bound Params are
recorded on :attr:Var.bound_param_names and
:meth:WarmProblem.declare_mutable / :meth:WarmProblem.update_param
raise for them — change bounds on a built problem with
:meth:WarmProblem.set_col_bounds.
add_cstr ¶
add_cstr(name: str, *, over: DataFrame | None = None, sense: str, lhs_terms: dict[str, Var | Expr | Param | int | float], rhs_terms: dict[str, Var | Expr | Param | int | float] | None = None) -> None
Add a constraint of the form Σ lhs_terms sense Σ rhs_terms.
Each term entry is either
- a
VarorExpr— variable contribution, or - a
Param,intorfloat— constant contribution.
The engine sorts variables and constants out per side, builds
(lhs_var − rhs_var) sense (rhs_const − lhs_const), and adds
the row to highspy at solve time. Labels (the dict keys) are
used in row names and diagnostics.
cstr_names ¶
All constraint family names currently registered, in declaration order. Useful for emission audits and debugging.
cstrs_named ¶
Return constraint metadata records matching name.
An exact-name match returns the single record; otherwise a prefix
match returns every record whose name starts with name + "_"
(so passing "minimum_uptime" returns both
minimum_uptime_linear and minimum_uptime_integer).
Each :class:CstrRecord carries:
* name: full registered name of the constraint family;
* over: the polars DataFrame of axis tuples (len(over)
is the row count);
* proto: the underlying _CstrProto (expr, sense,
rhs) for advanced introspection.
set_named_basis ¶
Record a warm-start basis to inject at the next solve (spec §4.1).
This is the in-process (non-save_memory) primitive: it only
RECORDS INTENT — it stores the carrier on self._warm_basis and
the policy on self._warm_basis_policy. It does NOT build or
inject anything. The solve-time hook (streaming path, just before
h.run()) constructs the sized HiGHS basis via
:func:polar_high._warm_basis.build_highs_basis and calls
h.setBasis.
policy="exact" transfers 1:1 by name and requires the carrier's
fingerprint to match this model's; a mismatch under "exact" is
expected to be caught by the hook, which then falls back safely to
a cold solve. policy="alien" maps shared names and lets HiGHS
repair the remainder.
basis_name_fingerprint ¶
Structural fingerprint of THIS model's LP name-set (spec §4.5).
A cheap, side-effect-free accessor the orchestrator uses as the
cache key for a captured :class:NamedBasis: it is the
:func:polar_high._warm_basis.basis_fingerprint of this model's
rendered column + constraint-row names, so a basis captured off one
build and re-injected into an identically-shaped build key on the
same string. It mirrors what :meth:write_mps stashes as
_last_mps_fingerprint (same basis_fingerprint(col_names,
m.row_names) inputs) but is callable without writing an MPS.
Uses the canonical matrix's names (m = self.canonicalise());
canonicalise is memoized, and LP names are Layer-2-invariant, so
the value is valid pre- or post-scaling and stable across repeated
calls. Returns None when the problem has been released or has no
columns (no meaningful key).
seed_primal ¶
seed_primal(values: dict[str, float], *, frame: Literal['user', 'scaled'], missing: str = 'skip') -> None
Record a partial primal seed to inject at the next solve.
Phase 3 Step 3a primitive. Like :meth:set_named_basis this only
RECORDS INTENT — it stores the seed on self._primal_seed and the
frame on self._primal_seed_frame. It does NOT build or inject
anything: the solve-time hook (streaming path, just before
h.run(), immediately after the basis block) resolves each name to
a column index, converts + sanitises the value, and calls
h.setSolution.
values maps column name (rendered exactly as
Solution.col_names — "name[d0,d1,...]" for indexed vars, the
bare name for scalars) to a numeric value. A partial seed is
legal and expected: names present in the model are seeded, names
absent are silently skipped (the subset behaviour, missing="skip").
frame is REQUIRED and has NO default (a silent default frame is a
corruption vector). (The :class:WarmProblem update_rhs /
update_obj_coef methods do default to frame="user": that is
backward compatible when no Layer-2 side vectors are set, and under
Layer 2 it is the deliberate fix — the values mean what they mean at
build.) frame="scaled" uses each value as-is in the
solver's post-Layer-2 scaled space. frame="user" converts with
the model's own Layer-2 side-vector
(x_scaled = x_user / _layer2_col_factor[col_id]; identity when
Layer 2 was not applied), so callers can seed in natural units.
Feasibility-safety: a primal seed is only a starting point for the solver — it never binds the optimum. A non-optimal (or even infeasible-looking) seed cannot change the objective HiGHS returns, with presolve on or off.
MUTUALLY EXCLUSIVE with a warm basis. setSolution and
setBasis clobber each other on the same handle (last writer wins;
setSolution invalidates a previously-set basis). For an LP a
basis dominates a partial seed, so the hook enforces the rule "if a
basis is set, it wins": when self._warm_basis is also set the seed
is skipped.
In-process (streaming) ONLY — there is no subprocess / MPS channel
for a primal seed; a save_memory solve ignores the seed.
A seed over integer columns is a MIP-start. (Noted for a future UC5; not implemented or specialised here — integer columns are seeded the same way and HiGHS treats them as a MIP-start.)
cstr_row_count ¶
Total LP-row count across all constraint families matching
name (exact or prefix; see :meth:cstrs_named). Returns 0
when no families match — letting callers distinguish "absent"
from "empty" without exception handling. A scalar constraint
(over=None) counts as one row.
add_obj_constant ¶
Accumulate a constant into the objective offset.
The constant is in the same units as the objective given to
:meth:set_objective (scaled by :meth:set_objective_scale like
the rest of the objective). It shows up in Solution.obj even
though no decision variable carries it. polar-high keeps it out of
HiGHS and adds it to the objective HiGHS reports, so HiGHS'
user_objective_scale and its MIP gaps see only the variable part
of the objective (a raw Solution.highs.getObjectiveValue()
excludes it). Used for pure-Param objective terms like an existing
entity's fixed cost.
set_objective_scale ¶
Declare that the objective given to :meth:set_objective and
:meth:add_obj_constant is factor times the objective in user
units, the units the caller thinks in (e.g. 1e-6 for a cost in currency built as
million currency).
Objective-valued solver options (targets, bounds, cutoffs, absolute
gaps; see :mod:polar_high.objective_options) are then read in user
units: polar-high multiplies them by factor (and by
HiGHS' 2**user_objective_scale) before they reach HiGHS, and
:meth:objective_transform gives the same mapping for other
solvers. Solution.objective_in_user_units is Solution.obj /
factor. factor must be finite and > 0; the default is 1.0.
objective_transform ¶
The :class:~polar_high.objective_options.ObjectiveTransform that
maps an objective value in user units to solver's value
for this Problem.
solver: "highs", "gurobi", "cplex", "xpress" or
"copt". io: "direct" (the solver gets the model through
its API, e.g. in-process HiGHS or polar_high.solvers.solve) or
"mps" (the solver reads a file from :meth:write_mps, which
does not carry the objective constant). options: the option
dict the solver gets; for HiGHS its user_objective_scale = N
gives the factor 2**N (None reads the options stored with
:meth:set_solver_options). Every other solver has factor 1,
whatever the options hold.
solve ¶
solve(*, options: dict | None = None, keep_solver: bool = False, streaming: bool = True, save_memory: bool = False, tmp_dir: str | PathLike | None = None, highs_names: str = 'always', pre_run_check: Callable[[dict, Highs], Mapping[str, Any] | None] | None = None) -> Solution
Solve the LP and return a :class:Solution.
Parameters¶
options
Per-call HiGHS options dict (overrides set_solver_options).
Objective-valued options (objective_target,
objective_bound, mip_abs_gap) are read in user units and
in the objective's sense and mapped to HiGHS' (see
:meth:set_objective_scale and
:mod:polar_high.objective_options); the dict is not changed.
keep_solver
When True, the live HiGHS instance is kept on the returned
:class:Solution so callers can inspect it post-solve (e.g.
sol.highs.writeModel("model.mps")). Default False —
the C-side LP storage is released as soon as primal/dual/
objective have been extracted.
streaming
When True (default), columns are added once via addCols
and each constraint family is emitted to HiGHS via addRows
immediately after its COO triples are built; the family's local
arrays then go out of scope before the next family is processed.
This caps peak memory at one family's COO + the running HiGHS
LP. When False, the entire model is assembled into a single
:class:highspy.HighsLp and loaded via passModel —
numerically identical results either way; False is mostly
useful for benchmarking the legacy path.
save_memory
Single one-shot knob that trades wall time for peak RSS. When
True, two things happen right before HiGHS.run():
1. polar-high's polars/numpy LP source-of-truth (constraint
family ``Expr.terms`` lists, ``_CstrProto.rhs`` Param
frames, objective ``_Term.lazy`` plans, the caller-side
``col_names`` / ``row_names`` lists) is dropped. Only the
per-:class:`Var` ``col_id`` frames survive — they are
needed by :meth:`Solution.value` to map column indices
back to user-space dim tuples.
2. The HiGHS instance is round-tripped through disk: the LP
is written to a temp MPS file, the original ``Highs()`` is
cleared and discarded, ``malloc_trim(0)`` is called (best
effort, Linux only), a fresh ``Highs()`` is created, the
same solver options are re-applied, and the MPS file is
read back. This resets the HiGHS allocator's high-water
mark — the C++ side accumulates ~5 GB of slack from the
incremental ``addRows`` loading path that ``readModel``
avoids by sizing once up front.
Cost: ~+90 s of MPS file I/O at N=3000 dense. Benefit: peak
RSS drops by ~5 GB on the same problem. Intended for one-
shot single-solve benchmarks where warm-start / re-solve
isn't needed. After the call returns, the :class:`Problem`
is in a "released" state and any further :meth:`solve` call
raises ``RuntimeError`` (the polar-side source AND the
original HiGHS instance with its basis have both been
discarded, so neither a fresh re-solve nor a WarmProblem-
style update is possible). Honoured only by the streaming
path; on ``streaming=False`` a warning is emitted and the
flag is ignored. Default ``False``.
tmp_dir
Directory for the temporary MPS file written by the
save_memory=True round-trip. None (default) uses
the system temp dir ($TMPDIR / /tmp). Set when the
caller wants the spill file on a specific volume (e.g. the
same filesystem as its workspace, or a per-job scratch
directory) instead of the system default. Ignored when
save_memory=False.
highs_names
When the column / row names are passed to HiGHS (see
:data:HIGHS_NAME_OPTIONS): "always" (default), "auto"
(just before run(), only if a name-writing HiGHS option is
on) or "never". HiGHS does not use names to solve, so the
solution is the same in every mode; leaving them out saves
HiGHS' copy of every name. Solution.col_names /
row_names are unchanged; Solution.highs_names_passed
says whether the kept HiGHS instance has them, and
:meth:Solution.pass_names_to_highs passes them later. A
save_memory solve always passes them: its MPS round trip
through HiGHS carries the names.
pre_run_check
Called once, right before HiGHS.run(), as
pre_run_check(ranges, highs): ranges is the
streamed_lp_ranges dict of the LP HiGHS holds, with the
scaling-check keys added (see :attr:Solution.streamed_lp_ranges:
col_bound_meaningful / row_bound_meaningful, (b_lo*,
b_hi) over the continuous-column bounds and the row bounds below
infinite_bound, the small end leaving out entries whose model-
unit magnitude is below residue_floor = 10 × the primal
feasibility tolerance in the effective options;
bound_rhs_residue (count, largest model-unit magnitude);
bound_rhs_beyond_infinite; residue_floor;
infinite_bound); highs is the live handle. It may set
options on highs (e.g. user_bound_scale), return a mapping
of HiGHS options to set, or raise — then no solve runs and the
exception propagates. The ranges also carry the
matrix_continuous / integer_matrix, cost_continuous /
integer_cost and integer_bound (min, max) pairs (HiGHS
multiplies an integer column's matrix entries and cost by
2**user_bound_scale). The names decision of
highs_names="auto" is taken after the hook, so an option it
sets that writes names is honoured. Honoured by the streaming
path only (ValueError with streaming=False); also taken by
:meth:build_only and by the first :meth:WarmProblem.solve.
build_only ¶
build_only(mps_path: str, *, options: dict | None = None, pre_run_check: Callable[[dict, Highs], Mapping[str, Any] | None] | None = None) -> dict[str, Any]
Build the LP into HiGHS, write to mps_path, release everything.
For callers that want to drive the actual solve out-of-process — typically a subprocess HiGHS reading the MPS file in a clean address space. After this call:
mps_pathexists and contains the LP in MPS format.- polar-side LP source-of-truth has been dropped
(:meth:
_release_python_lp_inputs). - The live :class:
highspy.Highsinstance has been torn down and the glibc allocator trimmed (Linux best-effort). self._releasedis True — further calls to :meth:solvewill raise.
What stays alive: self._vars and each :class:Var.frame,
so the caller can later construct a :class:Solution from
externally-produced col_value / row_dual arrays via
Solution(..., vars=dict(self._vars), ...).
Honoured only on the streaming path; the non-streaming
(passModel) path is not supported here. Raises
RuntimeError if called on an already-released Problem.
Parameters¶
mps_path
Where to write the MPS file. Caller owns the file —
polar-high will not delete it.
options
HiGHS solver options to apply during the build (mainly
presolve / solver / simplex_scale_strategy —
options that affect what writeModel serialises).
None uses :attr:_solver_options.
pre_run_check
As on :meth:solve: called once with the ranges of the built
LP and the HiGHS handle, before the MPS file is written; it may
raise (no file is written). The MPS file carries no options, so
the options the hook returns or sets on the handle are returned
for the caller to pass to its solver.
Returns¶
dict
The HiGHS options pre_run_check returned or set on the
handle (those set on the handle are read back from highspy's
HighsOptions record, so return an option to be sure it is
carried); empty without a hook.
canonicalise ¶
Build (or return cached) the canonical CSC matrix + metadata.
Idempotent: returns the cached _matrix unless
_canonical_dirty is set (which add_var / add_cstr
flip). Side vectors (_layer2_col_factor /
_layer2_row_factor) are baked into the returned arrays at
build time per orchestrator decision D8.
write_mps ¶
write_mps(path: str | PathLike, *, free_format: bool = True, column_order_strict: bool = True, emit_names: bool = True, release: bool = False, name: str = 'POLAR_HIGH', pre_run_check: Callable[[dict, Any], Mapping[str, Any] | None] | None = None) -> dict[str, Any]
Stream the LP/MIP to path as a free-format MPS file.
This is an in-house writer that consumes the polar-side LP
source-of-truth (_vars, _cstrs, _obj_terms) directly
and emits MPS without ever instantiating :class:highspy.Highs
or building an :class:LpView. The motivation is peak-memory:
HiGHS' own writeModel serialiser materialises ~20× more
transient state than the in-process LP itself, which OOMs on
very large LPs (~10 M rows × 5 M cols × 20 M nonzeros). This
writer targets ~2-3 GB peak on that same LP.
Parameters¶
path
Output MPS file path. Caller owns the file; this writer
does not delete on failure.
free_format
Reserved for future use. Currently always free-format MPS
(the modern default — HiGHS, Gurobi, CPLEX, Xpress all
accept it). Fixed 8-char-column MPS is not yet implemented.
column_order_strict
When True (default), all nonzeros for a single column
are emitted contiguously in the COLUMNS section, sorted by
(col_id, row_id). This is the spec-compliant ordering
every MPS reader accepts. False is currently not
implemented — see :ref:Open follow-ups in the module docs.
emit_names
When True (default), row and column names come from the
variable family name + dim tuple and the constraint family
name + dim tuple (matching the format used by
:meth:Problem.solve for Solution.col_names /
Solution.row_names). When False, generic
C0000001 / R0000001 names are emitted instead —
saves ~30-50% file size at the cost of losing the
dim-tuple → column mapping in the MPS itself. Callers
using emit_names=False must use index-based solution
readback (column 0 in the MPS is column 0 in the polars
Var.frame ordering, etc.). The objective row name is
always cost regardless — it's a reserved unique name.
release
When True, calls :meth:_release_python_lp_inputs
after the write completes. Mirrors the
solve(save_memory=True) semantics — drops the polars
LazyFrames, constraint family list, and rhs Param frames
while keeping self._vars (so solutions read back from an
external solver can still be mapped to user-space dim
tuples via the surviving Var.frame["col_id"] columns).
After release the :class:Problem is no longer solvable —
:meth:solve will raise.
name
MPS NAME header; default "POLAR_HIGH".
pre_run_check
As on :meth:solve, for a caller that solves the file in
another process: called once as pre_run_check(ranges, None)
before anything is written, with the ranges of the LP in the
file (the same keys as on :meth:solve, read from the
canonical matrix as a :class:WarmProblem build does). There
is no solver instance (None), so the hook returns the
options it wants; they are returned for the caller to pass to
its solver. An exception from it propagates and no file is
written.
Returns¶
dict
The options pre_run_check returned (empty without one).
Raises¶
RuntimeError
If called on a :class:Problem whose source has already
been released (by an earlier solve(save_memory=True),
build_only(...), or write_mps(..., release=True)).
ValueError
If any coefficient is NaN or infinite (silent filtering
would hide model bugs). The error message identifies at
least one offending (col_id, row_id) pair.
OSError
File I/O failures (passed through from open()).
Notes¶
- The MPS format has no portable encoding for an objective
constant offset. If
self._obj_offset != 0.0this method emits a :class:UserWarningand writes the LP without the offset. Solutions from any downstream solver will then be off byself._obj_offset— the caller must add it back manually if needed. - Integer columns are bracketed with
MARKER 'INTORG'/MARKER 'INTEND'in the COLUMNS section, per MPS convention. - The writer is single-pass per family and uses one global polars sort to put the COLUMNS section into column-major order. Streaming-engine fallback to disk activates automatically for triple counts that don't fit in RAM.
- Stashes
self._last_mps_fingerprint— a structural basis-fingerprint (:func:polar_high._warm_basis.basis_fingerprint) over the model's real column/constraint-row name-set — computed before anyrelease, and the same whetheremit_namesis True or False (the generic names are never hashed). Consumed by thesave_memorysubprocess warm-start arm to key a basis-cache file. - Every column is written in the COLUMNS section, in
col_idorder. A column with no matrix entry and no objective coefficient gets one explicitcost 0entry, so MPS readers keep it in place with its bounds and integrality (a column present only in BOUNDS is moved to the end or dropped by some readers, e.g. HiGHS).
SolveDiagnostics
dataclass
¶
SolveDiagnostics(model_status: HighsModelStatus, model_status_name: str, primal_feasible: bool, dual_feasible: bool, num_primal_infeasibilities: int, num_dual_infeasibilities: int, max_primal_infeasibility: float, max_relative_primal_infeasibility: float, max_dual_infeasibility: float, primal_dual_objective_error: float, objective_value: float, mip_gap: float = math.inf, user_bound_scale: int = 0, user_objective_scale: int = 0, user_scaled_num_primal_infeasibilities: int | None = None, user_scaled_max_primal_infeasibility: float | None = None, user_scaled_num_dual_infeasibilities: int | None = None, user_scaled_max_dual_infeasibility: float | None = None)
Normalised snapshot of a solve's HiGHS status + info counters.
Read once from a live highspy.Highs (a single getInfo() call
plus getModelStatus()), so the numbers are a consistent atomic
view of one solve. Consumers use this to decide whether a solve that
HiGHS could NOT certify as kOptimal (e.g. an interior-point solve
run without crossover, whose post-solve dual objective is slightly
inconsistent) is nonetheless usable in practice — the primal solution
is feasible and the primal--dual objective gap is small.
This is a pure fact carrier: it holds no policy and no thresholds. The accept/reject decision (and any tolerance) lives with the caller.
mip_gap is HiGHS info.mip_gap: the relative gap
|primal bound - dual bound| / |primal bound| of a MIP solve, as
HiGHS's log prints it and its mip_rel_gap stop tests it (it
includes the objective offset, and is unchanged by
user_objective_scale). inf for an LP, for a MIP without a
feasible solution (HiGHS's NaN there is normalised to inf),
and when not recorded (the default).
The plain infeasibility counters are HiGHS' own, and HiGHS measures them
on the solution it returns: after it has removed any user scaling
(user_bound_scale / user_objective_scale). Its feasibility
tolerance is not rescaled, so with user_bound_scale = N < 0 a primal
residual within the tolerance in the space HiGHS solved reads 2**-N
times larger after unscaling. The user_scaled_* fields give the
counters in the user-scaled space HiGHS solved — rebuilt exactly from
the unscaled solution, see :func:user_scaled_kkt_failures — and equal
the plain ones when no user scaling was applied. They are not recorded
(None) for a kOptimal solve with user scaling, which HiGHS
certified in the space it solved.
user_bound_scale / user_objective_scale are the exponents the
handle held at snapshot time (0 = none). The user_scaled_* fields
are None when not recorded (the default); the dual pair is also
None when the solve has no dual solution. The relative primal
infeasibility stays HiGHS' unscaled one.
Solution ¶
Solution(*, optimal: bool, obj: float, col_value: ndarray, row_dual: ndarray, col_names: list[str], row_names: list[str], vars: dict[str, Var], col_dual: ndarray | None = None, highs: Highs | None = None, streamed_lp_ranges: dict | None = None, highs_names_passed: bool = True, highs_name_passer: Callable[[], None] | None = None, obj_offset: float = 0.0, objective_scale: float = 1.0)
Read-only view of the solved LP. Look up variable values by
name; values come back as a polars frame (*dims, value).
objective_in_user_units
property
¶
obj in user units: obj divided by the
Problem's :meth:~Problem.set_objective_scale factor.
max_primal_infeasibility
property
¶
Largest primal-constraint violation in the returned solution.
A solver returns a vertex that sits within its feasibility tolerance
of the active constraints, so a hand-rolled feasibility re-check on the
solution (f <= cap, balance rows, …) must use a tolerance at least
this large — a hard-coded magic constant either masks real violations
or trips on the normal solver slack. HiGHS enforces feasibility on the
INTERNALLY-SCALED problem, so the unscaled slack reported here can
exceed :attr:primal_feasibility_tolerance.
0.0 when no live solver is attached (a synthesised Solution).
primal_feasibility_tolerance
property
¶
The solver's primal feasibility tolerance for this solve (the
nominal, scaled-problem tolerance; see
:attr:max_primal_infeasibility for the achieved unscaled slack).
0.0 when no live solver is attached (a synthesised Solution).
pass_names_to_highs ¶
Pass the LP names to the kept HiGHS instance, if the solve left
them out (highs_names="auto" / "never"). Idempotent; a
no-op when the names are already there or no instance is kept.
Call before highs.writeModel / writeSolution / writeBasis
when those files should carry the LP names.
solve_diagnostics ¶
Snapshot the solve's HiGHS status + feasibility/optimality counters.
Returns a :class:SolveDiagnostics read from the live solver, or
None when no queryable HiGHS handle is attached — i.e. a
synthesised :class:Solution (highs is None) OR a read-only
shim that reconstructs a solution from a solver .sol file
(the subprocess / commercial path) and does not implement
getInfo / getModelStatus. None means "cannot
diagnose", which callers must treat as un-verifiable — never as
an implicit pass.
Follows the highs is None → neutral guard of
:attr:max_primal_infeasibility, but extends it to the shim case
via hasattr so no AttributeError can leak from the
subprocess path.
The result is cached on the Solution (keyed on a weak reference to the handle, so the cache never keeps the handle alive), so a second call is free; it describes the solve this Solution came from.
value_wide ¶
value_wide(var_name: str, time_dims: tuple[str, ...] = ('d', 't'), solve_name: str | None = None) -> pl.DataFrame
Wide-form, flextool-compatible: time dims become rows, the remaining dims are encoded as a tuple-stringified column header.
For a 2-d variable like vq_state_up(n, d, t):
long : rows = (n, d, t, value)
wide : rows = (d, t) + one column per n (header = "west").
For a 5-d variable like v_flow(p, source, sink, d, t):
wide : rows = (d, t) + one column per (p, source, sink),
header = "('coal_plant', 'coal_market', 'west')"
to match flextool's MultiIndex parquet round-trip.
If solve_name is given, prepend a constant solve column
for fuller flextool-output compatibility.
constraint_dual ¶
Per-row dual values for a named constraint. Returns a frame
(over_dims..., dual) if the constraint had over= rows,
else a single-row scalar frame (dual,).
get_named_basis ¶
Extract the optimal basis as a name-keyed carrier (spec §4.1, §4.5).
Zips the live HiGHS basis (getBasis().col_status /
row_status) against this solution's rendered col_names /
row_names, storing each status as its integer enum value so
the carrier is portable across processes.
Synthetic row_<i> names are EXCLUDED from row_status — they
are positional, not stable across models (§4.5) — and from the
fingerprint, consistent with what the set-side computes.
Requires the solve to have retained the live HiGHS handle
(keep_solver=True, or a :class:WarmProblem solve); raises
:class:RuntimeError otherwise. Raises :class:ValueError if a
column name — or a non-synthetic row name — repeats (a duplicate
would silently overwrite its status in the dict).
WarmProblem ¶
Warm-update wrapper around a :class:Problem.
Build a :class:Problem as usual (add_var, add_cstr,
set_objective). Wrap with :class:WarmProblem, then alternate
update_* calls and solve() calls — the LP is built ONCE and
only the changed coefficients / RHS values are pushed to HiGHS
between solves.
Typical usage (re-solving the same LP with new data)::
wp = WarmProblem(p)
sol_0 = wp.solve()
for r in range(1, n_cases):
# user-space values on the build labels (or positional arrays)
wp.update_rhs("balance", demand_param_for_case[r])
wp.update_obj_coef("v_flow", cost_param_for_case[r])
sol_r = wp.solve()
The update_* calls are O(rows_or_cols_in_family); the solve()
benefits from HiGHS's hot-start (basis is preserved across calls).
Value frames. :meth:update_rhs, :meth:update_obj_coef and
:meth:update_obj_coef_array take values in user space by default
(frame="user"): the Layer-2 side vectors set on the Problem are
applied, exactly as at build, so the update means the same as
rebuilding with the new values. frame="scaled" writes values
as given. :meth:update_param recomputes cells from factors cached
in the scaled space, so it takes user-space Param values too. The
column-bound and cut APIs (:meth:fix_cols, :meth:fix_col_ids,
:meth:set_col_bounds, :meth:add_cut_row) write in the scaled
frame: the caller applies any Layer-2 transform itself.
The live LP keeps the labels of the Problem it was built from (the
Var frames and the HiGHS column / row names). The update_*
calls change values, never labels: a :class:Param is matched to rows
and columns by those build labels.
highs_names chooses when the column / row names go to HiGHS:
"always" (default; right after the build), "auto" (before a
run() at which an option in :data:HIGHS_NAME_OPTIONS is on) or
"never". HiGHS solves the same LP without them; leaving them out
saves its copy of every name. Solution.col_names / row_names
are the same in every mode, and :meth:pass_names_to_highs passes the
names on demand (e.g. before Solution.highs.writeModel).
problem
property
¶
The underlying :class:Problem.
Useful for diagnostics that need to inspect the un-built LP.
For coefficient ranges, prefer
:attr:Solution.streamed_lp_ranges on a returned
:class:Solution (populated during the streaming solve at zero
extra cost).
set_named_basis ¶
Record a warm-start basis to inject on the fresh build (spec §4.1).
RECORDS INTENT only — stores the carrier on self._warm_basis and
the policy on self._warm_basis_policy; it does NOT build or
inject anything here. The actual h.setBasis fires exactly once,
in :meth:_initial_build, immediately before the first solve's
h.run(). This is the in-process default-path equivalent of the
streaming basis hook in :meth:Problem._solve_streaming.
policy="exact" transfers 1:1 by name and requires the carrier's
fingerprint to match this model's; a mismatch under "exact" is
detected in _initial_build and safely falls back to a cold solve.
policy="alien" maps shared names and lets HiGHS repair the rest.
The inject fires only on the fresh build — _initial_build runs
once per WarmProblem lifetime (solve calls it only while
self._h is None), so a subsequent height-match reuse re-run never
re-injects.
update_rhs ¶
update_rhs(cstr_name: str, new_param: Param | float | int | ndarray, *, frame: Literal['user', 'scaled'] = 'user') -> None
Replace the RHS of constraint family cstr_name with values
drawn from new_param.
new_param may be a :class:Param whose dims match a subset
of the constraint's over= axis (broadcasting to the rest; the
values are matched by the over labels), a scalar (broadcast to
all rows), or a numpy array (positional — length must equal the
family's row count, in the order of the original over frame).
frame="user" (default; backward compatible when no Layer-2 side
vectors are set, a deliberate change under Layer 2, where earlier
releases wrote the values unscaled): the values are in user space, as
the RHS terms passed to :meth:Problem.add_cstr; the Layer-2 row
factor of each row is applied, exactly as at build.
frame="scaled": the values are already in the solver's scaled
space and are written as given.
The replace is total: the new vector is the whole RHS of each
row, so any constant the build put into the row (from other RHS
terms or constant LHS terms) must be included by the caller.
The small-value floor (Problem.coef_zero_threshold) is judged
on the user-space magnitude in both frames, as at build. ±inf
one-sided sentinels are preserved. With no side vectors set both
frames push the same arrays as earlier releases.
update_obj_coef ¶
update_obj_coef(var_name: str, new_param: Param | float | int, *, frame: Literal['user', 'scaled'] = 'user') -> None
Replace the objective coefficient on every column of var_name.
Assumes the objective contribution from var_name is exactly
coef[*dims] * var[*dims] for some coef Param; this method
OVERWRITES that coefficient via h.changeColsCost. If the
objective also has contributions from this variable through more
complex algebra (e.g. var * unitsize * slope), the update is
still valid as long as new_param carries the full product —
the caller is responsible for collapsing multi-Param products
ahead of the call.
This DOES NOT touch the cost coefficients of other variables.
frame="user" (default; backward compatible when no Layer-2 side
vectors are set, a deliberate change under Layer 2, where earlier
releases wrote the values unscaled): the coefficients are in user
space, as the objective terms passed to :meth:Problem.set_objective; the
Layer-2 column side vector is applied, exactly as at build.
frame="scaled": the coefficients are already in the solver's
scaled space and are written as given. With no side vectors set
both frames push the same values as earlier releases. Values of a
:class:Param are matched to the columns by the variable's dim
labels; columns without a match get 0.0.
update_obj_coef_array ¶
update_obj_coef_array(var_name: str, dim_tuples: list[tuple], values: ndarray, *, frame: Literal['user', 'scaled'] = 'user') -> None
Array-form of :meth:update_obj_coef.
dim_tuples is a list of dim-value tuples (one per cell) for
variable var_name; each tuple must have one entry per dim
in the var's declared signature. values is a same-length
numpy array of new objective coefficients.
The columns are resolved positionally: values[k] becomes the
new objective coefficient on the column whose dim-tuple is
dim_tuples[k]. Vectorised — a single changeColsCost
call regardless of cell count.
frame as in :meth:update_obj_coef: "user" (default)
applies the Layer-2 column side vector, "scaled" writes the
values as given.
fix_cols ¶
Fix the listed columns of var_name to the given values.
For each (dim_tuple, value) pair, sets both the column's
lower and upper bound to value (so the LP has no choice
but to set the column at that level). Used by the Lagrangian
primal-recovery step ("fix-and-resolve"). Vectorised — single
changeColsBounds call.
Scaled frame: values are written verbatim as column bounds of
the live (possibly Layer-2 scaled) model; any caller-side scaling
must be applied beforehand. Unlike :meth:update_obj_coef, there
is no frame="user" conversion.
fix_col_ids ¶
Fix columns BY RAW COL ID: set lower = upper = value for each
(col_id, value) pair.
The col-id-level counterpart of :meth:fix_cols — the same
vectorised changeColsBounds(lo=hi=value) write, but skipping
the per-call dim-tuple → col-id join for callers that already
hold the raw ids (resolve once via :meth:col_id_of_var, or use
ids returned by :meth:add_recourse_col). Typical consumer: a
cutting-plane driver temporarily pinning coupling columns.
Bounds are written verbatim into the live model — any caller-side
scaling must be applied to values beforehand (scaled frame;
there is no frame="user" conversion here).
get_col_bounds ¶
Read the current (lower, upper) bounds of the given raw col
ids from the live model.
Returns two float64 arrays aligned POSITIONALLY with col_ids
(any order, repeats allowed — HiGHS' getCols itself requires
an ordered duplicate-free index set, so the query goes through
the unique sorted set and gathers back to caller order).
Save/restore companion of :meth:set_col_bounds for temporary
pins made with :meth:fix_col_ids::
lo, hi = wp.get_col_bounds(ids) # save
wp.fix_col_ids(ids, values) # temporary pin
... wp.solve() ...
wp.set_col_bounds(ids, lo, hi) # restore
set_col_bounds ¶
Write (lower, upper) bounds onto the given raw col ids.
Restore companion of :meth:get_col_bounds (see there for the
save/pin/restore pattern). Vectorised — single
changeColsBounds call; ±inf entries round-trip unchanged
(HiGHS' bound sentinel IS the IEEE infinity).
Scaled frame: the bounds are written verbatim into the live model
(as read back by :meth:get_col_bounds); there is no
frame="user" conversion here.
update_coef ¶
Update a single (row, col) coefficient in the constraint
matrix. Use :meth:row_id_of_cstr / :meth:col_id_of_var to
resolve indices semantically.
add_cut_row ¶
Append a >= constraint row to the live (already-built) LP and
return the new row's id.
Appends Σ coefs[k] · x[col_ids[k]] >= lower to the live HiGHS
model via addRow. This is a POST-build mutation of the warm
_h handle (the same class of live edit as :meth:update_coef;
unlike :meth:update_rhs, which defaults to the user frame, the
row is written in the scaled frame); it deliberately bypasses the build-time
Problem.add_cstr DSL lock, which only guards the fixed-size
Layer-2 autoscale side vectors and is irrelevant once the model is
built. The caller is responsible for keeping the master autoscale
OFF (or for pre-scaling coefs) so the appended row lives on the
built columns' scale — there is no auto-scaling for appended rows.
col_ids may mix existing column ids (resolve via
:meth:col_id_of_var) and ids of columns previously appended with
:meth:add_recourse_col. A subsequent :meth:solve warm
re-optimises the grown model; the appended row's dual is then
readable on the returned :class:Solution by row_dual[row_id]
(read BY ROW ID, not via :meth:Solution.constraint_dual, which
only knows named build-time rows).
Bumps the cached _n_rows (the zero-fill fallback size in
:meth:solve) and appends a generated name to _row_names (kept
index-aligned so name-indexed reads stay correct). Returns the
row_id HiGHS assigned (== the pre-append getNumRow()).
add_recourse_col ¶
Append a single free/bounded column (e.g. a Benders recourse
η) to the live LP and return its col id.
Symmetric with :meth:add_cut_row: a POST-build addCol on the
warm _h handle, bumping the cached _n_cols (zero-fill
fallback size) and _col_names. ±np.inf bounds are mapped to
the HiGHS kHighsInf sentinel. The returned id can be referenced
from a later :meth:add_cut_row; its value is read on the
:class:Solution by col_value[col_id] (appended columns are not
in any Problem._vars frame, so :meth:Solution.value cannot see
them — read by id).
compact_cuts ¶
compact_cuts(solution: Solution, *, policy: str = 'slack', trial_col_values=None, tol_rel: float = 1e-07, verify: bool = True) -> dict
Prune retained cut rows to bound the active master LP.
A cutting-plane master (the Benders master) grows one >= cut row
per iteration via :meth:add_cut_row; left unpruned its warm
re-solve time grows super-linearly. Two selection policies are
available, both LB-safe (they only drop cuts that do not support the
certified lower bound) and both belted by the same verify-restore
rollback:
policy="slack" (default):
Drop the cut rows that are strictly slack (positive primal slack,
hence zero dual by complementary slackness) at solution,
keeping the binding ones. Classification is purely by PRIMAL
slack of the stored cut against solution.col_value: for a cut
Σ coefs·x >= lower the slack is Σ coefs·x - lower >= 0; a
cut with slack <= tol is BINDING (kept), one with
slack > tol is strictly slack (dropped). tol is a per-cut
relative tolerance scaled by the magnitude of lower and the
terms.
policy="dominance" (de Matos–Philpott–Finardi / Guigues
Limited-Memory Level-1):
At a DEGENERATE optimum almost every cut can be primal-binding
(slack ≈ 0), so slack-deletion prunes nothing even though many
cuts are redundant TIES at their recourse group's max value.
Dominance selection instead groups the retained cuts by the
recourse variable they bound (the single member of
col_ids ∩ _recourse_cols, registered by
:meth:add_recourse_col) and, over a WINDOW of recent master
trial points trial_col_values (a list of col_value arrays,
most-recent last; defaults to [solution.col_value]), keeps for
each group at each trial point the OLDEST cut that achieves the
group's max value there (within a relative tolerance). Every cut
not a keeper at any trial point is DROPPED — it is dominated (or a
redundant tie) at every trial point, so the outer approximation of
the recourse function at those points, and the LB it certifies,
is unchanged. A cut whose col_ids contains no recourse column
forms its own singleton group and is always kept (generic
non-Benders use degrades safely to "keep all").
When verify (default), after deleting rows the master is re-solved
and its objective compared to the incumbent solution.obj. A drift
larger than :data:_COMPACT_VERIFY_TOL (relative) means a dropped cut
was in fact supporting the optimum (a degenerate numerical edge) — the
deletion is ROLLED BACK by re-appending every dropped cut verbatim and
re-solving, and the returned dict reports restored=True with
dropped=0. On a well-behaved master the belt never fires.
Returns {"kept": int, "dropped": int, "restored": bool} — cut
counts AFTER the operation (kept == number of retained cut rows
that remain, dropped == number removed; on a restore kept is
the pre-compaction count and dropped is 0).
declare_mutable ¶
Declare a set of :class:Param names whose values should be
tracked into LP cells, so :meth:update_param can later push
new values into the live HiGHS instance via changeCoeff.
MUST be called BEFORE the first :meth:solve. Tracking is
opt-in: Params not declared here pay no bookkeeping cost.
Pass the same names that the Params carry on their .name
field — typically the FlexData attribute name ("p_inflow",
"p_penalty_up" etc.).
Raises ValueError for the name of a Param used as an
add_var lower / upper bound: bound Params are
resolved at add_var and not tracked (a later update would
leave the bounds stale) — use :meth:set_col_bounds instead.
set_output_flag ¶
Enable or disable HiGHS' native solve log for this problem.
enabled=False mutes the per-solve HiGHS output (the
output_flag HiGHS option) for THIS WarmProblem across all of
its cold / warm / retry solves — callers that drive many parallel
sub-solves (Benders regions, Lagrangian subproblems) use this to
keep stdout clean and emit their own concise progress log instead.
May be called either before the first :meth:solve (the flag is
applied when the HiGHS handle is built) or after it (applied to the
live handle immediately). The preference persists on the handle,
so a single call suffices for the whole lifetime.
update_param ¶
Replace the values of a tracked Param. Every LP cell whose
coefficient was originally a function of param_name is
re-computed from the new Param's values and pushed via
h.changeCoeff.
new_param must be either a scalar (broadcast to all tracked
cells) or a :class:Param whose dim signature matches the
signature recorded for that Param at build time.
Raises if param_name was not in :meth:declare_mutable's
list (silent corruption is worse than a hard error), or if it
names a Param used as an add_var bound (bound Params are
resolved at add_var and not tracked; use
:meth:set_col_bounds).
Scope, which the caller must respect:
- It updates tracked matrix cells only. Contributions of the
Param to the objective, to row bounds (RHS) or to column bounds
are NOT updated, nor are cells reached through an anonymous
composite Param that carries no link to
param_name. The caller must ensure the Param reaches the LP only through tracked cells; a Param with zero tracked cells is a silent no-op. - Values are matched to the cells by the Param's dim labels
(a join on the labels recorded at build); cells whose labels are
missing from
new_paramget0.0. A Param whose labels shifted (e.g. a rolled time axis) is therefore misplaced, not re-aligned by position. - Values are in user space; the cached factors carry the Layer-2 side vectors, and the small-value floor is judged on the user-space magnitude, as at build.
col_id_of_var ¶
Return the col_id(s) for a variable.
dims=None returns every col_id in the variable's family
(numpy array, ordered by the var's declaration order).
dims as a tuple of dim values returns the single col_id for
that one cell (a python int). dims as a dict
{dim_name: value} is a partial filter — returns a numpy
array of the matching col_ids.
row_id_of_cstr ¶
Return the row_id(s) for a constraint family. Mirrors
:meth:col_id_of_var.
solve ¶
solve(*, options: dict | None = None, retry_on_unknown: bool = False, pre_run_check: Callable[[dict, Highs], Mapping[str, Any] | None] | None = None) -> Solution
Solve the LP. First call builds the LP from scratch (same
pipeline as :meth:Problem.solve); subsequent calls just run
HiGHS again on the (possibly updated) live model.
options is honoured on the FIRST solve only — subsequent
solves use the same HiGHS instance. To change options on a
rebuilt LP, drop the WarmProblem and create a new one.
pre_run_check (see :meth:Problem.solve) is called on the FIRST
solve only, after the build and before the names decision and
run(), with the same ranges keys, built from the canonical
matrix this build holds. A re-solve of the built instance does not
call it: its ranges are not re-derived after updates.
retry_on_unknown (default False → byte-identical to the
legacy path for every existing caller) enables a warm-restart
retry: the solver runs WARM first (retaining the basis so dual
simplex hot-starts after rows appended via :meth:add_cut_row);
only if the warm run does NOT return a certified kOptimal —
i.e. a stale-basis transient (kUnknown / kSolveError) or a
spurious kUnbounded / kInfeasible / primal-infeasible miss — do we
drop the basis with clearSolver() and re-run ONCE (the proven
cold fallback) to re-certify the true status; if that cold run is
still kUnknown, it passes the instance's own LP back to HiGHS and
runs once more. Used by the Benders
master, which appends a cut each iteration; on a well-scaled
master the warm path stays kOptimal and the fallback never
fires.
pass_names_to_highs ¶
Pass the column / row names to the live HiGHS instance now.
Idempotent (a no-op once the names are there, which is always the
case under highs_names="always"). Call it before reading names
back from HiGHS or writing a HiGHS file that should carry them
(writeModel, writeSolution, writeBasis) when the
WarmProblem was built with "auto" or "never". Rows and
columns appended later (:meth:add_cut_row,
:meth:add_recourse_col) then pass their names as they are added.
solve_with_fallback ¶
Solve; on a non-optimal primary, retry ONCE under
fallback_options and restore the prior option values.
Runs :meth:solve first. If it certifies kOptimal the
solution is returned as (solution, "primary") and the options
are never touched. Otherwise each fallback_options entry is
applied to the live HiGHS handle via setOptionValue
(solve(options=...) is honoured on the FIRST solve only — a
live-handle write is the only way to change options on a built
model), the retained solver state is dropped with
clearSolver() (so options like solver / presolve take
effect from cold, and no stale basis biases the retry), the model
is re-run once, and the prior option values are RESTORED — a
later plain :meth:solve sees the original configuration.
Returns (solution, "fallback") with the retry's solution,
optimal or not: the caller decides what a still-failing fallback
means.
Objective-valued fallback options are read in user units (as at
the build), with the user_objective_scale
the retry runs under: the fallback's own value, else the live
handle's.
Raises ValueError on an unknown option name or a rejected
option value; any options already changed are restored first.
highs_options_need_names ¶
True when an option in :data:HIGHS_NAME_OPTIONS is on in h.
Reads the instance's effective option values, so options set by any
route (a stored options dict, a per-call dict, setOptionValue on
the live handle) are seen. An option this HiGHS build does not know
is ignored.
Lag ¶
Return an Expr that, for each (carry_dims, time_dim) in
lag_frame, references var at (carry_dims, lag_col).
Used for shifting variables in time, e.g. for storage state-change:
v_state[n, d, t] - v_state[n, d, t_prev]
= v_state - Lag(v_state, dtttdt, "t", "t_prev_within_timeset")
lag_frame carries the (d, t, t_prev) lookup; carry_dims are
the columns shared between var and lag_frame other than the
time dim itself (typically d).
Where ¶
Inner-join an Expr against frame. Two effects in one op:
- Filter — rows of the term whose shared-column values don't
appear in
frameare dropped (e.g.Where(v_flow, wind_only)keeps only the wind rows). - Map — any columns of
framethat the term doesn't already carry become new open dims of the resulting term (e.g.Where(v_flow, flow_to_n)whereflow_to_nhas columns(p, source, sink, n)addsnso the term can be bound to a constraint indexed by(n, t)).
The pure-filter case (no extras) does not bake the join into
t.lazy — it records frame into _Term.where_frames so
the LHS prune-down (_build_lhs_pruned_plan) can apply it at the
leaf-rebuild step (where the row count is bounded by the smaller of
Var.frame and the row_index key set). var_source / Param
chain / coef_scalar are preserved.
The map-effect case (extras non-empty) ALSO defers: frame is
recorded into _Term.where_map_frames as (frame_lf,
frozenset(extras)), the term's dims are extended with the extras,
and var_source / Param chain / coef_scalar / where_frames
are preserved. The deferred inner-join (which produces the extras
dim columns) is baked at leaf-rebuild time (or at Sum / Lag /
consumer fallback) via :func:_apply_where_map_frames. Param
multiplies AFTER the map-effect Where bake first iff their dims
overlap any pending extras (correctness); otherwise the deferral
propagates through.
Set POLAR_HIGH_DISABLE_WHERE_PUSHDOWN=1 to recover today's
behaviour (eager join + clear metadata) verbatim for BOTH branches.
Sum ¶
Aggregate an Expr. over lists the dims to sum out; the
remaining dims become the term's open dims. where is an index
frame that pre-filters the term frames (inner join on shared
columns) before the group-by-sum.
Sum(expr) with over=None collapses every open dim — useful
for a scalar (objective term, single-row constraint).
Any deferred :func:Where filters recorded on each term
(t.where_frames) are baked into t.lazy before the
aggregation — Sum collapses dims so the filter could not be
applied at the leaf level downstream. The result clears
var_source / where_frames (today's behaviour); the
per-term where kwarg is applied on top, unchanged.
user_scaled_kkt_failures ¶
user_scaled_kkt_failures(*, col_lower: ndarray, col_upper: ndarray, col_value: ndarray, row_lower: ndarray, row_upper: ndarray, row_value: ndarray, col_continuous: ndarray, user_bound_scale: int, user_objective_scale: int, primal_feasibility_tolerance: float, dual_feasibility_tolerance: float, col_dual: ndarray | None = None, row_dual: ndarray | None = None, sense: int = 1) -> tuple[int, float, int | None, float | None]
Primal and dual infeasibility counts and maxima of a solution in the user-scaled space HiGHS solved, from the unscaled model and solution it returns.
HiGHS' user_bound_scale = N multiplies the bounds and values of the
continuous columns and of every row by 2**N (integer columns keep
theirs), and user_objective_scale = M multiplies the duals by
2**M. After the solve it divides them back and recomputes its info
counters on the unscaled numbers, while its feasibility tolerance stays
as it was. Multiplying by a power of two is exact in floating point, so
this rebuilds the user-scaled numbers exactly and applies HiGHS' own
per-entry tests to them.
sense is the objective sense (+1 minimise, -1 maximise). The dual
pair is None when no dual solution is given. Returns
(num_primal, max_primal, num_dual, max_dual).
polar_high.parallel ¶
Deterministic parallel solving of independent :class:WarmProblems.
A small, domain-agnostic utility for cutting-plane / decomposition drivers
(e.g. Benders region recourse, Lagrangian subproblems) that need to solve N
independent subproblems each outer iteration. Each subproblem is its own
:class:~polar_high.engine.WarmProblem (its own HiGHS handle), and HiGHS'
run() releases the GIL, so a thread pool yields a real wall-clock speedup
over the sequential loop.
The thread-safety contract this module encapsulates (so the caller's algorithm code never touches the HiGHS-threading detail):
-
Single-threaded HiGHS scheduler. Every subproblem solve runs HiGHS with a single-threaded scheduler. This is required for (a) determinism — HiGHS is non-deterministic with
threads > 1— and (b) to avoidworkers × coresoversubscription. The process-global HiGHS scheduler is pinned to one thread ONCE up front via :func:prewarm_global_scheduler; the per-subproblemrun()calls then inherit that pinned pool. -
Sequential cold first build. The FIRST
WarmProblem.solve()builds the HiGHS model and, if it sees athreads/paralleloption, calls the process-globalresetGlobalScheduler— unsafe to run concurrently. This module therefore only parallelizes solves over WarmProblems that are ALREADY built (wp._h is not None); it raises if asked to fan out an unbuilt one. Callers must do the cold first build sequentially (or rely on the scheduler pre-pin) before calling :func:solve_indexed_parallel. -
Deterministic per-index collection. Results are collected into per-index slots and returned in index order, so the outcome is identical regardless of thread timing. Worker exceptions are re-raised in index order (the lowest failing index wins, matching the sequential loop).
workers <= 1 keeps a fully sequential path (no pool, no prewarm) that is
byte-identical to a plain for loop — so the parallel and sequential code
paths can be exercised against each other for a determinism gate.
resolve_worker_count ¶
Clamp a requested worker count to [1, n_items].
workers is None auto-resolves to min(n_items, cpu_count - 1) (at
least 1) — leave one core for the main thread / OS. A non-positive request
means sequential (1).
prewarm_global_scheduler ¶
Initialize HiGHS' process-global task scheduler ONCE, single-threaded,
so subsequent concurrent solves on distinct Highs instances need not
each call resetGlobalScheduler. Best-effort; returns False if any
highspy step fails (the caller then falls back to a sequential path).
Once this returns True the global scheduler is pinned to threads and
a subsequent run() with NO threads option inherits that pool — so
concurrent solves need not (and must not) pass threads per instance,
which would re-trigger resetGlobalScheduler and is unsafe to run
concurrently.
solve_indexed_parallel ¶
solve_indexed_parallel(warmproblems: Sequence[WarmProblem], fn: Callable[[int], T], *, workers: int | None = None) -> list[T]
Run fn(i) for every index i of warmproblems deterministically,
optionally across a thread pool, and return the results in index order.
fn(i) is the caller's per-subproblem work: typically it pins the i-th
:class:WarmProblem's coupling columns, calls wp.solve(), and extracts
a domain result (objective + dual slopes). Because each WarmProblem owns
its own HiGHS handle, fn(i) for distinct i touch disjoint state and
are safe to run concurrently — provided every WarmProblem is already built
(the cold first build calls the process-global resetGlobalScheduler and
must run sequentially up front). This helper enforces that precondition.
Parameters¶
warmproblems
The per-index :class:WarmProblems. Each MUST already be built
(solve() called at least once); a ValueError is raised
otherwise. Only used for the built-precondition check + count — the
actual solving is delegated to fn.
fn
fn(i) -> result for index i. May run on a worker thread; must
confine its mutations to the i-th WarmProblem's HiGHS handle.
workers
Effective worker count (see :func:resolve_worker_count). None
auto-resolves to min(n, cpu_count - 1). <= 1 runs a fully
sequential path on the calling thread (no pool, no scheduler prewarm) —
byte-identical to a plain for loop.
Returns¶
list
[fn(0), fn(1), ..., fn(n-1)] — always in index order, independent of
thread timing.