API reference¶
The full public surface, autogenerated from docstrings.
Top-level¶
polar-high — Python library for building indexed linear and mixed-integer programs in polars.
Problem ¶
LP container. Generic — no flextool-specific knowledge.
Construct an empty LP container.
Pure polar-high is a generic LP kernel; scaling decisions are
left to the caller. See :mod:polar_high.autoscale for the
opt-in autoscaler (Layer 1 detect + Layer 3 recommendation)
that callers (e.g. FlexTool) use to drive
user_bound_scale / user_objective_scale automatically.
dense_axes — the explicit client contract for the block-COO
LHS arm. When the client (e.g. FlexTool) declares the dense
trailing axes once here (e.g. Problem(dense_axes=("d", "t"))),
it makes a binding PROMISE about every frame it passes that
contains those columns:
the frame is globally lexicographically sorted by
``(other_dims_in_declared_order..., *dense_axes)`` — i.e. the
declared dense axes are the trailing sort keys, in the given
order, and the leading dims form a sorted prefix.
This lets block-COO slice the dense suffix of each Var with NO
re-sort (a re-sort would cost more than the multiply itself).
polar-high VERIFIES this promise cheaply (a single-pass monotonic
scan — see :func:_verify_dense_sorted) on every Var that the
block-COO arm classifies + fires on, and RAISES a clear
ValueError naming the Var if the client breaks it. Frames
that do not contain the dense axes (e.g. an investment Var
("p", "d") when dense_axes=("d", "t")) simply do not fire
block-COO and are unaffected. None (default) leaves the
block-COO arm dormant — it only fires once dense axes are declared
(here or via :meth:declare_dense_axes).
declare_dense_axes ¶
Declare the dense trailing axes for block-COO (see init contract).
Equivalent to passing dense_axes= to the constructor; provided so
callers that receive an already-constructed Problem (e.g. FlexTool's
build_flextool step, which builds the Problem first and populates
it afterwards) can declare them. Pass None to clear.
set_solver_options ¶
Store HiGHS options to be applied in solve(). Pass None
to clear. Keys are HiGHS canonical option names (presolve,
solver, parallel, time_limit etc); values must be
already coerced to the type HiGHS expects (str/int/float/bool).
Unknown keys are tolerated (a warning is emitted at solve time).
set_solver_option ¶
Set a single HiGHS option, leaving the rest untouched.
Convenience for callers that want to add one knob without
re-passing the whole dict (e.g. user_bound_scale set by
the autoscaler). Equivalent to a dict merge plus
:meth:set_solver_options.
get_solver_option ¶
Return the caller-set value of name (or None if unset).
Reads self._solver_options only — does NOT consult HiGHS
(the option may not have been pushed to a live Highs
instance yet). Returns None for unset options so callers
can use if get_solver_option(...) is not None to test
explicit setting.
add_cstr ¶
add_cstr(name: str, *, over: DataFrame | None = None, sense: str, lhs_terms: dict[str, Var | Expr | Param | int | float], rhs_terms: dict[str, Var | Expr | Param | int | float] | None = None) -> None
Add a constraint of the form Σ lhs_terms sense Σ rhs_terms.
Each term entry is either
- a
VarorExpr— variable contribution, or - a
Param,intorfloat— constant contribution.
The engine sorts variables and constants out per side, builds
(lhs_var − rhs_var) sense (rhs_const − lhs_const), and adds
the row to highspy at solve time. Labels (the dict keys) are
used in row names and diagnostics.
cstr_names ¶
All constraint family names currently registered, in declaration order. Useful for emission audits and debugging.
cstrs_named ¶
Return constraint metadata records matching name.
An exact-name match returns the single record; otherwise a prefix
match returns every record whose name starts with name + "_"
(so passing "minimum_uptime" returns both
minimum_uptime_linear and minimum_uptime_integer).
Each :class:CstrRecord carries:
* name: full registered name of the constraint family;
* over: the polars DataFrame of axis tuples (len(over)
is the row count);
* proto: the underlying _CstrProto (expr, sense,
rhs) for advanced introspection.
cstr_row_count ¶
Total LP-row count across all constraint families matching
name (exact or prefix; see :meth:cstrs_named). Returns 0
when no families match — letting callers distinguish "absent"
from "empty" without exception handling. A scalar constraint
(over=None) counts as one row.
add_obj_constant ¶
Accumulate a constant into the objective offset. HiGHS adds
this to the reported getObjectiveValue() after solve, so it
shows up in Solution.obj even though no decision variable
carries it. Used for pure-Param objective terms like the §8.1
existing-entity fixed cost.
solve ¶
solve(*, options: dict | None = None, keep_solver: bool = False, streaming: bool = True, save_memory: bool = False, tmp_dir: str | PathLike | None = None) -> Solution
Solve the LP and return a :class:Solution.
Parameters¶
options
Per-call HiGHS options dict (overrides set_solver_options).
keep_solver
When True, the live HiGHS instance is kept on the returned
:class:Solution so callers can inspect it post-solve (e.g.
sol.highs.writeModel("model.mps")). Default False —
the C-side LP storage is released as soon as primal/dual/
objective have been extracted.
streaming
When True (default), columns are added once via addCols
and each constraint family is emitted to HiGHS via addRows
immediately after its COO triples are built; the family's local
arrays then go out of scope before the next family is processed.
This caps peak memory at one family's COO + the running HiGHS
LP. When False, the entire model is assembled into a single
:class:highspy.HighsLp and loaded via passModel —
numerically identical results either way; False is mostly
useful for benchmarking the legacy path.
save_memory
Single one-shot knob that trades wall time for peak RSS. When
True, two things happen right before HiGHS.run():
1. polar-high's polars/numpy LP source-of-truth (constraint
family ``Expr.terms`` lists, ``_CstrProto.rhs`` Param
frames, objective ``_Term.lazy`` plans, the caller-side
``col_names`` / ``row_names`` lists) is dropped. Only the
per-:class:`Var` ``col_id`` frames survive — they are
needed by :meth:`Solution.value` to map column indices
back to user-space dim tuples.
2. The HiGHS instance is round-tripped through disk: the LP
is written to a temp MPS file, the original ``Highs()`` is
cleared and discarded, ``malloc_trim(0)`` is called (best
effort, Linux only), a fresh ``Highs()`` is created, the
same solver options are re-applied, and the MPS file is
read back. This resets the HiGHS allocator's high-water
mark — the C++ side accumulates ~5 GB of slack from the
incremental ``addRows`` loading path that ``readModel``
avoids by sizing once up front.
Cost: ~+90 s of MPS file I/O at N=3000 dense. Benefit: peak
RSS drops by ~5 GB on the same problem. Intended for one-
shot single-solve benchmarks where warm-start / re-solve
isn't needed. After the call returns, the :class:`Problem`
is in a "released" state and any further :meth:`solve` call
raises ``RuntimeError`` (the polar-side source AND the
original HiGHS instance with its basis have both been
discarded, so neither a fresh re-solve nor a WarmProblem-
style update is possible). Honoured only by the streaming
path; on ``streaming=False`` a warning is emitted and the
flag is ignored. Default ``False``.
tmp_dir
Directory for the temporary MPS file written by the
save_memory=True round-trip. None (default) uses
the system temp dir ($TMPDIR / /tmp). Set when the
caller wants the spill file on a specific volume (e.g. the
same filesystem as its workspace, or a per-job scratch
directory) instead of the system default. Ignored when
save_memory=False.
build_only ¶
Build the LP into HiGHS, write to mps_path, release everything.
For callers that want to drive the actual solve out-of-process — typically a subprocess HiGHS reading the MPS file in a clean address space. After this call:
mps_pathexists and contains the LP in MPS format.- polar-side LP source-of-truth has been dropped
(:meth:
_release_python_lp_inputs). - The live :class:
highspy.Highsinstance has been torn down and the glibc allocator trimmed (Linux best-effort). self._releasedis True — further calls to :meth:solvewill raise.
What stays alive: self._vars and each :class:Var.frame,
so the caller can later construct a :class:Solution from
externally-produced col_value / row_dual arrays via
Solution(..., vars=dict(self._vars), ...).
Honoured only on the streaming path; the non-streaming
(passModel) path is not supported here. Raises
RuntimeError if called on an already-released Problem.
Parameters¶
mps_path
Where to write the MPS file. Caller owns the file —
polar-high will not delete it.
options
HiGHS solver options to apply during the build (mainly
presolve / solver / simplex_scale_strategy —
options that affect what writeModel serialises).
None uses :attr:_solver_options.
canonicalise ¶
Build (or return cached) the canonical CSC matrix + metadata.
Idempotent: returns the cached _matrix unless
_canonical_dirty is set (which add_var / add_cstr
flip). Side vectors (_layer2_col_factor /
_layer2_row_factor) are baked into the returned arrays at
build time per orchestrator decision D8.
write_mps ¶
write_mps(path: str | PathLike, *, free_format: bool = True, column_order_strict: bool = True, emit_names: bool = True, release: bool = False, name: str = 'POLAR_HIGH') -> None
Stream the LP/MIP to path as a free-format MPS file.
This is an in-house writer that consumes the polar-side LP
source-of-truth (_vars, _cstrs, _obj_terms) directly
and emits MPS without ever instantiating :class:highspy.Highs
or building an :class:LpView. The motivation is peak-memory:
HiGHS' own writeModel serialiser materialises ~20× more
transient state than the in-process LP itself, which OOMs on
very large LPs (~10 M rows × 5 M cols × 20 M nonzeros). This
writer targets ~2-3 GB peak on that same LP.
Parameters¶
path
Output MPS file path. Caller owns the file; this writer
does not delete on failure.
free_format
Reserved for future use. Currently always free-format MPS
(the modern default — HiGHS, Gurobi, CPLEX, Xpress all
accept it). Fixed 8-char-column MPS is not yet implemented.
column_order_strict
When True (default), all nonzeros for a single column
are emitted contiguously in the COLUMNS section, sorted by
(col_id, row_id). This is the spec-compliant ordering
every MPS reader accepts. False is currently not
implemented — see :ref:Open follow-ups in the module docs.
emit_names
When True (default), row and column names come from the
variable family name + dim tuple and the constraint family
name + dim tuple (matching the format used by
:meth:Problem.solve for Solution.col_names /
Solution.row_names). When False, generic
C0000001 / R0000001 names are emitted instead —
saves ~30-50% file size at the cost of losing the
dim-tuple → column mapping in the MPS itself. Callers
using emit_names=False must use index-based solution
readback (column 0 in the MPS is column 0 in the polars
Var.frame ordering, etc.). The objective row name is
always cost regardless — it's a reserved unique name.
release
When True, calls :meth:_release_python_lp_inputs
after the write completes. Mirrors the
solve(save_memory=True) semantics — drops the polars
LazyFrames, constraint family list, and rhs Param frames
while keeping self._vars (so solutions read back from an
external solver can still be mapped to user-space dim
tuples via the surviving Var.frame["col_id"] columns).
After release the :class:Problem is no longer solvable —
:meth:solve will raise.
name
MPS NAME header; default "POLAR_HIGH".
Raises¶
RuntimeError
If called on a :class:Problem whose source has already
been released (by an earlier solve(save_memory=True),
build_only(...), or write_mps(..., release=True)).
ValueError
If any coefficient is NaN or infinite (silent filtering
would hide model bugs). The error message identifies at
least one offending (col_id, row_id) pair.
OSError
File I/O failures (passed through from open()).
Notes¶
- The MPS format has no portable encoding for an objective
constant offset. If
self._obj_offset != 0.0this method emits a :class:UserWarningand writes the LP without the offset. Solutions from any downstream solver will then be off byself._obj_offset— the caller must add it back manually if needed. - Integer columns are bracketed with
MARKER 'INTORG'/MARKER 'INTEND'in the COLUMNS section, per MPS convention. - The writer is single-pass per family and uses one global polars sort to put the COLUMNS section into column-major order. Streaming-engine fallback to disk activates automatically for triple counts that don't fit in RAM.
Var ¶
Var(name: str, dims: tuple[str, ...], frame: DataFrame, lower: float = 0.0, upper: float = float('inf'), integer: bool = False)
A variable family. frame carries columns *dims, col_id.
Var.frame stays an eager polars DataFrame — it's small (one row
per LP column), produced once in :meth:Problem.add_var, and
consumed by both flextool integration (v.frame["col_id"].unique())
and Problem.solve (col_id → bound/name lookups). Algebra ops on
Var lazify on the fly so the resulting _Term is lazy.
Param ¶
Param(dims: tuple[str, ...], frame: DataFrame | LazyFrame, name: str | None = None, _sources: list[tuple[Param, int]] | None = None, _value_scalar: float = 1.0)
A parameter table. frame carries columns *dims, value.
Stored internally as a polars.LazyFrame so that chained algebra
ops (Param * Param, Param + Param etc.) defer materialization
until a consumer reads .frame or the engine collects in
Problem.solve. The .frame property caches the eager
DataFrame on first read — flextool reads .frame.rename(...)
repeatedly off the same Param so we want that to be cheap.
name (optional) is a logical Param identifier (e.g. "p_inflow").
It is opt-in metadata used by :class:WarmProblem's Param-tracked
auto-update (declare_mutable / update_param). When unset,
Params are anonymous and carry no tracking overhead.
_sources records constituent named Params for composite results
of Param * Param / Param / Param. Each entry is
(param_name, dims_tuple, direction) where direction is +1 if the
Param contributes to the numerator and -1 if to the denominator.
Anonymous-only chains have _sources is None.
Expr ¶
A sum of terms (decision-variable contributions).
The terms can have different open-dim sets — they're concatenated,
not broadcast. Broadcasting happens once, at constraint emission,
via a join to the constraint's over= row index.
WarmProblem ¶
Warm-update wrapper around a :class:Problem.
Build a :class:Problem as usual (add_var, add_cstr,
set_objective). Wrap with :class:WarmProblem, then alternate
update_* calls and solve() calls — the LP is built ONCE and
only the changed coefficients / RHS values are pushed to HiGHS
between solves.
Typical rolling-horizon usage::
wp = WarmProblem(p)
sol_0 = wp.solve()
for r in range(1, n_rolls):
wp.update_rhs("balance", demand_param_for_roll[r])
wp.update_obj_coef("v_flow", cost_param_for_roll[r])
sol_r = wp.solve()
The update_* calls are O(rows_or_cols_in_family); the solve()
benefits from HiGHS's hot-start (basis is preserved across calls).
problem
property
¶
The underlying :class:Problem.
Useful for diagnostics that need to inspect the un-built LP.
For coefficient ranges, prefer
:attr:Solution.streamed_lp_ranges on a returned
:class:Solution (populated during the streaming solve at zero
extra cost).
update_rhs ¶
Replace the RHS of constraint family cstr_name with values
drawn from new_param.
new_param may be a :class:Param whose dims match a subset
of the constraint's over= axis (broadcasting to the rest), a
scalar (broadcast to all rows), or a numpy array (positional —
length must equal the family's row count, in the order of the
original over frame).
update_obj_coef ¶
Replace the objective coefficient on every column of var_name.
Assumes the objective contribution from var_name is exactly
coef[*dims] * var[*dims] for some coef Param; this method
OVERWRITES that coefficient via h.changeColsCost. If the
objective also has contributions from this variable through more
complex algebra (e.g. var * unitsize * slope), the update is
still valid as long as new_param carries the full product —
the caller is responsible for collapsing multi-Param products
ahead of the call.
This DOES NOT touch the cost coefficients of other variables.
update_obj_coef_array ¶
Array-form of :meth:update_obj_coef.
dim_tuples is a list of dim-value tuples (one per cell) for
variable var_name; each tuple must have one entry per dim
in the var's declared signature. values is a same-length
numpy array of new objective coefficients.
The columns are resolved positionally: values[k] becomes the
new objective coefficient on the column whose dim-tuple is
dim_tuples[k]. Vectorised — a single changeColsCost
call regardless of cell count.
fix_cols ¶
Fix the listed columns of var_name to the given values.
For each (dim_tuple, value) pair, sets both the column's
lower and upper bound to value (so the LP has no choice
but to set the column at that level). Used by the Lagrangian
primal-recovery step ("fix-and-resolve"). Vectorised — single
changeColsBounds call.
update_coef ¶
Update a single (row, col) coefficient in the constraint
matrix. Use :meth:row_id_of_cstr / :meth:col_id_of_var to
resolve indices semantically.
add_cut_row ¶
Append a >= constraint row to the live (already-built) LP and
return the new row's id.
Appends Σ coefs[k] · x[col_ids[k]] >= lower to the live HiGHS
model via addRow. This is a POST-build mutation of the warm
_h handle (the same class of live edit as :meth:update_rhs /
:meth:update_coef); it deliberately bypasses the build-time
Problem.add_cstr DSL lock, which only guards the fixed-size
Layer-2 autoscale side vectors and is irrelevant once the model is
built. The caller is responsible for keeping the master autoscale
OFF (or for pre-scaling coefs) so the appended row lives on the
built columns' scale — there is no auto-scaling for appended rows.
col_ids may mix existing column ids (resolve via
:meth:col_id_of_var) and ids of columns previously appended with
:meth:add_recourse_col. A subsequent :meth:solve warm
re-optimises the grown model; the appended row's dual is then
readable on the returned :class:Solution by row_dual[row_id]
(read BY ROW ID, not via :meth:Solution.constraint_dual, which
only knows named build-time rows).
Bumps the cached _n_rows (the zero-fill fallback size in
:meth:solve) and appends a generated name to _row_names (kept
index-aligned so name-indexed reads stay correct). Returns the
row_id HiGHS assigned (== the pre-append getNumRow()).
add_recourse_col ¶
Append a single free/bounded column (e.g. a Benders recourse
η) to the live LP and return its col id.
Symmetric with :meth:add_cut_row: a POST-build addCol on the
warm _h handle, bumping the cached _n_cols (zero-fill
fallback size) and _col_names. ±np.inf bounds are mapped to
the HiGHS kHighsInf sentinel. The returned id can be referenced
from a later :meth:add_cut_row; its value is read on the
:class:Solution by col_value[col_id] (appended columns are not
in any Problem._vars frame, so :meth:Solution.value cannot see
them — read by id).
declare_mutable ¶
Declare a set of :class:Param names whose values should be
tracked into LP cells, so :meth:update_param can later push
new values into the live HiGHS instance via changeCoeff.
MUST be called BEFORE the first :meth:solve. Tracking is
opt-in: Params not declared here pay no bookkeeping cost.
Pass the same names that the Params carry on their .name
field — typically the FlexData attribute name ("p_inflow",
"p_penalty_up" etc.).
set_output_flag ¶
Enable or disable HiGHS' native solve log for this problem.
enabled=False mutes the per-solve HiGHS output (the
output_flag HiGHS option) for THIS WarmProblem across all of
its cold / warm / retry solves — callers that drive many parallel
sub-solves (Benders regions, Lagrangian subproblems) use this to
keep stdout clean and emit their own concise progress log instead.
May be called either before the first :meth:solve (the flag is
applied when the HiGHS handle is built) or after it (applied to the
live handle immediately). The preference persists on the handle,
so a single call suffices for the whole lifetime.
update_param ¶
Replace the values of a tracked Param. Every LP cell whose
coefficient was originally a function of param_name is
re-computed from the new Param's values and pushed via
h.changeCoeff.
new_param must be either a scalar (broadcast to all tracked
cells) or a :class:Param whose dim signature matches the
signature recorded for that Param at build time.
Raises if param_name was not in :meth:declare_mutable's
list (silent corruption is worse than a hard error).
col_id_of_var ¶
Return the col_id(s) for a variable.
dims=None returns every col_id in the variable's family
(numpy array, ordered by the var's declaration order).
dims as a tuple of dim values returns the single col_id for
that one cell (a python int). dims as a dict
{dim_name: value} is a partial filter — returns a numpy
array of the matching col_ids.
row_id_of_cstr ¶
Return the row_id(s) for a constraint family. Mirrors
:meth:col_id_of_var.
solve ¶
Solve the LP. First call builds the LP from scratch (same
pipeline as :meth:Problem.solve); subsequent calls just run
HiGHS again on the (possibly updated) live model.
options is honoured on the FIRST solve only — subsequent
solves use the same HiGHS instance. To change options on a
rebuilt LP, drop the WarmProblem and create a new one.
retry_on_unknown (default False → byte-identical to the
legacy path for every existing caller) enables a warm-restart
retry: the solver runs WARM first (retaining the basis so dual
simplex hot-starts after rows appended via :meth:add_cut_row);
only if the warm run does NOT return a certified kOptimal —
i.e. a stale-basis transient (kUnknown / kSolveError) or a
spurious kUnbounded / kInfeasible / primal-infeasible miss — do we
drop the basis with clearSolver() and re-run ONCE (the proven
cold fallback) to re-certify the true status. Used by the Benders
master, which appends a cut each iteration; on a well-scaled
master the warm path stays kOptimal and the fallback never
fires.
Solution ¶
Solution(*, optimal: bool, obj: float, col_value: ndarray, row_dual: ndarray, col_names: list[str], row_names: list[str], vars: dict[str, Var], col_dual: ndarray | None = None, highs: Highs | None = None, streamed_lp_ranges: dict | None = None)
Read-only view of the solved LP. Look up variable values by
name; values come back as a polars frame (*dims, value).
max_primal_infeasibility
property
¶
Largest primal-constraint violation in the returned solution.
A solver returns a vertex that sits within its feasibility tolerance
of the active constraints, so a hand-rolled feasibility re-check on the
solution (f <= cap, balance rows, …) must use a tolerance at least
this large — a hard-coded magic constant either masks real violations
or trips on the normal solver slack. HiGHS enforces feasibility on the
INTERNALLY-SCALED problem, so the unscaled slack reported here can
exceed :attr:primal_feasibility_tolerance.
0.0 when no live solver is attached (a synthesised Solution).
primal_feasibility_tolerance
property
¶
The solver's primal feasibility tolerance for this solve (the
nominal, scaled-problem tolerance; see
:attr:max_primal_infeasibility for the achieved unscaled slack).
0.0 when no live solver is attached (a synthesised Solution).
value_wide ¶
value_wide(var_name: str, time_dims: tuple[str, ...] = ('d', 't'), solve_name: str | None = None) -> pl.DataFrame
Wide-form, flextool-compatible: time dims become rows, the remaining dims are encoded as a tuple-stringified column header.
For a 2-d variable like vq_state_up(n, d, t):
long : rows = (n, d, t, value)
wide : rows = (d, t) + one column per n (header = "west").
For a 5-d variable like v_flow(p, source, sink, d, t):
wide : rows = (d, t) + one column per (p, source, sink),
header = "('coal_plant', 'coal_market', 'west')"
to match flextool's MultiIndex parquet round-trip.
If solve_name is given, prepend a constant solve column
for fuller flextool-output compatibility.
constraint_dual ¶
Per-row dual values for a named constraint. Returns a frame
(over_dims..., dual) if the constraint had over= rows,
else a single-row scalar frame (dual,).
CstrRecord ¶
Read-only metadata for a registered constraint family.
Returned by :meth:Problem.cstrs_named for emission-introspection
tests. proto carries the LHS Expr, sense and rhs
structures; most callers only need over (whose height is the
row count) and name.
Sum ¶
Aggregate an Expr. over lists the dims to sum out; the
remaining dims become the term's open dims. where is an index
frame that pre-filters the term frames (inner join on shared
columns) before the group-by-sum.
Sum(expr) with over=None collapses every open dim — useful
for a scalar (objective term, single-row constraint).
Any deferred :func:Where filters recorded on each term
(t.where_frames) are baked into t.lazy before the
aggregation — Sum collapses dims so the filter could not be
applied at the leaf level downstream. The result clears
var_source / where_frames (today's behaviour); the
per-term where kwarg is applied on top, unchanged.
Where ¶
Inner-join an Expr against frame. Two effects in one op:
- Filter — rows of the term whose shared-column values don't
appear in
frameare dropped (e.g.Where(v_flow, wind_only)keeps only the wind rows). - Map — any columns of
framethat the term doesn't already carry become new open dims of the resulting term (e.g.Where(v_flow, flow_to_n)whereflow_to_nhas columns(p, source, sink, n)addsnso the term can be bound to a constraint indexed by(n, t)).
The pure-filter case (no extras) does not bake the join into
t.lazy — it records frame into _Term.where_frames so
the LHS prune-down (_build_lhs_pruned_plan) can apply it at the
leaf-rebuild step (where the row count is bounded by the smaller of
Var.frame and the row_index key set). var_source / Param
chain / coef_scalar are preserved.
The map-effect case (extras non-empty) ALSO defers: frame is
recorded into _Term.where_map_frames as (frame_lf,
frozenset(extras)), the term's dims are extended with the extras,
and var_source / Param chain / coef_scalar / where_frames
are preserved. The deferred inner-join (which produces the extras
dim columns) is baked at leaf-rebuild time (or at Sum / Lag /
consumer fallback) via :func:_apply_where_map_frames. Param
multiplies AFTER the map-effect Where bake first iff their dims
overlap any pending extras (correctness); otherwise the deferral
propagates through.
Set POLAR_HIGH_DISABLE_WHERE_PUSHDOWN=1 to recover today's
behaviour (eager join + clear metadata) verbatim for BOTH branches.
Lag ¶
Return an Expr that, for each (carry_dims, time_dim) in
lag_frame, references var at (carry_dims, lag_col).
Used for shifting variables in time, e.g. for storage state-change:
v_state[n, d, t] - v_state[n, d, t_prev]
= v_state - Lag(v_state, dtttdt, "t", "t_prev_within_timeset")
lag_frame carries the (d, t, t_prev) lookup; carry_dims are
the columns shared between var and lag_frame other than the
time dim itself (typically d).
prewarm_global_scheduler ¶
Initialize HiGHS' process-global task scheduler ONCE, single-threaded,
so subsequent concurrent solves on distinct Highs instances need not
each call resetGlobalScheduler. Best-effort; returns False if any
highspy step fails (the caller then falls back to a sequential path).
Once this returns True the global scheduler is pinned to threads and
a subsequent run() with NO threads option inherits that pool — so
concurrent solves need not (and must not) pass threads per instance,
which would re-trigger resetGlobalScheduler and is unsafe to run
concurrently.
resolve_worker_count ¶
Clamp a requested worker count to [1, n_items].
workers is None auto-resolves to min(n_items, cpu_count - 1) (at
least 1) — leave one core for the main thread / OS. A non-positive request
means sequential (1).
solve_indexed_parallel ¶
solve_indexed_parallel(warmproblems: Sequence[WarmProblem], fn: Callable[[int], T], *, workers: int | None = None) -> list[T]
Run fn(i) for every index i of warmproblems deterministically,
optionally across a thread pool, and return the results in index order.
fn(i) is the caller's per-subproblem work: typically it pins the i-th
:class:WarmProblem's coupling columns, calls wp.solve(), and extracts
a domain result (objective + dual slopes). Because each WarmProblem owns
its own HiGHS handle, fn(i) for distinct i touch disjoint state and
are safe to run concurrently — provided every WarmProblem is already built
(the cold first build calls the process-global resetGlobalScheduler and
must run sequentially up front). This helper enforces that precondition.
Parameters¶
warmproblems
The per-index :class:WarmProblems. Each MUST already be built
(solve() called at least once); a ValueError is raised
otherwise. Only used for the built-precondition check + count — the
actual solving is delegated to fn.
fn
fn(i) -> result for index i. May run on a worker thread; must
confine its mutations to the i-th WarmProblem's HiGHS handle.
workers
Effective worker count (see :func:resolve_worker_count). None
auto-resolves to min(n, cpu_count - 1). <= 1 runs a fully
sequential path on the calling thread (no pool, no scheduler prewarm) —
byte-identical to a plain for loop.
Returns¶
list
[fn(0), fn(1), ..., fn(n-1)] — always in index order, independent of
thread timing.
Modules¶
polar_high.engine ¶
Generic polars-backed LP kernel.
Three primitives — Var, Param, Sum — and one container
(Problem). Knows nothing about flextool, energy systems, or any
specific model. A constraint is built as either:
-
an
Exprproduced by overloaded operators (v <= cap,Sum(...) >= rhs,lhs.eq(rhs)), passed positionally toProblem.add_cstr; or -
a labelled
termsdict, summed across all entries, with an explicitsenseandrhs. Use this when a constraint is naturally a sum of named contributions (storage transitions, sink flow, source flow, slack — like flextool's nodeBalance_eq).
A variable is a polars frame (*dims, col_id) — one LP column per
row. A parameter is a polars frame (*dims, value). Var * Param
joins on shared dims and emits an Expr term (*union_dims, col_id,
coef). Sum(expr, over=…) group-by-sums one or more dims; the
remaining dims become the constraint's row dims when the term is bound
to over= at add_cstr time.
Param ¶
Param(dims: tuple[str, ...], frame: DataFrame | LazyFrame, name: str | None = None, _sources: list[tuple[Param, int]] | None = None, _value_scalar: float = 1.0)
A parameter table. frame carries columns *dims, value.
Stored internally as a polars.LazyFrame so that chained algebra
ops (Param * Param, Param + Param etc.) defer materialization
until a consumer reads .frame or the engine collects in
Problem.solve. The .frame property caches the eager
DataFrame on first read — flextool reads .frame.rename(...)
repeatedly off the same Param so we want that to be cheap.
name (optional) is a logical Param identifier (e.g. "p_inflow").
It is opt-in metadata used by :class:WarmProblem's Param-tracked
auto-update (declare_mutable / update_param). When unset,
Params are anonymous and carry no tracking overhead.
_sources records constituent named Params for composite results
of Param * Param / Param / Param. Each entry is
(param_name, dims_tuple, direction) where direction is +1 if the
Param contributes to the numerator and -1 if to the denominator.
Anonymous-only chains have _sources is None.
Var ¶
Var(name: str, dims: tuple[str, ...], frame: DataFrame, lower: float = 0.0, upper: float = float('inf'), integer: bool = False)
A variable family. frame carries columns *dims, col_id.
Var.frame stays an eager polars DataFrame — it's small (one row
per LP column), produced once in :meth:Problem.add_var, and
consumed by both flextool integration (v.frame["col_id"].unique())
and Problem.solve (col_id → bound/name lookups). Algebra ops on
Var lazify on the fly so the resulting _Term is lazy.
SumBlockMeta
dataclass
¶
SumBlockMeta(var_source: Var, param_sources: tuple[tuple[Param, int], ...], coef_scalar: float, where_frames: tuple[LazyFrame, ...] | None, where_map_frames: tuple[tuple[LazyFrame, frozenset[str]], ...] | None, reduce_dims: tuple[str, ...], keep: tuple[str, ...])
Inert reconstruction recipe captured at Sum-time (Phase C-2).
When a block-eligible term (var_source present, non-empty
param_sources list, non-empty over) is reduced by
:func:Sum, the aggregation clears var_source and survivor-
filters param_sources on the returned term, discarding the
information needed to rebuild the pre-Sum row_index → Var → P1 →
P2 … chain and reduce it in-block. :class:SumBlockMeta snapshots
that pre-Sum state so a future block-COO classifier (Phase C-3) can
evaluate the Sum-wrapped chain by rebuilding from leaves and reducing
within the block.
All fields are captured from the PRE-Sum term, before any clearing or survivor-filtering:
var_source— the originating :class:Var.param_sources— the FULL pre-Sumlist[(Param, direction)], NOT survivor-filtered: block-COO needs every factor, including Params whose dims are summed out (e.g.p_unitsizeoverp).coef_scalar— the cumulative constant scalar folded into coef.where_frames— the pre-Sum deferred pure-filter frames.where_map_frames— the pre-Sum deferred map-effect frames.reduce_dims— the dims summed out (the Sum'sover).keep— the post-Sum open dims (the returned term's dims).
This class is currently INERT: nothing reads it (confirmed by grep).
It is set ONLY at the point of reduction in :func:Sum, never
propagated through later :class:Expr ops or a nested / re-reduced
:func:Sum (those set it to None), so a stale recipe can never
attach to an already-reduced term.
Expr ¶
A sum of terms (decision-variable contributions).
The terms can have different open-dim sets — they're concatenated,
not broadcast. Broadcasting happens once, at constraint emission,
via a join to the constraint's over= row index.
CstrRecord ¶
Read-only metadata for a registered constraint family.
Returned by :meth:Problem.cstrs_named for emission-introspection
tests. proto carries the LHS Expr, sense and rhs
structures; most callers only need over (whose height is the
row count) and name.
Problem ¶
LP container. Generic — no flextool-specific knowledge.
Construct an empty LP container.
Pure polar-high is a generic LP kernel; scaling decisions are
left to the caller. See :mod:polar_high.autoscale for the
opt-in autoscaler (Layer 1 detect + Layer 3 recommendation)
that callers (e.g. FlexTool) use to drive
user_bound_scale / user_objective_scale automatically.
dense_axes — the explicit client contract for the block-COO
LHS arm. When the client (e.g. FlexTool) declares the dense
trailing axes once here (e.g. Problem(dense_axes=("d", "t"))),
it makes a binding PROMISE about every frame it passes that
contains those columns:
the frame is globally lexicographically sorted by
``(other_dims_in_declared_order..., *dense_axes)`` — i.e. the
declared dense axes are the trailing sort keys, in the given
order, and the leading dims form a sorted prefix.
This lets block-COO slice the dense suffix of each Var with NO
re-sort (a re-sort would cost more than the multiply itself).
polar-high VERIFIES this promise cheaply (a single-pass monotonic
scan — see :func:_verify_dense_sorted) on every Var that the
block-COO arm classifies + fires on, and RAISES a clear
ValueError naming the Var if the client breaks it. Frames
that do not contain the dense axes (e.g. an investment Var
("p", "d") when dense_axes=("d", "t")) simply do not fire
block-COO and are unaffected. None (default) leaves the
block-COO arm dormant — it only fires once dense axes are declared
(here or via :meth:declare_dense_axes).
declare_dense_axes ¶
Declare the dense trailing axes for block-COO (see init contract).
Equivalent to passing dense_axes= to the constructor; provided so
callers that receive an already-constructed Problem (e.g. FlexTool's
build_flextool step, which builds the Problem first and populates
it afterwards) can declare them. Pass None to clear.
set_solver_options ¶
Store HiGHS options to be applied in solve(). Pass None
to clear. Keys are HiGHS canonical option names (presolve,
solver, parallel, time_limit etc); values must be
already coerced to the type HiGHS expects (str/int/float/bool).
Unknown keys are tolerated (a warning is emitted at solve time).
set_solver_option ¶
Set a single HiGHS option, leaving the rest untouched.
Convenience for callers that want to add one knob without
re-passing the whole dict (e.g. user_bound_scale set by
the autoscaler). Equivalent to a dict merge plus
:meth:set_solver_options.
get_solver_option ¶
Return the caller-set value of name (or None if unset).
Reads self._solver_options only — does NOT consult HiGHS
(the option may not have been pushed to a live Highs
instance yet). Returns None for unset options so callers
can use if get_solver_option(...) is not None to test
explicit setting.
add_cstr ¶
add_cstr(name: str, *, over: DataFrame | None = None, sense: str, lhs_terms: dict[str, Var | Expr | Param | int | float], rhs_terms: dict[str, Var | Expr | Param | int | float] | None = None) -> None
Add a constraint of the form Σ lhs_terms sense Σ rhs_terms.
Each term entry is either
- a
VarorExpr— variable contribution, or - a
Param,intorfloat— constant contribution.
The engine sorts variables and constants out per side, builds
(lhs_var − rhs_var) sense (rhs_const − lhs_const), and adds
the row to highspy at solve time. Labels (the dict keys) are
used in row names and diagnostics.
cstr_names ¶
All constraint family names currently registered, in declaration order. Useful for emission audits and debugging.
cstrs_named ¶
Return constraint metadata records matching name.
An exact-name match returns the single record; otherwise a prefix
match returns every record whose name starts with name + "_"
(so passing "minimum_uptime" returns both
minimum_uptime_linear and minimum_uptime_integer).
Each :class:CstrRecord carries:
* name: full registered name of the constraint family;
* over: the polars DataFrame of axis tuples (len(over)
is the row count);
* proto: the underlying _CstrProto (expr, sense,
rhs) for advanced introspection.
cstr_row_count ¶
Total LP-row count across all constraint families matching
name (exact or prefix; see :meth:cstrs_named). Returns 0
when no families match — letting callers distinguish "absent"
from "empty" without exception handling. A scalar constraint
(over=None) counts as one row.
add_obj_constant ¶
Accumulate a constant into the objective offset. HiGHS adds
this to the reported getObjectiveValue() after solve, so it
shows up in Solution.obj even though no decision variable
carries it. Used for pure-Param objective terms like the §8.1
existing-entity fixed cost.
solve ¶
solve(*, options: dict | None = None, keep_solver: bool = False, streaming: bool = True, save_memory: bool = False, tmp_dir: str | PathLike | None = None) -> Solution
Solve the LP and return a :class:Solution.
Parameters¶
options
Per-call HiGHS options dict (overrides set_solver_options).
keep_solver
When True, the live HiGHS instance is kept on the returned
:class:Solution so callers can inspect it post-solve (e.g.
sol.highs.writeModel("model.mps")). Default False —
the C-side LP storage is released as soon as primal/dual/
objective have been extracted.
streaming
When True (default), columns are added once via addCols
and each constraint family is emitted to HiGHS via addRows
immediately after its COO triples are built; the family's local
arrays then go out of scope before the next family is processed.
This caps peak memory at one family's COO + the running HiGHS
LP. When False, the entire model is assembled into a single
:class:highspy.HighsLp and loaded via passModel —
numerically identical results either way; False is mostly
useful for benchmarking the legacy path.
save_memory
Single one-shot knob that trades wall time for peak RSS. When
True, two things happen right before HiGHS.run():
1. polar-high's polars/numpy LP source-of-truth (constraint
family ``Expr.terms`` lists, ``_CstrProto.rhs`` Param
frames, objective ``_Term.lazy`` plans, the caller-side
``col_names`` / ``row_names`` lists) is dropped. Only the
per-:class:`Var` ``col_id`` frames survive — they are
needed by :meth:`Solution.value` to map column indices
back to user-space dim tuples.
2. The HiGHS instance is round-tripped through disk: the LP
is written to a temp MPS file, the original ``Highs()`` is
cleared and discarded, ``malloc_trim(0)`` is called (best
effort, Linux only), a fresh ``Highs()`` is created, the
same solver options are re-applied, and the MPS file is
read back. This resets the HiGHS allocator's high-water
mark — the C++ side accumulates ~5 GB of slack from the
incremental ``addRows`` loading path that ``readModel``
avoids by sizing once up front.
Cost: ~+90 s of MPS file I/O at N=3000 dense. Benefit: peak
RSS drops by ~5 GB on the same problem. Intended for one-
shot single-solve benchmarks where warm-start / re-solve
isn't needed. After the call returns, the :class:`Problem`
is in a "released" state and any further :meth:`solve` call
raises ``RuntimeError`` (the polar-side source AND the
original HiGHS instance with its basis have both been
discarded, so neither a fresh re-solve nor a WarmProblem-
style update is possible). Honoured only by the streaming
path; on ``streaming=False`` a warning is emitted and the
flag is ignored. Default ``False``.
tmp_dir
Directory for the temporary MPS file written by the
save_memory=True round-trip. None (default) uses
the system temp dir ($TMPDIR / /tmp). Set when the
caller wants the spill file on a specific volume (e.g. the
same filesystem as its workspace, or a per-job scratch
directory) instead of the system default. Ignored when
save_memory=False.
build_only ¶
Build the LP into HiGHS, write to mps_path, release everything.
For callers that want to drive the actual solve out-of-process — typically a subprocess HiGHS reading the MPS file in a clean address space. After this call:
mps_pathexists and contains the LP in MPS format.- polar-side LP source-of-truth has been dropped
(:meth:
_release_python_lp_inputs). - The live :class:
highspy.Highsinstance has been torn down and the glibc allocator trimmed (Linux best-effort). self._releasedis True — further calls to :meth:solvewill raise.
What stays alive: self._vars and each :class:Var.frame,
so the caller can later construct a :class:Solution from
externally-produced col_value / row_dual arrays via
Solution(..., vars=dict(self._vars), ...).
Honoured only on the streaming path; the non-streaming
(passModel) path is not supported here. Raises
RuntimeError if called on an already-released Problem.
Parameters¶
mps_path
Where to write the MPS file. Caller owns the file —
polar-high will not delete it.
options
HiGHS solver options to apply during the build (mainly
presolve / solver / simplex_scale_strategy —
options that affect what writeModel serialises).
None uses :attr:_solver_options.
canonicalise ¶
Build (or return cached) the canonical CSC matrix + metadata.
Idempotent: returns the cached _matrix unless
_canonical_dirty is set (which add_var / add_cstr
flip). Side vectors (_layer2_col_factor /
_layer2_row_factor) are baked into the returned arrays at
build time per orchestrator decision D8.
write_mps ¶
write_mps(path: str | PathLike, *, free_format: bool = True, column_order_strict: bool = True, emit_names: bool = True, release: bool = False, name: str = 'POLAR_HIGH') -> None
Stream the LP/MIP to path as a free-format MPS file.
This is an in-house writer that consumes the polar-side LP
source-of-truth (_vars, _cstrs, _obj_terms) directly
and emits MPS without ever instantiating :class:highspy.Highs
or building an :class:LpView. The motivation is peak-memory:
HiGHS' own writeModel serialiser materialises ~20× more
transient state than the in-process LP itself, which OOMs on
very large LPs (~10 M rows × 5 M cols × 20 M nonzeros). This
writer targets ~2-3 GB peak on that same LP.
Parameters¶
path
Output MPS file path. Caller owns the file; this writer
does not delete on failure.
free_format
Reserved for future use. Currently always free-format MPS
(the modern default — HiGHS, Gurobi, CPLEX, Xpress all
accept it). Fixed 8-char-column MPS is not yet implemented.
column_order_strict
When True (default), all nonzeros for a single column
are emitted contiguously in the COLUMNS section, sorted by
(col_id, row_id). This is the spec-compliant ordering
every MPS reader accepts. False is currently not
implemented — see :ref:Open follow-ups in the module docs.
emit_names
When True (default), row and column names come from the
variable family name + dim tuple and the constraint family
name + dim tuple (matching the format used by
:meth:Problem.solve for Solution.col_names /
Solution.row_names). When False, generic
C0000001 / R0000001 names are emitted instead —
saves ~30-50% file size at the cost of losing the
dim-tuple → column mapping in the MPS itself. Callers
using emit_names=False must use index-based solution
readback (column 0 in the MPS is column 0 in the polars
Var.frame ordering, etc.). The objective row name is
always cost regardless — it's a reserved unique name.
release
When True, calls :meth:_release_python_lp_inputs
after the write completes. Mirrors the
solve(save_memory=True) semantics — drops the polars
LazyFrames, constraint family list, and rhs Param frames
while keeping self._vars (so solutions read back from an
external solver can still be mapped to user-space dim
tuples via the surviving Var.frame["col_id"] columns).
After release the :class:Problem is no longer solvable —
:meth:solve will raise.
name
MPS NAME header; default "POLAR_HIGH".
Raises¶
RuntimeError
If called on a :class:Problem whose source has already
been released (by an earlier solve(save_memory=True),
build_only(...), or write_mps(..., release=True)).
ValueError
If any coefficient is NaN or infinite (silent filtering
would hide model bugs). The error message identifies at
least one offending (col_id, row_id) pair.
OSError
File I/O failures (passed through from open()).
Notes¶
- The MPS format has no portable encoding for an objective
constant offset. If
self._obj_offset != 0.0this method emits a :class:UserWarningand writes the LP without the offset. Solutions from any downstream solver will then be off byself._obj_offset— the caller must add it back manually if needed. - Integer columns are bracketed with
MARKER 'INTORG'/MARKER 'INTEND'in the COLUMNS section, per MPS convention. - The writer is single-pass per family and uses one global polars sort to put the COLUMNS section into column-major order. Streaming-engine fallback to disk activates automatically for triple counts that don't fit in RAM.
Solution ¶
Solution(*, optimal: bool, obj: float, col_value: ndarray, row_dual: ndarray, col_names: list[str], row_names: list[str], vars: dict[str, Var], col_dual: ndarray | None = None, highs: Highs | None = None, streamed_lp_ranges: dict | None = None)
Read-only view of the solved LP. Look up variable values by
name; values come back as a polars frame (*dims, value).
max_primal_infeasibility
property
¶
Largest primal-constraint violation in the returned solution.
A solver returns a vertex that sits within its feasibility tolerance
of the active constraints, so a hand-rolled feasibility re-check on the
solution (f <= cap, balance rows, …) must use a tolerance at least
this large — a hard-coded magic constant either masks real violations
or trips on the normal solver slack. HiGHS enforces feasibility on the
INTERNALLY-SCALED problem, so the unscaled slack reported here can
exceed :attr:primal_feasibility_tolerance.
0.0 when no live solver is attached (a synthesised Solution).
primal_feasibility_tolerance
property
¶
The solver's primal feasibility tolerance for this solve (the
nominal, scaled-problem tolerance; see
:attr:max_primal_infeasibility for the achieved unscaled slack).
0.0 when no live solver is attached (a synthesised Solution).
value_wide ¶
value_wide(var_name: str, time_dims: tuple[str, ...] = ('d', 't'), solve_name: str | None = None) -> pl.DataFrame
Wide-form, flextool-compatible: time dims become rows, the remaining dims are encoded as a tuple-stringified column header.
For a 2-d variable like vq_state_up(n, d, t):
long : rows = (n, d, t, value)
wide : rows = (d, t) + one column per n (header = "west").
For a 5-d variable like v_flow(p, source, sink, d, t):
wide : rows = (d, t) + one column per (p, source, sink),
header = "('coal_plant', 'coal_market', 'west')"
to match flextool's MultiIndex parquet round-trip.
If solve_name is given, prepend a constant solve column
for fuller flextool-output compatibility.
constraint_dual ¶
Per-row dual values for a named constraint. Returns a frame
(over_dims..., dual) if the constraint had over= rows,
else a single-row scalar frame (dual,).
WarmProblem ¶
Warm-update wrapper around a :class:Problem.
Build a :class:Problem as usual (add_var, add_cstr,
set_objective). Wrap with :class:WarmProblem, then alternate
update_* calls and solve() calls — the LP is built ONCE and
only the changed coefficients / RHS values are pushed to HiGHS
between solves.
Typical rolling-horizon usage::
wp = WarmProblem(p)
sol_0 = wp.solve()
for r in range(1, n_rolls):
wp.update_rhs("balance", demand_param_for_roll[r])
wp.update_obj_coef("v_flow", cost_param_for_roll[r])
sol_r = wp.solve()
The update_* calls are O(rows_or_cols_in_family); the solve()
benefits from HiGHS's hot-start (basis is preserved across calls).
problem
property
¶
The underlying :class:Problem.
Useful for diagnostics that need to inspect the un-built LP.
For coefficient ranges, prefer
:attr:Solution.streamed_lp_ranges on a returned
:class:Solution (populated during the streaming solve at zero
extra cost).
update_rhs ¶
Replace the RHS of constraint family cstr_name with values
drawn from new_param.
new_param may be a :class:Param whose dims match a subset
of the constraint's over= axis (broadcasting to the rest), a
scalar (broadcast to all rows), or a numpy array (positional —
length must equal the family's row count, in the order of the
original over frame).
update_obj_coef ¶
Replace the objective coefficient on every column of var_name.
Assumes the objective contribution from var_name is exactly
coef[*dims] * var[*dims] for some coef Param; this method
OVERWRITES that coefficient via h.changeColsCost. If the
objective also has contributions from this variable through more
complex algebra (e.g. var * unitsize * slope), the update is
still valid as long as new_param carries the full product —
the caller is responsible for collapsing multi-Param products
ahead of the call.
This DOES NOT touch the cost coefficients of other variables.
update_obj_coef_array ¶
Array-form of :meth:update_obj_coef.
dim_tuples is a list of dim-value tuples (one per cell) for
variable var_name; each tuple must have one entry per dim
in the var's declared signature. values is a same-length
numpy array of new objective coefficients.
The columns are resolved positionally: values[k] becomes the
new objective coefficient on the column whose dim-tuple is
dim_tuples[k]. Vectorised — a single changeColsCost
call regardless of cell count.
fix_cols ¶
Fix the listed columns of var_name to the given values.
For each (dim_tuple, value) pair, sets both the column's
lower and upper bound to value (so the LP has no choice
but to set the column at that level). Used by the Lagrangian
primal-recovery step ("fix-and-resolve"). Vectorised — single
changeColsBounds call.
update_coef ¶
Update a single (row, col) coefficient in the constraint
matrix. Use :meth:row_id_of_cstr / :meth:col_id_of_var to
resolve indices semantically.
add_cut_row ¶
Append a >= constraint row to the live (already-built) LP and
return the new row's id.
Appends Σ coefs[k] · x[col_ids[k]] >= lower to the live HiGHS
model via addRow. This is a POST-build mutation of the warm
_h handle (the same class of live edit as :meth:update_rhs /
:meth:update_coef); it deliberately bypasses the build-time
Problem.add_cstr DSL lock, which only guards the fixed-size
Layer-2 autoscale side vectors and is irrelevant once the model is
built. The caller is responsible for keeping the master autoscale
OFF (or for pre-scaling coefs) so the appended row lives on the
built columns' scale — there is no auto-scaling for appended rows.
col_ids may mix existing column ids (resolve via
:meth:col_id_of_var) and ids of columns previously appended with
:meth:add_recourse_col. A subsequent :meth:solve warm
re-optimises the grown model; the appended row's dual is then
readable on the returned :class:Solution by row_dual[row_id]
(read BY ROW ID, not via :meth:Solution.constraint_dual, which
only knows named build-time rows).
Bumps the cached _n_rows (the zero-fill fallback size in
:meth:solve) and appends a generated name to _row_names (kept
index-aligned so name-indexed reads stay correct). Returns the
row_id HiGHS assigned (== the pre-append getNumRow()).
add_recourse_col ¶
Append a single free/bounded column (e.g. a Benders recourse
η) to the live LP and return its col id.
Symmetric with :meth:add_cut_row: a POST-build addCol on the
warm _h handle, bumping the cached _n_cols (zero-fill
fallback size) and _col_names. ±np.inf bounds are mapped to
the HiGHS kHighsInf sentinel. The returned id can be referenced
from a later :meth:add_cut_row; its value is read on the
:class:Solution by col_value[col_id] (appended columns are not
in any Problem._vars frame, so :meth:Solution.value cannot see
them — read by id).
declare_mutable ¶
Declare a set of :class:Param names whose values should be
tracked into LP cells, so :meth:update_param can later push
new values into the live HiGHS instance via changeCoeff.
MUST be called BEFORE the first :meth:solve. Tracking is
opt-in: Params not declared here pay no bookkeeping cost.
Pass the same names that the Params carry on their .name
field — typically the FlexData attribute name ("p_inflow",
"p_penalty_up" etc.).
set_output_flag ¶
Enable or disable HiGHS' native solve log for this problem.
enabled=False mutes the per-solve HiGHS output (the
output_flag HiGHS option) for THIS WarmProblem across all of
its cold / warm / retry solves — callers that drive many parallel
sub-solves (Benders regions, Lagrangian subproblems) use this to
keep stdout clean and emit their own concise progress log instead.
May be called either before the first :meth:solve (the flag is
applied when the HiGHS handle is built) or after it (applied to the
live handle immediately). The preference persists on the handle,
so a single call suffices for the whole lifetime.
update_param ¶
Replace the values of a tracked Param. Every LP cell whose
coefficient was originally a function of param_name is
re-computed from the new Param's values and pushed via
h.changeCoeff.
new_param must be either a scalar (broadcast to all tracked
cells) or a :class:Param whose dim signature matches the
signature recorded for that Param at build time.
Raises if param_name was not in :meth:declare_mutable's
list (silent corruption is worse than a hard error).
col_id_of_var ¶
Return the col_id(s) for a variable.
dims=None returns every col_id in the variable's family
(numpy array, ordered by the var's declaration order).
dims as a tuple of dim values returns the single col_id for
that one cell (a python int). dims as a dict
{dim_name: value} is a partial filter — returns a numpy
array of the matching col_ids.
row_id_of_cstr ¶
Return the row_id(s) for a constraint family. Mirrors
:meth:col_id_of_var.
solve ¶
Solve the LP. First call builds the LP from scratch (same
pipeline as :meth:Problem.solve); subsequent calls just run
HiGHS again on the (possibly updated) live model.
options is honoured on the FIRST solve only — subsequent
solves use the same HiGHS instance. To change options on a
rebuilt LP, drop the WarmProblem and create a new one.
retry_on_unknown (default False → byte-identical to the
legacy path for every existing caller) enables a warm-restart
retry: the solver runs WARM first (retaining the basis so dual
simplex hot-starts after rows appended via :meth:add_cut_row);
only if the warm run does NOT return a certified kOptimal —
i.e. a stale-basis transient (kUnknown / kSolveError) or a
spurious kUnbounded / kInfeasible / primal-infeasible miss — do we
drop the basis with clearSolver() and re-run ONCE (the proven
cold fallback) to re-certify the true status. Used by the Benders
master, which appends a cut each iteration; on a well-scaled
master the warm path stays kOptimal and the fallback never
fires.
Lag ¶
Return an Expr that, for each (carry_dims, time_dim) in
lag_frame, references var at (carry_dims, lag_col).
Used for shifting variables in time, e.g. for storage state-change:
v_state[n, d, t] - v_state[n, d, t_prev]
= v_state - Lag(v_state, dtttdt, "t", "t_prev_within_timeset")
lag_frame carries the (d, t, t_prev) lookup; carry_dims are
the columns shared between var and lag_frame other than the
time dim itself (typically d).
Where ¶
Inner-join an Expr against frame. Two effects in one op:
- Filter — rows of the term whose shared-column values don't
appear in
frameare dropped (e.g.Where(v_flow, wind_only)keeps only the wind rows). - Map — any columns of
framethat the term doesn't already carry become new open dims of the resulting term (e.g.Where(v_flow, flow_to_n)whereflow_to_nhas columns(p, source, sink, n)addsnso the term can be bound to a constraint indexed by(n, t)).
The pure-filter case (no extras) does not bake the join into
t.lazy — it records frame into _Term.where_frames so
the LHS prune-down (_build_lhs_pruned_plan) can apply it at the
leaf-rebuild step (where the row count is bounded by the smaller of
Var.frame and the row_index key set). var_source / Param
chain / coef_scalar are preserved.
The map-effect case (extras non-empty) ALSO defers: frame is
recorded into _Term.where_map_frames as (frame_lf,
frozenset(extras)), the term's dims are extended with the extras,
and var_source / Param chain / coef_scalar / where_frames
are preserved. The deferred inner-join (which produces the extras
dim columns) is baked at leaf-rebuild time (or at Sum / Lag /
consumer fallback) via :func:_apply_where_map_frames. Param
multiplies AFTER the map-effect Where bake first iff their dims
overlap any pending extras (correctness); otherwise the deferral
propagates through.
Set POLAR_HIGH_DISABLE_WHERE_PUSHDOWN=1 to recover today's
behaviour (eager join + clear metadata) verbatim for BOTH branches.
Sum ¶
Aggregate an Expr. over lists the dims to sum out; the
remaining dims become the term's open dims. where is an index
frame that pre-filters the term frames (inner join on shared
columns) before the group-by-sum.
Sum(expr) with over=None collapses every open dim — useful
for a scalar (objective term, single-row constraint).
Any deferred :func:Where filters recorded on each term
(t.where_frames) are baked into t.lazy before the
aggregation — Sum collapses dims so the filter could not be
applied at the leaf level downstream. The result clears
var_source / where_frames (today's behaviour); the
per-term where kwarg is applied on top, unchanged.
polar_high.parallel ¶
Deterministic parallel solving of independent :class:WarmProblems.
A small, domain-agnostic utility for cutting-plane / decomposition drivers
(e.g. Benders region recourse, Lagrangian subproblems) that need to solve N
independent subproblems each outer iteration. Each subproblem is its own
:class:~polar_high.engine.WarmProblem (its own HiGHS handle), and HiGHS'
run() releases the GIL, so a thread pool yields a real wall-clock speedup
over the sequential loop.
The thread-safety contract this module encapsulates (so the caller's algorithm code never touches the HiGHS-threading detail):
-
Single-threaded HiGHS scheduler. Every subproblem solve runs HiGHS with a single-threaded scheduler. This is required for (a) determinism — HiGHS is non-deterministic with
threads > 1— and (b) to avoidworkers × coresoversubscription. The process-global HiGHS scheduler is pinned to one thread ONCE up front via :func:prewarm_global_scheduler; the per-subproblemrun()calls then inherit that pinned pool. -
Sequential cold first build. The FIRST
WarmProblem.solve()builds the HiGHS model and, if it sees athreads/paralleloption, calls the process-globalresetGlobalScheduler— unsafe to run concurrently. This module therefore only parallelizes solves over WarmProblems that are ALREADY built (wp._h is not None); it raises if asked to fan out an unbuilt one. Callers must do the cold first build sequentially (or rely on the scheduler pre-pin) before calling :func:solve_indexed_parallel. -
Deterministic per-index collection. Results are collected into per-index slots and returned in index order, so the outcome is identical regardless of thread timing. Worker exceptions are re-raised in index order (the lowest failing index wins, matching the sequential loop).
workers <= 1 keeps a fully sequential path (no pool, no prewarm) that is
byte-identical to a plain for loop — so the parallel and sequential code
paths can be exercised against each other for a determinism gate.
resolve_worker_count ¶
Clamp a requested worker count to [1, n_items].
workers is None auto-resolves to min(n_items, cpu_count - 1) (at
least 1) — leave one core for the main thread / OS. A non-positive request
means sequential (1).
prewarm_global_scheduler ¶
Initialize HiGHS' process-global task scheduler ONCE, single-threaded,
so subsequent concurrent solves on distinct Highs instances need not
each call resetGlobalScheduler. Best-effort; returns False if any
highspy step fails (the caller then falls back to a sequential path).
Once this returns True the global scheduler is pinned to threads and
a subsequent run() with NO threads option inherits that pool — so
concurrent solves need not (and must not) pass threads per instance,
which would re-trigger resetGlobalScheduler and is unsafe to run
concurrently.
solve_indexed_parallel ¶
solve_indexed_parallel(warmproblems: Sequence[WarmProblem], fn: Callable[[int], T], *, workers: int | None = None) -> list[T]
Run fn(i) for every index i of warmproblems deterministically,
optionally across a thread pool, and return the results in index order.
fn(i) is the caller's per-subproblem work: typically it pins the i-th
:class:WarmProblem's coupling columns, calls wp.solve(), and extracts
a domain result (objective + dual slopes). Because each WarmProblem owns
its own HiGHS handle, fn(i) for distinct i touch disjoint state and
are safe to run concurrently — provided every WarmProblem is already built
(the cold first build calls the process-global resetGlobalScheduler and
must run sequentially up front). This helper enforces that precondition.
Parameters¶
warmproblems
The per-index :class:WarmProblems. Each MUST already be built
(solve() called at least once); a ValueError is raised
otherwise. Only used for the built-precondition check + count — the
actual solving is delegated to fn.
fn
fn(i) -> result for index i. May run on a worker thread; must
confine its mutations to the i-th WarmProblem's HiGHS handle.
workers
Effective worker count (see :func:resolve_worker_count). None
auto-resolves to min(n, cpu_count - 1). <= 1 runs a fully
sequential path on the calling thread (no pool, no scheduler prewarm) —
byte-identical to a plain for loop.
Returns¶
list
[fn(0), fn(1), ..., fn(n-1)] — always in index order, independent of
thread timing.