-
Notifications
You must be signed in to change notification settings - Fork 518
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding (parameterized) linear programming dual transformation! #3402
Open
emma58
wants to merge
53
commits into
Pyomo:main
Choose a base branch
from
emma58:lp-dual
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,401
−18
Open
Changes from all commits
Commits
Show all changes
53 commits
Select commit
Hold shift + click to select a range
c2b42b5
Initializing dual transform
emma58 8fccf37
Whoops, adding LP dual transform and tests
emma58 40925d0
Merge branch 'main' into lp-dual
emma58 ab58d80
First draft of 'mixed' LP dual as well as adding option for parameter…
emma58 601471f
Merge branch 'linear-walker-wrt' into lp-dual
emma58 16fdff4
Beginning of mixed form LP dal, no parameterized, with tests that don…
emma58 a1cb793
Fixing a bug with the direction of constraints being transposed, also…
emma58 eb44531
Making Constraint expressions prettier and testing them
emma58 8420e05
black
emma58 d702d57
Adding a test that we can recover the primal from the dual with the t…
emma58 ad0ef8e
Testing that dual values reported by solver and what we get from solv…
emma58 06c87c9
Not bothering to transpose primal constraint matrix because we don't …
emma58 e237311
Centralizing the var list domain validator in pyomo/util
emma58 d3fdccb
Modularizing standard form a little so that I can override the parts …
emma58 73e07f0
Draft of scipy-style CSR and CSC matrices that accomodate pyomo expre…
emma58 27e09f5
Rewriting LP dual code to actually use a CSC matrix sanely
emma58 fd276d0
whoops, missing file from two commits ago
emma58 ff38080
Implementing todense, making the data structures in CSC and CSR numpy…
emma58 8d582c9
Start on unit tests for parameterized standard form
emma58 4c32954
Blacking
emma58 8f87684
More unit tests (and whitespace)
emma58 ced3faa
Adding a test for todense and removing my hacky debugging in paramete…
emma58 0c9567a
Making _csc_to_nonnegative_vars a method on the class so that it can …
emma58 d0d9abf
Testing all the config variations for parameterized standard form
emma58 39f6e40
Black
emma58 7173461
Renaming _get_data_list to _to_vector
emma58 0a5663a
Partial fix merge of templatized-writer branch. Standard repn works b…
emma58 89defeb
Generalizing standard form to handle parameterized standard form again
emma58 18c848c
black
emma58 6548f29
NFC: cleaning up comments
emma58 40fbad3
Resolving merge conflicts with the create_csc abstraction
emma58 22a1f79
Merge branch 'main' into lp-dual
emma58 470de80
Adding a copy of scipy's sum duplicates code, but I think we can chea…
emma58 d1bf1b4
Merge branch 'templatized-writer' into lp-dual
emma58 3edc964
Removing an unused method
emma58 53ea29e
Merging with templatized-writer branch
emma58 c52a915
Fixing conflicts with paramaterized linear walker PR
emma58 d86e00b
Adjusting parameterized standard form to match the changes in standar…
emma58 c569897
black
emma58 70d1fbe
Converting the c matrix dense for now to get things working
emma58 d7d3b74
Adding a test of a parameterized dual
emma58 82326af
black
emma58 76ac53d
Merging main
emma58 6f5790a
Converting to var_recorder argument for parameterized standard form
emma58 139d307
black
emma58 2f40eb9
More black
emma58 5a25e2a
NFC: fixing comment typo
emma58 c2c5c0b
Adding docstrings for mapping API and clearning up some prepositional…
emma58 7238c31
Adding tests for paraemeterized dual from solving primal and dual, ad…
emma58 d914e52
Black
emma58 d92d4df
NFC: fixing two typos
emma58 662fbfb
Reverting unnecessary formatting changes in standard form
emma58 8f07edb
Merge branch 'main' into lp-dual
emma58 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -22,4 +22,5 @@ | |
add_slack_vars, | ||
scaling, | ||
logical_to_linear, | ||
lp_dual, | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,261 @@ | ||
# ___________________________________________________________________________ | ||
# | ||
# Pyomo: Python Optimization Modeling Objects | ||
# Copyright (c) 2008-2024 | ||
# National Technology and Engineering Solutions of Sandia, LLC | ||
# Under the terms of Contract DE-NA0003525 with National Technology and | ||
# Engineering Solutions of Sandia, LLC, the U.S. Government retains certain | ||
# rights in this software. | ||
# This software is distributed under the 3-clause BSD License. | ||
# ___________________________________________________________________________ | ||
|
||
from pyomo.common.autoslots import AutoSlots | ||
from pyomo.common.collections import ComponentMap | ||
from pyomo.common.config import ConfigDict, ConfigValue | ||
from pyomo.common.errors import MouseTrap | ||
from pyomo.common.dependencies import scipy | ||
from pyomo.core import ( | ||
ConcreteModel, | ||
Block, | ||
Var, | ||
Constraint, | ||
Objective, | ||
TransformationFactory, | ||
NonNegativeReals, | ||
NonPositiveReals, | ||
maximize, | ||
minimize, | ||
Reals, | ||
) | ||
from pyomo.opt import WriterFactory | ||
from pyomo.repn.standard_repn import isclose_const | ||
from pyomo.util.var_list_domain import var_component_set | ||
|
||
|
||
class _LPDualData(AutoSlots.Mixin): | ||
__slots__ = ('primal_var', 'dual_var', 'primal_constraint', 'dual_constraint') | ||
|
||
def __init__(self): | ||
self.primal_var = {} | ||
self.dual_var = {} | ||
self.primal_constraint = ComponentMap() | ||
self.dual_constraint = ComponentMap() | ||
|
||
|
||
Block.register_private_data_initializer(_LPDualData) | ||
|
||
|
||
@TransformationFactory.register( | ||
'core.lp_dual', 'Generate the linear programming dual of the given model' | ||
) | ||
class LinearProgrammingDual(object): | ||
CONFIG = ConfigDict("core.lp_dual") | ||
CONFIG.declare( | ||
'parameterize_wrt', | ||
ConfigValue( | ||
default=None, | ||
domain=var_component_set, | ||
description="Vars to treat as data for the purposes of taking the dual", | ||
doc=""" | ||
Optional list of Vars to be treated as data while taking the LP dual. | ||
For example, if this is the dual of the inner problem in a multilevel | ||
optimization problem, then the outer problem's Vars would be specified | ||
in this list since they are not variables from the perspective of the | ||
inner problem. | ||
""", | ||
), | ||
) | ||
|
||
def apply_to(self, model, **options): | ||
raise MouseTrap( | ||
"The 'core.lp_dual' transformation does not currently implement " | ||
"apply_to since it is a bit ambiguous what it means to take a dual " | ||
"in place. Please use 'create_using' and do what you wish with the " | ||
"returned model." | ||
) | ||
|
||
def create_using(self, model, ostream=None, **kwds): | ||
"""Take linear programming dual of a model | ||
Returns | ||
------- | ||
ConcreteModel containing linear programming dual | ||
Parameters | ||
---------- | ||
model: ConcreteModel | ||
The concrete Pyomo model to take the dual of | ||
ostream: None | ||
This is provided for API compatibility with other writers | ||
and is ignored here. | ||
""" | ||
config = self.CONFIG(kwds.pop('options', {})) | ||
config.set_value(kwds) | ||
|
||
if config.parameterize_wrt is None: | ||
std_form = WriterFactory('compile_standard_form').write( | ||
model, mixed_form=True, set_sense=None | ||
) | ||
else: | ||
std_form = WriterFactory('parameterized_standard_form_compiler').write( | ||
model, wrt=config.parameterize_wrt, mixed_form=True, set_sense=None | ||
) | ||
return self._take_dual(model, std_form) | ||
|
||
def _take_dual(self, model, std_form): | ||
if len(std_form.objectives) != 1: | ||
raise ValueError( | ||
"Model '%s' has no objective or multiple active objectives. Cannot " | ||
"take dual with more than one objective!" % model.name | ||
) | ||
primal_sense = std_form.objectives[0].sense | ||
|
||
dual = ConcreteModel(name="%s dual" % model.name) | ||
# This is a csc matrix, so we'll skip transposing and just work off | ||
# of the columns | ||
A = std_form.A | ||
c = std_form.c.todense() | ||
dual_rows = range(A.shape[1]) | ||
dual_cols = range(A.shape[0]) | ||
dual.x = Var(dual_cols, domain=NonNegativeReals) | ||
trans_info = dual.private_data() | ||
for j, (primal_cons, ineq) in enumerate(std_form.rows): | ||
# maximize is -1 and minimize is +1 and ineq is +1 for <= and -1 for | ||
# >=, so we need to change domain to NonPositiveReals if the product | ||
# of these is +1. | ||
if primal_sense * ineq == 1: | ||
dual.x[j].domain = NonPositiveReals | ||
elif ineq == 0: | ||
# equality | ||
dual.x[j].domain = Reals | ||
trans_info.primal_constraint[dual.x[j]] = primal_cons | ||
trans_info.dual_var[primal_cons] = dual.x[j] | ||
|
||
dual.constraints = Constraint(dual_rows) | ||
for i, primal in enumerate(std_form.columns): | ||
lhs = 0 | ||
for j in range(A.indptr[i], A.indptr[i + 1]): | ||
coef = A.data[j] | ||
primal_row = A.indices[j] | ||
lhs += coef * dual.x[primal_row] | ||
|
||
if primal.domain is Reals: | ||
dual.constraints[i] = lhs == c[0, i] | ||
elif primal_sense is minimize: | ||
if primal.domain is NonNegativeReals: | ||
dual.constraints[i] = lhs <= c[0, i] | ||
else: # primal.domain is NonPositiveReals | ||
dual.constraints[i] = lhs >= c[0, i] | ||
else: | ||
if primal.domain is NonNegativeReals: | ||
dual.constraints[i] = lhs >= c[0, i] | ||
else: # primal.domain is NonPositiveReals | ||
dual.constraints[i] = lhs <= c[0, i] | ||
trans_info.dual_constraint[primal] = dual.constraints[i] | ||
trans_info.primal_var[dual.constraints[i]] = primal | ||
|
||
dual.obj = Objective( | ||
expr=sum(std_form.rhs[j] * dual.x[j] for j in dual_cols), | ||
sense=-primal_sense, | ||
) | ||
|
||
return dual | ||
|
||
def get_primal_constraint(self, model, dual_var): | ||
"""Return the primal constraint corresponding to 'dual_var' | ||
Returns | ||
------- | ||
Constraint | ||
Parameters | ||
---------- | ||
model: ConcreteModel | ||
A dual model returned from the 'core.lp_dual' transformation | ||
dual_var: Var | ||
A dual variable on 'model' | ||
""" | ||
primal_constraint = model.private_data().primal_constraint | ||
if dual_var in primal_constraint: | ||
return primal_constraint[dual_var] | ||
else: | ||
raise ValueError( | ||
"It does not appear that Var '%s' is a dual variable on model '%s'" | ||
% (dual_var.name, model.name) | ||
) | ||
|
||
def get_dual_constraint(self, model, primal_var): | ||
"""Return the dual constraint corresponding to 'primal_var' | ||
Returns | ||
------- | ||
Constraint | ||
Parameters | ||
---------- | ||
model: ConcreteModel | ||
A primal model passed as an argument to the 'core.lp_dual' transformation | ||
primal_var: Var | ||
A primal variable on 'model' | ||
""" | ||
dual_constraint = model.private_data().dual_constraint | ||
if primal_var in dual_constraint: | ||
return dual_constraint[primal_var] | ||
else: | ||
raise ValueError( | ||
"It does not appear that Var '%s' is a primal variable on model '%s'" | ||
% (primal_var.name, model.name) | ||
) | ||
|
||
def get_primal_var(self, model, dual_constraint): | ||
"""Return the primal variable corresponding to 'dual_constraint' | ||
Returns | ||
------- | ||
Var | ||
Parameters | ||
---------- | ||
model: ConcreteModel | ||
A dual model returned from the 'core.lp_dual' transformation | ||
dual_constraint: Constraint | ||
A constraint on 'model' | ||
""" | ||
primal_var = model.private_data().primal_var | ||
if dual_constraint in primal_var: | ||
return primal_var[dual_constraint] | ||
else: | ||
raise ValueError( | ||
"It does not appear that Constraint '%s' is a dual constraint on " | ||
"model '%s'" % (dual_constraint.name, model.name) | ||
) | ||
|
||
def get_dual_var(self, model, primal_constraint): | ||
"""Return the dual variable corresponding to 'primal_constraint' | ||
Returns | ||
------- | ||
Var | ||
Parameters | ||
---------- | ||
model: ConcreteModel | ||
A primal model passed as an argument to the 'core.lp_dual' transformation | ||
primal_constraint: Constraint | ||
A constraint on 'model' | ||
""" | ||
dual_var = model.private_data().dual_var | ||
if primal_constraint in dual_var: | ||
return dual_var[primal_constraint] | ||
else: | ||
raise ValueError( | ||
"It does not appear that Constraint '%s' is a primal constraint on " | ||
"model '%s'" % (primal_constraint.name, model.name) | ||
) | ||
Comment on lines
+167
to
+261
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The code in all of these is basically exactly the same minus the thing that you are getting. You could abstract this by doing something like:
|
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think
MouseTrap
is the right error here.NotImplementedError
would make more sense.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MouseTrap
inherits from (and is a special case of)NotImplementedError