finally updating EDI

doc/OnlineDocs/contributed_packages/edi/blackboxconstraints.rst

@@ -116,7 +116,9 @@ The ``BlackBox`` is extremely flexible, but here we present standard usage for a
 
 The ``BlackBox`` method assumes to take in the inputs as arguments in the order defined during the ``__init__()`` method.  Note that the method assumes inputs **with units** and expects outputs **with units**.  In general, the units on inputs and outputs need not be in any specific system, but should be convertible (ex, meters and feet) to whatever has been specified as the input units when defining in the ``__init__()`` function.  
 
-The unit handling system in Pyomo can be rough at times, and so the BlackBox function is expected to return values that are ``pint.Quantity`` types.  These are obtained using the ``pyo.as_quantity()`` function.
+Various unpacking schemes are enabled by default via the ``parse_inputs`` function.  Use of this function is not necessary, but provides for the parsing of index argumented lists (ex: ``function(x1, x2, x3)``) and keyword argumented dictionaries (ex: ``function({'x2':x2, 'x1':x1, 'x3',x3})``), along with a few other possibilities.
+
+The unit handling system in Pyomo can be rough at times, and so best practice is generally for the BlackBox function is expected to return values that are ``pint.Quantity`` types.  These are obtained using the ``pyo.as_quantity()`` function.
 
 Since the units cannot be assumed on input, the first step in any black box is to convert to the model units:
 
@@ -200,7 +202,7 @@ In the event that the inputs and/or outputs are non-scalar, then outputs should
     du_dx[0,0,1,0,0] # derivative of u[0,0,1] with respect to x[0,0]
     du_dx[0,0,0,1,1] # derivative of u[0,0,0] with respect to x[1,1]
 
-Note that this may change in the future, as developers are unsatisfied with extensions of this method to second order and higher derivatives.
+Note that this may change in the future, as developers are currently unsatisfied with extensions of this method to second order and higher derivatives.
 
 The BlackBox_Standardized Method
 ********************************
@@ -267,28 +269,18 @@ The MultiCase Method
 ********************
 The ``MultiCase`` method provides a native capability to call the ``BlackBox`` method across multiple inputs simultaneously.  This function is **not** vectorized in the base class and is **not** optimized for performance.  If you wish to have a high performance vectorized function, you will need to implement your own method.
 
-The native ``MultiCase`` function can take in a variety of different input formats:
-
-
-
-
+Inputs to the ``MultiCase`` funciton should be a list of cases, which can be packed in any form accepted by the ``BlackBox_Standardized`` method.  Overloading these functions may allow different forms of unpacking scheme.
 
-
-
-The output is a list of ``NamedTuple`` objects that are output from the ``BlackBox_Standardized`` method.
+The output is a list of ``NamedTuple`` objects that are output from the ``BlackBox_Standardized`` method.  If overloading, you may choose to output via a differnt packing scheme.
 
 Below is an example of overriding the default ``MultiCase`` method:
 
 
-
-
-
-
-
-
-
-
-
+.. literalinclude:: ../../../../pyomo/contrib/edi/tests/test_docSnippets.py
+    :language: python 
+    :dedent: 8
+    :start-after: # BEGIN: RuntimeConstraints_Snippet_13
+    :end-before: # END: RuntimeConstraints_Snippet_13
 
 
 Including a Black-Box in an EDI Formulation
@@ -408,7 +400,7 @@ Tips
 * Embrace units.  They will save you so many times, it is well worth the minor additional overhead
 * Pyomo units work slightly diffenrently than pint (for those with pint experience), but those differences should be hidden from the model creator for the most part
 * It is common to use this framework to call to a piece of software external to python
-* See the :doc:`advanced <./advancedruntimeconstraints>` documentation for extra tips and tricks
+* A model summary can be printed by calling ``print(model_instance.summary)``
 
 
 Known Issues

pyomo/contrib/edi/README.md

@@ -3,7 +3,6 @@
 The Pyomo Engineering Design Interface (EDI) is a lightweight wrapper on the Pyomo language that is targeted at composing engineering design optimization problems.  The language and interface have been designed to mimic many of the features found in [GPkit](https://github.com/convexengineering/gpkit) and [CVXPY](https://github.com/cvxpy/cvxpy) while also providing a simple, clean interface for black-box analysis codes that are common in engineering design applications.
 
 ## TODO
-- Fix broken unit tests
 - Add unit tests for the BlackBox_Standardized method
 - Add unit tests for the MultiCase method
 - Add documentation for the MultiCase method

pyomo/contrib/edi/__init__.py

@@ -24,7 +24,7 @@
 # #     pass
 
 # Recommended just to build all of the appropriate things
-import pyomo.environ
+import pyomo.environ as pyo
 
 # Import the relevant classes from Formulation
 try:
@@ -48,3 +48,35 @@
 except:
     pass
     # in this case, the dependencies are not installed, nothing will work
+
+
+# the printer that does units ok
+import copy
+from pyomo.core.base.units_container import _PyomoUnit
+from pyomo.core.expr.numeric_expr import NPV_ProductExpression, NPV_DivisionExpression
+from collections import namedtuple
+import numpy as np
+
+
+def recursive_sub(x_in):
+    x = list(copy.deepcopy(x_in))
+    for i in range(0, len(x)):
+        if isinstance(x[i], _PyomoUnit):
+            x[i] = '1.0*' + str(x[i])
+        elif (
+            isinstance(
+                x[i], (NPV_ProductExpression, NPV_DivisionExpression, np.float64)
+            )
+            or x[i] is None
+        ):
+            if pyo.value(x[i]) == 1:
+                x[i] = '1.0*' + str(x[i])
+            else:
+                x[i] = str(x[i])
+        else:
+            x[i] = recursive_sub(list(x[i]))
+    return x
+
+
+def ediprint(x):
+    print(recursive_sub(x))

pyomo/contrib/edi/blackBoxFunctionModel.py

@@ -24,6 +24,8 @@
 from pyomo.core.base.units_container import as_quantity
 from pyomo.common.dependencies import attempt_import
 
+from pyomo.core.expr.ndarray import NumericNDArray
+
 # from pyomo.common.numeric_types import RegisterNumericType
 try:
     import pint
@@ -451,7 +453,7 @@ def fillCache(self):
                 modelOutputUnits = opt.units
                 outputOptimizationUnits = optimizationOutput.get_units()
                 vl = valueList[i]
-                if isinstance(vl, pyomo.core.expr.numeric_expr.NumericNDArray):
+                if isinstance(vl, NumericNDArray):
                     validIndexList = optimizationOutput.index_set().data()
                     for j in range(0, len(validIndexList)):
                         vi = validIndexList[j]
@@ -519,9 +521,7 @@ def fillCache(self):
                         ptr_col += 1
                         ptr_row_step = 1
 
-                    elif isinstance(
-                        jacobianValue_raw, pyomo.core.expr.numeric_expr.NumericNDArray
-                    ):
+                    elif isinstance(jacobianValue_raw, NumericNDArray):
                         jshape = jacobianValue_raw.shape
 
                         if isinstance(oopt, pyomo.core.base.var.ScalarVar):
@@ -594,7 +594,7 @@ def fillCache(self):
 
                 ptr_row += ptr_row_step
 
-            print(outputJacobian.dtype)
+            # print(outputJacobian.dtype)
             # print(sps.coo_matrix(outputJacobian))
             # outputJacobian = np.zeros([3,1])
             self._cache['pyomo_jacobian'] = sps.coo_matrix(
@@ -612,6 +612,11 @@ def BlackBox(*args, **kwargs):
     # These functions wrap black box to provide more functionality
 
     def BlackBox_Standardized(self, *args, **kwargs):
+        runCases, extras = self.parseInputs(*args, **kwargs)
+        if len(runCases) != 1:
+            # This is actually a MultiCase, run the correct function instead
+            return self.MultiCase(*args, **kwargs)
+
         # --------
         # Run
         # --------
@@ -628,13 +633,16 @@ def BlackBox_Standardized(self, *args, **kwargs):
         iptTuple = namedtuple('iptTuple', inputNames)
 
         structuredOutput = {}
-        structuredOutput['values'] = len(self.outputs) * [None]
-        structuredOutput['first'] = len(self.outputs) * [len(self.inputs) * [None]]
-        structuredOutput['second'] = len(self.outputs) * [
-            len(self.inputs) * [len(self.inputs) * [None]]
-        ]
+        structuredOutput['values'] = np.zeros(len(self.outputs)).tolist()
+        structuredOutput['first'] = np.zeros(
+            [len(self.outputs), len(self.inputs)]
+        ).tolist()
+        structuredOutput['second'] = np.zeros(
+            [len(self.outputs), len(self.inputs), len(self.inputs)]
+        ).tolist()
 
         opt = toList(opt_raw, ['values', 'first', 'second'])
+        # print(opt)
 
         # --------
         # Values
@@ -655,6 +663,9 @@ def BlackBox_Standardized(self, *args, **kwargs):
                 structuredOutput['values'][i] = self.convert(
                     opt_values[i], self.outputs[i].units
                 )
+            elif isinstance(self.outputs[i].size, int):
+                for ix in range(0, self.outputs[i].size):
+                    structuredOutput['values'][i][ix] = opt_values[i][ix]
             else:
                 listOfIndices = list(
                     itertools.product(*[range(0, n) for n in self.outputs[i].size])
@@ -708,8 +719,11 @@ def BlackBox_Standardized(self, *args, **kwargs):
 
             for i, vl in enumerate(opt_first):
                 for j, vlj in enumerate(opt_first[i]):
-                    # print(opt_first)
-                    # print(opt_first[i])
+                    # from pyomo.contrib.edi import ediprint
+                    # print()
+                    # ediprint(opt_first)
+                    # ediprint(opt_first[i])
+                    # ediprint(opt_first[i][j])
                     # print(self.outputs)
                     # print(self.inputs)
                     # print(i)
@@ -720,35 +734,61 @@ def BlackBox_Standardized(self, *args, **kwargs):
                             self.outputs[i].units / self.inputs[j].units,
                         )
                     elif self.outputs[i].size == 0:
+                        # print(self.inputs[j].size)
+                        if isinstance(self.inputs[j].size, int):
+                            sizelist = [self.inputs[j].size]
+                        else:
+                            assert isinstance(self.inputs[j].size, list)
+                            sizelist = self.inputs[j].size
                         listOfIndices = list(
-                            itertools.product(
-                                *[range(0, n) for n in self.inputs[j].size]
-                            )
+                            itertools.product(*[range(0, n) for n in sizelist])
                         )
+                        # print(listOfIndices)
                         for ix in listOfIndices:
+                            if len(ix) == 1:
+                                ix = ix[0]
                             structuredOutput['first'][i][j][ix] = opt_first[i][j][
                                 ix
                             ]  # unit conversion handled automatically by pint
                     elif self.inputs[j].size == 0:
+                        if isinstance(self.outputs[i].size, int):
+                            sizelist = [self.outputs[i].size]
+                        else:
+                            assert isinstance(self.outputs[i].size, list)
+                            sizelist = self.outputs[i].size
                         listOfIndices = list(
-                            itertools.product(
-                                *[range(0, n) for n in self.outputs[i].size]
-                            )
+                            itertools.product(*[range(0, n) for n in sizelist])
                         )
                         for ix in listOfIndices:
+                            if len(ix) == 1:
+                                ix = ix[0]
                             structuredOutput['first'][i][j][ix] = opt_first[i][j][
                                 ix
                             ]  # unit conversion handled automatically by pint
                     else:
+                        if isinstance(self.inputs[j].size, int):
+                            sizelist_inputs = [self.inputs[j].size]
+                        else:
+                            assert isinstance(self.inputs[j].size, list)
+                            sizelist_inputs = self.inputs[j].size
+
+                        if isinstance(self.outputs[i].size, int):
+                            sizelist_outputs = [self.outputs[i].size]
+                        else:
+                            assert isinstance(self.inputs[j].size, list)
+                            sizelist_outputs = self.outputs[i].size
+
                         listOfIndices = list(
                             itertools.product(
                                 *[
                                     range(0, n)
-                                    for n in self.outputs[i].size + self.inputs[j].size
+                                    for n in sizelist_outputs + sizelist_inputs
                                 ]
                             )
                         )
                         for ix in listOfIndices:
+                            if len(ix) == 1:
+                                ix = ix[0]
                             structuredOutput['first'][i][j][ix] = opt_first[i][j][
                                 ix
                             ]  # unit conversion handled automatically by pint
@@ -826,6 +866,7 @@ def convert(self, val, unts):
 
         try:
             val = val * pyomo_units.dimensionless
+            val = pyo.value(val) * pyomo_units.get_units(val)
         except:
             pass  ## will handle later
 
@@ -836,23 +877,25 @@ def convert(self, val, unts):
                 pyomo.core.expr.numeric_expr.NPV_ProductExpression,
             ),
         ):
-            return pyomo_units.convert(val, unts)
-        elif isinstance(val, pyomo.core.expr.numeric_expr.NumericNDArray):
+            rval = pyomo_units.convert(val, unts)
+            return pyo.value(rval) * pyomo_units.get_units(rval)
+        elif isinstance(val, NumericNDArray):
             shp = val.shape
             ix = np.ndindex(*shp)
             opt = np.zeros(shp)
             for i in range(0, np.prod(shp)):
                 ixt = next(ix)
                 opt[ixt] = pyo.value(pyomo_units.convert(val[ixt], unts))
             return opt * unts
+        # elif isinstance(val, (list, tuple)):
         else:
             raise ValueError('Invalid type passed to unit conversion function')
 
     def pyomo_value(self, val):
         try:
             return pyo.value(val)
         except:
-            if isinstance(val, pyomo.core.expr.numeric_expr.NumericNDArray):
+            if isinstance(val, NumericNDArray):
                 shp = val.shape
                 ix = np.ndindex(*shp)
                 opt = np.zeros(shp)
@@ -867,7 +910,6 @@ def pyomo_value(self, val):
     # ---------------------------------------------------------------------------------------------------------------------
     def parseInputs(self, *args, **kwargs):
         args = list(args)  # convert tuple to list
-
         inputNames = [self.inputs[i].name for i in range(0, len(self.inputs))]
 
         # ------------------------------
@@ -917,7 +959,8 @@ def parseInputs(self, *args, **kwargs):
                         raise ValueError(
                             "Invalid data type in the input list.  Note that BlackBox([x1,x2,y]) must be passed in as BlackBox([[x1,x2,y]]) or "
                             + "BlackBox(*[x1,x2,y]) or BlackBox({'x1':x1,'x2':x2,'y':y}) or simply BlackBox(x1, x2, y) to avoid processing singularities.  "
-                            + "Best practice is BlackBox({'x1':x1,'x2':x2,'y':y})"
+                            + "Best practice is BlackBox({'x1':x1,'x2':x2,'y':y}).  For multi-case input, be sure to wrap even single inputs in lists,"
+                            + ' ex: [[x],[x],[x]] and not [x,x,x].'
                         )
                 return dataRuns, {}
 
@@ -1067,7 +1110,7 @@ def sizeCheck(self, size, ipval_correctUnits):
                         'Size did not match the expected size %s (ie: Scalar)'
                         % (str(size))
                     )
-            elif isinstance(szVal, pyomo.core.expr.numeric_expr.NumericNDArray):
+            elif isinstance(szVal, NumericNDArray):
                 shp = szVal.shape
                 if isinstance(size, (int, float)):
                     size = [size]
@@ -1123,7 +1166,7 @@ def sanitizeInputs(self, *args, **kwargs):
 
             ipval = inputDict[name]
 
-            if isinstance(ipval, pyomo.core.expr.numeric_expr.NumericNDArray):
+            if isinstance(ipval, NumericNDArray):
                 for ii in range(0, len(ipval)):
                     try:
                         ipval[ii] = self.convert(ipval[ii], unts)  # ipval.to(unts)
@@ -1144,7 +1187,7 @@ def sanitizeInputs(self, *args, **kwargs):
 
             # superseded by the custom convert function
             # if not isinstance(ipval_correctUnits, (pyomo.core.expr.numeric_expr.NPV_ProductExpression,
-            #                                        pyomo.core.expr.numeric_expr.NumericNDArray,
+            #                                        NumericNDArray,
             #                                        pyomo.core.base.units_container._PyomoUnit)):
             #     ipval_correctUnits = ipval_correctUnits * pyomo_units.dimensionless
 
@@ -1198,7 +1241,7 @@ def checkOutputs(self, *args, **kwargs):
         #         ipval_correctUnits = ipval
 
         #     if not isinstance(ipval_correctUnits, (pyomo.core.expr.numeric_expr.NPV_ProductExpression,
-        #                                            pyomo.core.expr.numeric_expr.NumericNDArray,
+        #                                            NumericNDArray,
         #                                            pyomo.core.base.units_container._PyomoUnit)):
         #         ipval_correctUnits = ipval_correctUnits * pyomo_units.dimensionless