Skip to content

Commit

Permalink
add highs docs
Browse files Browse the repository at this point in the history
  • Loading branch information
trevorb1 committed May 3, 2024
1 parent 9e0914b commit b1f9c6f
Show file tree
Hide file tree
Showing 7 changed files with 421 additions and 32 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Changelog
===========================
- Lock pandas to 2.1.4 or later
- Capital Investment result calculation fixed
- Add HiGHS as a solver

Version 1.1.2
=============
Expand Down
282 changes: 281 additions & 1 deletion docs/_static/otoole_figures.drawio

Large diffs are not rendered by default.

Binary file modified docs/_static/overview_v2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/_static/workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 3 additions & 2 deletions docs/convert.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ See :py:func:`otoole.convert.convert` for more details
Converting solver results to a folder of CSV files
--------------------------------------------------

The ``convert_results`` function creates a folder of CSV result files from a CBC_, CLP_,
The ``convert_results`` function creates a folder of CSV result files from a CBC_, CLP_, HiGHS_,
Gurobi_ or CPLEX_ solution file::

>>> from otoole import convert_results
Expand All @@ -34,7 +34,7 @@ See :func:`otoole.convert.convert_results` for more details
Reading solver results into a dict of Pandas DataFrames
-------------------------------------------------------

The ``read_results`` function reads a CBC_, CLP_,
The ``read_results`` function reads a CBC_, CLP_, HiGHS_,
Gurobi_ or CPLEX_ solution file into memory::

>>> from otoole import read_results
Expand Down Expand Up @@ -68,3 +68,4 @@ You can use the :py:func:`otoole.convert.write` function to write data out to di
.. _CLP: https://github.com/coin-or/Clp
.. _CPLEX: https://www.ibm.com/products/ilog-cplex-optimization-studio/cplex-optimizer
.. _Gurobi: https://www.gurobi.com/
.. _HiGHS: https://ergo-code.github.io/HiGHS/dev/
129 changes: 118 additions & 11 deletions docs/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ Solver Setup
Objective
~~~~~~~~~

Install GLPK_ (required) and CBC_ (optional) to use in the otoole examples.
While ``otoole`` does not require a solver, these examples will use the free
and open source solvers GLPK_ and CBC_.
Install GLPK_ (required), CBC_ (optional), and HiGHS_ (optional) to use in the otoole
examples. While ``otoole`` does not require a solver, these examples will use the free
and open source solvers GLPK_, CBC_, and HiGHS_.

1. Install GLPK
~~~~~~~~~~~~~~~~
Expand All @@ -50,8 +50,6 @@ your Python packages, you can install GLPK via the command::

$ conda install -c conda-forge glpk

2. Test the GLPK install
~~~~~~~~~~~~~~~~~~~~~~~~
Once installed, you should be able to call the ``glpsol`` command::

$ glpsol
Expand All @@ -61,7 +59,7 @@ Once installed, you should be able to call the ``glpsol`` command::
.. TIP::
See the `GLPK Wiki`_ for more information on the ``glpsol`` command

3. Install CBC
2. Install CBC
~~~~~~~~~~~~~~

CBC_ is a free and open-source mixed integer linear programming solver. Full
Expand All @@ -84,8 +82,6 @@ your Python packages, you can install CBC via the command::

$ conda install -c conda-forge coincbc

4. Test the CBC install
~~~~~~~~~~~~~~~~~~~~~~~
Once installed, you should be able to directly call CBC::

$ cbc
Expand All @@ -99,6 +95,60 @@ Once installed, you should be able to directly call CBC::

You can exit the solver by typing ``quit``

3. Install HiGHS
~~~~~~~~~~~~~~~~

HiGHS_ is a free and open-source mixed integer linear programming solver. HiGHS_ can be
run through the command line or through one of their
`API interfaces <https://ergo-code.github.io/HiGHS/dev/interfaces/python/>`_. See the
`HiGHS install instructions <https://ergo-code.github.io/HiGHS/dev/installation/>`_ for full
details on both methods. Here we will cover abbreviated instructions for CLI and Python
execution.

CLI Install
+++++++++++

To install the HiGHS executable, install a pre-compiled binary as directed by
the `HiGHS install documentation <https://ergo-code.github.io/HiGHS/dev/installation/#Precompiled-Binaries>`_.

Extract the binary with the following command on MacOS/Linux::

$ tar -xzf filename.tar.gz

Navigate to the ``./bin/`` folder and run HiGHS from the command line::

$ ./highs

.. TIP::
To call HiGHS_ from anywhere in the command line, add the path to the execultable
to your environment variables.

Once installed, you should be able to directly call HiGHS_::

$ highs
Please specify filename in .mps|.lp|.ems format.

Python Install
++++++++++++++

In Python, simple install HiGHS through pip with::

$ pip install highspy

Once installed, you should be able to see ``highspy`` in your environment::

$ pip show highspy
Name: highspy
Version: 1.5.3
Summary: Python interface to HiGHS
Home-page: https://github.com/ergo-code/highs
Author:
Author-email:
License: MIT
Location: /home/xxx/.local/lib/python3.10/site-packages
Requires:
Required-by:

Input Data Conversion
---------------------

Expand Down Expand Up @@ -145,7 +195,7 @@ Process Solutions from Different Solvers
Objective
~~~~~~~~~

Process solutions from GLPK_, CBC_, Gurobi_, and CPLEX_. This example assumes
Process solutions from GLPK_, CBC_, HiGHS_, Gurobi_, and CPLEX_. This example assumes
you have an existing GNU MathProg datafile called ``simplicity.txt`` (from the
previous example).

Expand Down Expand Up @@ -175,7 +225,62 @@ save the solution as ``simplicity.sol``. Use otoole to create a folder of CSV re

$ otoole results cbc csv simplicity.sol results csv data config.yaml

3. Process a solution from Gurobi
3. Process a solution from HiGHS (CLI)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use GLPK_ to build the model and save the problem as ``simplicity.lp``. Use HiGHS_ to solve the model and
save the solution as ``simplicity.sol``. Use otoole to create a folder of CSV results called ``results/``.

HiGHS_ has the ability to write solutions in a variety of formats; ``otoole`` will process the
``kSolutionStylePretty`` solution style. We pass this into the HiGHS_ solver through an
`options file <https://ergo-code.github.io/HiGHS/dev/options/intro/#Options-file>`_. First, create the options file::

$ touch highs_options.txt

And add the following option to the file::

write_solution_style = 1

Next, we can follow a similar process to processing results from other solvers::

$ glpsol -m OSeMOSYS.txt -d simplicity.txt --wlp simplicity.lp --check

$ highs --model_file simplicity.lp --solution_file simplicity.sol --options_file="highs_options.txt"

$ otoole results highs csv simplicity.sol results csv data config.yaml

.. NOTE::
Run the following command to see all the options available to pass into highs in the options file::

$ highs --options_file=""

4. Process a solution from HiGHS (Python)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use `HiGHS Python API`_ to solve a model, and use otoole's Python API to create a folder of CSV results called ``results/``.
HiGHS can process models in both ``.mlp`` and CPLEX ``.lp`` format. This example will assume you have a model file called ``simplicity.lp``
already created. This can be created through GLPK following the first command in the previous example.

First, install HiGHS and otoole into your Python environment::

$ pip install highspy
$ pip install otoole

Next, use HiGHS to solve the model and write a solution file::

import highspy
h = highspy.Highs()
h.readModel("simplicity.lp")
h.run()
h.writeSolution("simplicity.sol", 1)

.. warning::
The HiGHS_ solution style **must be** solution style ``1`` (ie. ``kSolutionStylePretty``)

Finally, use otoole to convert the solution file into a folder of result CSVs::

from otoole import convert_results
convert_results("config.yaml", "highs", "csv", "simplicity.sol", "results", "csv", "data")

1. Process a solution from Gurobi
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use GLPK_ to build the model and save the problem as ``simplicity.lp``. Use Gurobi_ to solve the model and
save the solution as ``simplicity.sol``. Use otoole to create a folder of CSV results called ``results/`` from the solution file::
Expand All @@ -186,7 +291,7 @@ save the solution as ``simplicity.sol``. Use otoole to create a folder of CSV re

$ otoole results gurobi csv simplicity.sol results csv data config.yaml

4. Process a solution from CPLEX
6. Process a solution from CPLEX
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use GLPK_ to build the model and save the problem as ``simplicity.lp``. Use CPLEX_ to solve the model and
save the solution as ``simplicity.sol``. Use otoole to create a folder of CSV results called ``results/`` from the solution file::
Expand Down Expand Up @@ -504,3 +609,5 @@ will also flag it as an isolated fuel. This means the fuel is unconnected from t
.. _Anaconda: https://www.anaconda.com/
.. _Gurobi: https://www.gurobi.com/
.. _graphviz: https://www.graphviz.org/download/
.. _HiGHS: https://ergo-code.github.io/HiGHS/dev/
.. _HiGHS Python API: https://ergo-code.github.io/HiGHS/dev/interfaces/python/
36 changes: 18 additions & 18 deletions docs/functionality.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ to turbo-charge your modelling, are described on this page!
As shown in the diagram, ``otoole`` deals primarily with data before and after OSeMOSYS.
Data work prior to the generation of a datafile, which is read in by GLPK_, is called
pre-processing. Anything that happens with the immediate outputs of a solver, such as
the recommended open-source solvers GLPK_, CBC_, or the proprietary solvers CPLEX_ and
the recommended open-source solvers GLPK_, CBC_, HiGHS_, or the commercial solvers CPLEX_ and
Gurobi_, is called results and post-processing.

.. image:: _static/workflow.png
Expand Down Expand Up @@ -71,11 +71,10 @@ Overview
With small OSeMOSYS models, it is normally fine to use the free open-source GLPK_ solver.
If you do, then OSeMOSYS will write out a full set of results as a folder of CSV files.
As you progress to larger models, the performance constraints of GLPK_ quickly become
apparent. CBC_ is an alternative open-source solver which offers better performance than
GLPK_ and can handle much larger models. However, CBC_ has no way of knowing how to write
out the CSV files you were used to dealing with when using GLPK_. ``otoole`` to the rescue!
apparent. CBC_ and HiGHS_ are alternative open-source solvers which offer better performance than
GLPK_ and can handle much larger models.

``otoole`` currently supports using GLPK_, CBC_, CPLEX_ or Gurobi_ with all versions of
``otoole`` currently supports using GLPK_, CBC_, HiGHS_, CPLEX_ or Gurobi_ with all versions of
GNU MathProg OSeMOSYS - the long, short and fast versions.

The long version includes all results as variables within the formulation, so the
Expand All @@ -94,21 +93,21 @@ Gurobi_ or CPLEX_ solution file together with the input data::

$ otoole results --help
usage: otoole results [-h] [--glpk_model GLPK_MODEL] [--write_defaults]
{cbc,cplex,gurobi} {csv} from_path to_path {csv,datafile,excel} input_path config
{cbc,cplex,glpk,gurobi,highs} {csv} from_path to_path {csv,datafile,excel} input_path config

positional arguments:
{cbc,cplex,glpk,gurobi} Result data format to convert from
{csv} Result data format to convert to
from_path Path to file or folder to convert from
to_path Path to file or folder to convert to
{csv,datafile,excel} Input data format
input_path Path to input_data
config Path to config YAML file

optional arguments:
-h, --help show this help message and exit
--glpk_model GLPK_MODEL GLPK model file required for processing GLPK results
--write_defaults Writes default values
{cbc,cplex,glpk,gurobi,highs} Result data format to convert from
{csv} Result data format to convert to
from_path Path to solver solution file
to_path Path to file or folder to convert to
{csv,datafile,excel} Input data format
input_path Path to input_data
config Path to config YAML file

options:
-h, --help show this help message and exit
--glpk_model GLPK_MODEL GLPK model file required for processing GLPK results
--write_defaults Writes default values

.. versionadded:: v1.0.0
The ``config`` positional argument is now required
Expand Down Expand Up @@ -218,3 +217,4 @@ the rest of the model::
.. _Gurobi: https://www.gurobi.com/
.. _`OSeMOSYS Repository`: https://github.com/OSeMOSYS/OSeMOSYS_GNU_MathProg/tree/master/scripts
.. _graphviz: https://graphviz.org/
.. _HiGHS: https://ergo-code.github.io/HiGHS/dev/

0 comments on commit b1f9c6f

Please sign in to comment.