An Open Energy Modeling update

open-energy-modeling  · 

Author: Oscar Dowson, Ivet Galabova, Joaquim Dias Garcia, Julian Hall

We’re now four months into our Open Energy Modeling project. This blog post is a summary of some of things that we have been up to.

Community engagement

We have increased engagement between the developers of JuMP and HiGHS and the open energy modeling community. We have done this by holding fortnightly stakeholder calls, publishing bi-monthly updates on the JuMP blog (like this one), and interacting with the developers of energy modeling frameworks by opening issues and pull requests on GitHub. As a general comment, we (the developers of JuMP and HiGHS) now have a much better understanding of the types of problems that are of interest to energy modelers, and these problems have “unusual” characteristics that are not typical of the wider optimization community.

If you are an open energy modeller who uses JuMP or HiGHS and you want to stay in touch with our progress or provide us with feedback and examples, write to jump-highs-energy-models@googlegroups.com. We’d love to hear how you are using JuMP or HiGHS to solve problems related to open energy modelling.

JuMP-dev

We held an energy modeling session at JuMP-dev 2024, which was held in July in Montreal, Canada. All talks from the workshop are available on YouTube. We published a report summarizing energy modeling talks from all past JuMP-dev workshops. We plan to hold further sessions at the HiGHS Workshop in July and JuMP-dev 2025 in November.

open-energy-modeling-benchmarks

In collaboration with energy modelers from GenX, Sienna, Tulipa, and others, we built open-energy-modeling-benchmarks. This GitHub repository is both a collection of energy-focused benchmark instances for MIP solvers, and a collection of scripts for the end-to-end profiling of JuMP-based energy modeling frameworks.

Open Energy Transition

We have started a collaboration with Open Energy Transition (OET) on their solver-benchmark project. In particular, we have shared our benchmark instances with them and adopted their metadata format for structuring benchmarks so that OET can import any future updates to our benchmarks with minimal effort. In addition, we have incorporated their benchmarks from PyPSA into our analysis of HiGHS, and we have maintained a back-and-forth dialog on how to appropriately benchmark MIP solvers. We have also chimed in on some issues for their linopy modeling framework (linopy#402).

Profiling

We used HiGHS to profile the benchmark instances from open-energy-modeling-benchmarks, OET, and Hans Mittelmann’s MIPLIB. These benchmarks revealed a significant difference between the performance of HiGHS on the general purpose MIPLIB set and the performance of HiGHS on the energy-focused models. We will have more to share publicly at a later date.

Performance improvements

Using the initial results of our benchmarks, we have begun implementing performance improvements to HiGHS. As one example, a model from Tulipa that did not solve in a time limit of 3 hours can now be solved in 250 seconds (HiGHS#2049).

Critical bugs

Working with the developers of Sienna and Tulipa, we found and fixed a number of critical bugs in HiGHS. This included multiple segfaults (HiGHS#1990, HiGHS#2109), a case where HiGHS failed to detect that the problem did not have a solution (HiGHS#1962), and a case where HiGHS returned an incorrect solution because the energy modeler was running with a set of non-default options in an attempt to improve performance (HiGHS#1935).

As part of this work, we implemented a new way to debug the interaction of JuMP and HiGHS that allowed us to reproduce and fix a segfault that had been undiagnosed for one year (HiGHS.jl#258).

GenX

Working with the developers of GenX, we identified a number of possible performance improvements to the GenX codebase (GenX#773). As a result, we made a number of changes to both GenX (GenX#815) and JuMP (MutableArithmetics#302), with the result that GenX models are now 12% faster to build.

Sienna

Working with the developers of Sienna, we identified a number of performance bottlenecks in their code and in JuMP:

The net impact of these changes is that the time spent by Sienna outside of the HiGHS solver is reduced by half. The overall benchmark time depends on the specific model and how long HiGHS takes to solve the model. Currently, the total runtime improvement is approximately 5% faster with 15% less memory required, but this percentage will increase as HiGHS becomes faster.

Working with the developers of Sienna, we identified a number of common Julia and JuMP-related anti-patterns in the Sienna code base (PowerSimulations#1218). This included unnecessary array copies when indexing, suboptimal loop order when iterating over matrices, and type stability issues when re-using variable names within a function. Although each of these issues have a minor runtime impact, their cumulative effect can be large, if hard to benchmark. Therefore, our suggested best practice is to avoid these issues in the first place. In addition to providing guidance to the Sienna developers, we will update the JuMP documentation to better highlight these anti-patterns for the benefit of all energy system developers (JuMP#2348).

We also identified that an optional check of the input data was a critical performance bottleneck in Sienna’s optimization pipeline. The check iterated over the data in HiGHS to identify common mistakes such as mis-matched units, or trivially infeasible solutions such as initial generator setpoints outside their operating bounds. The way this algorithm was implemented in HiGHS meant that the time taken increased with the square of the problem size, so larger models were particularly affected. We rewrote the algorithm so that the time taken now increases linearly with problem size and, in our initial benchmarking, the check went from taking minutes to never taking more than a few seconds (HiGHS#2114). Benchmarking this in the context of an end-to-end run of Sienna will require a new release of HiGHS.

Tulipa

Working with the developers of Tulipa, we identified a bottleneck in their model building pipeline that was caused by the informative string names of variables and constraints used when debugging. Turning off these names when not debugging led to a 30% decrease in the time taken to build the model (Tulipa#936).

SpineOpt

Working with the developers of SpineOpt, we identified an issue with their code base that leads to non-deterministic model creation (SpineOpt#1143). This can mean that, all else equal, the same model solved on two different machines can return different solutions. This can make benchmarking and reproducibility difficult. The Spine developers are investigating.

HiGHS.jl

The profiling of energy system models revealed that adding variables to a HiGHS model can be a bottleneck. We implemented changes to JuMP and HiGHS.jl so that adding bounded variables is now three times faster than before (HiGHS.jl#248).

Other changes

We have extended the local testing of HiGHS, making it possible to run unit tests from an external repository. The external unit tests are built with CMake and Catch2, consistent with the ones already in HiGHS. It is also possible to use private problem instances for additional checks, as we consider introducing modifications to the code. A thorough analysis of updates is very helpful as we implement more major updates to the solution algorithms. The separation of new tests from the HiGHS code allows for more comprehensive testing with no additional code shipped to the users, keeping the solver lightweight and easy to integrate within other systems.

The code coverage functionality in HiGHS was updated and coverage reports are now generated and uploaded to Codecov on each PR to our development and master branches (HiGHS#2112, HiGHS#2138).

We have developed local benchmarking infrastructure for use during the development of HiGHS. Running specific test sets, we record output from the HiGHS library, as well as collecting additional debugging and development output, to test the correctness and speed of HiGHS and any potential improvements. For some of the experiment results presented here, a dedicated local server was used, with no other tasks running, required for reliable timing results. The new benchmark infrastructure will allow us to gather such results much faster, in an automated and reproducible way.