The Benchmarks submodule

To aid the development of efficient solver wrappers, MathOptInterface provides a suite of benchmarks in the MOI.Benchmarks submodule.

import PkgPkg.add("BenchmarkTools")import BenchmarkTools

Benchmarking a solver wrapper

Benchmarking a wrapper follows a two-step process.

First, prior to making changes, create a baseline for the benchmark results on a given benchmark suite as follows:

# You must load BenchmarkTools.jl to enable MOI.Benchmarksimport BenchmarkTools# Replace `SolverPackage` with your choice of solverusing SolverPackageimport MathOptInterface as MOIsuite = MOI.Benchmarks.suite() do    SolverPackage.Optimizer()endMOI.Benchmarks.create_baseline(    suite, "current"; directory = "/tmp", verbose = true)

Use the exclude argument to Benchmarks.suite to exclude benchmarks that the solver doesn't support.

Second, after making changes to the package, re-run the benchmark suite and compare to the prior saved results:

import BenchmarkToolsusing SolverPackageimport MathOptInterface as MOIsuite = MOI.Benchmarks.suite() do    SolverPackage.Optimizer()endMOI.Benchmarks.compare_against_baseline(    suite, "current"; directory = "/tmp", verbose = true)

This comparison will create a report detailing improvements and regressions.