Containers
JuMP provides specialized containers similar to AxisArrays that enable multi-dimensional arrays with non-integer indices.
These containers are created automatically by JuMP's macros. Each macro has the same basic syntax:
@macroname(model, name[key1=index1, index2; optional_condition], other stuff)The containers are generated by the name[key1=index1, index2; optional_condition] syntax. Everything else is specific to the particular macro.
Containers can be named, for example, name[key=index], or unnamed, for example, [key=index]. We call unnamed containers anonymous.
We call the bits inside the square brackets and before the ; the index sets. The index sets can be named, for example, [i = 1:4], or they can be unnamed, for example, [1:4].
We call the bit inside the square brackets and after the ; the condition. Conditions are optional.
In addition to the standard JuMP macros like @variable and @constraint, which construct containers of variables and constraints respectively, you can use Containers.@container to construct containers with arbitrary elements.
We will use this macro to explain the three types of containers that are natively supported by JuMP: Array, Containers.DenseAxisArray, and Containers.SparseAxisArray.
Array
An Array is created when the index sets are rectangular and the index sets are of the form 1:n.
julia> Containers.@container(x[i = 1:2, j = 1:3], (i, j))2×3 Matrix{Tuple{Int64, Int64}}: (1, 1) (1, 2) (1, 3) (2, 1) (2, 2) (2, 3)The result is a normal Julia Array, so you can do all the usual things.
Slicing
Arrays can be sliced
julia> x[:, 1]2-element Vector{Tuple{Int64, Int64}}: (1, 1) (2, 1)julia> x[2, :]3-element Vector{Tuple{Int64, Int64}}: (2, 1) (2, 2) (2, 3)Looping
Use eachindex to loop over the elements:
julia> for key in eachindex(x) println(x[key]) end(1, 1)(2, 1)(1, 2)(2, 2)(1, 3)(2, 3)Get the index sets
Use axes to obtain the index sets:
julia> axes(x)(Base.OneTo(2), Base.OneTo(3))Broadcasting
Broadcasting over an Array returns an Array
julia> swap(x::Tuple) = (last(x), first(x))swap (generic function with 1 method)julia> swap.(x)2×3 Matrix{Tuple{Int64, Int64}}: (1, 1) (2, 1) (3, 1) (1, 2) (2, 2) (3, 2)Tables
Use Containers.rowtable to convert the Array into a Tables.jl compatible Vector{<:NamedTuple}:
julia> table = Containers.rowtable(x; header = [:I, :J, :value])6-element Vector{@NamedTuple{I::Int64, J::Int64, value::Tuple{Int64, Int64}}}: (I = 1, J = 1, value = (1, 1)) (I = 2, J = 1, value = (2, 1)) (I = 1, J = 2, value = (1, 2)) (I = 2, J = 2, value = (2, 2)) (I = 1, J = 3, value = (1, 3)) (I = 2, J = 3, value = (2, 3))Because it supports the Tables.jl interface, you can pass it to any function which accepts a table as input:
julia> import DataFrames;julia> DataFrames.DataFrame(table)6×3 DataFrame Row │ I J value │ Int64 Int64 Tuple…─────┼────────────────────── 1 │ 1 1 (1, 1) 2 │ 2 1 (2, 1) 3 │ 1 2 (1, 2) 4 │ 2 2 (2, 2) 5 │ 1 3 (1, 3) 6 │ 2 3 (2, 3)DenseAxisArray
A Containers.DenseAxisArray is created when the index sets are rectangular, but not of the form 1:n. The index sets can be of any type.
julia> x = Containers.@container([i = 1:2, j = [:A, :B]], (i, j))2-dimensional DenseAxisArray{Tuple{Int64, Symbol},2,...} with index sets: Dimension 1, Base.OneTo(2) Dimension 2, [:A, :B]And data, a 2×2 Matrix{Tuple{Int64, Symbol}}: (1, :A) (1, :B) (2, :A) (2, :B)Slicing
A Containers.DenseAxisArray can be sliced:
julia> x[:, :A]1-dimensional DenseAxisArray{Tuple{Int64, Symbol},1,...} with index sets: Dimension 1, Base.OneTo(2)And data, a 2-element Vector{Tuple{Int64, Symbol}}: (1, :A) (2, :A)julia> x[1, :]1-dimensional DenseAxisArray{Tuple{Int64, Symbol},1,...} with index sets: Dimension 1, [:A, :B]And data, a 2-element Vector{Tuple{Int64, Symbol}}: (1, :A) (1, :B)Looping
Use eachindex to loop over the elements:
julia> for key in eachindex(x) println(x[key]) end(1, :A)(2, :A)(1, :B)(2, :B)Get the index sets
Use axes to obtain the index sets:
julia> axes(x)(Base.OneTo(2), [:A, :B])Broadcasting
Broadcasting over a Containers.DenseAxisArray returns Containers.DenseAxisArray:
julia> swap(x::Tuple) = (last(x), first(x))swap (generic function with 1 method)julia> swap.(x)2-dimensional DenseAxisArray{Tuple{Symbol, Int64},2,...} with index sets: Dimension 1, Base.OneTo(2) Dimension 2, [:A, :B]And data, a 2×2 Matrix{Tuple{Symbol, Int64}}: (:A, 1) (:B, 1) (:A, 2) (:B, 2)Access internal data
Use Array(x) to copy the internal data array into a new Array:
julia> Array(x)2×2 Matrix{Tuple{Int64, Symbol}}: (1, :A) (1, :B) (2, :A) (2, :B)To access the internal data without a copy, use x.data.
julia> x.data2×2 Matrix{Tuple{Int64, Symbol}}: (1, :A) (1, :B) (2, :A) (2, :B)Tables
Use Containers.rowtable to convert a Containers.DenseAxisArray into a Tables.jl compatible Vector{<:NamedTuple}:
julia> table = Containers.rowtable(x; header = [:I, :J, :value])4-element Vector{@NamedTuple{I::Int64, J::Symbol, value::Tuple{Int64, Symbol}}}: (I = 1, J = :A, value = (1, :A)) (I = 2, J = :A, value = (2, :A)) (I = 1, J = :B, value = (1, :B)) (I = 2, J = :B, value = (2, :B))Because it supports the Tables.jl interface, you can pass it to any function which accepts a table as input:
julia> import DataFrames;julia> DataFrames.DataFrame(table)4×3 DataFrame Row │ I J value │ Int64 Symbol Tuple…─────┼──────────────────────── 1 │ 1 A (1, :A) 2 │ 2 A (2, :A) 3 │ 1 B (1, :B) 4 │ 2 B (2, :B)Keyword indexing
If all axes are named, you can use keyword indexing:
julia> x[i = 2, j = :A](2, :A)julia> x[i = :, j = :B]1-dimensional DenseAxisArray{Tuple{Int64, Symbol},1,...} with index sets: Dimension 1, Base.OneTo(2)And data, a 2-element Vector{Tuple{Int64, Symbol}}: (1, :B) (2, :B)SparseAxisArray
A Containers.SparseAxisArray is created when the index sets are non-rectangular. This occurs in two circumstances:
An index depends on a prior index:
julia> Containers.@container([i = 1:2, j = i:2], (i, j))JuMP.Containers.SparseAxisArray{Tuple{Int64, Int64}, 2, Tuple{Int64, Int64}} with 3 entries: [1, 1] = (1, 1) [1, 2] = (1, 2) [2, 2] = (2, 2)The [indices; condition] syntax is used:
julia> x = Containers.@container([i = 1:3, j = [:A, :B]; i > 1], (i, j))JuMP.Containers.SparseAxisArray{Tuple{Int64, Symbol}, 2, Tuple{Int64, Symbol}} with 4 entries: [2, A] = (2, :A) [2, B] = (2, :B) [3, A] = (3, :A) [3, B] = (3, :B)Here we have the index sets i = 1:3, j = [:A, :B], followed by ;, and then a condition, which evaluates to true or false: i > 1.
Slicing
Slicing is supported:
julia> y = x[:, :B]JuMP.Containers.SparseAxisArray{Tuple{Int64, Symbol}, 1, Tuple{Int64}} with 2 entries: [2] = (2, :B) [3] = (3, :B)Looping
Use eachindex to loop over the elements:
julia> for key in eachindex(x) println(x[key]) end(2, :A)(2, :B)(3, :A)(3, :B)julia> for key in eachindex(y) println(y[key]) end(2, :B)(3, :B)If you use a macro to construct a Containers.SparseAxisArray, then the iteration order is row-major, that is, indices are varied from right to left. As an example, when iterating over x above, the j index is iterated, keeping i constant. This order is in contrast to Base.Array, which iterates in column-major order, that is, by varying indices from left to right.
Broadcasting
Broadcasting over a Containers.SparseAxisArray returns a Containers.SparseAxisArray:
julia> swap(x::Tuple) = (last(x), first(x))swap (generic function with 1 method)julia> swap.(y)JuMP.Containers.SparseAxisArray{Tuple{Symbol, Int64}, 1, Tuple{Int64}} with 2 entries: [2] = (:B, 2) [3] = (:B, 3)Tables
Use Containers.rowtable to convert the Containers.SparseAxisArray into a Tables.jl compatible Vector{<:NamedTuple}:
julia> table = Containers.rowtable(x; header = [:I, :J, :value])4-element Vector{@NamedTuple{I::Int64, J::Symbol, value::Tuple{Int64, Symbol}}}: (I = 2, J = :A, value = (2, :A)) (I = 2, J = :B, value = (2, :B)) (I = 3, J = :A, value = (3, :A)) (I = 3, J = :B, value = (3, :B))Because it supports the Tables.jl interface, you can pass it to any function which accepts a table as input:
julia> import DataFrames;julia> DataFrames.DataFrame(table)4×3 DataFrame Row │ I J value │ Int64 Symbol Tuple…─────┼──────────────────────── 1 │ 2 A (2, :A) 2 │ 2 B (2, :B) 3 │ 3 A (3, :A) 4 │ 3 B (3, :B)Keyword indexing
If all axes are named, you can use keyword indexing:
julia> x[i = 2, j = :A](2, :A)julia> x[i = :, j = :B]JuMP.Containers.SparseAxisArray{Tuple{Int64, Symbol}, 1, Tuple{Int64}} with 2 entries: [2] = (2, :B) [3] = (3, :B)Forcing the container type
Pass container = T to use T as the container. For example:
julia> Containers.@container([i = 1:2, j = 1:2], i + j, container = Array)2×2 Matrix{Int64}: 2 3 3 4julia> Containers.@container([i = 1:2, j = 1:2], i + j, container = Dict)Dict{Tuple{Int64, Int64}, Int64} with 4 entries: (1, 2) => 3 (1, 1) => 2 (2, 2) => 4 (2, 1) => 3You can also pass DenseAxisArray or SparseAxisArray.
How different container types are chosen
If the compiler can prove at compile time that the index sets are rectangular, and indexed by a compact set of integers that start at 1, Containers.@container will return an array. This is the case if your index sets are visible to the macro as 1:n:
julia> Containers.@container([i=1:3, j=1:5], i + j)3×5 Matrix{Int64}: 2 3 4 5 6 3 4 5 6 7 4 5 6 7 8or an instance of Base.OneTo:
julia> set = Base.OneTo(3)Base.OneTo(3)julia> Containers.@container([i=set, j=1:5], i + j)3×5 Matrix{Int64}: 2 3 4 5 6 3 4 5 6 7 4 5 6 7 8If the compiler can prove that the index set is rectangular, but not necessarily of the form 1:n at compile time, then a Containers.DenseAxisArray will be constructed instead:
julia> set = 1:31:3julia> Containers.@container([i=set, j=1:5], i + j)2-dimensional DenseAxisArray{Int64,2,...} with index sets: Dimension 1, 1:3 Dimension 2, Base.OneTo(5)And data, a 3×5 Matrix{Int64}: 2 3 4 5 6 3 4 5 6 7 4 5 6 7 8What happened here? Although we know that set contains 1:3, at compile time the typeof(set) is a UnitRange{Int}. Therefore, Julia can't prove that the range starts at 1 (it only finds this out at runtime), and it defaults to a DenseAxisArray. The case where we explicitly wrote i = 1:3 worked because the macro can "see" the 1 at compile time.
However, if you know that the indices do form an Array, you can force the container type with container = Array:
julia> set = 1:31:3julia> Containers.@container([i=set, j=1:5], i + j, container = Array)3×5 Matrix{Int64}: 2 3 4 5 6 3 4 5 6 7 4 5 6 7 8Here's another example with something similar:
julia> a = 11julia> Containers.@container([i=a:3, j=1:5], i + j)2-dimensional DenseAxisArray{Int64,2,...} with index sets: Dimension 1, 1:3 Dimension 2, Base.OneTo(5)And data, a 3×5 Matrix{Int64}: 2 3 4 5 6 3 4 5 6 7 4 5 6 7 8julia> Containers.@container([i=1:a, j=1:5], i + j)1×5 Matrix{Int64}: 2 3 4 5 6Finally, if the compiler cannot prove that the index set is rectangular, a Containers.SparseAxisArray will be created.
This occurs when some indices depend on a previous one:
julia> Containers.@container([i=1:3, j=1:i], i + j)JuMP.Containers.SparseAxisArray{Int64, 2, Tuple{Int64, Int64}} with 6 entries: [1, 1] = 2 [2, 1] = 3 [2, 2] = 4 [3, 1] = 4 [3, 2] = 5 [3, 3] = 6or if there is a condition on the index sets:
julia> Containers.@container([i = 1:5; isodd(i)], i^2)JuMP.Containers.SparseAxisArray{Int64, 1, Tuple{Int64}} with 3 entries: [1] = 1 [3] = 9 [5] = 25The condition can depend on multiple indices, the only requirement is that it is an expression that returns true or false:
julia> condition(i, j) = isodd(i) && iseven(j)condition (generic function with 1 method)julia> Containers.@container([i = 1:2, j = 1:4; condition(i, j)], i + j)JuMP.Containers.SparseAxisArray{Int64, 2, Tuple{Int64, Int64}} with 2 entries: [1, 2] = 3 [1, 4] = 5