Entropy Maximization

Here is a constrained entropy maximization problem:

$$$\begin{array}{ll} \text{maximize} & -\sum_{i=1}^n x_i \log x_i \\ \text{subject to} & \mathbf{1}' x = 1 \\ & Ax \leq b \end{array}$$$

where $x \in \mathbf{R}^n$ is our optimization variable and $A \in \mathbf{R}^{m \times n}, b \in \mathbf{R}^{m}$.

To solve this, we can simply use the entropy operation Convex.jl provides.

using Convex, SCS

n = 25;
m = 15;
A = randn(m, n);
b = rand(m, 1);

x = Variable(n);
problem = maximize(entropy(x), sum(x) == 1, A * x <= b)
solve!(problem, SCS.Optimizer; silent = true)
Problem statistics
problem is DCP         : true
number of variables    : 1 (25 scalar elements)
number of constraints  : 2 (16 scalar elements)
number of coefficients : 391
number of atoms        : 6

Solution summary
termination status : OPTIMAL
primal status      : FEASIBLE_POINT
dual status        : FEASIBLE_POINT
objective value    : 3.1232

Expression graph
maximize
└─ sum (concave; real)
└─ entropy (concave; real)
└─ 25-element real variable (id: 113…489)
subject to
├─ == constraint (affine)
│  └─ + (affine; real)
│     ├─ sum (affine; real)
│     │  └─ …
│     └─ [-1;;]
└─ ≤ constraint (affine)
└─ + (affine; real)
├─ * (affine; real)
│  ├─ …
│  └─ …
└─ 15×1 Matrix{Float64}

evaluate(x)
25-element Vector{Float64}:
0.04073644905190682
0.0495185014520281
0.03871603394974694
0.019100323731602392
0.015864046392695366
0.026989270634199203
0.037749244416734606
0.03376883373222509
0.055361785157703715
0.049483462595470665
⋮
0.035957758305254654
0.09575936166405558
0.08303645075153872
0.04549096367850228
0.02776053175370694
0.024692168317226152
0.023808041919073125
0.03961346086566964
0.04209321629313468