Entropy Maximization

Here is a constrained entropy maximization problem:

\[\begin{array}{ll} \text{maximize} & -\sum_{i=1}^n x_i \log x_i \\ \text{subject to} & \mathbf{1}' x = 1 \\ & Ax \leq b \end{array}\]

where $x \in \mathbf{R}^n$ is our optimization variable and $A \in \mathbf{R}^{m \times n}, b \in \mathbf{R}^{m}$.

To solve this, we can simply use the entropy operation Convex.jl provides.

using Convex, SCS

n = 25;
m = 15;
A = randn(m, n);
b = rand(m, 1);

x = Variable(n);
problem = maximize(entropy(x), sum(x) == 1, A * x <= b)
solve!(problem, SCS.Optimizer; silent = true)

Problem statistics
  problem is DCP         : true
  number of variables    : 1 (25 scalar elements)
  number of constraints  : 2 (16 scalar elements)
  number of coefficients : 391
  number of atoms        : 6

Solution summary
  termination status : OPTIMAL
  primal status      : FEASIBLE_POINT
  dual status        : FEASIBLE_POINT
  objective value    : 3.1232

Expression graph
  maximize
   └─ sum (concave; real)
      └─ entropy (concave; real)
         └─ 25-element real variable (id: 122…096)
  subject to
   ├─ == constraint (affine)
   │  └─ + (affine; real)
   │     ├─ sum (affine; real)
   │     │  └─ …
   │     └─ [-1;;]
   └─ ≤ constraint (affine)
      └─ + (affine; real)
         ├─ * (affine; real)
         │  ├─ …
         │  └─ …
         └─ 15×1 Matrix{Float64}

evaluate(x)

25-element Vector{Float64}:
 0.04073644905190679
 0.04951850145202862
 0.03871603394974409
 0.01910032373160382
 0.015864046392696576
 0.02698927063419836
 0.03774924441673821
 0.03376883373222556
 0.05536178515769542
 0.049483462595468916
 ⋮
 0.03595775830525436
 0.09575936166405809
 0.08303645075153616
 0.04549096367849687
 0.027760531753709343
 0.024692168317228064
 0.0238080419190761
 0.03961346086567168
 0.04209321629313606

This page was generated using Literate.jl.