Entropy Maximization

Here is a constrained entropy maximization problem:

\[\begin{array}{ll} \text{maximize} & -\sum_{i=1}^n x_i \log x_i \\ \text{subject to} & \mathbf{1}' x = 1 \\ & Ax \leq b \end{array}\]

where $x \in \mathbf{R}^n$ is our optimization variable and $A \in \mathbf{R}^{m \times n}, b \in \mathbf{R}^{m}$.

To solve this, we can simply use the entropy operation Convex.jl provides.

using Convex, SCS

n = 25;
m = 15;
A = randn(m, n);
b = rand(m, 1);

x = Variable(n);
problem = maximize(entropy(x), sum(x) == 1, A * x <= b)
solve!(problem, SCS.Optimizer; silent = true)

Problem statistics
  problem is DCP         : true
  number of variables    : 1 (25 scalar elements)
  number of constraints  : 2 (16 scalar elements)
  number of coefficients : 391
  number of atoms        : 6

Solution summary
  termination status : OPTIMAL
  primal status      : FEASIBLE_POINT
  dual status        : FEASIBLE_POINT
  objective value    : 3.1232

Expression graph
  maximize
   └─ sum (concave; real)
      └─ entropy (concave; real)
         └─ 25-element real variable (id: 135…360)
  subject to
   ├─ == constraint (affine)
   │  └─ + (affine; real)
   │     ├─ sum (affine; real)
   │     │  └─ …
   │     └─ [-1;;]
   └─ ≤ constraint (affine)
      └─ + (affine; real)
         ├─ * (affine; real)
         │  ├─ …
         │  └─ …
         └─ 15×1 Matrix{Float64}

evaluate(x)

25-element Vector{Float64}:
 0.04073644905190682
 0.0495185014520281
 0.03871603394974694
 0.019100323731602392
 0.015864046392695366
 0.026989270634199203
 0.037749244416734606
 0.03376883373222509
 0.055361785157703715
 0.049483462595470665
 ⋮
 0.035957758305254654
 0.09575936166405558
 0.08303645075153872
 0.04549096367850228
 0.02776053175370694
 0.024692168317226152
 0.023808041919073125
 0.03961346086566964
 0.04209321629313468

This page was generated using Literate.jl.