On Minimization of Entropy Functionals Under Moment Constraints

Imre Csiszár
Alfréd Rényi Institute of Mathematics

May 16, 2008
Elements of Information Theory Workshop
Stanford University

The following problem is addressed: For a strictly convex and differentiable function γ on the open interval (0, +∞), with γ (0) defined by continuity, and a measurable vector valued function φ = (φ0 , . . . , φd ) on a σ-finite measure space, with φ0 identically 1, minimize the integral of γ (g) for functions g ≥ 0 such that the integral of gφ equals a given vector a. Special cases include maximization of Shannon or Burg entropy, and minimization of Lp norm, under moment constraints. The problem will be treated in its relationship with Bregman distances, covering also cases when only a generalized minimizer exists that does not satisfy the constraints. Recent results of Csisz´ ar and Mat´ uˇs, not requiring the usual constraint qualification (Slater condition), will also be presented. The main tools are convex duality, and the concept of convex core of a measure introduced by Csisz´ ar and Mat´ uˇs.