"A General Minimax Result for Relative Entropy"
David Haussler, Submitted to IEEE Transactions on Information Theory
[postscript]
Abstract:
Suppose nature picks a probability measure P_theta on a complete
separable metric space X at random from a measurable set
P_Theta = {P_theta : theta \in Theta}. Then, without knowing
theta, a statistician picks a measure Q on X. Finally, the
statistician suffers a loss D(P_\theta||Q), the relative entropy
between P_theta and Q. We show that the minimax and maximin
values of this game are always equal, and there is always a minimax
strategy in the closure of the set of all Bayes strategies. This
generalizes previous results of Gallager, and Davisson and Leon-Garcia.