On the Conjecture of Berry Regarding a Bernoulli Two-Armed Bandit

Abstract

In this paper, we study an independent Bernoulli two-armed bandit with unknown parameters ρ and λ, where ρ and λ have a pair of priori distributions such that dR(ρ)=CRρr0(1−ρ)r0′dμ(ρ),dL(λ)=CLλl0(1−λ)l0′dμ(λ) and μ is an arbitrary positive measure on [0,1]. Berry proposed the conjecture that, given a pair of priori distributions (R,L) of parameters ρ and λ, the arm with R is the current optimal choice if r0+r0′l0+l0′ and the expectation of ρ is not less than that of λ. We give an easily verifiable equivalent form of Berry’s conjecture and use it to prove that Berry’s conjecture holds when R and L are two-point distributions as well as when R and L are beta distributions and the number of trials N≤r0r0′+1.

Keywords

• Bernoulli two-armed bandit
• stochastically maximizing
• prior distributions
• Bayesian decision theory