Probability Puzzle: Expected Number of Expression Evaluations

Source: Asked for a data scientist position at a technology startup in financial services sector in London

Problem:

Given python code to calculate maximum in an array of integers x

def find_max(x):
    
    max_num = x[0]
    
    for i in x[1:]:
        if i > max_num:
            max_num = i << Expected number of times this expression was evaluated

     return max_num
Calculate the expected number of times the expression max_num = i was evaluated, given that array x was taken from a uniform random distribution.



Solution:

Highlight the part between the * symbols for the answer.
* Answer: H_n - 1 = 1/2 + 1/3 + ... + 1/n, which is about ln(n) - 1 + gamma (roughly log n updates).

Reason: the line max_num = i executes on iteration i exactly when the i-th element is the largest of the first i elements. For i.i.d. draws (with ties having probability zero, e.g. continuous or effectively continuous distributions), each of the first i elements is equally likely to be the maximum, so this happens with probability 1/i. By linearity of expectation the expected number of executions is the sum over i = 2..n of 1/i.

(Caveat from suyash in the comments: for a discrete uniform range with many repeats, ties make the probability smaller than 1/i and the answer drops - in the extreme of a one-value range it is 0.)

Solution by Sandeep and an Anonymous commenter from the comments, with suyash's discrete-range caveat. *

Comments

  1. Anonymous3:14 AM, August 03, 2013

    answer is 1/2 + 1/3 + 1/4...+ 1/n

    probability that j-th element is the maximum till now = 1/j

    sum over all such j's from 2 to n

    ReplyDelete
  2. vikas9:47 AM, August 03, 2013

    Assuming total no of elements N. (N+1)/2

    ReplyDelete
  3. Sandeep11:36 AM, August 03, 2013

    Assumption : Elements come in random order.
    Consider the ith iteration, the probability that the ith element is the largest amongst the first i elements is simply 1/i since all are equally likely to be largest. So E[no. of times the line is evaluated for ith iteration] = 1/i = E_i. So,
    E[total number of assignments to var max_num] = summation of E_i for i=1 to n. This is approximately LogN for large N.

    ReplyDelete
  4. sumit somani2:34 PM, August 03, 2013

    n/2 by there are infinitely many integers on either side of any integer x and thus probability that next number in array is greater than current maximum is 1/2. Thus expected number is n/2

    ReplyDelete
  5. suyash3:16 PM, August 04, 2013

    Since we are talking about an integer array, the uniform distribution from which the elements are picked is discrete in nature. And if the range of this distribution is finite, the statement "the probability that the ith element is the largest amongst the first i elements is simply 1/i since all are equally likely to be largest" is not true.

    Consider, for example, a range consisting of only a single integer. Then the expected value is clearly 0. Or a range containing less than LogN integers, then the expected value is clearly less than LogN. The argument holds, however, if the probability of repitition of elements in the array is negligible, which is true if the range is infinite or the array contains real numbers instead of integers.

    ReplyDelete
  6. Unknown4:58 PM, October 05, 2015

    This comment has been removed by the author.

    ReplyDelete

Post a Comment

Popular posts from this blog

Polya's Urn Problem: Expected Balls in the Smaller Urn

Puzzle: There are two urns with one ball each. Each of subsequent n-2 balls is placed into one of these urns, with probability proportional to the number of balls already in that urn. What is the expected number of balls in the smaller sized urn? Source: P. Winkler's Puzzles book. (Chapter: Probability). Solution: Highlight the part between the * symbols for the answer. * This problem can be reformulated as the following problem. Suppose I have a stack of black cards and one red card. Initially I take red card in my hand. Now I add black cards randomly between any two cards (so, initially its either above or below red). Note that the probability that I add the card above the red card, when x-1 is the number of cards above red and y-1 is the number of cards below red is x/(x+y). Let the problem be if red card is dividing the black cards into two sets, what is the expected number of black cards in the smaller section. So, we see that the two problems are equivalent. No...
Read more

Lion and Man in a Circular Cage: Can the Lion Catch the Tamer?

Source: Asked to me by Pramod Ganapathi (PhD Student at Stony Brook University) Problem: A lion and a lion tamer are enclosed within a circular cage. If they move at the same speed but are both restricted by the cage, can the lion catch the lion tamer? (Represent the cage by a circle, and the lion and lion tamer as two point masses within it.) Solution: Highlight the part between the * symbols for the answer. * Answer: surprisingly, no - the tamer can avoid being caught forever. This is the famous Lion and Man problem, posed in the 1930s and settled by Besicovitch; the proof appears in Littlewood's "A Mathematician's Miscellany" (and opens Bollobas's "The Art of Mathematics"). What the lion CAN do (Rado's strategy): move to the center, then always stay on the radius through the tamer, using any spare speed to move outward. Since the lion runs on a smaller circle, it can match the tamer's angular position while steadily increasing its radiu...
Read more

Expected Tosses for Consecutive Heads: 2 Heads vs 3 Heads Puzzle

Let's say A keep tossing a fair coin, until he get 2 consecutive heads, define X to be the number of tosses for this process; B keep tossing another fair coin, until he get 3 consecutive heads, define Y to be the number of the tosses for this process. 1) Calculate P{X>Y} 2) What's the expected value of X 3) What's the expected value of Y This is probably the hardest puzzle I have ever put on my blog. Hence, I will post its solution in the post directly rather than on comment. Solution: 1) (Solved by me finally after 13 months :)) Make a state diagram. Let the state be (a,b) where a is the number of consecutive heads streak "A" is on currently and b is the number of consecutive heads streak "B" is on currently. So, (0,3) (1,3) are final accepted states and (2,0) (2,1) (2,2) (2,3) are failure states. If you get tails, your contribution to the state reaches to "0" f(State) = P(X>Y | "State" configuration initially) ...
Read more