Probability Puzzle: Expected Number of Expression Evaluations
Source: Asked for a data scientist position at a technology startup in financial services sector in London
Problem:
def find_max(x):
max_num = x[0]
for i in x[1:]:
if i > max_num:
max_num = i << Expected number of times this expression was evaluated
return max_num
Calculate the expected number of times the expression max_num = i was evaluated, given that array x was taken from a uniform random distribution.
Solution:
Highlight the part between the * symbols for the answer.
* Answer: H_n - 1 = 1/2 + 1/3 + ... + 1/n, which is about ln(n) - 1 + gamma (roughly log n updates).
Reason: the line max_num = i executes on iteration i exactly when the i-th element is the largest of the first i elements. For i.i.d. draws (with ties having probability zero, e.g. continuous or effectively continuous distributions), each of the first i elements is equally likely to be the maximum, so this happens with probability 1/i. By linearity of expectation the expected number of executions is the sum over i = 2..n of 1/i.
(Caveat from suyash in the comments: for a discrete uniform range with many repeats, ties make the probability smaller than 1/i and the answer drops - in the extreme of a one-value range it is 0.)
Solution by Sandeep and an Anonymous commenter from the comments, with suyash's discrete-range caveat. *
answer is 1/2 + 1/3 + 1/4...+ 1/n
ReplyDeleteprobability that j-th element is the maximum till now = 1/j
sum over all such j's from 2 to n
Your answer looks correct...
DeleteAssuming total no of elements N. (N+1)/2
ReplyDeleteAssumption : Elements come in random order.
ReplyDeleteConsider the ith iteration, the probability that the ith element is the largest amongst the first i elements is simply 1/i since all are equally likely to be largest. So E[no. of times the line is evaluated for ith iteration] = 1/i = E_i. So,
E[total number of assignments to var max_num] = summation of E_i for i=1 to n. This is approximately LogN for large N.
n/2 by there are infinitely many integers on either side of any integer x and thus probability that next number in array is greater than current maximum is 1/2. Thus expected number is n/2
ReplyDeleteSince we are talking about an integer array, the uniform distribution from which the elements are picked is discrete in nature. And if the range of this distribution is finite, the statement "the probability that the ith element is the largest amongst the first i elements is simply 1/i since all are equally likely to be largest" is not true.
ReplyDeleteConsider, for example, a range consisting of only a single integer. Then the expected value is clearly 0. Or a range containing less than LogN integers, then the expected value is clearly less than LogN. The argument holds, however, if the probability of repitition of elements in the array is negligible, which is true if the range is infinite or the array contains real numbers instead of integers.
This comment has been removed by the author.
ReplyDeleteCorrect (for distinct values): the assignment at element j happens iff x[j] is the max of the first j, probabilit
ReplyDeleteNo - that would be right if each new element beat the current max with probability 1/2, but that probability at s
ReplyDeleteCorrect - expected assignments = sum_{i=2..n} 1/i = H_n - 1, which is Theta(log n). (Fine print: distinct values;
ReplyDeleteThe 1/2 logic fails: the next element is NOT an independent 50-50 against the current max - the max of the first
ReplyDeleteExcellent subtlety and correct - with ties, P(new max at position i) = P(x_i > max of first i-1) < 1/i, and for
ReplyDelete(For the record: the harmonic-number answer, H_n - 1, is the correct one for distinct values.)
ReplyDelete