Romanian Informatics Olympiad - Modified Huffman Encoding

Source: (Romanian Informatics Olympiad ONI'03, extended team selection)

File:Huffman tree 2.svg

 ( ^ Representative diagram of Huffman Encoding. No relevance in the problem)

 Problem:
A telegraph machine can transmit only lines and dots; it takes 2 seconds to transmit a line, but only 1 second to transmit a dot. We generally want to transmit texts containing letters of the English alphabet, and digits (so we have N<=36 symbols in total). Therefore, a prefix-free encoding using lines and dots is needed. Given the frequencies of the N symbols in a large text, find the minimum time it takes to transmit the text using a suitable encoding. The solution should run in O(N^4) time, and use O(N^3) space.


Solution:

Highlight the part between the * symbols for the answer.
* Official solution (as posted by JDGM from the Olympiad authors): this is Huffman coding with unequal letter costs (dot = 1s, line = 2s), which - unlike the equal-cost case - has no known greedy solution, but does have a polynomial DP.

Think of the code tree as a modified Huffman tree where a "line" child sits 2 levels below its parent. First observation (greedy exchange argument): in an optimal tree, reading the leaves in level order, the symbol frequencies must be non-increasing. So sort the symbols by frequency; only the tree's SHAPE is left to determine.

Build the tree top-down with DP. The state: (number of internal nodes one level above the frontier, number of internal nodes two levels above, number of leaves placed on upper levels). Each expansion choice (open an internal node, or close a leaf and assign the next-most-frequent symbol to it) costs frequency x depth, and the depth is implicit in the state. With N <= 36 symbols the state space is O(N^3) and transitions O(N), giving the required O(N^4) time / O(N^3) space.

Solution by JDGM from the comments (the problem author's official write-up). *

Comments

  1. JDGM1:04 PM, January 19, 2013

    This one has been untouched almost three weeks, so here's the solution in the author's words:

    "This is a special case of Huffman coding with unequal letter costs (a polynomial solution is not known for the general case). We need to construct a modified Huffman tree, in which the right child of a node is considered at depth 2 below the node itself. If we take the level traversal of the leaves in this tree, the symbols must appear in decreasing order of frequency (by a greedy exchange argument). So we must only determine the topology of the tree. This can be done by conceptually building the tree from root to leaves using a dynamic programming algorithm; the state is given by the number of internal nodes one and two levels above, and the total number of leaves on upper levels. Note that we can construct the algorithm so that it doesn't need to include the level in the state (this is needed to get the required complexity). Constant factors are significantly improved if we implement this using a memorized recursive call (it turns out only about 10% of the array needs to be computed)."

    RIP Mihai.

    ReplyDelete

Post a Comment

Popular posts from this blog

Polya's Urn Problem: Expected Balls in the Smaller Urn

Puzzle: There are two urns with one ball each. Each of subsequent n-2 balls is placed into one of these urns, with probability proportional to the number of balls already in that urn. What is the expected number of balls in the smaller sized urn? Source: P. Winkler's Puzzles book. (Chapter: Probability). Solution: Highlight the part between the * symbols for the answer. * This problem can be reformulated as the following problem. Suppose I have a stack of black cards and one red card. Initially I take red card in my hand. Now I add black cards randomly between any two cards (so, initially its either above or below red). Note that the probability that I add the card above the red card, when x-1 is the number of cards above red and y-1 is the number of cards below red is x/(x+y). Let the problem be if red card is dividing the black cards into two sets, what is the expected number of black cards in the smaller section. So, we see that the two problems are equivalent. No...
Read more

Lion and Man in a Circular Cage: Can the Lion Catch the Tamer?

Source: Asked to me by Pramod Ganapathi (PhD Student at Stony Brook University) Problem: A lion and a lion tamer are enclosed within a circular cage. If they move at the same speed but are both restricted by the cage, can the lion catch the lion tamer? (Represent the cage by a circle, and the lion and lion tamer as two point masses within it.) Solution: Highlight the part between the * symbols for the answer. * Answer: surprisingly, no - the tamer can avoid being caught forever. This is the famous Lion and Man problem, posed in the 1930s and settled by Besicovitch; the proof appears in Littlewood's "A Mathematician's Miscellany" (and opens Bollobas's "The Art of Mathematics"). What the lion CAN do (Rado's strategy): move to the center, then always stay on the radius through the tamer, using any spare speed to move outward. Since the lion runs on a smaller circle, it can match the tamer's angular position while steadily increasing its radiu...
Read more

Expected Tosses for Consecutive Heads: 2 Heads vs 3 Heads Puzzle

Let's say A keep tossing a fair coin, until he get 2 consecutive heads, define X to be the number of tosses for this process; B keep tossing another fair coin, until he get 3 consecutive heads, define Y to be the number of the tosses for this process. 1) Calculate P{X>Y} 2) What's the expected value of X 3) What's the expected value of Y This is probably the hardest puzzle I have ever put on my blog. Hence, I will post its solution in the post directly rather than on comment. Solution: 1) (Solved by me finally after 13 months :)) Make a state diagram. Let the state be (a,b) where a is the number of consecutive heads streak "A" is on currently and b is the number of consecutive heads streak "B" is on currently. So, (0,3) (1,3) are final accepted states and (2,0) (2,1) (2,2) (2,3) are failure states. If you get tails, your contribution to the state reaches to "0" f(State) = P(X>Y | "State" configuration initially) ...
Read more