Part I: Iterative Algorithms & Loop InvariantsChapter: 1: Measures of Progress and Loop InvariantsSection: 1.1: The Steps to Develop an Iterative Algorithm

1. Loop Invariants

(a) What is a loop invariant?

• Answer: A picture of what must be true at the top of the loop

(b) What is the formal statement that an algorithm is correct for a problem.

• Answer: If for every legal input instance, the required output is produced. If the inputinstance does not meet the preconditions, then all bets are off. Formally, we express this as

〈pre−cond〉 & codealg ⇒ 〈post−cond〉

This correctness is only with respect to this specifications.

(c) What are the formal mathimatical things involving loop invariants that must be proved to proveif your program exits then it obtains the post condition?

i.

• Answer: 〈pre−cond〉 & codepre−loop ⇒ 〈loop−invariant〉ii.

• Answer: 〈loop−invariant′〉 & not 〈exit−cond〉 & codeloop ⇒ 〈loop−invariant′′〉iii.

• Answer: 〈loop−invariant〉 & 〈exit−cond〉 & codepost−loop ⇒ 〈post−cond〉(d) What is the loop invariant for the GCD alg?

• Answer: On input 〈a, b〉, the algorithm maintains two variables x and y such thatGCD(x, y) = GCD(a, b).

(e) What is the loop invariant for a dynamic programming algorithm?

(f) What is the loop invariant for the binary search?

• Answer: If the thing being searched for is anywhere, then then it is in this narrowed sublist.

(g) What is the loop invariant for insertion sort?

2. Loop Invariants: In high school you would have learned the algorithm Gaussian Elimination for solvingsystems of linear equations and for inverting a matrix. Here we are to think of this algorithm as aniterative algorithm with a loop invariant. (Don’t panic. No linear algebra or memory of the algorithmis needed.) The input in an n× n-matrix M that is invertible and the output is its inverse N , i.e. then × n-matrix such that M × N = I. Here I is the identity matrix, i.e. that for which M × I = M .As a huge hint, the loop invariant is that the algorithm maintains matrices M ′ and N ′ for whichM ′ = M × N ′. (I recall drawing these two matrices next to each other with vertical line betweenthem.)

(a) Initializing:

i. For the iterative algorithm meta-algorithm, what is the formal statement that needs to beproved about the code before the loop?

• Answer: You must prove that the initial code establishes the loop invariant. The formalstatement that must be true is 〈pre−cond〉 & codepre−loop ⇒ 〈loop−invariant〉

ii. Give the code that goes before the loop for this matrix inverting algorithm.

• Answer: M ′ = M and N ′ = I.

iii. Prove this formal statement for this matrix inverting algorithm.

• Answer: The loop invariant M ′ = M ×N ′ follows trivially because M ′ = M = M × I =M ×N ′.

(b) The main loop:

i. For the meta-algorithm, what are the two things required of each iteration?

• Answer: Make progress and maintain the loop invariant.

ii. For the iterative algorithm meta-algorithm, what is the formal statement that needs to beproved about the code within the loop?

• Answer: You must maintain the loop invariant. The formal statement is〈loop−invariant′〉 & not 〈exit−cond〉 & codeloop ⇒ 〈loop−invariant′′〉

iii. Give a few sentences giving the intuition about what the code within the loop for this matrixinverting algorithm must accomplish and what needs to be proved about it. You do not needto specify any details of the linear algebra.

• Answer: The code in the loop must take two matrices M ′ and N ′ and produce two newmatrices M ′′ and N ′′.We must prove that if the loop invariant M ′ = M ×N ′ is true for M ′ and N ′, then it istrue for M ′′ and N ′′, namely M ′′ = M ×N ′′.You do not need to get this, but M ′′ is produced from M ′ by adding a multiple of onerow to another row in such a way that M ′′ has one more zero elements than M ′. N ′′ isproduced by doing the same row operation on N ′.Also progress must be made. You do not need to get this, but the measure of progresswill be the number of zero of elements of M ′. This number will go up by at least oneeach iteration.

(c) Ending:

i. For the meta-algorithm, what is the formal statement that needs to be proved about the codeafter the loop?

• Answer: In this step, you must ensure that once the loop has exited you will be able tosolve the problem. 〈loop−invariant〉 & 〈exit−cond〉 & codepost−loop ⇒ 〈post−cond〉

ii. What should the exit condition be for this matrix inverting algorithm?

• Answer: Exit when M ′ = I (or when M ′ has an all zero row).

iii. Give the code that goes after the loop for this matrix inverting algorithm.

• Answer: Set N = N ′ and return it.

iv. Prove this formal statement for this matrix inverting algorithm.

• Answer: By the loop invariant M ′ = M ×N ′, by the exit condition M ′ = I, and by thepost code N = N ′. It follows that M ×N = M ×N ′ = M ′ = I. Hence, as required, theinverse N of M is returned.

Section: 1.2: More About The StepsSection: 1.3: Different Types of Iterative AlgorithmsSection: 1.4: Typical ErrorsEnd of Chapter Questions

3. Sums

2

algorithm Eg(n)

〈pre−cond〉: n is an integer.

〈post−cond〉: Prints “Hi”s.

begins = 0i = 0loop

〈loop−invariant〉exit when s ≥ n

i = i + 1s = s + 1

iput “Hi”

end loopend algorithm

(a) Give a good loop invariant.

• Answer: s =∑i

j=11j . Initially true with s = i = 0 and maintained with i′′ = i′ + 1 and

s′′ = s′ + i′′ = [∑i′

j=11j ] + i′′ =

∑i′′

j=11j .

(b) How is the value of s not a reasonable measure of progress for Eg? How is it reasonable? If youcan think of one, what would be a better measure of progress?

• Answer: Officially, we said that the measure of progress should decrease by at least one eachstep. This one increases by only 1

i . The increasing is not a problem. Increasing by 1i2 would

be a problem, but it seems increasing by 1i is not. We will in fact use s to prove the program

halts. If you want an officially good measure of progress note that because s = loge i, we havei = es and hence, M(i) = n− es = n− i decreases from n to 0 by one each iteration.

(c) Give the theta of the running time of Eg.

• Answer: Size = log n = s. n = 2s. When the program exists s = sumij=1

1j = loge i+Θ(1) ≥

n. Hence, T ime = i = en+Θ(1) = e2s+Θ(1) = 22Θ(s)

, which is double exponential.

4. (25 marks) The Carnival Coin Game:

You are running a game booth at your local village carnival. In your game you lay out an array of $1coins on a table. For a $5 charge, anyone can challenge you to a game. In this game, you and yourcustomer alternately pick coins from the table, either 1 or 2 at a time. If your customer can make youpick up the last coin, he walks away with all the coins. You graciously allow your customer to go first.

Being an enterprising sort, you want to arrange the game so that you will always win. To do this, youwrite an iterative algorithm CarniCoin(n) that tells you how many coins to pick up on every turn. Thealgorithm takes one parameter n, the number of coins on the table at the beginning of the game.

The algorithm is

while there is more than one coin left on the tableThe customer picks up 1 or 2 coins.You do the opposite: if the customer picked up 1, you pick up 2, and vice-versa.

end while

(a) (5 marks) What is the postcondition for your algorithm?

• Answer: One coin remains on the table, and it is the customer’s turn.

(b) (5 marks) What is the loop invariant for your algorithm?

3

• Answer: The number i of coins on the table satisfies i mod 3 = 1.

(c) (5 marks) What precondition must you place on n in order to guarantee that you will win everytime?

• Answer: n > 0 and n = 1 mod 3.

(d) (10 marks) Prove the correctness of your algorithm. Be sure to explicitly include all steps requiredto prove correctness.

• Answer:The algorithm is

i = nwhile i > 1

i = i − 3end while

The loop invariant is maintained because 3 coins are picked up on each iteration, so that thenumber of coins on the table, mod 3, is maintained.The loop invariant is true on entry by virtue of the precondition. As a measure of progress,we can use the coins remaining on the table, which, as shown above, decreases by 3 on everyiteration, and will therefore meet the exit condition in finite time.On exit, i ≤ 1 and i mod 3 = 1. Since i decreases by 3 every iteration, i > −2. Thus i = 1and the postcondition is satisfied.

Chapter: 2: Examples Using More of The Input Loop InvariantSection: 2.1: Colouring the PlaneSection: 2.2: Deterministic Finite AutomatonSection: 2.3: More of the Input vs More of the Output

5. Loop Invariants (32 marks)Longest Contiguous Increasing Subsequence (LCIS): The input consists of a sequence A[1..n] of integersand we want to find a longest contiguous subsequence A[k1..k2] such that the elements are strictlyincreasing. For example, the optimal solution for [5, 3, 1, 3, 7, 7, 9, 8] is [1, 3, 7].

(a) (5 marks) Provide a tight lower bound on the running time for this problem. Prove that youranswer is indeed a tight lower bound.

• Answer: The problem takes at least Ω(n) time because the input needs to be read. Let Abe an arbitrary algorithm. Give it the input consisting of all zero. If it does not read someinteger, change this integer to a 1 (If it is the first integer change it to -1). It answers thesame longest length in both cases, however, in the first the longest has length one an in thesecond two. Therefore, it is wrong in at least one of these cases.

(b) (5 marks) Specify an appropriate loop-invariant, measure of progress and exit condition for aniterative solution to this problem. No explanations are necessary.

• Answer: LI: We maintain an over all optimal LCIS found so far and the LSIS that we arecurrently working on.A[k1 . . . k2] is an LCIS of A[1 . . . i] and A[p . . . i] is the LCIS of A[1 . . . i] that ends at i.Note the first LI is “More of the Input”, namely if we consider the prefix A[1 . . . i] to be theentire input then we have the solution.Most people missed the second one, but it is needed if the LI is to be maintained.Measure of Progress: the number i of elements read. Many gave that i increases. This isprogress, but not a measure.Exit Condition: When all the elements have been read because i = n.

4

(c) (10 marks) Provide concise pseudo-code for an algorithm LCIS(A,n) that returns the indicesk1, k2 of the LCIS of A[1 . . . n] uses the loop-invariant, measure of progress and exit condition youspecified above. Assume n ≥ 1.

• Answer:I thought more about why LI are important to me. It is a life philosophy. It is about feelinggrounded. Most of the code I marked today makes me feel ungrounded. It cycles, but I dontknow what the variables mean, how they fit together, where the algorithm is going, or howto start thinking about it. Loop invariants are about starting my day at home, where I knowwhat is true and what things mean. Loop invariants are also about returning back full circleback to my safe home at the end of my day

algorithm [k1, k2] =LCIS(A,n)

〈pre−cond〉: A[1 . . . n] is an array of integers

〈post−cond〉: A[k1 . . . k2] is an LCIS of A[1 . . . n]

begink1 = k2 = p = i = 1loop

〈loop−invariant〉: A[k1 . . . k2] is an LCIS of A[1 . . . i] andA[p . . . i] is the LCIS of A[1 . . . i] that ends at i

i = i + 1if( A[i−1] ≥ A[i] ) then

p = iend ifif (q − p) > (k2 − k1)

k1 = pk2 = q

end ifend if

end algorithm

(d) (6 marks) Provide an informal argument for how your code is able to maintain the loop invariant.A formal proof is not required.

• Answer: 〈loop−invariant′〉 & not 〈exit−cond〉 & codeloop ⇒ 〈loop−invariant′′〉Suppose at the beginning of an iteration, the LI is true for the current values, i.e. A[k′

1 . . . k′2]

is an LCIS of A[1 . . . i′] and A[p′ . . . i′] is the LCIS of A[1 . . . i′] that ends at i′.The first block of code either extends the LCIS that we are currently working on fromA[p′ . . . i′] to A[p′ . . . i′+1] or shrinks it to contain only the one element A[i′ +1] depend-ing on whether or not A[i′+1] is bigger than A[i′]. This is achieved by moving its end pointfrom i′ to i′′ = i′+1 and either leaving its begging at p′′ = p′ or moving it to p′′ = i′′. Eitherway, the LI is maintained that A[p′′ . . . i′′] is the LCIS of A[1 . . . i′′] that ends at i′′.The second block of code updates the over all LCIS found so far simply by moving it to thecurrent one, if the current one has grown to be larger. Either way, the LI is maintained thatA[k′′

1 . . . k′′2 ] is an LCIS of A[1 . . . i′′]

(e) (6 marks) What are the other key steps in proving your algorithm correct? List and provide aconcise proof for each.

• Answer:Establishing the LI: 〈pre−cond〉 & codepre−loop ⇒ 〈loop−invariant〉.On entry, k1 = k2 = p = i = 1. Thus the first part of the loop-invariant requires that A[1] bean LCIS of A[1], which is trivially true. The second part of the loop invariant requires thatA[1] is the LCIS of A[1] that ends at location 1, which is also trivially true.Making Progress: i increments by 1 on each iteration. Program exits when i = n.Establishing Post-Condition: 〈loop−invariant〉 & 〈exit−cond〉 & codepost−loop ⇒〈post−cond〉. When i = n, the first LI gives that A[k1 . . . k2] is an LCIS of A[1 . . . n], whichis the post condition.

5

6. Suppose you start with the number 1 and it costs you one to add two numbers together that youalready have. (Numbers can be reused.) Once you have produced a number, you can use it manytimes. Given an integer k as input, design an algorithm to produce k as cheaply as possible. Whatdoes it cost? The space requirement is how many numbers you need to keep at any given time. Tryto get an algorithm with minimum space. Hint 1: Think of the input k as an n = log2(k) bit binarystring. (Consider reading it forward and backward). Hint 2: Try “More of the Input,” “More of theOutput,” and “Narrowing the Search Space.” One of these three works fairly easily. (10 bonus pointif you can do it in time T (k) = n + Θ( n

log n ).)

• Answer: You definitely don’t want to produce 1, 2, 3, 4, . . . , k by adding 1 each time, but thiswould cost k − 1 = 2n.

The More of the Input technique has the loop invariant state that if we consider the first i bits ofthe input instance to be the entire input instance, then for this we have a solution. As an example,suppose k was 1101010002 in binary and i = 4. Then the prefix would be k′ = 11012 in binary.The loop invariant says that somehow we have produced that value. Each step of this algorithmis to extend the prefix by one bit and then to produce the solution for the longer instance fromthe solution for the shorter instance. In our example, the next bit is a zero. Including this givesthe longer instance k′′ = 110102. Note that this change doubled the value of the instance, namelyk′′ = 2 · k′ = k′ + k′. The loop invariant says that we have produced k′. We can produce k′′

simply by adding this k′ to itself. On the other hand, if the next bit is a 1, then k′ = 11012 wouldextend to k′′ = 110112. This gives k′′ = k′ + k′ + 1. Given that we have k′ and we can keep 1around, we can easily produce k′′. This algorithm does at most two additions for each bit of k.The space is only two because it only needs to keep 1 and the current k′.

This same algorithm could be thought of recursively. If k is even, then recursively produce k2 and

then add this to itself to give k. If k is odd, then produce ⌊k2 ⌋ and then compute k = ⌊k

2⌋+⌊k2⌋+1.

Another useful technique is to simply try to build big numbers as quickly as possible. From 1,produce 2, 4, 8, ... by adding the last number to itself. Then to produce a arbitrary k, look at thevalue k in binary. For example, k = 180 is 101101002 in binary because 180 = 128 + 32 + 16 + 4.This tells us how to combine the powers of 2 that we have produced to give us k. The time ofthis algorithm is still at most two times the number of bits in k. However, it requires log k space.

As said, these three algorithms use the same amount amount of time, namely T (n) = 2 log2 k = 2n.No algorithm can do better then n = log2 k additions because no algorithm can more than doublethe largest value, and hence after i iterations, the largest value is at most 2i.

If you care about the extra factor of 2 between the upper and the lower bound, lets try a littleharder. Let b be a parameter that we will later set to 1

2 log n. The algorithm first produces1, 2, 3, 4, . . . , 2b by adding 1 each time. This takes time 2b. All of the intermediate values are kept.Note that thinking of each of these values as a binary string, we have produced every possible bbit string.

The next stage of the algorithm breaks the string of n bits representing k into nb blocks of length

b. The algorithm is the same as the more of the input algorithm given above except that eachiteration considers not just the next bit but the next block. The loop invariant says that we haveproduced the value k′ formed from the first i blocks of the instance. To produce value k′′ formedfrom the first i + 1 blocks of the instance, we must first multiply k′ by 2b to add b zeros andthen we must add on the next block. Multiplying by two just requires adding the number byitself. Hence, multiplying by 2b just requires repeating this b times. Adding on the next blockjust requires one more addition. This is because in the first stage of the algorithm we producedevery possible b bit string. We need only add on the correct one. There are n

b iterations in thissecond stage of the algorithm and each iteration requires b + 1 additions.

The total time of the algorithm is T (n) = 2b + nb · (b + 1) = n + n

b + 2b. Setting b = 12 log n gives

T (n) = n + 2 nlog n +

√n ∈ n + Θ( n

log n ).

7. Let G = (V, E) be a directed weighted graph with possible negative weights and k ≤ n some integer.Find an algorithm that finds for each pair of nodes u, v ∈ V a path from u to v with the smallest total

6

weight from amongst those paths that contains exactly k edges. Here a path may visit a node morethan once. Do the best you can, but ideally the running time is O(n3 log n). A hint is that a smallestweighted path with i+j edges must be composed of a smallest weighted path with i edges and onewith j edges.

• Answer: In order to solve the problem on input instance k, the “More of the Input” techniquesuggests solving it first for smaller integers first. Suppose for each pair of nodes u, v ∈ V , wealready have that wu,v,i is the weight of the smallest weighted path from u to v containing exactlyi edges and that wu,v,j the same for those containing exactly j edges. Then a smallest weightedpath from u to v containing exactly i+j edges must be composed of a smallest weighted pathwith i edges from u to some node b and then a smallest weighted path with j edges from b ontov. We need to only try all nodes b and see which one works best. This gives the recurrencewu,v,i+j = minb(wu,b,i + wb,v,i). Doing this for each pair u and v takes O(n3) time. We can thenbuild up paths with k edges in the same way as done in the previous exercise. The total time isthen O(n3 log n).

Chapter: 3: Abstract Data TypesSection: 3.1: Specifications and Hints at ImplementationsSection: 3.2: Link List ImplementationSection: 3.3: Merging With a QueueSection: 3.4: Parsing With a StackChapter: 4: Narrowing the Search Space: Binary SearchSection: 4.1: Binary Search TreesSection: 4.2: Magic SevensEnd of Chapter Questions

8. (For the Engineers, By Jeff)

Imagine a wire that over time carries a 0/1 signal. A front-end processor has sampled it at a rate of1GHz (ie every nano second) and given you an array A of length n. A[i] = 0 if there is a 0 on thewire at time i nsecs and A[i] = 1 if there is a 1 on the wire at that time. A transition (or ”edge”) isdefined to be a place in time where the signal transitions from one value to another. This may be of”positive polarity”, i.e. from 0 to 1 or of ”negative polarity” meaning from 1 to 0. The time of thetransition is the average of these two indexes. For example, if A[7] = 0 and A[8] = 1 then we say thatwe ”time-stamp” this transition with the time t = 7.5 nsecs. Our goal, given such an array A is to findthe time-stamp of one such transition (if it exists). If there are many such transitions in A, then wecan return any one of them.

(a) Give a lower bound that any algorithm that solves this problem, on the worst case input, requiresn time to find the answer. Imagine this as a game in which some adversary gives you an algorithmand you must find an input array A for which this algorithm queries A n times before finding atransition. Your solution will describe how you will use the actions of the algorithm to producethe input. (We did this in class to prove that adding two n bit numbers requires Ω(n) time.) Beclear how you handle the following circular argument. You dont know what input instance to givethe algorithm until you know what the algorithm does. But you dont know what the algorithmdoes until you give it an input instance and run it.

• Answer: The input instance we consider will either be all zeros or have exactly one one.We ”run” the algorithm without actually giving it a fixed input instance. Each time thealgorithm queries A at some index i, we respond by saying telling it that A[i] = 0. With thisinformation, the algorithm can continue on just as if it did have an actual input instance.If the algorithm stops before reading each array location then it can’t know whether or notthere is a transition because we will ether give him the all zero instance or the instance witha one in one of the locations he has not read.

7

(b) Now let us add the additional constraint that at the beginning and end of the time interval , thewire has a different signal, i.e. A[1] 6= A[n]. Note that this assumption ensures that there is atleast one transition, though it could still have many transitions. Design a sublinear time recursivealgorithm which given the 0/1 array A returns the time t of a transition. Be very clear aboutyour pre and post conditions and that you are giving your friend a smaller instance that meetsthe precondition. Though you must be clear what the algorithm is, actual code is not needed.Give and solve the recurrence relation for its running time.

• Answer: The precondition is that we are given an array A and two indexes p and q for whichA[p] 6= A[q]. We query A[m] for m = p+q

2 . Then we recurse on A[p..m] if A[p] 6= A[m] andrecurse on A[m..q] if A[m] 6= A[q]. By the precondition on our instance, one of these mustbe true (otherwise A[p] = A[m] and A[m] = A[q] giving A[p] = A[q].) When q = p + 1, wehave a transition A[p] 6= A[p + 1] at t = p + 1

2 . The recurrence relation for the running timeis T (n) = T (n/2) + 1 = Θ(log n). It is not needed, but the master theorem can be used asfollows. The time for the top level is f(n) = 1 = n0. Hence c = 0. The time for the base

cases is Θ(nlog alog b ) = Θ(n

log 1log 2 ) = Θ(n0) = 1. Because the top and bottom are the same, we

know that T (n) = Θ(log(n)× f(n)) = Θ(log n).

Chapter: 5: Iterative Sorting AlgorithmsSection: 5.1: Bucket Sort by HandSection: 5.2: Counting Sort (A Stable Sort)Section: 5.3: Radix Sort

9. (10 marks) RadixSort sorts d-digit numbers:

loop i = 1 to dstable sort cards with respect to the ith least significant digit.

end loop

At every iteration the following assertion is true: “The input is stable-sorted with respect to the ith

least-significant digit.” Is this a good loop-invariant? Why or why not?

• Answer: No this is not a good loop invariant. It is true but not enough to know that the inputis stable-sorted with respect to JUST the ith least-significant digit. It is a more of the input typeof loop invariant. If you consider each number to be just its least-significant i digits then the listis sorted stably.

Section: 5.4: Radix/Counting SortChapter: 6: Euclid’s GCD Algorithm

10. (Amortized Analysis) Suppose we have a binary counter such that the cost to increment the counteris equal to the number of bits that need to be flipped. For example, incrementing from 100, 111, 1112

to 101, 000, 0002 costs 7. Consider a sequence of n increments increasing the counter from zero to n.Some of these increments, like that from 101, 111, 0102 to 101, 111, 0112 costs only one. Others, likethat from 111 . . .1112 = 2⌊log2 n⌋ − 1 ≈ n to 1000 . . .0002 costs O(log n). This one is the worst case.Given that we generally focus on worst case, one would say that operations cost O(log n). It might bemore fair, however, to consider the average case. If a sequence of n operations requires at most T (n)

time, then we say that each of the operations has amortized cost T (n)n .

(a) Suppose the counter begins at zero and we increment it n times. Show that the amortized costper increment is just O(1).

Hint: Let Tx,j = 1 if incrementing value x requires flipping the jth bit. Note∑n

x=1

∑j Tx,j =∑

j

∑nx=1 Tx,j.

8

• Answer: The jth bit flips every 2j iterations for a total of ⌊ n2j ⌋ flips. The total cost is

T (n) =∑n

x=1

∑j Tx,j =

∑j

∑nx=1 Tx,j =

∑log nj=0 ⌊ n

2j ⌋ =∑log n

j′=0 2j′ . This sum is geometricand hence is dominated by the largest term. The largest term comes from the first bit beingflipped each iteration for a total of n times. This gives that T (n) = O(n) and the amortized

cost per operation is T (n)n = O(1).

(b) Suppose that we want to be able to both increment and decrement the counter. Starting with acounter of zero, give a sequence of n of operations where each is either an increment and decrementoperation, that gives the highest amortized cost per operation (within Θ). (Don’t have the countergo negative.) Compute this amortized cost.

• Answer: First increment the counter from zero to 111 . . .1112 = 2⌊log2(n/2)⌋ − 1 ≈ n2 . Then

have n2 operations alternating between incrementing and decrementing the counter between

111 . . . 1112 and 1000 . . .0002. Each of these costs ⌊log2(n/2)⌋. Hence, the total cost isT (n) = Θ(n

2 ) + n2 · ⌊log2(n/2)⌋ = Θ(n logn) giving an amortized cost of Θ(log n).

(c) To fix the problem from part (b), consider the following redundant ternary number system. Anumber is represented by a sequence of trits X = xn−1 . . . x1x0, where each xi is 0, +1, or -1. The value of the number is

∑n−1i=0 xi · 2i. For example, X = 1012 is 5 because 4 + 1 = 5.

However, X = (1)(1)(−1)2 is also 5 because 4 + 2− 1 = 5. Also X = (1)(−1)(0)(−1)(1)(1)(1) =64− 32− 8 + 4 + 2 + 1 = 31.

The process of incrementing a ternary number is analogous to that operation on binary numbers.One is added to the low order trit. If the result is 2, then it is changed to 0, and a carry ispropagated to the next trit. This process is repeated until no carry results. For example, X =(1)(−1)(0)(−1)(1)(1)(1) = 31 increments to X +1 = (1)(−1)(0)(0)(0)(0)(0) = 64−32 = 32. Notethis is a change of 8−4−2−1 = 1. Decrementing a number is similar. One is subtracted from thelow order trit. If it becomes -2 then it is replaced by 0, and a borrow is propagated. For example,X + 1 = (1)(−1)(0)(0)(0)(0)(0) decrements to X = (1)(−1)(0)(0)(0)(0)(−1) = 64− 32− 1 = 31.Note that this increment followed by a decrement resulted in a different representation of thenumber 31 than the one we started with.

The cost of an increment or a decrement is the number of trits that change in the process. Our goalis to prove that for any sequence n of operations where each is either an increment and decrementoperations, starting with a counter of zero, the amortized cost per operation is at most O(1). LetT (t) be the total number of trit changes that have occurred during the first t operations. Let Z(t)be the number of non-zero trits in our counter after the t operations. For example, if the resultingcounter is X = (1)(−1)(0)(−1)(0)(1)(1) then Z(t) = 5. We will define P (t) = T (t) + Z(t) to beour measure of progress or potential function at time t.

Bound the maximum amount that P (t) can change in any one increment or decrement operation.

• Answer: We will prove that each operation increases P (t) by at most 2. Consider any oneoperation. Any time a trit changes and causes a carry increases T (t) by one because anothertrit has changed and decreases Z(t) by one because this trit then changes from a non-zeroto a zero trit. The total effect is that P (t) = T (t) + Z(t) does not change at all. Only thelast and final trit change does not cause a carry. It increases T (t) by 1 and can increase Z(t)by at most 1, because of a possible change from a zero to a non-zero trit. Hence, the totalchange in P (t) resulting from one operation is at most 2.

(d) Use induction (or loop invariants) to bound P (t). In doing so, prove that for any sequence n ofoperations where each is either an increment and decrement operations, starting with a counterof zero, the amortized cost per operation is at most O(1).

• Answer: The induction hypothesis or loop invariant is that P (t) ≤ 2t. Note that P (0) = 0because no changes have been made yet and all trits are zero. This is maintained becauseP (t) increases by at most 2 each operation, namely P (t + 1) ≤ P (t) + 2 ≤ 2t + 2 = 2(t + 1).Z(n) is not negative. Hence, T (n) = P (n)− Z(n) ≤ P (n) ≤ 2n and the amortized cost is at

most T (n)n ≤ 2.

9

11. Shifted Streams: You receive two streams of data, A[1], A[2], A[3], . . . and B[1], B[2], B[3], . . .. Once anumber arrives you cannot request it again. You must output a stream of data containing the valuesA[1] + B[1], A[2] + B[2], A[3] + B[3], . . .. Once you output it you do not need to store it. There arethree things that make this more challenging.1) The B data streams is shifted in time by some amount s.2) You have only a limited amount of memory M [0], M [1], M [2], M [3], . . . , M [m] of which you want touse as little as possible.3) You have only a limited amount of time between data numbers arriving.

(a) Give the loop invariant. (I give the first point.)

i. The numbers A[1], A[2], A[3], . . . , A[t] have arrived.The numbers B[1], B[2], B[3], . . . , B[t− s] have arrived.

ii. What values have been outputted?

• Answer: The numbers A[1] + B[1], A[2] + B[2], A[3] + B[3], . . . , A[t− s] + B[t− s] havebeen outputted.

iii. Which values do you want to be remembering in memory? (Use as little memory as possible.)

• Answer: The last s values of A are saved. Specifically, A[t− s + 1], A[t− s + 2], . . . A[t].

iv. In which memory cell do you save each such value? (You want to move things as little aspossible.)

• Answer: The A data stream is stored in cyclically in s memory cells so that the last svalues are saving. Specifically, for i = t− s + 1, .., t, value A[i] is stored in M [i mod s].

(b) I taught a few “types” of loop invariants? Which type is this? Why? Include the definition.

• Answer: “More of the Input” because if you consider a prefix of the input stream as theinput, we have produced the output. Similarly, it could be “More of the Output”.

(c) What code establishes this loop invariant?

• Answer: For i = 1, ..s, value A[i] is stored in M [i mod s]. With t = s, the loop invariant istrue.

(d) What code maintains this loop invariant?

• Answer: At time t,Read values A[t] and B[t− s] arrive.Get value A[t− s] from memory M [t− s mod s] = M [t mod s].Output the value A[t− s] + B[t− s].Store value A[t] is stored in M [t mod s].

(e) In order to prove that the loop invariant has been maintained, what is the formal statement ofwhat you prove? (No need to prove it.)

• Answer: 〈loop−invariant′〉 & not 〈exit−cond〉 & codeloop ⇒ 〈loop−invariant′′〉

Part II: RecursionChapter: 7: Abstractions, Techniques, and TheorySection: 7.1: Thinking about RecursionSection: 7.2: Looking Forwards vs BackwardsSection: 7.3: With a Little Help from Your FriendsSection: 7.4: The Towers of HanoiSection: 7.5: Check List for Recursive AlgorithmsSection: 7.6: The Stack FrameSection: 7.7: Proving Correctness with Strong InductionEnd of Chapter Questions

10

12. (15 marks) Recursion: Define a Recursive Integer List (RIL) to be a list of a finite number of objects,where each object is either an integer or is an RIL. For example, L = [3, 2, [4, 3], [[3, 9], [], 8], 7] is anRIL. You are allowed the following operations:

(a) If L is an RIL and i is an integer, then L[i] is the ith object in the list L. For our above example,L[1] = 3, L[2] = 2, L[3] = [4, 3], L[4] = [[3, 9], [], 8], L[5] = 7, and L[6] returns an error. More overL[4][1] = [3, 9] and L[4][1][2] = 9.

(b) Similary, if L is an RIL, then L[i..j] is the RIL consisting of the ith through the jth objects inthe list L. For our above example, L[2..4] = [2, [4, 3], [[3, 9], [], 8]]. More over, L[2..2] and L[2..1]return the RILs [2] and [].

(c) If L is an RIL, then |L| returns the number of objects in the list L. For our above example,|L| = 5. More over, |L[4][2]| = 0.

(d) If Obj is an object, then (Obj == Integer) returns true if the object is an integer and (Obj ==RIL) returns true if it is an RIL. In our example, (L[1] == Integer) is true and (L[3] == Integer)is false. Note that 2 is an integer, while [2] is a RIL containing one object which happens to bethe integer 2.

Your task is to take as input an RIL L and return the sum of all integers any where in L. In ourexample, SUM(L) should return 3 + 2 + 4 + 3 + 3 + 9 + 8 + 7 = 39. You may use a loop if you reallymust, but it is better if you don’t.

• Answer:

algorithm Sum (L)

〈pre−cond〉: L is an RIL.

〈post−cond〉: Returns the sum of all integers in L

beginif(|L| = 0)

return(0)else

if( L[1] == Integer )first = L[1]

elsefirst = Sum(L[1])

endifreturn(first + Sum(L[2..|L|]))

end algorithm

algorithm Sum (L)

〈pre−cond〉: L is an RIL.

〈post−cond〉: Returns the sum of all integers in L

beginsum = 0loop i = 1 . . . |L|

if( L[i] == Integer )sum = sum + L[i]

elsesum = sum + Sum(L[i])

endifend loopreturn(sum)

end algorithm

algorithm Sum (Obj)

〈pre−cond〉: Obj is either an integer or is an RIL.

〈post−cond〉: Returns the sum of all integers in Obj

beginif( Obj == Integer )

return( Obj )else

L = Objsum = 0loop i = 1 . . . |L|

sum = sum + Sum(L[i])end loopreturn(sum)

endifend algorithm

11

13. (13 marks) Recursion

algorithm A (a, b)

〈pre−cond〉: a and b are integers.

〈post−cond〉: Returns ??

beginif( a = b ) then

return( 0 )elseif( a < b ) then

return(−A(b, a))else

return(1 + A(a− 1, b) )endif

end algorithm

algorithm A (a, b)

〈pre−cond〉: a and b are integers.

〈post−cond〉: Returns ??

beginif( a = b ) then

return( 0 )elseif( a < b ) then

return(−A(b, a))else

return(A(a− 1, b− 1) )endif

end algorithm

(a) Circle the output of this program.

i. a + b

ii. a− b

iii. max(a, b);

iv. Does not halt.

• Answer: a− b

(b) Prove either that this program halts or that it doesnot.

• Answer: Consider the “size” of the input to bea−b if a ≥ b and b−a+1 otherwise. Every friendgets a smaller instance and if the size is zero thana base case is reached.

(c) If the program halts, prove that its output is whatyou claim.

(a) Circle the output of this program.

i. a + b

ii. a− b

iii. max(a, b);

iv. Does not halt.

• Answer: Does not halt.

(b) Prove either that this program halts or that it doesnot.

• Answer: On instance 〈a, b〉 = 〈2, 1〉, the subin-stances will be 〈1, 0〉 , 〈0,−1〉 , 〈−1,−2〉 , . . ..

(c) If the program halts, prove that its output is whatyou claim.

• Answer: Consider instance 〈a, b〉. By way of strong induction, assume that A returns a − b forevery instance smaller.If a = b than the result is 0, which is a− b.If a < b than the result is −A(b, a) = −[b− a] = a− b.else the result is 1 + A(a− 1, b) = 1 + [(a− 1)− b] = a− b.Either way, our result is a− b.

Chapter: 8: Some Simple Examples of Recursive AlgorithmsSection: 8.1: Sorting and Selecting AlgorithmsSection: 8.2: Operations on Integers

14. Recursion: Suppose you are given two n digit numbers (base 10), X = x1x2 . . . xn and Y = y1y2 . . . yn

and you want to multiply them.

(a) In one sentence explain what is the running time of the kindergarten algorithm is and why.

• Answer: It adds X times which is 10n.

(b) In one sentence explain what is the running time of the high school algorithm is and why.

• Answer: ∀i, j ∈ [n] it computes xi × yj . Hence the time is O(n2).

12

(c) In three short sentences describe the slow recursive algorithm for multiplying. (The one whosetime is the same as the high school algorithm.)

• Answer: Break X and Y into pieces X = X1X2 and Y = Y1Y2. Get four friends to multiplyX1 × Y1, X1 × Y2, X2 × Y1, and X2 × Y2. Shift their answers and add.

(d) In one sentence, what is the key trick used to reducing the time.

• Answer: Reduce the number of friends from four to three.

(e) Give the recurrence relation for this recursive algorithm. Solve it.

• Answer: T (n) = 3T (n/2) +O(n) = O(nc) where c = log(a)/ log(b) = log(3)/ log(2).

(f) Give all the key ideas in thinking abstractly about a recursive program?

• Answer: If your input is sufficiently small solve it yourself. Otherwise, construct instancesfor friends that are smaller then your instance and that meet the preconditions. Withoutyou worrying how, assume that they can solve their instance. Combine the solutions to theirinstances to obtain a solution to your instance.

15. GCD and Finite Fields.

(a) (2 marks) Given integers a and b, what does the generalized GCD algorithm return and with whatpost condition.

• Answer: a · s + b · t = g = gcd(a, b)

(b) (2 marks) How is this used to find the inverse of u mod p, where p is a prime.

• Answer: Let a = u and b = p. This gives u · s + p · t = gcd(a, p) = 1 or u · s = 1 mod p.Hence s is the inverse.

(c) (2 marks) Why are the integers mod n not a field when n is not prime?

• Answer: If a · b = n, then a · b = 0 mod n. In a field this should only be true if a or b arezero.

16. Finite Fields.

(a) Find the inverse of 20 in Z67. To show your work, make a table with columns u, v, r, s, and tand a row for each stack

• Answer: Sorry for not doing the table, but the inverse is −10 = 57 because 20 ∗ (−10) =−200 = 1− 3× 67 = 1− 201.

(b) Given as input I = 〈a, b, p〉 compute ab mod p. The algorithm is in the notes. Do not copy thealgorithm. The section reference (and necessary changes) is sufficient. What is the number of bitoperations needed as a function of n = |I| = log2(a) + log2(b) + log2(p)?

• Answer: The algorithm is in Operations on Integers. It requires Θ(log b) multiplications.All multiplications are mod p and hence require Θ(log p) bit operations for a total of T (n) =Θ(log b) ·Θ(log p) = Θ(n2).

(c) Given as input I = 〈a, c, p〉 solve ab ≡mod p c for b. This is called discrete log. What is the bestalgorithm that you can come up with in 15min. (Do not cheat and spend more time than this.)What is the number of bit operations needed as a function of n = |I| = log2(a)+log2(c)+log2(p)?

• Answer: The best know algorithm requires Θ(b) = Θ(2n) Θ(b) = Θ(2n) multiplications. Itis the bases of multiplications. It is the bases of various cryptographic systems.

Exercise 0.0.1 In friends level of abstracting recursion, you can give your friend any legal instancethat is smaller than yours according to some measure as long as you solve on your own any instancethat is sufficiently small. For which of these algorithms has this been done? If so what is your measureof the size of the instance? On input instance 〈n, m〉, either bound the depth to which the algorithm

13

recurses as a function of n and m or prove that there is at least one path down the recursion tree thatis infinite.

algorithm Ra(n, m)

〈pre−cond〉: n & m ints.

〈post−cond〉: Say Hi

beginif(n ≤ 0 and m ≤ 0)Print(”Hi”)

elseRa(n− 1, m)Ra(n, m− 1)Ra(n− 2, m− 2)

end ifend algorithm

Change base case condition in theRa to each of the following.

(b) if(n ·m ≤ 0)(c) if(n ·m ≤ 10)

• Answer:

(a) This does not halt following only first friend because m never gets small.

(b) If n = m are initially odd, then this does not halt following only third friend because n =m = 1,−1,−3,−5 . . . and the n ·m stays positive.

(c) Halts. Size(n, m) = n + m gets smaller by at least one and at most 4 for each friend. When0 ≤ Size(n + m) ≤ 3 at most one of n and m are negative so we cant get a negative times anegative gives us a positive and n ·m is at most 10. Initially, the size is n + m so it will be 0in at most n+m time steps. Hence, this is a bound on the depth. Note the time is not n ·m,not n, and not m.

Section: 8.3: Ackermann’s FunctionEnd of Chapter Questions

17. Quick Sort:

(a) [4 apples] Describe the quick sort algorithm using the “friends” paradigm. Assume that you havepivot available as a subroutine.

(b) [2 apples] What is the time complexity of the algorithm when the pivot is selected randomly. Giveyour best approximation of its recurrence relation. What does it evaluate to?

18. You have a bag of n nuts and a bag of n bolts. The nuts and bolts are all of different sizes, but for eachnut there is a matching bolt (and vice versa). What you want to do is to match them. Unfortunately,you cant compare a nut to a nut or a bolt to a bolt (your eyes are too weak). Instead, what you cando is to compare a nut to a bolt and see if the bolt is too small, too big, or just right. The problemcan be solved by comparing each nut to each bolt but this would require O(n2).

(a) I want you to describe a simple algorithm which uses randomness and recursion. It should be verysimilar to Quick Sort which partitions with a pivot. Be sure to mention all the key requirementsof the friends analogy described in class. (Hint: Use the fact that every nut matches some bolt).(Nine sentences.)

• Answer: Choose a random nut. Compare it to each bolt, partitioning them into those thatare smaller, a perfect match, and larger. At least one bolt is a perfect match. Compare it toeach nut, partitioning them into those that are smaller and larger. Match this nut and bolt.The nuts and bolts that are smaller then this pair must meet the precondition, namely youhave a bag of n′ nuts and a bag of n′ bolts and for each nut there is a matching bolt (and

14

vice versa). This subinstance is also smaller. Hence, get a friend to match them. Similarly,those that are larger.

(b) Explain intuitively what the recurrence relation for the expected running time of this algorithmis. Explain intuitively what the solution for this recurrence relation is.

• Answer: We expect each of the two subinstance to be half the size. We expect the largerof the two to be of size 3

4n. If this was always the case, the recurrence relation would beT (n) = T (3

4 ) + T (14 ) + n. This evaluates to T (n) = Θ(n log n) because there will be Θ(log n)

levels of recursion and the sum work in each level will be Θ(n).

(c) Explain intuitively what the worst cast running time of this algorithm is and why.

• Answer: In the worst case, the subinstances are always of size n − 1 and zero. This givesT (n) = T (n− 1) + n = Θ(n2).

19. For many computational problems there is a trade off between the time and the amount of memoryneeded. There are algorithms that use minimal time but lots of memory and algorithms that useminimal memory but lots of time. One way of modeling this is by a game of pebbling the nodes ofa directed acyclic graph (DAG). Each node represents a value that needs to be computed. Having apebble on a node represents the fact that that value is stored in memory. Note the game has one pebblefor each of the memory registers that is available to the algorithm. Pebbles can be removed from a nodeand placed else where. This corresponds to replacing the value in a register with a newly computedvalue. However, there is a restriction. If there are edges from each of the nodes In(v) = u1, u2, . . . , udto v, then there needs to be a pebble on these nodes u1, u2, . . . , ud before you can put a pebble on v.This is modeling the fact that you need the values associated with u1, u2, . . . , ud in order to be able tocompute the value associated with v. In contrast, a pebbles can be placed at any time on a leaf of theDAG because these nodes have no incoming edges. These correspond to the easy to compute values.The input to the pebbling problem specifies a DAG and one of its nodes. The goal is to place a pebbleon this node.

u1

u2

2u

1uchilderen and

Have pebbles on both

can place pebbleon parent v.

v

d=2

Get a pebble to the top.

You will be required to solve the problem corresponding to pebbling the following DAG. The DAGin consideration will have a rectangle of nodes w wide and h high. Here h much smaller than w,but w is much smaller than 2h. For each node v (not on the bottom row), there will be d nodesIn(v) = u1, u2, . . . , ud on the previous row with directed edges to v. Here, d is some small value.

You are to write both a iterative and a recursive algorithm for this problem.

(a) Iterative:

i. Describe an extremely simple iterative algorithm that puts a pebble on the specified node asquickly as possible. The algorithm may use as many pebbles (memory) as needed, but doesnot redo any work. Being an iterative algorithm, what is the algorithm’s loop invariant?

• Answer: This algorithm starts by placing a pebble on each of the nodes on the bottomrow. This can be done because each such node has in-degree zero. The loop invariantis that after r iterations, there is a pebble on each of the nodes on the rth row from thebottom. The step of the next iteration is to place pebble on each of the nodes on ther+1st row from the bottom. This can be done because each such node v has pebbles onall of its in-neighbors In(v) = u1, u2, . . . , ud on the previous row. The step ends byremoving the pebbles from the rth row so that they can be reused. Note that progresshas been made while maintaining the loop invariant. The exit condition is when r = h atwhich time the task of placing a pebble on the top row has been accomplished.

15

ii. How much time do you for this iterative programming algorithm? Given that you are able toreuse pebbles, how many do you really need?

• Answer: The time required is O(hw) and the number of pebbles is 2w.

(b) Recursive:

i. Suppose your task is to place a pebble on some node v which is r rows from the bottom.Describe a recursive backtracking algorithm to accomplish this task that re-does work asnecessary, but uses as few pebbles as possible. Being a recursive algorithm, describe thisalgorithm using the friends paradigm.

• Answer: My task is to place a pebble on some node v which is r rows from the bottom.For each of the d nodes ui ∈ In(v) with an edge to v, I ask a friend to place a pebble onnode ui. Once, there is a pebble on each node in In(v), I place a pebble on node v.

ii. Let T ime(r) and Pebbles(r) be the time and the numbers of pebbles used by your algorithmto go from there being no pebbles on the DAG to placing a pebble on one node r rows fromthe bottom. Give and solve a recurrence relation for each of these.The hardest part of this pebble question is the recurrence relation for the number of pebbles.Remember to reuse the pebbles.

• Answer: The total time used to complete my task is T ime(r) = d × T ime(r − 1) + 1,because each of my d friends uses T ime(r−1) time and I move one pebble. This evaluatesto T ime(r) ≈ d× T ime(r − 1) = dr−1.The number of pebbles I will need is Pebbles(r) = Pebbles(r − 1) + (d − 1). Note that Imust leave a pebble on node ui before moving on to node ui+1, but the rest of the pebblescan be reused. The time at which I need the most pebbles is when pebbling node the dth

node in In(v). At this point I need to have d− 1 pebbles on the previous nodes in I(v)and Pebbles(r− 1) pebbles that I allow this last friend to use for a total of Pebbles(r) =Pebbles(r− 1) + (d− 1) pebbles. This evaluates to Pebbles(r) = (d− 1)× (r − 1) + 1.

(c) Conclusion: Compare and contrast the time and the space used by these two algorithms. Re-member that w is much bigger than h and d.

• Answer: The iterative algorithm uses time w × h which is the number of nodes, while therecursive algorithm uses exponential time. On the other hand, the iterative algorithm usestime Θ(w) pebbles which is more than the Θ(dh) used recursive algorithm because we weregiven that w is much bigger than h and d.

(d) (This question was not asked for, but I am including the answer for student’s information.) Onceyou learn recursive back tracking and dynamic programming, explain how your two pebblingalgorithms correspond to these concepts.

• Answer: A recursive backtracking algorithm has an instance to solve and gets his friends tosolve subinstances for him, who in turn get their friends to solve their subinstances for them.This sets up an entire recursive tree of subinstances to be solved. Some of these subinstancesthat are solved by different friend’s friend’s friends are in fact the exact same subinstances.Hence, the tree of subinstances can be collapsed into a DAG. Each node represents a subin-stance that needs to be solved. If the subinstance associated with node v asks friends tosolve the subinstance associated with nodes In(v) = u1, u2, . . . , ud, then there are edgesfrom each of these nodes to v, specifying that they each need to be solved before v can besolved. The leaves of the DAG correspond to the easy to compute base cases. The goal isto pebble the node corresponding to the initial instance of the problem. The recursive backtracking algorithm gets friends to solve subinstances even if they have been solved before.This corresponds to the recursive algorithm above. It uses exponential time and memoryproportional to the depth of recursion. The Dynamic Programming algorithm assigns one“friend” to each subinstance and solves each once starting the base cases and ending with theorigin instance. This corresponds to the iterative algorithm above. It uses time and memoryproportional to the number of subinstances.

16

v

u2

u1

d=2

20. Recall the pebbling game from an earlier question. You can freely put a pebble on a leaf. If there areedges from each of the nodes In(v) = u1, u2, . . . , ud to v, then there needs to be a pebble on thesenodes u1, u2, . . . , ud before you can put a pebble on v.

Consider a rooted tree such that each node (except leaves) has d children pointing at it. The height ish.

you can place pebble on parent v.Have pebbles on all d=3 children

u3

u21

u

v

Get a pebble to the top.

(a) Give the Theta of the total number n of nodes in the tree. Show your math and give the intuition.

• Answer: There are di nodes at level i. n =∑h

i=0 di = Θ(dh), because it is dominated by thebiggest term.

(b) What is the connection between pebbles and memory?What does a node in the graph represent?What does an edge in the graph represent?

• Answer: Each node represents a value (solution to a subinstance) that needs to be computed.Having a pebble on a node represents the fact that that value is stored in memory. If thereare edges from each of the nodes In(v) = u1, u2, . . . , ud to v, then there needs to be apebble on these nodes u1, u2, . . . , ud before you can put a pebble on v. This is modeling thefact that you need the values associated with u1, u2, . . . , ud in order to be able to computethe value associated with v.

(c) What is the minimum amount of time needed to get a pebble to the root even if you have lots ofpebbles? Why?

• Answer: The time needed is n = Θ(dn), because each node will need to be pebbled.

(d) Briefly describe the recursive algorithm for placing a pebble on the root using as few pebbles aspossible. Describe this algorithm using the friends paradigm.

• Answer: My task is to place a pebble on some node v which is r rows from the bottom. Foreach of the d nodes ui ∈ In(v) with an edge to v, I ask a friend to place a pebble on node ui.Once, there is a pebble on each node in In(v), I place a pebble on node v.

(e) What is the number of pebbles for this recursive algorithm to get a pebble to the root? Give andsolve the recurrence relation.

17

• Answer: The number of pebbles I will need is Pebbles(r) = Pebbles(r − 1) + (d − 1) =(d − 1)× (r − 1) + 1. Note that I must leave a pebble on node ui before moving on to nodeui+1, but the rest of the pebbles can be reused.

(f) What is the running time of this recursive algorithm? Compare it to that for the minimum time.

• Answer: The time needed is still n = Θ(dn), because each node will need to be pebbled.

(g) Imagine you are tracing out this recursive algorithm. Stop the tracing at the point in time atwhich you need the most pebbles. Put an X on the nodes that have pebbles on them at that time.

Chapter: 9: Recursion on TreesSection: 9.1: Tree Traversals

21. Postfix Transversals: Suppose you are deep within the computation of “Postfix Traversal of a BinaryTree”.

(a) (5 marks) Give the output of a postfix traversal of the tree in the next question.

• Answer: 7821 9043 65

(b) (3 marks) Who stores a pointer to the current node in the tree?

• Answer: The current stack frame stores a pointer to the current node in the tree.

(c) (3 marks) Who stores pointers that allow you to move down a path in the tree?

• Answer: The node in the tree stores pointers that allow you to move down a path in thetree.

(d) (3 marks) Who stores pointers that allow you to move up the tree to the root?

• Answer: Stack frames that are pushed on the the stack of stack frames and waiting to finishtheir execution store pointers that allow you to move up the tree to the root.

Section: 9.2: Simple Examples

22. (15 marks) Recursion

Design a recursive algorithm that, given a binary search tree T , and a key key, returns two numberskey− and key+ equalling the largest key in T less than or equal to key and the smallest key in Tstrictly greater than key, respectively. If key is smaller than all keys in T , then key− = −∞ isreturned. Similarly, If key is greater than or equal to all keys in T , then or key+ = +∞ is returned.

Please provide the pseudocode for your algorithm.

• Answer:

algorithm Bracket(T, key)

〈pre−cond〉: T is a BST, key is a real number

〈post−cond〉: Returns two numbers key− and key+ equalling the largest key in T less than keyand the smallest key in T greater than key, respectively. If key is smaller than all keys in T ,then key− = −∞ is returned. Similarly, If key is greater than or equal to all keys in T , thenor key+ = +∞ is returned.

beginif T = ∅ then return (−∞,∞)elseif T.key ≤ key

(key−, key+) = Bracket(N.right, key, m)key− = max(key−, T.key)

elseif T.key > key

18

(key−, key+) = Bracket(N.left, key, m)key+ = max(key+, T.key)

endend algorithm

23. Given a binary search tree T , an key k and an integer b, return two sets Ssmaller and Slarger. Ssmaller

is to contain the largest b elements of T that are smaller or equal to k. If |Ssmaller | < b this is becausethere are no more elements in T than this that are smaller than k. Similarly define Slarger except itis looking for b elements that are strictly larger than k. Feel free to use the notations |S|, S = S1 ∪ S2

and S = S1 + element.

Recurrence relation and running time.

Argue whether or not the running time is O(depth(T ) + b) for every tree T and every value b. If thistime is not quite right, what is a better expression for the time. If the input b is given as a log(b) bitstring then is this time complexity exponential?

• Answer: No the time is O(depth(T ) + min(n, b))

24. Define the distance between two nodes x and y in a binary to be the number of edges in the uniquepath from x to their lowest common ancestor and then back down to y. For example, the distancebetween 7 and 9 is 6 because of the six edges in the path 〈7, 2, 1, 3, 4, 0, 9〉. Write a recursive programthat is given a binary tree and outputs two values. The first, dist, is the maximum distance betweenany two nodes in the tree. It also outputs the depth (height) of the tree in edges. The outputs for thetrees rooted at various nodes are given to the right. (Not to be handed in, but check your code onthese inputs.) Also provide a quick paragraph containing the friend’s explanation of the algorithm.

0

987

1 4

<dist=2,depth=1>because of <7,2,8>

<dist=6,depth=3>

<dist=6,depth=4>because of <7,2,1,3,4,0,9>

because of <7><dist=0,depth=0>

because of <7,2,1,3,4,0,9>

because of <7,2,8> or <1,2,8><dist=2,depth=2>

6

2

3

5

algorithm MaxDistance(tree)

〈pre−cond〉: tree is a binary tree.

〈post−cond〉: dist is the maximum distance (in edges) between two nodes in treeand depth is its depth measured in edges.

begin

• Answer:

if( tree = emptyT ree )return( 〈−1,−1〉 )

else〈distleft, deptleft〉 = MaxDistance(leftSub(tree))〈distright, deptright〉 = MaxDistance(rightSub(tree))dist = Max(deptleft + deptright + 2, distleft, distright)dept = Max(deptleft, deptright) + 1

19

return( 〈dist, dept〉) )end if

end algorithm

• Answer: The key idea in a friend’s algorithm is that we are given a tree as an instance. We giveour left subtree to one friend and our right to another. Because these are smaller valid instanceswe can trust our friend to give us the right answer for their instance. Our job is to construct thecorrect answer for our instance from their answers. This was all I was looking for in the question.I, however, will continue giving you my thinking.

It is best to start by pretending that we have a general big tree with two subtrees. Our depth isdetermined by the deeper of our subtrees. Our tree is one deeper than this. This gives dept =Max(deptleft, deptright) + 1. For example, the depth of the tree rooted at node 5 is dept =Max(4, 0) + 1 = 5.

The maximum distance (in edges) between two nodes in our tree is a little harder. There are twocases depending on whether the root is involved in the maximum path. If the root is involved inthe maximum path, then our answer would be the sum of distance from the root to the deepestnode x in the left subtree plus that to the deepest node y in the right subtree. This first distanceis the depth (in edges) of the left subtree plus 1 for the edge from the root to the root of the leftsubtree. This gives dist = deptleft + deptright + 2.

If the root is not involved in the maximum path, then the two nodes x and y are contained in thesame subtree and our answer is then the same as that given by our friend for the maximum pathlength in this subtree. This gives dist = Max(distleft, distright).

Our answer is the maximum of these two cases. This gives that dist = Max(deptleft +deptright +2, distleft, distright). For example, the dist of the tree rooted at node 5 is dist = Max(3 + 0 +2, 6, 0) = 6 and the dist of the tree rooted at node 3 is dist = Max(2 + 2 + 2, 2, 2) = 6.

This completes tree with both a left and a right subtree. Before continuing let us try of figure outanswer empty tree. If it is done correctly it can save us from having lots of strange cases. Thedepth of the tree rooted at node 2 is 1. The depth of the tree rooted at node 7 is 0. Followingthis pattern let us make the depth of the empty tree -1. For the tree rooted at node 7, the longestpath is that from node 7 to node 7. It has one node and dist = 0 edges. For the empty tree, thelongest path has zero nodes and say dist = −1 or dist = −∞ edges.

So as to not have lots of cases lets test our code as it is on other examples. If the treeonly has one subtree, like that rooted at node 1, then dept = Max(deptleft, deptright) + 1 =Max((−1), deptright) + 1 = deptright + 1 works fine and so does dist = Max(deptleft +deptright + 2, distleft, distright) = Max((−1) + deptright + 2, (−1), distright) = Max(deptright +1, (−1), distright).

When the tree consists of a single node, dept = Max(deptleft, deptright)+1 = Max((−1), (−1))+1 = 0 and dist = Max(deptleft + deptright + 2, distleft, distright) = Max((−1) + (−1) +2, (−1), (−1)) = 0.

This covers all cases.

Section: 9.3: Generalizing the Problem Solved

25. (15 marks) Recall that an AVL tree is a binary search tree with the property that every node has abalance factor of -1, 0, or 1, where its balance factor is the difference between the height of its left andits right subtrees. You are to give pseudocode for an recursive algorithm that is given a binary treeand returns whether or not it is an AVL tree. You do not have to prove its correctness. Also describethe algorithm using a paragraph includin all required aspects of loop invariants, of recursive/friends,and of subroutines.

• Answer: In addition to whether or not the tree is an AVL tree, the routine will return the heightof the tree. If our instance tree has no nodes than we return that it is an AVL tree of height 0.

20

Otherwise, we ask one friend about the left subtree and another about the right. They tell us theheights hleft and hright of these and whether they are AVL trees. If either of the subtrees arenot AVL trees or |hleft − hright| > 1, then we return that our tree is not an AVL. Otherwise, wereturn that it is an AVL tree with height h = max(hleft, hright) + 1.

algorithm IsAV Ltree (tree)

〈pre−cond〉: tree is a binary tree.

〈post−cond〉: The output indicates whether it is an AVL tree.It also gives the height the tree.

beginif(tree = emptyT ree) then

return 〈Y es, 0〉else

〈leftIs, leftHeight〉= IsAV Ltree(leftSub(tree))〈rightIs, rightHeight〉 = IsAV Ltree(rightSub(tree))height = max(leftHeight, rightHeight)+ 1.if( leftIs and rightIs and |leftHeight− rightHeight| ≤ 1 ) then

return 〈Y es, height〉else

return 〈No, height〉end if

end ifend algorithm

Section: 9.4: Heap Sort and Priority Queues

26. Binary Trees

(a) Give the definition of a binary tree

• Answer: A binary tree is either the empty tree or consists a node with a left and right binarytree.

(b) What additional requirements do binary search trees have?

• Answer: For each node, all nodes in its left subtree are smaller or equal to it and all nodesin its right subtree are greater or equal.

(c) Given a set of n integers, how long does it take to put them into a binary search tree. Why?

• Answer: It takes Θ(n logn) time because you must sort them.

(d) What additional requirements do Heaps have?

• Answer: For each node, all nodes in its subtrees are smaller or equal to it.

(e) Given a set of n integers, how long does it take to put them into a heap. Why?

• Answer: It takes Θ(n) time. No more because you don’t really sort them. No less becauseyou do need to look at each integer.

(f) Give two applications of Heaps.

• Answer: Heap Sort and Priority Queues

27. Heaps

(a) What is the ordering imposed on the elements of a max heap?

• Answer: A node’s value must be greater than or equal to the values of its children.

21

(b) Given a binary maxheap A, provide concise pseudocode for an O(1) algorithm that prints thebiggest 3 elements in the heap in non-increasing order. You may make the following assumptions:

i. The heap is implemented as a linear array.

ii. Operations LEFT (i) and RIGHT (i) return the indices of the left and right subtree of thenode stored at index i of the array.

iii. The heap contains at least 10 nodes.

• Answer:

Print A[1]l = LEFT [1]r = RIGHT [1]If A[l] > A[r]

larger = lsmaller = r

Elsesmaller = llarger = r

Print A[larger]Print max(A[smaller], A[LEFT [larger]], A[RIGHT [larger]])

(c) Consider the problem of determining the smallest element in a binary maxheap. Derive a tightlower bound for this problem and prove that it is a tight lower bound.

• Answer:One of the smallest elements of a maxheap must be a leaf. (Otherwise, there must be anon-leaf that is smaller than one of its descendants, which means the tree is not a maxheap.)Thus it is sufficient to search all leaves.Is it necessary? Suppose there exists an algorithm that does not search all leaves. Give ita maxheap A and check which leaf it misses. Let’s call this leaf i If it reports this A[i] asthe minimum, set the value of another leaf to A[i]− 1. The algorithm will be wrong on thisinput. If it reports another value A[j] as the minimum, set A[i] = A[j] − 1. the algorithmwill be wrong on this input. Thus a correct algorithm must check all leaves.An n-node maxheap has ⌈n/2⌉ leaves. Thus the problem of finding the smallest element in abinary maxheap is Θ(n).

Section: 9.5: Representing Expressions with TreesSection: 9.6: Pretty Tree Print

End of Chapter Questions

28. Binary Trees (20 marks)

Consider a binary tree of integers T . Consider a path s from the root of T to any leaf of T . Let ms

be the minimum element on the path. We define the maxmin value of T as the maximum ms for allpaths s from the root of T to all leaves of T .

Examples:

• In Tree (a), the smallest element on the first path 〈7, 3, 5〉 is 3. The smallest element on the secondpath 〈7, 3, 2〉, is 2. The smallest element on the third path 〈7, 4, 8〉 and fourth path 〈7, 4, 5〉 is 4.The largest of these four smallest values 3, 2, 4, 4 is 4.

• In Tree (b), the smallest element on the path 〈7, 2〉 is 2 and the smallest element on the path〈7, 4〉, is 4. The largest of these two smallest values 2, 4 is 4.

• In Tree (c), the smallest element on the path 〈7, 2〉 is 2 and the smallest element on the path〈7, 9〉, is 7. The largest of these two smallest values 2, 7 is 7.

22

e)

Return ?Return 7Return 7Return 4Return 4

a)d)c)b)

18

4

62

3

7

7

2 4

7

2 9

68

4

25

3

7

7

8

• In Tree (d), there is only one path 〈7, 8〉. Its minimum is 7. The maximum of this one value is 7.

(a) (2 marks) What is the maxmin value for Tree (e).

• Answer: Value 4.

(b) (7 marks) Let T be the root of a binary tree of integers. Let T.val be the integer-valued key atthe root. Let mL be the maxmin value of the subtree rooted at T’s left child. Let mR be themaxmin value of the subtree rooted at T’s right child. Write an expression for the maxmin valuemT of the tree rooted at T in terms of T.val, mL and mR.

• Answer: mT = max(min(mL, T.val), min(mR, T.val))

(c) If we want this general code to work for a tree consisting of a single node, what answer shouldthe empty tree return? Does ∞ work? Why? Does −∞ work? Why?

• Answer: Yes, ∞ works. If the tree consists of one node, the root element is returned asrequired.max( min(MaxMin(leftSub(tree)), root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )= max( min(∞, root(tree)), min(MaxMin(∞, root(tree)) )= max( root(tree), root(tree) ) = root(tree)

No, −∞ does not work. If the tree consists only of 7, then there is only one path whoseminimum is 7. However, the following incorrect thing is returned.max( min(MaxMin(leftSub(tree)), root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )= max( min(−∞, 7), min(−∞, 7) ) = max(−∞,−∞) = −∞.

(d) If we want this general code to work for a tree with no left subtree, what answer should the emptytree return? Does ∞ work? Why? Does −∞ work? Why?

• Answer: No, ∞ does not work. If the root is 7, left subtree is empty, and the right is 2, thenthere is only one path whose minimum is 2. However, the following incorrect thing is returned.max( min(MaxMin(leftSub(tree)), root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )= max( min(∞, 7), min(2, 7) ) = max(7, 2) = 7.It mistakenly thinks the of path to the empty tree as a path.

Yes, −∞ works. If the left subtree is empty, the following correct thing is returned.max( min(MaxMin(leftSub(tree)), root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )= max( min(−∞, root(tree)), min(MaxMin(rightSub(tree)), root(tree)) )= max( −∞, min(MaxMin(rightSub(tree)), root(tree)) )= min(MaxMin(rightSub(tree)), root(tree))

(e) Give the entire code which works on every type of input.

• Answer:

if(tree = emptyT ree) thenreturn −∞

elseif(rightSub(tree) = emptyT ree and leftSub(tree) = emptyT ree) then % tree is a leafreturn rootNode(tree)

23

elsereturn max( min(MaxMin(leftSub(tree)), root(tree)),

min(MaxMin(rightSub(tree)), root(tree)) )endif

end alg

(f) Give all the key ideas in thinking abstractly about a recursive program?

• Answer: If your input is sufficiently small solve it yourself. Otherwise, construct instancesfor friends that are smaller then your instance and that meet the preconditions. Withoutyou worring how, assume that they can solve their instance. Combine the solutions to theirinstances to obtain a solution to your instance.

29.

9 marks: Finding the kth Smallest Element: Write a re-cursive program, which given a binary search tree and aninteger k returns the kth smallest element from the tree.(Code is fine.)

D 15

k=3

B 4

A 8

C 10

kth smallest = 10

• Answer:

algorithm Smallest(tree, k)

〈pre−cond〉: tree is a binary search tree and k > 0 is an integer.

〈post−cond〉: Outputs the kth smallest element s and the number of elements n.

beginif( tree = emptyT ree ) then

result( 〈NotPossible, 0〉 )else

〈sl, nl〉 = Smallest(leftSub(tree), k)% There are nl + 1 nodes before the right subtree〈sr, nr〉 = Smallest(rightSub(tree), k− (nl + 1))n = nl + 1 + nr

if( k ∈ [1..nl] )thens = sl

elseif( k = nl + 1 )thens = root(tree)

elseif( k ∈ [nl + 2..n] )thens = sr

else thens = OutOfRange

endifresult( 〈s, n〉 )

end ifend algorithm

Chapter: 10: Recursive ImagesSection: 10.1: Drawing a Recursive Image from a Fixed Recur-sive and Base Case ImagesSection: 10.2: Randomly Generating a MazeChapter: 11: Parsing with Context-Free GrammarsEnd of Part Questions

24

30. (20 marks) Recall the Tower of Hanoi problem discussed in class. The task is to move n rings, each ofa different diameter, from a source peg Source to a destination peg Dest. A third peg Spare may beused in the process. The rings start off on Source, stacked according to size, with the largest on thebottom. The rings are moved one at a time, and a ring can never be put on a ring that is smaller thanit. In class we discussed a recursive algorithm to solve this problem:

algorithm TowerofHanoi(n, Source, Spare, Dest)

〈pre−cond〉: The n smallest rings are stacked on Source

〈post−cond〉: The n smallest rings are stacked on Dest

beginTowerofHanoi(n− 1, Source, Dest, Spare)Move the nth-largest ring directly from Source to Dest in one moveTowerofHanoi(n− 1, Spare, Source, Dest)

end algorithm

(a) (10 marks) Derive a tight bound on the complexity of the problem (as a function of n).

• Answer: The recurrence relation of the optimal solution is

T (n) = 2T (n− 1) + 1

= 2 [2T (n− 2) + 1] + 1 = 4T (n− 2) + 3

= 2 [2 [2T (n− 2) + 1] + 1] + 1 = 8T (n− 3) + 7

...

= 2iT (n− i) + 2i − 1

= 2n−1T (1) + 2n−1 − 1

= 2n − 1

(b) (5 marks) Suppose that you are now allowed to move 10 of the disks at a time. How will thischange the asymptotic complexity of the problem (as a function of n)?

• Answer: Method 1. T (n) = 2T (n − 10) + 1 = O(2n10 ). My work of 1 is the work of me

moving the last block of 10 largest disks.Method 2. Put the disks into blocks which you move together. There are n

10 blocks. This isjust like the normal problem when there are n

10 disks. This gives T (n) = O(2n10 ).

(c) (5 marks) Suppose that you are now allowed to move n10 of the disks at a time, where n is the

original number of disks. How will this change the asymptotic complexity of the problem (as afunction of n)?

• Answer: T (n′) = 2T (n′− n10 )+1 = O(210) = O(1), where n is the original n. (This does not

make a good recursive algorithm, because it should not know the size of the original instance.)If we put the disks into blocks which you move together, then there are 10 blocks. This isjust like the normal problem when there are 10 disks. This gives T (n) = O(210) = O(1).

(d) The goal is to design an algorithm for Towers of Hanoi with more towers/stacks. Review (3101notes) the Hanoi alg with three towers. Remember to trust the friend. I move n disks fromtowersource to towerdestination using three towers as follows. I ask my friend to move the smallestn−1 disks from towersource to towerspare. On my own, I move the largest disk from towersource totowerdestination. Finally, I ask my friend to move the n−1 disks from towerspare to towerdestination.

Let Friendk be the following routine that is able to move a stack of disks from a source to adestination tower using k towers. It has a friend move the n−ck(n) smallest disks onto a differenttower. (Here ck(n) is a parameter chosen later.) It has another friend move the ck(n) remaining

25

disks to the desired location. Finally, it has yet another friend move the n− ck(n) smallest disksonto the desired location. Let Tk(n) be the required time. For example, the routine describedabove is Friend3 with c3(n) = 1.

i. Give the recurrence relation for the required time Tk(n). Note Tk(n) will be a function of n,c(n), and possibly other Tk′ .

• Answer: The first friend has all k towers available to him to move the n− ck(n). Hence,his time is Tk(n− ck(n)). The second friend can only use k− 1 towers because we cannotuse the tower holding the smaller disks. Hence, his time is Tk−1(ck(n)). The third friendstime is again Tk(n−ck(n)). This gives The time required is Tk(n) = 2·Tk(n−c)+Tk−1(c).

ii. Now assume that you must move n disks using four towers. Are you allowed to ask Friend4

to move n− 1 disks using four towers? Are you allowed to ask him to move n disks? Are youallowed to ask Friend3 to move n disks using three towers? Why?

• Answer: Friend4 is a recursive call and must be given a smaller instance. Hence, he canmove n − 1 but not n disks. Friend3 is a call to a routine that is already written so canbe called on any size instance. If you consider Friend(n, k) to be one recursive routineon two inputs, then we can consider the instance 〈2n, k − 1〉 to be “smaller” than 〈n, k〉.

iii. Algorithms often run the fastest when you cut the input in half. Hence, let us set c(n) = n2 .

Determine its running time Tk(n)(n). Hint: For an easier approximation ignore the influenceof k(n).

• Answer: The running time is at most T (n) = 3T (n2 ) = Θ(n

log alog b ) = Θ(n

log 3log 2 ).

iv. Again consider the algorithms with c(n) = n2 . What is the number k(n) of towers needed to

make sure each friend at each level of recursion has a spare tower. Prove this in two ways.First unwind the algorithm to deterine how many towers is needed. The second way is tostate in the precondition of the problem that in order to move n disks, k(n) towers is needed.Then prove that the tasks that you give your friends meet this precondition.

• Answer: Unwinding we see that there are log2 n levels of recurssion, each of which usesan extra tower. Hence, k(n) = log2(n) + 1 towers are needed. The only tricky case is thesecond friend. When moving the n

2 largest disks, he cannot use the tower containing then2 smallest disks. Hence, he only has log2(n) + 1 − 1 = log2(

n2 ) + 1 towers available to

him. However, given he only has n2 disks to move, this is exactly the number of towers

he needs to be meeting the preconditions of the problem.

v. Consider the following game. You start with n disks. Each step your remove c(n′) disks fromyour current number n′. We need to compute the number of steps h until there are no disksleft. It is easy to see that if c is a constant then h = n

c steps are needed. If c(n′) shrinks asn′ shrinks then fewer disks are removed each step and hence more than h = n

c(n) steps are

needed. However, until half the disks are gone, at least c(n2 ) disks are removed each step.

Hence at most h′ = n/2c(n/2) levels are needed until half the disks are gone. The total number

of steps is then at most h ≤ ∑log2 ni=1

n/2i

c(n/2i) . If c(n) = n2 then give the Theta of this sum.

Again for c(n) = n1/4.

• Answer: If c(n) = n2 , then h ≤ ∑log2 n

i=1n/2i

c(n/2i) =∑log2 n

i=1 2 = 2 log2 n. If c(n) = n1/4,

then the terms in∑log2 n

i=1n/2i

c(n/2i) change by some constant factor and hence this sum

is geometric. This means that the sum is dominated by the biggest term, giving h ≤Θ( n/2

c(n/2) ) = Θ( nc(n)), which is tight with our lower bound bound.

vi. Solve T4 as a function of n and c(n). (Assume h = nc(n) ). What is the number of base cases

and what is the work at the top level? Ignoring the other levels for a minute, set c(n) tooptimize running time for these two levels. Then when considering all levels, approximateT4(n).

• Answer: The time with three towers is T3(n) = 2 · T3(n − 1) + 1 = 2n. For k = 4,this is T4(n) = 2 · T4(n − c) + T3(c) = 2 · T4(n − c) + Θ(2c). The number of base

26

cases is 2h where h = nc(n) is the number of levels of recursion. The amount of work at

the top level is 2c. We set c(n) to balance these two amounts, i.e. 2n/c = 2c, givingc(n) =

√n. Because there are h levels, each with the same work 2n/c = 2c, the total work

is T4(n) = h2c =√

n2√

n ≈ 2√

n.

vii. Repeat this to compute T5(n) and Tk(n). To make it easy ignore low order factors, eg.n log n ≈ n and nc · 2n ≈ 2n. For what value of k(n) does this time become a polynomial?(For this value of k(n) do not expect to get a good approximation because we were ignoringfactors as we went along.) Compare this ti your result in (b).

• Answer: Similarly, T5(n) = 2 ·T5(n−c)+T4(c) ≈ 2 ·T5(n−c)+2√

c. Setting the number

of base cases 2n/c to be equal to top level work 2c1/2

, gives c = n2/3 and T5(n) ≈ 2n1/3

.

Suppose by induction Tk′(n) ≈ 2n1/(k′−2)

). Then Tk(n) = 2 · Tk(n − c) + Tk−1(c) ≈2 · Tk(n − c) + 2c1/(k−3)

. Setting the number of base cases 2n/c to be equal to top level

work 2c1/(k−3)

, gives c = n(k−3)/(k−2) and Tk(n) ≈ 2n1/(k−2)

. For k = log2(n) + 2, note

that 2log2(n) = n and hence n1/ log2(n) = 2 and Tk(n) ≈ 2n1/(k−2)

= 2n1/ log2(n)

= 22 = 4.Clearly this is a bad approximation of the time, but it does say that this time is nowlikely a polynomial.

Part III: Optimization ProblemsChapter: 12: Definition of Optimization ProblemsChapter: 13: Graph Search AlgorithmsSection: 13.1: A Generic Search Algorithm

31. (10 marks) Graph searching:

(a) (2 marks) Briefly describe an algorithm which is given a directed graph and a destination node tand outputs all nodes v for which there is a path from node v to t. You can use as a subroutineany algorithm discussed in class.

• Answer: Turn the edges around and call a generic search algorithm (e.g., BFS) to output allnodes reachable from t.

(b) (8 marks) Let G be a directed graph where each node of G also has a value:e.g., node 1 mighthave value 100, node 2 might have value 50, etc. Given an O(|E| + |V |log|V |) time algorithmthat computes, for every node, the maximum value reachable from that node. For instance, if Gis strongly-connected (every nodes is reachable from every node), then for every node this will bethe maximum value in the entire graph. Hint: one worthwhile preprocessing step is to sort nodesby value.

• Answer: First modify your algorithm for Part (b) so that it marks nodes once they arehandled, and doesn’t explore marked nodes on subsequent searches. Now sort the nodes indecreasing order of value (Θ(|V |log|V |) time). Starting with the most valuable node, runyour modified search algorithm to find all nodes that can reach this most valuable node, andrecord this maximum value for each of these nodes. Now repeat the search from nodes oflesser value in descending order. Since vertices are only handled once, this search step takesT (|E|, |V |) = Θ(|E|), so the total time required is T (|E|, |V |) = Θ(|E|+ |V |log|V |).

Section: 13.2: Breadth-First Search/Shortest PathsSection: 13.3: Dijkstra’s Shortest-Weighted Paths Algorithm

32. Fixed edge on a shortest pathGiven a directed, weighted graph G with non-negative weights (weights can be zero), vertices s, t andan edge (u, v) in G:

27

(a) Describe (in English or pseudo-code) how to determine whether (u, v) occurs on every path ofminimal cost from s to t.

• Answer: First use Dijkstra’s algorithm to compute the cost d of the shortest path from s tot. If d =∞, then there is no path of minimal cost from s to t, and it is true that (u, v) occurson every min cost path from s to t.Now consider the graph G′ obtained from G by removing the edge (u, v). Use Dijkstra’salgorithm again to compute the cost d′ of the shortest path from s to t in G′. If d 6= d′ returntrue, otherwise return false.To see that the algorithm works, first assume that the edge (u, v) does appear on every min-cost path from s to t in G. That is, (u, v) appears on every path of cost d from s to t. Hencethe min-cost path found on G′ cannot be of cost d, because it does not contain (u, v). Henced′ 6= d, and the algorithm returns true in this case.If there is a min-cost path from s to t not containing (u, v), then the same path would be ashortest path in G′, so d′ = d, and the algorithm returns false in this case.The algorithm calls Dijikstra’s algorithm twice, and can be implemented in O(E log V ).

(b) Describe (in English or pseudo-code) how to determine whether (u, v) occurs on some path ofminimal cost from s to t.

• Answer: Once again, we run Dijkstra’s algorithm from s to compute the weights of the shortestpaths d(s, t) and d(s, u). If d(s, t) =∞, then there is no min cost path from s to t, and it is falsethat there is such a path through (u, v).

If d(s, t) <∞, we run Dijkstra’s algorithm from v to compute d(v, t). We return true if d(s, t) =d(s, u) + w(u, v) + d(v, t), and false otherwise.

To show that the algorithm works, suppose that there is a min-cost path p = s→ . . .→︸ ︷︷ ︸p1

u →

v→ . . .→︸ ︷︷ ︸p2

t through (u, v). We know that a subpath of a shortest path must be a shortest

path, hence p1 is a shortest path from s to u, and p2 is a shortest path from v to t. We havew(p) = d(s, t), w(p1) = d(s, u) and w(p2) = d(v, t). So d(s, t) = w(p) = w(p1)+w(u, v)+w(p2) =d(s, u) + w(u, v) + d(v, t), and the algorithm returns true in this case.

In the opposite direction, suppose that the algorithm returns true. Then we have d(s, t) = d(s, u)+w(u, v)+d(v, t). Denote by p1 a shortest path from s to u, and p2 a shortest path from v to t. Thenw(p1) = d(s, u), and w(p2) = d(v, t). Take the path p = s→ . . .→︸ ︷︷ ︸

p1

u → v→ . . .→︸ ︷︷ ︸p2

t. p obviously

contains the edge (u, v), and w(p) = w(p1)+w(u, v)+w(p2) = d(s, u)+w(u, v)+d(v, t) = d(s, t),so p is a min-cost path containing (u, v), which proves that there exists such a path.

The algorithm calls Dijikstra’s algorithm twice, and can be implemented in O(E log V ).

Section: 13.4: Depth-First SearchSection: 13.5: Recursive Depth-First SearchSection: 13.6: Linear Ordering of a Partial OrderEnd of Chapter Questions

33. Given a directed graph and an edge 〈u, v〉 give an algorithm to find (if possible) a cycle that contains〈u, v〉. Prove that the algorithm works.

• Answer: To find a cycle use depth first search starting with the edge 〈u, v〉. If the node u isrefound while the nodes u and v are still on the stack then the contents of the stack u, v, . . . ,and then back to u forms a cycle. Conversely, suppose there is a cycle u, v, . . . , w, u. Before v isremoved from the stack, w must be found and the edge 〈w, u〉 handled. Hence, the node u is willbe refound and the cycle returned.

28

Exercise 0.0.2 (See solution in Section ??) A Boolean circuit can be represented as an acyclic di-rected graph in which every node is labeled with one of 0, 1, AND, OR, NOT , and edges are calledinputs. See Figure 1. Nodes with labels 0 or 1 (input nodes) have no edges (inputs) going into them,nodes with label NOT have one input, and nodes with labels AND and OR can have two or moreinputs. The unique node of out-degree 0 is called the “output node”. Give an algorithm to evaluate acircuit, producing the value of the output node according to the usual rules. The recursive algorithmin Example ??.1 for evaluating an expression can solve this problem. Give some thought to why thisalgorithm takes linear time on a tree and exponential time on a graph. Then come up with an O(|E|)time algorithm that uses topological sort. The exponential recursive algorithm is called Recursive Back-tracking and the linear time algorithm is called Dynamic Programming. Try to understand these ideason your own and then read about them in Chapters ?? and ??.

NOT

y

AND AND

OR

yx

x

NOT

y=1x=0

AND AND

NOT

OR

NOT

Figure 1: A circuit for (x AND y) OR(¬x) OR (¬x AND ¬y) and a graph representing it. With x = 0 andy = 1 this evaluates to (0 AND 1) OR(¬0) OR (¬0 AND ¬1) = (0) OR(1) OR (0) = 1

0.0.2 Run topological sort on the graph to find a total order of the nodes. Computing the value of the nodesin this order, starting with the input nodes and ending with the output node. By the definition of thetotal order, when it is time to compute the value of a node, all the node’s inputs will already havebeen computed.

Chapter: 14: Network Flows and Linear ProgrammingSection: 14.1: A Hill Climbing Algorithm with a Small LocalMaximumSection: 14.2: The Primal-Dual Hill Climbing MethodSection: 14.3: The Steepest Ascent Hill Climbing AlgorithmSection: 14.4: Linear ProgrammingEnd of Chapter Questions

34. Network Flow: Consider the following flow network with source s and terminal (or sink) t. The labelson edges are of the form “f/c′′ where f is the current flow and c is the capacity of the edge. This flowis feasible.

0/2

dc4/9

5/5

7/17

5/7

6/6

t2/2

3/32/2

6/10

ba

s

ba

st

c ddc

ts

a b

(a) Trace one iteration of the network flow algorithm. Use the nodes in the middle fig to give theaugmentation graph for this flow. Make the necessary changes to the left figure to give the newflow.

29

(b) Prove that there is no flow that is better than this current flow. (Use the right figure if you like.)

• Answer: The value of this flow value is f(s, a) + f(s, c) = 6 + 8 = 14.The min-cut (U, V ), where U = s , a , c , d. The min-cut capacity isc(a, b) + c(d, b) + c(d, t) = 7 + 2 + 5 = 14. No flow can exceed this cut.

Augmenting Path2

0/2

0/2

New Flow

5

5

V

U

9

87

1

2

6

3

8+2=10

2

Augmenting Graph

6

22

Cut

5

4

6+2=8

4

2 3 27

105

2

6

Augmenting GraphGiven Flow

ba

st

c d

ba

st

c d

s

a b

6/10

2/23/3

2/2 t

6/6

5/7

7/17

5/5

4/9c d

dc

5/5

6/6

t2/22/2

ba

s

8/10

6/9

1/3

7/7

9/17

ba

st

c d

7/7

dc

5/5

t2/2

ba

s

35. 7 marks: Network Flows

(a) 3 marks: In class, what was used to prove that when Ford-Fulkerson’s network flow algorithmstops with a flow, it has the optimal flow?

• Answer: The algorithm provides a matching min cut that acts as an upper bound

(b) 4 marks: Starting with the flow given below, complete the network flow algorithm.

36. (3 marks) Flows

Recall that the flow of a network is defined as the flow out of the source. Suppose you have a networkwhere flow conservation does not apply (i.e., for nodes other than the source and sink, the flow in maynot equal the flow out). Would the maxflow-mincut theorem still apply? Why or why not?

• Answer: The maxflow-mincut theorem would no longer apply. The max flow is equal to the sumof the capacities leaving s. This is also the value of a cut. Hence, the min cut is smaller thanequal to this. Consider a graph with edge 〈s, v〉 having f = c = 2 and 〈v, t〉 having f = 0 andc = 1. Then the max flow is 2 and the min cut is 1.

37. (15 marks) Suppose you are given a real-valued network flow instance, an optimal flow for it, and anedge 〈u, v〉. Though the current flow through the edge 〈u, v〉 does not exceed its capacity, the owner ofthis edge has bribed the authorities to decrease the flow through this edge as much as possible whilemaintaining the validity and the optimality of the overall flow. You are to design an efficient algorithmto achieve this. Explain your algorithm using the following instance as an example. A formal proof ofcorrectness is not required.

s5/5

7/7

2/10

t5/11

Optimal Flow

2/6

u

v

w

• Answer:

30

There are at least two reasonable solutions to this problem.

Solution 1.

The algorithm constructs in the normal way the augmenting graph for the current flow. Clearly,because the current flow is optimal, there is no path from s to t. Instead find a cycle in theaugmentation graph that contains the edge 〈v, u〉. Find the smallest augmenting value on thiscycle. Modify the current flow along this cycle as in the network flow algorithm. Repeat thisprocess until there is no such cycle. To find the cycle, use depth first search starting with theedge 〈v, u〉. There is a cycle containing this edge if and only if the node v is refound while thenodes v and u are still on the stack.

6/10

1/5 1/11

v

u

w

v

u

w

v

u6/6

44

7

2 2

8

5

65

s t

Augmentation Graph Augmentation Cycle New Optimal Flow

v

s 7/7 tts

Solution 2.

Denote the original flow network as G and the optimal flow on this network as f∗. Our algorithmconstructs a modified flow network G0 in which the capacity of edge 〈u, v〉 is set to 0. (Whileby convention we normally include only edges with positive capacity, we make an exception inthis case for the sake of the proof.) The algorithm then uses an efficient maxflow algorithm (e.g.,Edmonds-Karp) to recompute the optimal flow f0 for the modified network. We then form afinal network G′ in which the capacity of 〈u, v〉 is set to f∗ − f0. Running an efficient max flowalgorithm on this network computes the optimal flow of value f∗ with minimum flow on 〈u, v〉.The proof is as follows. First suppose that f0 = f∗, i.e., 〈u, v〉 is not required to achieve themaximum flow. Then clearly the optimal solution is to set the capacity of 〈u, v〉 to f∗ − f0 = 0,to ensure no flow on 〈u, v〉.Otherwise, f0 must be less than f∗. Since the original flow network G has a max flow of f∗, bythe max-flow min-cut theorem, the capacity of the minimum cut of G is also f∗. Similarly, thecapacity of the minimum cut of G0 is f0. Since the only difference between G∗ and G0 is thecapacity of edge 〈u, v〉, 〈u, v〉 must be in all min cuts of G0, and all cuts that do not contain 〈u, v〉must have capacity of at least f∗. By increasing the capacity of 〈u, v〉 by f∗ − f0, we increasethe capacity of all min cuts of G0 to f0 + f∗ − f0 = f∗, and any max flow solution will entail aflow of f∗ − f0 through 〈u, v〉. If we increase the capacity of 〈u, v〉 by less, than the capacity ofthese min cuts will be less than f∗. Thus f∗ − f0 is the minimum flow on 〈u, v〉 that achieves anoptimal network flow.

38. Once your hill climbing algorithm finds an optimal solution for the given Network Flow instance,describe how you would take “steps” that move you around on top of this plateau from one optimalsolution to another. As with a hill climbing step, this step will look at the augmentation graph of thecurrent flow. Clearly, however, you will not be able to find an st- path in this augmentation graph.What do you look for instead? (If you are really bold, attempt to prove that every optimal solutioncan be reached with such steps.)

• Answer: Same solution for the last question.

39. Short Answers

(a) (2 marks) What is the purpose of considering the Dual of a Linear Program?

(b) (2 marks) What is the Steepest Accent version of the Network Flows algorithm?

(c) (2 marks) What is the measure of progress for this Steepest Accent Network Flows algorithm thatwe used and how much does it change each iteration.

31

40. Recall the Network Flow problem. Suppose that in addition to edge capacities, we wanted to allowvertex capacities too. For example, we might want to say that vertex u can have at most 5 units offlow passing through it and vertex v can have at most 3. Show how to convert the maximum flowproblem with both node and edge capacities to the standard maximum flow problem that is not toomuch larger. In other words, imagine we had an instance with both kinds of constraints, and we wantedto solve it using a black-box program that only solved the standard edge-capacity problem.

• Answer: Replace each node u in the network with two nodes u0 and u1 and a directed edge〈u0, u1〉 with capasity equal to the capasity of the node. Replace each directed edge 〈u, v〉 in thenetwork with the directed edge 〈u1, v0〉 with the same capasity. It is easy to see that any flowin the original network translates to a flow in the new network and visa versa. My answer hereshould be a little longer but I think you have it.

Chapter: 15: Greedy AlgorithmsSection: 15.1: Abstractions, Techniques and TheorySection: 15.2: ExamplesSubsection: 15.2.1: Example: The Job/Event Scheduling Prob-lemSubsection: 15.2.2: Example: The Interval Cover ProblemSubsection: 15.2.3: Example: The Minimum-Spanning-TreeProblem

41. The fractional-knapsack problem is the same as the integer-knapsack problem except that there isa given amount of each object and any fractional amount of each it can be put into the knapsack.Develop a quick greedy algorithm for this problem.

42. [2 + 2 + 4 = 8 marks] Greedy Algorithms:

(a) Trace out Minimal Spanning Tree on the following example. Darken the selected edges.

42 99

7589

855627

97

16

81

61

1253

95

67

64

84

• Answer:

42 99

7589

855627

97

16

81

61

1253

95

67

64

84

32

(b) DFS can be used to find a Minimum Spanning Tree (MST) of a given connected graph, if all edgeshave equal weight.

• Answer: T

(c) In general, what action does a greedy algorithm take each iteration?

• Answer: Commits or rejects the next best object.

(d) What is the generic loop invariant for a greedy algorithm?

• Answer: There exists an opt solution consistent with choices made so far.

(e) Briefly explain how we prove that this loop invariant is maintained.

• Answer: Massage the opt solution consistent with previous choices into one consistent withthe new choice as well.

(f) In both Dynamic Programming and Greedy Algorithms the question for the little bird needs tobe answered. What is the difference between how this is done in these two settings?

• Answer:

i. In Dynamic Programming all options are tried. The best of the best for each of theseoptions is taken. In a Greedy Algorithm one choice is made and committed to.

ii. In Dynamic Programming the choice is about whether or not to take the last object, whilein a Greedy Algorithm the “best” object is taken.

43. Bottleneck Spanning Trees (25 marks)

A bottleneck spanning tree of an undirected graph G is a spanning tree of G whose largest edge weightis minimum over all spanning trees of G.

(a) (10 marks) Prove that a minimum spanning tree is a bottleneck spanning tree.

• Answer: Let T = (V, ET ) be an MST of G = (V, E), and let T ′ = (V, ET ′) be a bottleneckspanning tree of G. Suppose T is not a bottleneck spanning tree. Then there exists an edge(x, y) ∈ ET : w(x, y) > w(x′, y′)∀(x′, y′) ∈ ET ′ .Consider a cut (X, Y ) of the graph G, x ∈ X, y ∈ Y , such that (x, y) is the only edge in T thatcrosses the cut. This cut partitions T into two subtrees TX and TY . Since (x, y) /∈ T ′, theremust be at least one other edge (x′, y′), x′ ∈ X, y′ ∈ Y , crossing the cut such that (x′, y′) ∈ T ′

and (x′, y′) /∈ T .The edge (x′, y′) forms a cycle with the path from (x′ to y′) in T . Thus we can form a newspanning tree T by replacing x, y by (x′, y′) in T . Since w(x′, y′) < w(x, y), w(T ) < w(T ).However, this is not possible, since T is a minimum spanning tree. Thus our supposition thatT is not a bottleneck spanning tree must be wrong: if T is a minimum spanning tree, it mustalso be a bottleneck spanning tree.

(b) (15 marks) Your company needs to lease access to a communications network linking a largenumber of Canadian cities. The network can be modeled as a connected, undirected graph withvertices representing cities and edges representing direct communication links between cities. Eachedge provides a maximum data rate, or bandwidth. You need to be able to move data from anyone of the cities to any other, but you do not necessarily need every link. Since you will be tariffedaccording to the number of links you use, you want to minimize the number of links you lease. Inother words, you seek a spanning tree of the graph.

Since your company provides broadband services, you wish to maximize bandwidth. On anypath connecting two cities in the network, the bandwidth of the path is limited by the minimumbandwidth edge on the path. Your goal is to determine the spanning tree that connects eachpair of cities by a maximum bandwidth path. Your tree will have the property that adding anyadditional edges from the graph will not produce a path with higher bandwidth.

Design an algorithm to solve this problem and prove that it is correct.

33

• Answer: We assign each edge in the graph G a weight equal to the inverse bandwidth. Thenwe compute (using, e.g., Kruskal’s or Prim’s algorithm) a minimum spanning tree for thisgraph. We claim that the resulting tree will have the desired properties.The proof is similar to the proof for Part (a). Let T = (V, ET ) be an MST of G = (V, E).Consider an arbitrary pair of vertices x, y ∈ V , and the path p connecting these vertices inT . Let (xm, ym) ∈ p denote the minimum bandwidth (bottleneck) edge in p. Consider a cut(X, Y ) of the graph G, x, xm ∈ X, y, ym ∈ Y , such that (xm, ym) is the only edge in T thatcrosses the cut. This cut partitions T into two subtrees TX and TY .By way of contradiction, suppose that p is not a maximum bandwidth path in G. That meansthat there exists another path p′ ∈ G from x to y of greater bandwidth. Since (xm, ym) /∈ p′,there must exist a different edge (x′, y′) ∈ p′, x′ ∈ X, y′ ∈ Y crossing the cut (X, Y ), suchthat (x′, y′) /∈ T . This edge forms a cycle with the path from x′ to y′ in T . Thus a newspanning tree T can be formed by replacing (xm, ym) by (x′, y′) in T .Note that since the bandwidth of (x′, y′) is greater than the bandwidth of (xm, ym), w(x′, y′) <w(xm, ym). Thus w(T ) < w(T ). This is a contradiction, since T is a minimum spanning tree.Thus our supposition that p is not a maximum bandwidth path must be wrong: all paths inT must be maximum bandwidth paths.

End of Chapter Questions

44. ** The question is already in the notes in some form. Here is an answer. Truth is that I did not likeit as much as I thought I would. **

We proved that the greedy algorithm works for the making change problem, when the denominationsof the coins are 25, 10, 5, and 1, but it does not work when the denominations are 4, 3, and 1 cent.More generally, suppose the coin denominations are c1 > c2 > . . . > cr = 1 (order taken by the greedyalgorithm). An interesting problem is determining whether the greedy algorithm works for these. Acomplete answer to this question is too hard. However, we can answer some questions about this.

(a) Suppose for each i, each coin ci is an integer multiple of the next smaller ci+1, eg 120, 60, 12, 3, 1.This question will proof that for every set of coins with this property and for every input amountto make up, that the greedy algorithm works.

Add: Suppose you have a “second” coin worth 1, a “minute” coin worth 60, an hour coin worth“360” a day coin worth “24*360”. If you are making up some amount with the minimum numberof coins, will you ever have 73 minute coins? If you add up the minutes and the second in theoptimal, what is the biggest this sum will be?

i. Consider such coin denominations. Consider an amount to be made with these coins. Consideran optimal solution optSLI held by the fairy god mother. For each i ∈ [1, r], let ni be thenumber of coins of denomination ci. Prove a useful property of ni.

• Answer: We will prove that for each i ∈ [2, r], the number ni of coins of denomination ci

is in the range [0, ci−1

ci− 1]. By way of contradiction, if ni ≥ ci−1

ci, then we could remove

ci−1

cicoins of denomination ci and replace them with one coins of denomination ci−1. The

total value does not change because ci−1

ci× ci = 1 × ci−1. The total number of coins

decreases by ci−1

ci− 1 > 0. This contradicts that what the fairy god mother holds is an

optimal solution. We are using here that fact that ci−1

ciis an integer.

ii. Again, consider the optSLI held by the fairy god mother that has the property stated in thelast question. For each i ∈ [2, r + 1], let ai =

∑j∈[i,r] nj × cj be the total value of the coins

with denomination ci or smaller. Prove by induction on i, some useful property of ai.

• Answer: We will prove that for each i ∈ [2, r + 1], that ai ≤ ci−1 − 1. The base case,when i = r + 1, must have ni = 0 coins with denomination ci or smaller because thereare no coins with denomination cr+1 or smaller. Now by way of induction, assume thatai+1 ≤ ci−1. By definition of ai, we have that ai =

∑j∈[i,r] nj×cj = ni ·ci+ai+1. By our

34

previous question and by the induction assumption, this is at most [ ci−1

ci−1]·ci+[ci−1] =

ci−1 − 1. Hence, the inductive hypothesis is true for i. By way of induction, it is true forall i ∈ [2, r + 1].

iii. Suppose that the fairy god mother holds an optimal optSLI consistent with all the choicesmade by the Algorithm so far. Suppose the algorithm in the next iteration commits to acoin of denomination ck. Explain how to massage optSLI into optSours and prove that thisis a valid solution, is consistent both with the choices made previously by the algorithm andwith this new choice, and is optimal. Might there be two different optimal solutions to thisproblem?

• Answer: We will prove that the number ni of each denomination in the optimal solutionis unique by showing that except for may be swapping one coin of denomination ci foranother of the same denomination, we do not need to massage the fairy god mother’ssolution optSLI at all.Let n′

i be the number of coins of denomination ci in the fairy god mother’s solution optSLI

excluding those that the algorithm has committed to so far. Our goal is to prove thatn′

k ≥ 1, i.e. that optSLI contains a coin of denomination ck that has not previously beencommitted to by the algorithm. We massage optSLI into optSours by replacing this ck

with the one that the algorithm committed to.Let A be the remaining amount required by the algorithm, i.e. the total amount requiredby the input minus the value of the coins committed to so far. Because the algorithm didnot commit to a coin with denomination ck−1 but did commit to a coin with denominationck, we know that ck ≤ A < ck−1. Because the remaining amount for the algorithm andthe adversary is the same, we have that A =

∑i∈[1,r] n

′i · ci.

Because A < ck−1, we know that know that the fairy god mother is not holding any largecoins that we do not know about, i.e. for i ≤ k − 1, we have that n′

i = 0.Let a′

k+1 =∑

j∈[k+1,r] n′j × cj be the total amount of the coins with of denomination

ck+1 or smaller in optSLI excluding those that have been committed to. By the previousquestion, we have ak+1 ≤ ck − 1. Hence, these coins are not enough to make up therequired A ≥ ck. We can conclude that the fairy god mother, just like the algorithm,must be holding another ck coin.

(b) When the coins are 4, 3, and 1. The problem seems to be that the 4 and the 3 coin are too close.Suppose for each i, each coin denomination is at least twice the previous, i.e. ci ≥ 2ci+1. Onemight then argue that if a c1 coin could be taken that it should be taken, otherwise at least twoc2 needs to be taken instead.

i. If possible, proof that for every set of coins with this property and for every input amountto make up, that the greedy algorithm works. Hint: Follow similar steps as done in the lastquestion.

• Answer: It is not possible. See the next question.

ii. If possible, give a set of coins with this property and an input amount for which the greedyalgorithm does not work. Hint: Try three coin denominations c1, c2, and c3 = 1. Beforechoosing the values for c1 and c2 decide how many of each coins will be in the greedy solutionand how many will be in the optimal solution. Write down some requirements on the valuesc1 and c2 and then solve for them.

• Answer: How silly of me. 2a asks about 25, 10, 1. It is a fine counter example. Maybeyou will find my answer interesting.The greedy algorithm will commit to one of the largest coins c1. Then there is not enoughamount remaining to take a c2. Hence, many 1 coins will be needed to make up the rest.Use n to denote this number. The optimal, needing to be different will not take a c1 butwill take three c2 coins. The reason for three is that we know that c1 is at least twice c2.This gives that Amount = c1 +n = 3c2. Because the greedy did not take c2, we have thatn < c2. Because the greedy solution is worse, we have that 1 + n > 3. Our requirementson the coin denominations are that c1 > 2c2 and that c2 > 2c3 = 2.

35

To get 1 + n > 3, set n = 3. To get n < c2, set c2 = n + 1 = 4, which as needed is morethan twice c3 = 1. Subbing this into c1 + n = 3c2 gives c1 = 9 which as needed is morethan twice c2 = 4. Amount = c1 + n = 3c2 = 12.If the coins are 9, 4, 1, each coin is more than twice the previous and the greedy algorithmdoes not work for the amount 12. The greedy takes 9 + 1 + 1 + 1 while the optimum is4 + 4 + 4.

45. (20 marks) Minimum Spanning Trees

*** The MST on this graph is in text. Changing weights is in text exercise 16.2.3 ExLabel 57 ***

(a) (2 marks) Prove that, in general, an undirected graph may have more than one minimum spanningtree (MST).

• Answer: This graph has 3 MSTs.

1

11

(b) (6 marks) Prove that, if all weights of an undirected graph G are unique, then the MST of thegraph is unique.

• Answer: Let T be an MST of G produced by Kruskal’s algorithm. Suppose there is also adifferent MST T ′. Since T and T ′ have the same number of edges, there must be at least oneedge (u, v) in T ′ but not in T . Consider a cut (U, V ) of G, u ∈ U, v ∈ V such that (u, v) isthe only edge in T ′ that crosses the cut. There must be another edge (x, y) ∈ T, (x, y) /∈ T ′

that also crosses the cut. Note that the procedure followed by Kruskal’s algorithm guaranteesthat (x, y) is a light edge crossing this cut. Augmenting T ′ with this edge (x, y) creates analternate path from u to v in T ′ through (x, y) that forms a cycle with the edge (u, v). Thusa new spanning tree T ∗ can be formed by replacing (u, v) in T ′ with (x, y). Since the weightsare unique, w(x, y) < w(u, v), thus w(T ∗) < w(T ′), which means T ′ could not have been anMST. Thus T must be the unique MST of G.

(c) (4 marks) On the graph below, darken the edges that are part of the MST. (Pencil recommended.)

42 99

7589

855627

97

16

81

61

1253

95

67

64

84

• Answer:

36

42 99

7589

855627

97

16

81

61

1253

95

67

64

84

(d) (2 marks) Suppose we decide to change the weight 56 to any real number from −∞ to +∞. Whatis the interval of values that it could be changed to without making your MST invalid?

• Answer: The weight 56 can be changed to any number in the half-open interval (−∞, 81].

(e) (2 marks) Answer the same question for the edge with weight 85.

• Answer: The weight 85 can be changed to any number in the half-open interval [61,∞).

(f) (4 marks) State the general principal that determines the range over which the weight of an edgecan be changed without invalidating an MST.

• Answer: The weight of an edge (u, v) that is part of an MST can be lowered by any amountwithout invalidating the MST. Deleting (u, v) partitions the graph into two sets of nodes.Currently (u, v) must be the lightest of the edges that crosses this cut. Its weight can beraised as long as this is the case. Hence, it can be raised to equal the weight of the secondlightest such edge. Said another way, it can also be raised to equal the weight of the lightestedge that is not in the MST, but if added to the MST would form a cycle with (u, v) in theMST.Similarly, the weight of an edge (u, v) that is not part of an MST can be raised by any amountwithout invalidating the MST. Add (u, v). This creates a cycle. Currently (u, v) must be theheaviest edge in that cycle. Its weight can be lowered as long as this is the case. Hence, itcan be lowered to equal the weight of the second heaviest such edge. Said another way, it canalso be lowered to equal the weight of the heaviest edge (that is in the MST and would forma cycle with (u, v) in the MST if (u, v) were added to the MST.

46. (MST) Our goal is to find a shortest path from node s to node t in an undirected graph. Our algorithmof choice is to find a minimal spanning tree and then to return the path from s to t in this tree. Givea graph with n nodes for which the ratio between the length of the path returned and the length ofthe optimal path is as bad as possible.

• Answer: The graph has a single edge from s to t with weight 1. This is the optimal path. Thereis also a path with n− 1 edges from s to t where each edge has weight 1− ǫ. The MST will takeall the edges in this path because they are the lightest. It will not take the single edge from s to tbecause it creates a cycle. Hence, the algorithms solution is of weight (1− ǫ) ∗ (n− 1). One couldprove that this is the worst case.

47. Unique Solution/Profiles: Given an undirected graph with weights on the edges, a solution to the MSTproblem is a subset of the edges that forms a spanning tree. Define the profile of a solution to bemulti-set consisting of the weights of the edges in the solution sorted so that we no longer know whichedges they came from. For example, if the solution consists of three edges with weights 2, 7, and 2.

37

Then the profile would be [2, 2, 7]. Prove that every input instance has a unique optimal profile, i.e.every optimal solution has the same profile. Go on to prove that if the instance graphs has a differentweight on each edge, that the graph has a unique MST. Hint: Watch the optimal solution held by thefairy god mother.

• Answer: Consider a computation of the greedy algorithm on some instance graph. We provedthat it returns an optimal solution, which we will denote with optSgreedy . Let Pgreedy be itsprofile. I claim that every optimal solution for this graph has this same profile. To prove thisconsider any optimal solution. Lets call it optSother and its profile Pother. We must prove thatPother = Pgreedy . To do this, we will trace out our proof that the greedy algorithm works. Tobegin we give optSother to the fairy god mother to witness the fact that the loop invariant is truewhen the greedy algorithm first gets to the top of the loop, i.e. that there is an optimal solutionconsistent with every thing committed to so far by the algorithm. We know that optSother isconsistent with this because the algorithm has not committed to any edges yet. Each iteration,the prover instructs the fairy god mother to make a change to her optimal solution so that itremains a valid optimal solution and so that it remains consistent with everything committed toby the greedy algorithm. In the end, she holds the same optimal solution optSgreedy produced bythe greedy algorithm. Now lets pay attention to the profile of the optimal solution held by thefairy god mother. The prover tells the fairy god mother to remove one edge and replace it withanother. We proved that the weight of the added edge cant be larger than that of the removededge. Also it cant be smaller or else what the fairy god mother had before was not optimal.Hence, the weights of these swapped edges must have been the same. This mean that the profilethat she has does not change. If the profile never changes it follows that her first profile Pother isthe same as each of her intermediate profiles PLI which is the same as her final profile Pgreedy .This concludes the proof. If the instance graphs has a different weight on each edge, then theunique optimal profile uniquely specifies the edges in the optimal solution.

48. Greedy Algorithms: You are standing on the bank of a river that you must cross by jumping on steppingstones which are spread in a line across the river. Though you can’t jump far, you want to jump onas few stones as possible. An instance to the problem specifies 〈d0, d1, d2, . . . , dn〉, where d0 = 0 isyour location, di for i ∈ [1, .., n− 1] is a real number giving the distance in meters that the ith stone isfrom you, and dn is the distance of the opposite shore. Assume that 0 = d0 ≤ d1 ≤ d2 ≤ . . . ≤ dn. Asolution is the subset S ⊆ [0, .., n] of the stones that you jump on. Locations 0 and n must be included.The constraint is that if you jump from stone i to stone j then dj − di ≤ 1. The cost of a solution,which is its size |S|, should be minimized.

(a) (3 marks) What action does your greedy algorithm take each iteration? (Imagine that you arecrossing a river on stones. How would you choose which of the stones to jump on? (Or if lost,what actions does a general greedy algorithm take each iteration?)

• Answer: Suppose that i is the last stone jumped on. In a greedy way, jump on the stone jthat is as far from i as you can jump, i.e. the largest j for which dj ≤ di + 1.

(b) (3 marks) What is the loop invariant for your (and any) a greedy algorithm?

• Answer: There exists an opt solution consistent with choices made so far.

(c) (10 marks) Explain how we prove that this loop invariant is maintained. Who constructs what?How is it constructed? What needs to be proven about it? Prove these things. (Again if lostanswer these things for a general greedy algorithm.)

• Answer: The fairy good mother is holding an optimal solution consistent with the choicesmade so far, i.e. she has a set of stone that she jumps on which includes the stones thatalgorithm has jumped on up to and including stone i.The prover tells her how to modify her solution to be with the new choice as well. He saysto let j′ be the next stone after stone i that she jumps on. He tells her to replace j′ with j.This is still consistent with the previous choices made by the algorithm because we onlychanged stones after i. It is also consistent with the new choice.

38

It is still optimal because its cost is the same.It is still valid. Only two jumps have changed. That from i to j and that from j to the nextnode k that is the fairy god mother’s solution. The jump from i to j is valid because thealgorithm does it. Because j is the furthest stone reachable from i and because j′ is reachableform i, it follows that dj′ ≤ dj . The jump from j to k must be valid because it is no longerthan the jump from j′ to k and the fairy god mother does this.The existence of this proves that the loop invariant is still true after going around the loop.

49. When asked to explain how we prove that this loop invariant is maintained, the following answers weregiven. What, if anything is wrong with them.

(a) We prove it is consistent, optimal, and valid.

(b) We must prove that the choices made by the algorithm are consistent, optimal, and valid.

(c) the algorithm’s solution

(d) What the algorithm has done so far is optimal.

(e) If you consider the instance to be the part of our instance that we have seen so far, then we havean optimal solution.

(f) The Prover compares the Fairy Godmother’s optimal solution to what the algorithm has done.

• Answer:

(a) Don’t say ”it” without saying what ”it” is.

(b) We tell the fairy god mother to change her solution OPTLI into OPTours. It is OPTours, thechoices made by the algorithm, that we must prove are consistent with the choices made sofar by the algorithm, valid, i.e. has no conflicts within itself, and optimal, i.e. there is not abetter valid solution out there.

(c) Dont say ”the algorithm’s solution”. The algorithm have made some commitments already,but this does not yet constitute a valid solution.

(d) Dont say ”What the algorithm has done so far is optimal”. ”Optimal” is a property of fullsolutions and we dont have one yet.

(e) The ”More of the Input Loop Invariant” does not work here. Saying that what you haveis optimal within this part of the instance is only a ”local” property. It wont necessarilyguarantee global optimality. Eg. When looking for the best path from s to t, taking the bestedge out of s, may give you the best path to this node u, but this is not part of the best pathto t.

(f) The prover is unable to see the Fairy Godmother’s optimal solution.

50. (20 marks) (See matching DP algorithm) You are robbing a warehouse filled with sealed boxes. Eachbox is specified by 〈wi, ci〉, where wi is how heavy the box is and ci is its capacity in terms of howmuch weight can sit on top of it. You must put all of the boxes on the truck, but the base of the truckis small. Hence, you must stack all the boxes into one tall pile. Your job is to determine in which orderto stack the boxes so that nothing breaks. The rule is that for each box i, the sum of the weights ofthe boxes j above it cannot exceed the amount that can be placed on box i. More formally, if Si is theset of boxes to be stacked on box i, then ∀i, ∑

j∈Siwj ≤ ci. Assume that the truck itself has infinite

capacity. A solution is consists of a permutation of all of the boxes. It is valid (and of value one) ifevery box can be put on the truck in this order with nothing breaking. Otherwise, the solution is notvalid (and has value zero). Your goal is to find a valid solution.

Being a robber, you are greedy. You run into the warehouse and grab the “best” box based on somegreedy criteria. After grabbing the box, you put it on top of the stack you are building.

(a) (5 marks) One obvious algorithm is to grab the box with the largest weight wi, because clearlyyou want the heavy boxes on the bottom. Prove that this greedy algorithm may fail to find avalid solution when one exists.

39

• Answer: Putting the heaviest box on the bottom may be a mistake. Suppose w1 = 3, c1 = 1,w2 = 2, and c2 = 3. For this instance, there is a valid solution (Box 1 on Box 2), but thegreedy algorithm fails, because the heaviest box (Box 1) lacks sufficient capacity for Box 2.

(b) (5 marks) Another obvious algorithm is to grab the box that can hold the most weight ci, becauseclearly you want these on the bottom of the stack. Prove that this greedy algorithm also fails.

• Answer: Putting the largest ci box on the bottom may be a mistake. Suppose that w1 = 1,c1 = 3, w2 = 100, and c2 = 2. For this instance, there is a valid solution (Box 1 on Box 2),but the greedy algorithm fails, because the box with largest capacity (Box 1) lacks sufficientcapacity for Box 2.

(c) (6 marks) A third obvious algorithm is to grab any box that can support the set of boxes thatremain after this box is taken. Note that a box does not have to support itself. Prove that if sucha box exists, then the box with maximum sum wi + ci satisfies this constraint.

• Answer: Let S be the set of boxes remaining. Suppose box i has the largest sum wi + ci.Suppose that box j can support the set of boxes S − j. We prove that it follows that boxi can hold the set of boxes S − i. By the second statement cj ≥

∑k∈S−j wk. Adding wj

to both sides yields cj + wj ≥∑

k∈S wk. Combining this with the first statement gives thatwi + ci ≥ cj + wj ≥

∑k∈S wk. Subtracting wi from both sides yields ci ≥

∑k∈S−i wk as

required.

(d) There may be many boxes that can support the set of boxes that remain after this box is taken.(The last question proves that one such box is the one with largest wi + ci, but there may beothers.) How do we know which of these choices will lead to a overall solution that stack all of theboxes? This third greedy algorithm does not worry about this, but just grabs any one that hasthis property. We will no prove that this greedy algorithm always gives a correct way of stackingthe boxes (assuming one exists).

i. (5 marks) What is the loop invariant for your greedy algorithm?

• Answer: There exists an optimal solution consistent with what the algorithm has doneso far.

ii. (25 marks) Prove that the algorithm maintains this loop invariant. Make sure that you specifyall steps.

• Answer: Assume that there exists an optimal solution consistent with what the algorithmhas done during the first t steps. Let optSLI be one such optimal solution. (You canassume the fairy god mother is holding it.)The algorithm chooses box some box i that can support the set of boxes that remain afterthis box is taken. Perhaps the fairy god mother took box j at this step instead. Thismeans that box i must be higher up in her stack. Tell her to take box i out of the stackand insert it into the t + 1st place, directly below box j. We need to prove that this is aconsistent and valid solution.optSLI was consistent with the first t boxes and we maintain this consistency by notchanging these boxes.It was valid. We have to prove that it is still valid. Nothing changes for the first t boxesand things either stay the same or get better for box j and those above it because if theysupported box i, they no longer do. What remains is to show that box i does not breakfrom this new load. However, be definition box i can support the set of boxes that remainafter i box is taken.

(e) (5 marks) The correctness of this algorithm depends on the fact that it only needs to work if it ispossible to take all of the boxes. Prove that if the goal is to take as many boxes as possible thengreedily taking the box with the largest wi + ci could be a mistake.

• Answer: Suppose that w1 = 4, c1 = 1, w2 = 2, c2 = 2, w3 = 2, and c3 = 2. This criterionwould lead to selecting box 1 first, and only that box could be loaded. Selecting either box 2or 3 first would allow the other to be loaded as well.

40

Chapter: 16: Recursive BacktrackingSection: 16.1: Recursive Backtracking AlgorithmsSection: 16.2: The Steps in Developing a Recursive Backtrack-ingSection: 16.3: Pruning BranchesSection: 16.4: SatisfiabilitySection: 16.5: ScrabbleEnd of Chapter QuestionsChapter: 17: Dynamic Programming Algorithms

Exercise 0.0.3 Shortest Prefix: The input consists of an array A[1..n], with each element from0, 1, 2, 3. The output is a matrix B, such that for each x, y, z ∈ [−n..n], B[x, y, z] = i when A[1..i]is the shortest prefix of A that has x more 1’s than 0’s, y more 2’s than 0’s, and z more 3’s than 0’s.The running time must be O(n). You may assume that memory is initially zeroed.

• Answer: Start with the matrix B[−n..n,−n..n,−n..n] all zero. Scan the array from left to rightmaintaining the number of each of the values in the prefix A[1..i]. Let x be the number more 1’sthan 0’s, y the number more 2’s than 0’s, and z the number more 3’s than 0’s. If B[x, y, z] = 0,set B[x, y, z] = i. The running time must be O(n).

Exercise 0.0.4 Repeat the same, but memory initially has arbitrary values. The trick is to use a datastructure other than an matrix so that it can still be done in time O(n).

• Answer: Store in a set the tuples 〈x, y, z〉 that have such an i. The abstract data types describedin the notes states that this can be done in O(1) time each.

Exercise 0.0.5 Longest Balanced Substring: Again, the input consists of an array A[1..n], with eachelement from 0, 1, 2, 3. The output now is a longest contiguous subarray A[i..j] with equal number of0’s, 1’s, 2’s, and 3’s. The running time must be O(n). Hint: Use the solution to Exercise 0.0.3.

• Answer: Let X be the number more 1’s than 0’s in the entire array A, Y the number more2’s than 0’s, and Z the number more 3’s than 0’s. Scan the array from right to left maintainingthe number of each of the values in the postfix A[j..n]. Let x2 be the number more 1’s than0’s, y2 the number more 2’s than 0’s, and z2 the number more 3’s than 0’s in A[j..n]. If thesubstring A[i+1..j − 1] is the have the same number of each value, then the prefix A[1..i] musthave differences x1 = X−x2, y1 = Y −x2, z1 = Z− z2. B[x1, y1, zi] gives the smallest i for whichthis is case. This gives the longest balanced substring A[i+1..j − 1] for this j. Take the longestoverall values of j.

Exercise 0.0.6 Longest “#0−#1 = 2 mod 5” Block: This time the input consists of an array A[1..n],with each element 0 or 1. The output is a longest contiguous subarray A[i..j] whose number of occur-rences of 0’s minus its number of occurrences of 1’s is 2 mod 5. Design and analyze an incrementalalgorithm to solve this problem in O(n) worst-case time. Hint: A similar technique works as did forExercise 0.0.5.

• Answer: Below we offer two solutions. They are essentially saying the same thing.

Solution 1: We define modality of a 0/1 array X to be the number of 0-bits minus the numberof 1-bits in X modulo 5, and we denote it with M(X). For instance, M([1, 1, 0, 1, 0, 1]) = (2 −4) mod 5 = +3. We will call a 0/1 array with modality 2 feasible. So, we are trying to find thelongest feasible subarray (LFS) of A[1..n].

The algorithm we develop below is an example of an iterative incremental algorithm where we needto strengthen the loop invariant (i.e., the inductive hypothesis), so that we can maintain it from

41

one iteration to the next, and at the end it is strong enough to imply the desired postcondition.As we scan through array A, we not only maintain the longest feasible subarray of the scannedportion, but we also maintain additional information which is given by the auxiliary index-arrayB[0..4] as explained below.

We use the following facts about prefixes of array A:

Fact 1: For any indices 1 ≤ j < i ≤ n we have M(A[j+1..i]) = M(A[1..i])−M(A[1..j]) mod 5.

Corollary 1: Subarray A[j+1..i] is feasible if and only if M(A[1..j]) = M(A[1..i])− 2 mod 5.

Fact 2: Longest feasible subarray of A[1..i] is either the longest feasible subarray of A[1..i − 1]or the longest feasible suffix of A[1..i], whichever is longer.

The following is the algorithm based on the above ideas. IndexRange eventually denotes theindex range of LFS of array A. IndexRange = [null] means A has no feasible subarray.

algorithm LongestFeasibleSubarray(A, n)

〈pre−cond〉: A[1..n] is an array of n 0/1 bits.

〈post−cond〉: A[IndexRange] is the longest feasible subarray of A[1..n]

begin3 for m = 0..4 do B[m]← +∞4 MaxLength← 0; Modality← 0; IndexRange← [null]5 for i = 1..n do6

〈loop−invariant〉:(i) A[IndexRange] = LFS(A[1..i− 1]) of length MaxLength(ii) For each m ∈ 0..4, A[1..B[m]] is the shortest prefix of A with modality m.If such a prefix does not exist, then B[m] = +∞.(iii) Modality = M(A[1..i− 1])

10 if( A[i] = 0 ) then Modality←Modality + 1 mod 5 else Modality←Modality − 1 mod 511 if( B[Modality] = +∞ ) then B[Modality]← i12 j ← B[Modality − 2 mod 5]13 if( i− j > MaxLength ) then14 MaxLength← i− j15 IndexRange← [j+1..i]

end ifend for16 return IndexRangeend algorithm

Proof of Correctness:

〈preCond〉 & codeA ⇒ 〈loop − invariant〉:Clearly LI(i), LI(ii) and LI(iii) hold for i = 1.

〈loop − invariant′〉 & not 〈exitCond′〉 & codeB ⇒ 〈loop − invariant′′〉:Suppose LI(i), (ii), and (iii) hold just before an arbitrary iteration i. We need to show theyhold at the end of that iteration, that is, at the beginning of iteration i+1. LI(iii) and line10 imply that LI(iii) is maintained correctly. Line 11 maintains the validity of LI(ii), sinceextending the prefix A[1..i−1] to A[1..i] can only affect array B at the entry m = M(A[1..i]) =Modality. If B[Modality] is already not +∞, then we should leave it intact; it alreadyindicates the shortest prefix with modality Modality. However, if B[Modality] is +∞, thenclearly A[1..i] is the shortest prefix with modality Modality. Lines 12-15 guaranty LI(i) ismaintained by using Fact 2 and Corollary 1 and LI. Let m = Modality−2 mod 5. Line 12 setsj = B[m]. Corollary 1 and LI(ii-iii) imply that A[j+1..i] is the longest feasible suffix of A[1..i]at lines 13-15. This, LI(i), and Fact 2 imply that the longer of A[IndexRange] and A[j+1..i] isthe longest feasible subarray of A[1..i]. (Note that on line 12, if j = B[Modality−2 mod 5] =+∞, then that and LI(iii)” imply that A[1..i] does not have a feasible suffix. It would also

42

make the if-condition of line 13 fail. This is the reason for leaving the undefined entries of Bto be +∞.)

〈loop − invariant〉 & 〈exitCond〉 & codeC ⇒ 〈postCond〉:We exit the for-loop at the end of iteration i = n. This can be interpreted as the beginningof iteration i = n+1. That condition and LI(i) imply the Postcondition. So, we see that toget to the postcondition we need LI(i), but to maintain LI(i) during the iterations, we needto strengthen LI by the added information provided in LI(ii-iii).

Termination:The algorithm clearly takes O(n) time.

Solution 2: The following is another almost identical solution that has the small advantage thatit is closer to the hint to read section 12.1.1 of [HTA]. As before, we read one item at a time.After reading A[1..i], we maintain not only the longest feasible subarray of the scanned portion,but we also maintain the longest one from amongst those that ends in A[i], because though shortnow, it might be extended to a longest one. However, when we are in the middle of working ona subarray, its modality so far might not be 2. Hence, for each c ∈ [0..4], we let jc be such thatA[jc..i] is the longest subarray ending in A[i] that has modality c. If such a subarray does notexist then jc = nil. Initially i = 0 and j0 = 1, because there is such a subarray of length zero withmodality zero, but for other c, jc = nil. We maintain the loop invariant as follows. If we readA[i+1] = 0, then the longest subarray ending in A[i] that has modality c extends to the longestsubarray ending in A[i+1] that has modality c+1. Hence jc+1 mod 5 = jc. Similarly, if we readA[i+1] = 1, jc−1 mod 5 = jc. If after doing this we have j0 = nil, then we change this to j0 = i+2because A[i+2..i+1], though of length zero is the longest subarray ending in A[i+1] that hasmodality 0. If the longest one A[j2..i] with modality 2 that we are working becomes longer thanour longest, then this new one becomes the longest. In the end, we have the longest.

Section: 17.1: Start by Developing a Recursive BacktrackingSection: 17.2: The Steps in Developing a Dynamic Program-ming AlgorithmSection: 17.3: Subtle Points

Exercise 0.0.7 (See solution in Section ??) Two Friends Per Bird Answer: Now suppose that in thebird and friend’s version of the dynamic programming algorithm, when given an instance of size n,the bird is asked a question with K possible answers and then two friend each are asked to solve asubinstance that is never any bigger than size n

2 . (See ”Chains of Matrix Multiplications”). Againbriefly describe what needs to changed so that the dynamic program also outputs the number of optimalsolutions.

• Answer: Given a bird’s answer k, I ask two friends to solve a subinstance and then take thesetwo solutions to produce my solutions. Let numk,1 be the number of different optimal solutionsthat the first of my friends might give me and numk,2 be the number that the second might give.There are then numk = numk,1×numk,2 ways for me to combine one solution from the first andone solutions from the second to give me an optimal solution consistent with this bird’s answer k.Generally, these are all distinct solutions. Again the number of optimal solutions to my problemis num =

∑k∈k | optCostk=optCost numk.

Exercise 0.0.8 (See solution in Section ??) Now let Num(n) denote the maximum number of optimalsolutions that any dynamic program as described in the last question might output given an instanceof size n. Give the recurrence relation for Num(n). In order to solve it, assume that Num(n) = bn−1

for some constant b and solve for b.

43

• Answer: num =∑

k∈k | optCostk=optCost numk,1 × numk,2. The maximum this can be is

Num(n) =∑

k∈[1..K] Num(

n2

)× Num

(n2

)= K × Num(n

2 )2. Plugging in Num(n) = bn−1

(and Num(

n2

)= b

n2 −1) gives that

Num(n) = K ×Num(

n2

)2

bn−1 = K ×[b

n2 −1

]2

bn−1 = K × bn−2

b = K.Giving that Num(n) = Kn−1. It is interesting that is basically what we had before.

Subsection: 17.3.1: The Question For the Little BirdSubsection: 17.3.2: Subinstances and SubsolutionsSubsection: 17.3.3: The Set of SubinstancesSubsection: 17.3.4: Decreasing Time and SpaceSubsection: 17.3.5: Counting The Number of SolutionsSubsection: 17.3.6: The New CodeEnd of Chapter QuestionsChapter: 18: Examples of Dynamic ProgramsSection: 18.1: The Longest Common Subsequence ProblemSection: 18.2: More of the Input Iterative Loop InvariantPerspectiveSection: 18.3: A Greedy Dynamic Program: The WeightedJob/Activity Scheduling ProblemSection: 18.4: The Solution Viewed as a Tree: Chains of MatrixMultiplicationsSection: 18.5: Generalizing the Problem Solved: Best AVL Tree

Section: 18.6: All Pairs Shortest Paths with Negative CyclesSection: 18.7: All Pairs Using Matrix MultiplicationSection: 18.8: Parsing with Context-Free GrammarsSection: 18.9: More ExamplesEnd of Chapter Questions

51. You are very lucky to have a time machine bring you the value each day of m stocks for the next T days.The input to your problem consists of a table Price(t, s) which gives for each t ∈ [1..T ] and s ∈ [1..m]the price of buying one share of stock s on day t. Buying stocks costs an overhead of 3%. Hence, if

you buy p dollars worth of s on day t, then you can sell them on day t′ for p · (1 − 0.03) · Price(t′,s)Price(t,s) .

You have one dollar on day 1 and must sell all your stock on day T . How should you buy and sell tomaximize your profits?

Because you know exactly what the stocks will do, there is no advantage in owning more than onestock at a time. To make the problem easier, assume that at each point time there is at least onestock not going down and hence at each point in time you alway own exactly one stock. A solutionwill be viewed a list of what you buy and when. More formally, a solution is a sequence of pairs〈ti, si〉, meaning that stock si is bought on day ti and sold on day ti+1. (Here i ≥ 1, t1 = 1 andtlast+1 = T .) For example, the solution 〈〈1, 4〉 , 〈10, 8〉 , 〈19, 2〉〉 means that on day 1 you put your onedollar into the 4th stock, on day 10 you sell all of this stock and buy the 8th stock, on day 19 yousell and buy the 2nd stock, and finally on day T you sell this last stock. The value of this solution is

Πi

[(1− 0.03) · Price(ti+1,si)

Price(ti,si)

]= 1 · (1− 0.03) · Price(10,4)

Price(1,4) · (1− 0.03) · Price(19,8)Price(10,8) · (1− 0.03) · Price(T,2)

Price(19,2) .

Three dynamic programming algorithms occure to me.

44

(a) The most obvious one has mT subinstances (nodes in G) and mT bird answers (degree of nodesin G) given a time of O(m2T 2).

(b) The second algorithm decreases the number of bird answers (degree of nodes in G) to m given atime of O(m2T ).

(c) The third algorithm instead decreases the number of subinstances (nodes in G) to T given a timeof O(mT 2).

Clearly the first is the worst. Whether the second or third is best depends on whether there are morestocks m, or more time steps T . Answers each of the following questions (except for the first) threetimes, one for each of these algorithms.

(a) Suppose at first that there is not the 3% overhead for selling stock. Give an easy algorithm forknowing what and when to buy and sell.

• Answer: Since there is not cost in buying and selling, we might as well sell and buy everyday. Hence, each day t sell the stock you have and buy the stock st that makes the biggest

profit Price(t+1,st)Price(t,st)

between day t and day t + 1.

(b) Read Edmonds’ notes chapter 19.9 which describes how to design a dynamic program via reduc-tions. This has not been covered in either class. Follow the steps in this chapter to reduce thisnew stock problem to the longest paths problem.

i. Given the table of prices Price(t, s), how do you map this to a graph G? A path is a sequenceof nodes which will correspond to a sequence of states that you might be in during your stockbuying. Hence, each node of G will represent one such state. However, you what as few nodesas possible. Hence, you must identify a small set of bench mark states that are of interest.What are these states/nodes? What is the start node s and the finishing node t in yourgraph?

• Answer: In the first and second algorithms, there is a node us,t for each stock s ∈ [1..m]and each day t ∈ [1..T ]. Being in this state means that on day t you own stock s.In the third algorithm, **************************** REWRITE*********************If you pause your buying-selling sequence after you buy a stock, then your state wouldbe determined by the current day ti and the stock si that you own. However, we chooseto pause after you sell the stock. Hence, your state is determined by the current day. Wewill have a node in the graph for each day t ∈ [1..T ]. Standing on node t means that itis day t, you have just sold some stock and you are about buy something else. The startnode s is day 1 and the finishing node t is day T .

• Answer: If you pause your buying-selling sequence after you buy a stock, then your statewould be determined by the current day ti and the stock si that you own. However, wechoose to pause after you sell the stock. Hence, your state is determined by the currentday. We will have a node in the graph for each day t ∈ [1..T ]. Standing on node t meansthat it is day t, you have just sold some stock and you are about buy something else. Thestart node s is day 1 and the finishing node t is day T .

ii. What are the edges between the nodes?

• Answer: The edges out of node t consist of your options. For each stock s ∈ [1..m] andeach day t′ ∈ [t + 1..T ] put an edge from node t to node t′.

iii. What is the length of each edge?

• Answer: The length of this edge will be log[(1 − 0.03) · Price(t′,s)

Price(t,s)

].

iv. Given the longest path from s to t in your graph G, how does this tell you what and whento buy stocks?

• Answer: If your ith edge in the path goes from node t to node t′ and buys and sells stocks then our ith item in our solution will be 〈ti, si〉 = 〈t, s〉.

45

v. The value optCostpath of the path is the sum of the lengths of the edges in it. The value ofyour resulting stock solution is optCoststock as defined above. Show the relationship betweenoptCostpath and optCoststock

• Answer: log [optCoststock] = log[Πi

[(1 − 0.03) · Price(ti+1,si)

Price(ti,si)

]]=

∑i log

[(1− 0.03) · Price(ti+1,si)

Price(ti,si)

]= optCostpath.

(c) Design for a dynamic programming algorithm for this stock buying problem.

i. What are the subinstances? What is problem is each of these subinstances needing to havesolved? How many subinstances are there?

• Answer: For each day t ∈ [1..T ], there is a subinstance that asks what to buy and sellso that on day t you sell your last stock and maximize the amount of money you have onthat day. There are T such subinstances.

ii. Given you are to solve one of these subinstances, what do you ask the bird and what do yougive your friend? How many bird answers are there?

• Answer: If you end on day t′, ask the bird for the last item 〈ti, si〉 in your solution,namely what stock si ∈ [1..m] you buy last and what day ti ∈ [1..t′− 1] you should by it.She answers 〈t, s〉 You ask your friend to solve the subinstance that ends on day t. Thenumber of bird answers is mT .

iii. Give the code

• Answer:

algorithm Stocks (Price)

〈pre−cond〉: An instance consists of the price Price(t, s) of stock s on day t.

〈post−cond〉: optSol is an optimal valid schedule of what to buy on what day.

begin% Table: optSol[t] stores an optimal schedule ending on day t and costSol[t] its cost.table[1..T ] optCost, birdAdvice

% Base Case: The only base case is for the optimal set ending on day t = 1.It’s solution consists of the empty set with value 1.

% optSol[1] = ∅optCost[1] = 1birdAdvice[1] = ∅

% General Cases: Loop over subinstances in the table.for t′ = 2 to T

% Solve instance 〈t′〉.% Try each possible bird answer.for each t ∈ [1..t′ − 1]

for each s ∈ [1..m]% The bird and Friend Alg: I want to finish on day t′. I ask the bird what stock

si ∈ [1..m] I should buy last and what day ti ∈ [1..t′−1] I should by it. She answers〈t, s〉 I ask my friend to solve the subinstance that ends on day t.

% optSol〈t,s〉 = optSol[t] + 〈t, s〉optCost〈t,s〉 = optCost[t] ·

[(1− 0.03) · Price(t′,s)

Price(t,s)

]

end for% Having the best, optSol〈t,s〉, for each bird’s answer 〈t, s〉, we keep the best of these

best.〈tmax, smax〉 = “a 〈t, s〉 that maximizes optCost〈t,s〉”% optSol[t′] = optSol〈tmax,smax〉optCost[t′] = optCost〈tmax,smax〉

46

birdAdvice[t′] = 〈tmax, smax〉end foroptSol = SchedulingWithAdvice (Price, birdAdvice)return 〈optSol, optCost[T ]〉

end algorithm

52. The input to this problem is a pair of strings A = 〈a1, . . . , an〉 and B = 〈b1, . . . , bm〉. A solution isa sequence of operations to convert A into B. The allowed operations are as follows. For a cost of 3you can delete any letter. For a cost of 4 you can insert a letter in any position. For a cost of 5 youcan replace any letter by any other letter. The goal is to find a way to convert A into B which is ascheap as possible. For example, you can convert A = abcabc to B = abacab via the following sequence:abcabc at a cost of 5 can be converted to abaabc, which at cost of 3 can be converted to ababc, whichat cost of 3 can be converted to abac, which at cost of 4 can be converted to abacb, which at cost of 4can be converted to abacab. Thus the total cost for this conversion would be 19. This is almost surelynot the cheapest possible conversion.

(a) As a hint to designing the dynamic programming algorithm, think of the process of conversionfrom A into B as an iterative process that goes from the left to the right. Each step the littlebird will tell you for free which operation to take. What would the loop invariant of this processbe? What possible steps would be taken to make progress while maintaining the loop invariant?

• Answer: The loop invariant of process of converting A into B is that the letters 〈a1, . . . , ai〉have already been converted into 〈b1, . . . , bj〉. Then this conversion is extended by either

– deleting the letter ai+1 giving a conversion from 〈a1, . . . , ai+1〉 into 〈b1, . . . , bj〉.– inserting the letter bj+1 giving a conversion from 〈a1, . . . , ai〉 into 〈b1, . . . , bj+1〉.– changing the letter ai+1 into bj+1 giving a conversion from 〈a1, . . . , ai+1〉 into〈b1, . . . , bj+1〉.

– if ai+1 = bj+1, then this letter can be left unchanged giving a conversion from〈a1, . . . , ai+1〉 into 〈b1, . . . , bj+1〉.

(b) You are trying to find an optimal solution for converting A = 〈a1, . . . , an〉 into B = 〈b1, . . . , bm〉.What small question do you ask the little bird?

• Answer: We ask the bird “Which is that last type of operation in the optimal conversion?”

(c) What subinstance do now ask your friend? How do you construct an optimal solution for yourinstance out of the optimal solution for his subinstance? How do construct your cost from his?

• Answer:

– If the bird answers (k = 1) that the last operation was the deletion of the letter an,then we know that our optimal solution consists of the optimal way of converting from〈a1, . . . , an−1〉 into 〈b1, . . . , bm〉 followed by this last deletion. Hence, we ask our friendthe subinstance of how to optimally convert 〈a1, . . . , an−1〉 into 〈b1, . . . , bm〉. Our solutionis simply his followed by this last deletion. Our cost is his plus 3.

– If the bird answers (k = 2) that the last operation was the insertion of the letter bm,then we know that our optimal solution consists of the optimal way of converting from〈a1, . . . , an〉 into 〈b1, . . . , bm−1〉 followed by this last insertion. Hence, we ask our friendthe subinstance of how to optimally convert 〈a1, . . . , an〉 into 〈b1, . . . , bm−1〉. Our solutionis simply his followed by this last insertion. Our cost is his plus 4.

– If the bird answers (k = 3) that the last operation was the change of the letter an intobm, then we know that our optimal solution consists of the optimal way of convertingfrom 〈a1, . . . , an−1〉 into 〈b1, . . . , bm−1〉 followed by this last change. Hence, we ask ourfriend the subinstance of how to optimally convert 〈a1, . . . , an−1〉 into 〈b1, . . . , bm−1〉. Oursolution is simply his followed by this last conversion. Our cost is his plus 5.

47

– If an = bm, then we can automatically assume that the bird answers (k = 4) that thelast operation was to leave this letter unchanged. In this case, we know that our optimalsolution consists of the optimal way of converting from 〈a1, . . . , an−1〉 into 〈b1, . . . , bm−1〉followed by this last step. Hence, we ask our friend the subinstance of how to optimallyconvert 〈a1, . . . , an−1〉 into 〈b1, . . . , bm−1〉. Our solution is simply his followed by leavingthis last letter alone. Our cost is the same as his.

(d) Starting with some input instance A = 〈a1, . . . , an〉 and B = 〈b1, . . . , bm〉, what is the set ofinstances that might get asked in the recursive back tracking algorithm of some friend’s friend’sfriend?

• Answer: The set of subinstances will be 〈〈a1, . . . , ai〉 , 〈b1, . . . , bj〉〉 | i ≤ n, j ≤ m.(e) What is the table indexed by, what is stored in it, and what order should it be filled in?

• Answer: The table having one entry for each subinstance which can be indexed by i and j.The tables will be cost[0..n, 0..m] and birdAdvice[0..n, 0..m]. Generally, the table is filled inSmallest to largest, but the obvious order of looping for i = 0 to n and j = 0 to m respectsthe dependencies between the instances as well.

(f) Give the code for the dynamic programming algorithm. You can use sentences in some places toreplace some blocks of code. (Do not include the code that constructs the solution.)

• Answer:algorithm Convert (〈〈a1, . . . , an〉 , 〈b1, . . . , bm〉〉)〈pre−cond〉: An instance consists of two strings A and B.

〈post−cond〉: optSol is an optimal way of converting A into B.

begintable[−1..n,−1..m] birdAdvice, cost% Base Cases. (I am being a bit fancy here to save code.)cost[−1,−1] = 0for i = 0 to n

cost[i,−1] =∞end forfor j = 0 to m

cost[−1, j] =∞end fora0 = 0; b0 = 0% Loop over subinstances in the table.for i = 0 to n

for j = 0 to m% Fill in entry 〈i, j〉if ai = bj then

birdAdvice[i, j] = 4 % leave alonecost[i, j] = cost[i− 1, j − 1]

else% Try possible bird answers.% cases k = 1, 2, 3% Get help from friend

cost1 = cost[i− 1, j] + 3cost2 = cost[i, j − 1] + 4cost3 = cost[i− 1, j − 1] + 5

% end cases% Take the best bird answer.kmax = “a k ∈ [1, 2, 3] that maximizes costk”birdAdvice[i, j] = kmax

cost[i, j] = costkmax

48

end ifend for

end foroptSol = ConvertWithAdvice (〈〈a1, . . . , an〉 , 〈b1, . . . , bm〉〉 , birdAdvice)return 〈optSol, cost[n, m]〉

end algorithm

(g) What is the running time to fill in the table?

• Answer: As is usual, the running time is “the number of bird answers” times “the numberof subinstances” which is 3× n2 = Θ(n2).

(h) Given the table storing the cost of the optimal solutions and the table of bird’s answers, how doyou construct the solution for the instance A = 〈a1, . . . , an〉 and B = 〈b1, . . . , bm〉.• Answer: See notes how use the bird’s advice to either iteratively or recursively follow one

path down the recursive back tracking algorithm.

53. Consider an arbitrary dynamic programming algorithm that given an instance returns one of thepossibly many optimal solutions.

(a) Suppose that in the bird and friend’s version of the dynamic programming algorithm, when givenan instance of size n, the bird is asked a question with K possible answers and then one friend isasked to solve a subinstance that is never any bigger than size n− 1. Briefly describe what needsto changed so that in addition to outputting one of the optimal solutions to the given instance,the dynamic program also outputs the number of possible optimal solutions. Your change shouldnot significantly change the running time.

• Answer: If I want to count the number of fruit in a bowl, I can ask one friend to count forme the number of red fruit and another the number of green fruit and another the numberof orange. My answer is the sum of these three. If all the orange fruit is rotten, then I mightnot include the number of orange fruits in my sum. One complication is if there are somefruits that are half red and half green. These might get double counted. I want to make surethat my question “colour” disjointly partitions the fruit according to the answer given.To be sure that we do not double count some optimal solutions we need that the set ofsolutions consistent with one bird answer is disjoint from those consistent with another birdanswer. For example, if the bird’s answer tells us the last edge of the optimal path, thenthose paths ending in the first edge are clearly different than the paths ending in the second.On the other hand, for the longest common subsequence problem, the bird’s answers were“the last letter of X is excluded” and “the last letter of Y is excluded”. There are solutionsfor which both are true. This causes a problem when counting solutions. Hence, we needto change the dynamic programming algorithm so that each solution only has one valid birdanswer. For example, with the longest common subsequence problem, we could change thebird’s answers to being either “the last letter of X is excluded” or “the last letter of X isincluded”.In the new dynamic programming algorithm, each friend, in addition to the cost of the optimalsolution for his subinstance and the bird’s advice, stores the number of optimal solutions forthe subinstance. This is computed as follow. Consider me to be one of these friends with oneof these SubInstances. I try each of the K possible bird’s answers. When trying k, I find thebest solution to my instance from amongst its solutions that are consistent with this bird’sanswer k. I do this by asking a friend of mine to solve some subinstance. This friend tells methe number of optimal solutions to this subinstance. Let numk be this number. Generally,there is a one-to-one mapping between my friend’s optimal solutions and solutions to myinstance that are consistent with this bird’s answer k. Hence, I know that there are numk ofthese. As before let optCostk be the cost of the best solution to my instance from amongstits solutions that are consistent with this bird’s answer k. I compute the cost of my optimalsolution simply by optCost = maxk∈[K] optCostk. There may be a number of bird’s answers

49

that lead to this same optimal cost optCost. Each of these lead to optimal solutions. Wewere careful that the set of solutions consistent with one bird answer is disjoint from thatconsistent with another bird answer. Hence, the total number of optimal solutions to myinstance is num =

∑k∈k | optCostk=optCost numk.

(b) Let Num(n) denote the maximum number of possible optimal solutions that any dynamic programas described above might output given an instance of size n. Give and solve the recurrence relationfor Num(n).

• Answer: num =∑

k∈k | optCostk=optCost numk. The maximum this can be is Num(n) =∑k∈[1..K] Num(n− 1) = K ×Num(n− 1) = Kn.

(c) Suppose that there were this maximum number of solutions. What is the minimal number of bitsneeded to describe each with a unique string? How would you describe a solution as a sequenceof bird’s answers? How does the length of this description compare to the minimum length?

• Answer: If you have N objects, one way to distinguish them is to allocate to each a numberfrom one to N . This description requires log2(N) bits. It can be proved that no descriptioncan be shorter than this. Given that there may be N = Kn solutions, they need a descriptionof length at least log2(N) = log2 (Kn) = log2(K) × n. One way of identifying a solution isby listing the sequence of answers the bird provides in a 20-questions fashion to you, to yourfriend, to his friend, and so on. Given that there are N ′ = K different bird answers, theyneed a description of length at least log2(N

′) = log2(K) = log2(K). If each subinstance isone shorter than the instance, then they will have lengths n, n − 1, n − 2, . . . , 1 for a totalof n questions to the little bird. Therefore, it takes log2(K)× n to list all the birds answers〈k1, . . . , kn〉.

(d) Why can the program in polynomial time output the number of optimal solutions but not be ableto output all of these optimal solutions?

• Answer: There may be Kn of them. Hence, it takes exponential time to output them all.However, the number of bits to represent this number is only log2 (Kn) = log2(K)×n = Θ(n).Hence, this number can be outputted.

(e) Now suppose that in the bird and friend’s version of the dynamic programming algorithm, whengiven an instance of size n, the bird is asked a question with K possible answers and then twofriend each are asked to solve a subinstance that is never any bigger than size n

2 . (See ”Chainsof Matrix Multiplications”). Again briefly describe what needs to changed so that the dynamicprogram also outputs the number of optimal solutions.

• Answer: Given a bird’s answer k, I ask two friends to solve a subinstance and then takethese two solutions to produce my solutions. Let numk,1 be the number of different optimalsolutions that the first of my friends might give me and numk,2 be the number that the secondmight give. There are then numk = numk,1 × numk,2 ways for me to combine one solutionfrom the first and one solutions from the second to give me an optimal solution consistentwith this bird’s answer k. Generally, these are all distinct solutions. Again the number ofoptimal solutions to my problem is num =

∑k∈k | optCostk=optCost numk.

(f) Now let Num(n) denote the maximum number of optimal solutions that any dynamic programas described in the last question might output given an instance of size n. Give the recurrencerelation for Num(n). In order to solve it, assume that Num(n) = bn−1 for some constant b andsolve for b.

• Answer: num =∑

k∈k | optCostk=optCost numk,1 × numk,2. The maximum this can be is

Num(n) =∑

k∈[1..K] Num(

n2

)×Num

(n2

)= K ×Num(n

2 )2. Plugging in Num(n) = bn−1

(and Num(

n2

)= b

n2 −1) gives that

Num(n) = K ×Num(

n2

)2

bn−1 = K ×[b

n2 −1

]2

50

bn−1 = K × bn−2

b = K.Giving that Num(n) = Kn−1. It is interesting that is basically what we had before.

54. (22 marks) Dynamic Programming:

Consider an encoding of characters into binary strings in which differ-ent characters may be encoded into strings of different length. We maydescribe such an encoding by a table with two columns, where each char-acter on the left is encoded by the bit string given on the right. See theexample on the right.

Example Decoding TableCharacter Encoding

A 0B 10C 01D 11

Using this table the encoding of ACDC is 0011101 (which comes from 0 01 11 01 , but the decoderis not given the separation between the code-words). Notice that there may potentially be severaldecodings of a given bit string. For instance, 001110 can be decoded either as AADB or as ACDA.Your task is to devise an efficient algorithm which, given an encoded string 〈a1, . . . , an〉 ∈ 0, 1nand a decoding table, decodes the string into one of its possible string of characters. For example, if〈a1, . . . , an〉 = 001110 and the DecodingTable is as above, then optSol is either AADB or ACDA.

(a) (2 marks) Give all ways to decode 0100.

• Answer: ABA = 0 10 0, and CAA = 01 0 0.

(b) (2 marks) We say that the question asked of the little bird must be ”small”. What is the definitionhere of ”small”? Give an example of a big question. Give an example of a big question.

• Answer: The number of different answers that she might give. “What is the optimal solu-tion?” is big. “What is the first edge?” is small.

(c) (2 marks) When we ask a (recursive) friend a question, what are the requirements on this question.

• Answer: Meets the preconditions to the same problem and is smaller.

(d) (5 marks) Suppose that you have a trusting bird to whom you can ask a small question and youhave a trusting friend to whom you can ask a recursive question. Give a fast algorithm for thisproblem. How do you construct your solution?

• Answer: The algorithm is almost identical to Pretty Print. The question to the bird withPP is “How many words do I put on the last line?” With Decoding, it is “How many bits ofthe input do I use in the code for the last letter?” With an answer k, the you take the last kbits of your input and look them up in the decoding table. If no letter corresponds to them,then you tell the bird that she is wrong. If say letter C corresponds to them, then deletethese k bits from your input and give the rest to your friend. He decodes this. Our solutionis the friend’s decoded message concatenated with the bird’s letter C.

(e) (2 marks) Do we have a bird that we can trust? If so why can we trust her? If not how do wechange the algorithm to deal with this? What are we doing each step? How do we conclude?

• Answer: We cannot trust the bird. Hence, we try each bird answer. For each, we find thebest solution consistent with this bird’s answer. Then we take the best of these best.

(f) (2 marks) Do we have a friend that we can trust? If so why can we trust him? If not how do wechange the algorithm to deal with this?

(g) (2 marks) Why can you trust the friend to give the correct answer?

• Answer: He is a version of ourselves, that can be given a smaller instance of the sameproblem. We can trust him because strong induction tells us that we can.

(h) (2 marks) Starting with some input instance 〈a1, a2, . . . , an〉, what is the set of instances thatmight get asked in the recursive back tracking algorithm of some friend’s friend’s friend . . .?

• Answer: S = 〈a1, a2, . . . , ai〉 | i ∈ [1..n].

51

(i) (3 marks) In a general dynamic program, what is the table indexed by? In a fast dynamic program,what does it store? Why do we not store the obvious thing?

• Answer: Indexed by the subinstances. Stores the bird’s advice and the cost of the optimalsolution. We do not store the entire optimal solution because it likely has size Θ(n) andpassing it from friend to friend can add an extra factor of Θ(n) to the running time.

(j) (2 marks) Does a dynamic program recurse? Does it have ”friends”? Explain.

• Answer: It has a friend for each subinstance, but it does not recurse. When we ask a friend,he looks his answer up in the table that he had previously computed.

(k) (6 marks) Give the code for the dynamic programming algorithm. You can use sentences in someplaces to replace some blocks of code. (Do not include the code that constructs the solution.)

algorithm Decode (〈a1, . . . , an〉 , DecodingTable)

〈pre−cond〉: An instance consists of a coded message ~a and a DecodingTable.

〈post−cond〉: optSol is a decoded messages.For example, if 〈a1, . . . , an〉 = 001110 and the DecodingTable is as above, then optSol iseither AADB or ACDA.The program also returns the ”cost” of the solution, which is 1 if it is a valid decodingand 0 if it is not.

begin

• Answer:

table[0..n] birdAdvice, cost% Base Cases.birdAdvice[0] = ∅cost[0] = 1% Loop over subinstances in the table.for i = 1 to n

% Fill in entry 〈i〉 for 〈a1, a2, . . . , ai〉.% Try possible bird answers, k = number of bits in the last letter.for k = 1 to K K is the length of the longest code for a letter

if( 〈ai−k+1, ai−k+2, . . . , ai〉 is the code for a letter C ) then% Get help from friend% subSolk = subSol[i−mk] + Ccostk = cost[i−mk]% i.e. ours is valid if our friend’s is valid.

elsecostk = 0 % Bird was wrong.

end ifend for% Take the best bird answer.kmax = “a k that maximizes costk”birdAdvice[i] = kmax

% subSol[i] = subSolkmax

cost[i] = costkmax

end foroptSol = DecodeWithAdvice (〈a1, . . . , an〉 , code, birdAdvice)return 〈optSol, cost[n]〉

end algorithm

(l) (2 marks) What is the running time to fill in the table?

• Answer: As is usual, the running time is “the number of bird answers” times “the numberof subinstances” which is Θ(D × n).

52

55. Consider the recurrence relation T (0) = T (1) = 2 and for n > 1 T (n) =∑n−1

i=1 T (i) × T (i − 1). Weconsider the problem of computing T(n) from n.

(a) Show that if you implement this recursion directly in say the C programming language, that theprogram would use exponentially, in n many arithmetic operations.

• Answer: I have (n − 1) × 2 friends. Most of them have Θ(n) friends. Most of them haveΘ(n) friends. Hence, the total number of friends will be nΘ(n).

(b) Explain how to use dynamic programming that only uses O(n2) arithmetic operations for thisproblem.

• Answer: There are only n subinstances to solve, namely T (n′) for n′ ∈ [0, n]. A dynamicprogramming solves these in the order n′ = 0..n so that nobody has to wait and stores thesolutions in a table so that they do not need to be recomputed. The straight code then loopsthrough each T (n′) and for each loops through i ∈ [1, n′ − 1] for a total of Θ(n2) time.

(c) Give an algorithm for this problem that only uses O(n) arithmetic operations.

• Answer: T (n) = T (n−1)×T (n−2)+∑n−2

i=1 T (i)×T (i−1) = T (n−1)×T (n−2)+T (n−1) =T (n− 1)× (T (n− 2) + 1). Computing each T (n′) using this algorithm from the solutions inthe table requires only constant time per n′.

(d) Which of these approximates T (n) the closest?

nΘ(n) 2Θ(n) 22Θ(n)

• Answer: As said, T (n) ≈ T (n− 1)2. Unwinding this gives,

T (n) ≈[T (n− 2)2

]2 ≈ T (n− 2)(22) ≈

[T (n− 3]2

](22) ≈ T (n − 3)(23) ≈ ... ≈ T (n− i)(2

i) ≈T (0)(2

n) ≈ 22Θ(n)

56. Dynamic Programming: (See greedy version of this.) The box question in the last assignment questionproved that the three greedy algorithms given do not correctly choose which boxes to take. Thismotivates trying dynamic programming. We will generalize the problem further by having values onthe boxes.

You are robbing a warehouse filled with sealed boxes Bi for i ∈ [n]. Each box is specified by 〈wi, ci, vi〉,where wi is how heavy the box is, ci is its capacity in terms of how much weight can sit on top of it,and vi is how much money the box is worth. You are able to put one stack of boxes on the truck.A solution specifies the set S of boxes to take and the order to take them. A solution is valid if∀i ∈ S,

∑〈j above i〉 wj ≤ ci. The value of a solution is

∑i∈S vi. The goal is to maximize this value.

Hint 1: You might want to change the precondition so that an instance also provides a value c whichis the maximum weight that can go onto the truck.

Hint 2: As motivated by the last question, you might as well assume that all the boxes are orderedby wi + ci. Then for any subset S that you take, they are already in the correct order.

Hint 3: Instead of “asking the little bird” about the last box, ask her whether the first box, i.e. theone with the largest wi + ci, should be included in the solution.

Design a dynamic programming algorithm for this box stacking problem.

(a) For each bird answer, what subinstance do you ask the friend? How do you use this for youranswer?

• Answer: Suppose that our instance specifies a capacity c of the truck and a set of remainingboxes that could go on this truck. We sort these remaining boxes by wi + ci and ask the birdwhether the first box in this sequence should go on the truck. If she says take this first boxBi, then remove it from the set of boxes and put it on the truck. All the other boxes stayunchanged. What changes is the capacity c of the truck. If wi > c, then the bird was wrong.

53

Otherwise, the truck now has remaining capacity c − wi and box bi, which will be beneathany remaining boxes, has capacity ci. Hence, the effective capacity is the minimum of thesetwo. The instance that we give our friend is that with truck capacity min(c−wi, ci) and thesame boxes except for this first one removed. Our answer is the same plus adding box Bi andits value vi back in. If she says no, simply delete box Bi. Our answer is the same.The recurrence relation is as follows. If wi ≤ c, then

Cost[c, i] = Max(Cost[min(c− wi, ci), i + 1] + vi, Cost[c, i + 1])

If wi > c, then

Cost[c, i] = Cost[c, i + 1]

(b) What are the subinstances? What is the problem that must be solved for each of these subin-stances? Which subinstance is our original instance?

• Answer: For each i ∈ [n] and each c ∈ [cmax], the subinstance 〈c, i〉 asks which of the boxesBi, Bi+1, . . . , Bn to take on a truck that can hold weight c. Here cmax =

∑i wi. Our instance

is 〈n, cmax〉 because we can consider all n boxes and the truck need not hold more than theweight of all the boxes.

(c) Provide the pseudo code for your algorithm.

• Answer:

algorithm StackBoxes (〈w1, c1, v1〉 , . . . , 〈wn, cn, vn〉)〈pre−cond〉: n boxes. wi is how heavy the box is, ci is its capacity in terms of how much

weight can sit on top of it, and vi is how much money the box is worth.

〈post−cond〉: optSol is an optimal way to stack some of the boxes so as to maximize thetotal value of the boxes taken. optCost is its price.

begincmax =

∑i wi

Sort boxes by wi + ci

% Table: optSol[c, i] would store an optimal solution given a truck with capacity c and whenconsidering only the last i boxes,but actually we store only the bird’s advice for the subinstance and the cost of its solution.

table[0..cmax, 1..n + 1] birdAdvice, cost

% Base Cases: The base cases are when the number of boxes is zero.For each, the solution is the empty truck with cost zero.

loop c = 0..cmax

% optSol[c, n + 1] = ∅cost[c, n + 1] = 0birdAdvice[c, n + 1] = ∅

end loop

% General Cases: Loop over subinstances in the table.loop i = n to 1

loop c = 0 to cmax

% Solve instance 〈c, i〉 and fill in the table.% The bird and Friend Alg: The bird tells us either (1) exclude the ith item from

the truck or (2) include it. Either way, we remove this last object, but in case (2)we decrease the capacity of the truck to min(ci, c−wi). Then we ask the friendfor an optimal packing of the resulting subinstance. He gives us (1) optSol[c, i+1]or (2) optSol[min(ci, c−wi), i + 1] which he had stored in the table. If the birdhad said we were to include the ith item, then we add this item to the friend’ssolution. This gives us optSolk which is a best stacking for 〈c, i〉 from amongstthose consistent with the bird’s answer.

54

% Try each possible bird answers.% cases k = 1, 2 where 1=exclude 2=include

% optSol1 = optSol[c, i + 1]cost1 = cost[c, i + 1]if(c− wi ≥ 0) then

% optSol2 = optSol[min(ci, c− wi), i + 1] ∪ icost2 = cost[min(ci, c− wi), i + 1] + vi

else% Bird was wrong% optSol2 =?cost2 = −∞

end if% end cases% Having the best, optSolk, for each bird’s answer k,

we keep the best of these best.kmax = “a k that maximizes costk”% optSol[c, i] = optSolkmax

cost[c, i] = costkmax

birdAdvice[c, i] = kmax

end forend foroptSol = StackBoxesWithAdvice (〈w1, c1, v1〉 , . . . , 〈wn, cn, vn〉 , birdAdvice)return 〈optSol, cost[cmax, 1]〉

end algorithm

(d) What is its running time?

• Answer: # subinstances × # bird answers = n · cmax × 2.

(e) What time complexity of this algorithm?

• Answer: Exponential, because cmax may be exponential in the number of bits to encode theinstance.

57. (30 marks) Dynamic Programming

You are given a sequence s of characters and your goal is to find a longest subsequence p of s thatis a palindrome. A palindrome is a string which is the same as its reverse. For example, if s =PROFESSOR, the optimal solution is p = ROSSOR, because ROSSOR is the longest string that isboth a subsequence of PROFESSOR (PROFESSOR), and a palindrome (the reverse of p is p itself).Your goal is to write a dynamic programming algorithm to solve this problem. (Caution: do not tryto use the LCS algorithm on s and its reverse. It does not work.)

In the following, we will let si denote the ith character of s, and sij denote the (consecutive) substringof s starting at character i and ending at character j, including all characters in between. For example,if s = PROFESSOR, then s5 =E and s4,7 = FESS.

(a) If (the bird) were to tell you that the last character of s is not in the optimal solution p, howwould you use this information to solve the problem.

• Answer: I would give my friend s with this last character removed. My answer would be thesame as his.

(b) If (the bird) were to tell you that both the first and the last characters of s are in the optimalsolution p, how would you use this information to solve the problem.

• Answer: If these characters are different, then I would tell her that she was wrong. If theyare the same, I would give my friend s with both of them removed. My answer would be thesame as his with the first and last added back.

(c) What is a third answer that (the bird) might give so that all possiblities are covered?

55

• Answer: The first character of s is not in the optimal solution p.

(d) (6 marks) Consider the first and last characters of the input string s, and whether they are partof a longest palindromic subsequence p of s. What are the three possibilities?

• Answer:

i. s1 = sn and both are part of a longest palindromic subsequence of s.

ii. s1 6= sn, and there is a longest palindromic subsequence of s that does not include s1.

iii. s1 6= sn, and there is a longest palindromic subsequence of s that does not include sn.

(e) (3 marks) How would you describe the set of subinstances that must be solved in order to solvethe original instance? If the original instance is of length n, how many of these subinstances arethere?

• Answer: The set of sij , 1 ≤ i ≤ j ≤ n. There are

n∑

i=1

(n− i+1) =

n∑

i=1

i =1

2n(n+1) of these.

(f) (8 marks) Let A(i, j) be the length of a longest palindromic subsequence of sij . Use the threepossibilities identified in Part (a) to define a recurrence relation for A(i, j). Be sure to includethe base cases.

• Answer:

A(i, j) =

0 i > j

1 i = j

2 + A(i + 1, j − 1) i < j and si = sj

maxA(i + 1, j), A(i, j − 1) otherwise

(g) (2 marks) In what order must the table for A(i, j) be filled so that the answers to subinstancesare always available when needed?

• Answer: Only elements on or above the main diagonal must be filled. These must be enteredso that elements below and to the left of the current element are available when the currentelement is processed. One valid method is to fill the table diagonally, starting with j = i, andprogressing through j = i + 1, j = i + 2...j = i + n − 1. Another method, slightly easier tocode, is to fill the upper triangle row-wise, starting from the bottom row.

(h) (4 marks) Fill in the table A(i, j) for the input instance s =PROFESSOR. (Pencil recommended.)

i\j P R O F E S S O R

PROFESSOR

• Answer:

56

i\j P R O F E S S O RP 1 1 1 1 1 1 2 4 6R 0 1 1 1 1 1 2 4 6O 0 0 1 1 1 1 2 4 4F 0 0 0 1 1 1 2 2 2E 0 0 0 0 1 1 2 2 2S 0 0 0 0 0 1 2 2 2S 0 0 0 0 0 0 1 1 1O 0 0 0 0 0 0 0 1 1R 0 0 0 0 0 0 0 0 1

(i) Provide the pseudocode for a dynamic programming algorithm that fills in table A(i, j).

algorithm Palindrome(s)

〈pre−cond〉: s is a string of n characters

〈post−cond〉: A(i, j) is a table of the length of the longest palindromic subsequence of sij .

beginfor i = 2 to n

A(i, i− 1) = 0endfor i = 1 to n

A(i, i) = 1endfor k = 1 to n− 1

for i = 1 to n− kj=i+kif si = sj

then A(i, j) = A(i + 1, j − 1) + 2else A(i, j) = max(A(i + 1, j), A(i, j − 1))

endend

end algorithm

(j) What is the running time of your algorithm?

• Answer: Θ(n2)

(k) (7 marks) Provide the pseudocode for a recursive algorithm that recovers an actual longest palin-dromic subsequence of an input string s, given s and the filled-in table A(i, j) for s.

• Answer: To print out one possible longest palindrome given A we call PrintOpt(A, 1, n),where PrintOpt is:

PrintOpt(A, i, j)if i > j then return end ifif i = j then

print si

returnend ifif si = sj then

print si

PrintOpt(i+1,j-1)print sj

else if A(i + 1, j) ≥ A(i, j − 1) thenPrintOpt(i+1,j)

elsePrintOpt(i,j-1)

end if

57

58. Dynamic programmingThe SubsetSum problem is defined as follows: given positive integers a1..an, and a positive integer t,find a subset S of (1 . . . n) such that Σi∈Sai = t, if it exists.

(a) Design a dynamic programming algorithm to solve the SubsetSum decision problem. In otherwords, your algorithm will report whether a solution exists, but not what it is. You may use thesteps and language set out by James, those set out by Jeff, or some combination. Explain in pointform English the key table (array) used, the recurrence relation used to fill it in (as in recursivebacktracking, for those in Jeff’s class), and the order in which it is filled. Also provide concisepseudocode for your algorithm.

• Answer:

Array:A[i, τ ] is a 2D array over i ∈ [0, ..., n], τ ∈ [0, ..., t], specifying the maximum

∑j∈S aj such

that∑

j∈S aj ≤ τ and j ≤ i∀j ∈ S. An S such that Σi∈Sai = t exists iff A[n, t] = t.

Recurrence:A[0, τ ] = A[i, 0] = 0∀τ ∈ [0, ..., t], i ∈ [0, ..., n].

A[i, τ ] = A[i− 1, τ ] if ai > τ= maxA[i− 1, τ ], ai + A[i− 1, τ − ai] if ai ≤ τ

Pseudocode:

A[0, τ ] = A[i, 0] = 0 ∀τ ∈ [0, ..., t], i ∈ [0, ..., n]for i = 0 : n

for τ = 0 : tif ai ≤ τ

A[i, τ ] = maxA[i− 1, τ ], ai + A[i− 1, τ − ai]else

A[i, τ ] = A[i− 1, τ ]end

endend

(b) Which problem studied in class is SubsetSum the most similar to? How?

• Answer: Knapsack: the ai map to the weights of the items, t maps to the capacity of theknapsack. Instead of maximizing value, here our goal is to get as close to t as possible, whichcorresponds to filling the knapsack as close to full as possible.

(c) In one or two sentences explain why the main dynamic programming algorithm does not constructthe actual solution and how the solution is constructed at the end.

• Answer: The dynamic programming algorithm constructs a table of sums. The value inA(n, t) tells us whether a sum totalling t exists, but it does not directly tell us what combi-nation of the ai total that sum.Just as for the knapsack problem, we can traverse the table to recover the subset. We startby scanning the last column of the table from the (n, t) entry. A change from entry (i, t) toentry (i − 1, t) indicates that ai is included in the solution, and ai is thus added to S. Wethen shift our scan point left in the table by ai and resume our scan from (i− 1, t− ai). Thisprocess iterates until we hit a 0 entry.

– Answer:

PrintOpt(i, t)if i = 0 return

58

if A[i, t] = A[i− 1, t]Printopt (i− 1, t)

ElsePrintopt(i− 1, t− ai)Print ai

(d) Running Time:

i. What is the running time of the main dynamic programming algorithm as a function of nand t? Why?

• Answer: Initialization: Θ(n) + Θ(t).Construction of A : Θ(nt).Total Running Time: Θ(nt).

ii. What is the running time of the main dynamic programming algorithm as a function of thesize of the input? Why?

• Answer:Input size is proportional to n+ log t. Thus running time is exponential in the input size.

iii. What is the running time of the algorithm that constructs the solution as a function of n andt? Why?

• Answer: Recovery of S : Θ(n), since we traverse A one row at a time, from row n to (atworst) row 0.

Chapter: 19: Reductions and NP-CompletenessSection: 19.1: Satisfiability Is At Least As Hard As Any Opti-mization ProblemSection: 19.2: Steps to Prove NP-CompletenessSection: 19.3: Example: 3-Colouring is NP-CompleteSection: 19.4: An Algorithm for Bipartite Matching using theNetwork Flow Algorithm

59. (15 marks) Reductions: The optimization problem Palg is specified by defining its instances Ialg, itssolutions Salg for each such instance, and the cost cost(Salg) of each such solution. Given an instanceIalg the goal is to find an optimal solution Salg. Similarly, the goal of the optimization problem Poracle,given an instance Ioracle, is to find an optimal solution Soracle. In a reduction, you design a fastalgorithm Algalg for Palg using an assumed fast algorithm Algoracle for Poracle. In the three mainreductions done this term, the problem pairs [Palg to Poracle] were [Boy & Girls Marriage to NetworkFlows], [Software to Min Cut], and [Circuit Satisfiability to 3-Colouring].

(a) (4 marks) Outline the basic code for Algalg solving Palg using the supposed algorithm Algoracle

supposedly solving Poracle as a subroutine.

• Answer:

algorithm Algalg(Ialg)

〈pre−cond〉: Ialg is an instance of Palg.

〈post−cond〉: Determine whether Ialg has a solution Salg and if so returns it.

beginIoracle = InstanceMap(Ialg)〈ansoracle, Soracle〉 = Algoracle(Ioracle)if( ansoracle = Y es) then

ansalg = Y esSalg = SolutionMap(Soracle)

else

59

ansalg = NoSalg = nil

end ifreturn(〈ansalg, Salg〉)

end algorithm

(b) (3 marks) What steps do you have to take to prove that this reduction is correct?

• Answer: We have to prove that Algalg works if Algoracle works. For this we prove

i. Given an instance Ialg, InstanceMap(Ialg) maps this to a valid instance Ioracle.

ii. If Soracle is a solution for Ioracle than Salg = SolutionMap(Soracle) is a solution for Ialg

whose cost is just as good.

iii. If Salg is a solution for Ialg than Soracle = ReverseSolutionMap(Salg) is a solution forIoracle whose cost is just as good.

(c) (4 marks) Give two purposes of reductions.

i.

ii.

• Answer:

i. Designing new algorithms

ii. Arguing that a problem is hard or easy.

iii. identifying equivalence classes of problems.

(d) (2 marks) What does it mean for a problem to be in NP (Non-Deterministic Polynomial Time)?

• Answer: Given an instance and a solution, it is easy to test its validity.

(e) (2 marks) What does it mean for a problem to be NP-Complete.

• Answer: It is as hard as any problem in NP. (and is in NP)

60. Reductions

(a) There are two opposite purposes for reductions.As an example of the first purpose, we designed an agorithm for the Elephant problem (ProblemA) using an algorithm for the Longest Weighted Path Problem (Problem B).As an example of the second purpose, we designed an algorithm for the Circuit Sat Problem(Problem A) using an algorithm for the 3-Colouring Problem (Problem B).State and contrast these two purposes. Be sure to be clear about what this reduction says aboutProblem A and about Problem B.

• Answer: The first provided a new algorithm for problem A using a known algorithm forproblem B. The second argued that there is likely not a poly-time algorithm problem Bbecause there is likely not one for problem A.

(b) What are the formal requirements for a problem to be NP-Complete?

• Answer: Given an instance and a potential solution, there is a poly time algorithm thatchecks if it is a valid solution

61. (8 marks) Reductions and NP-Completeness

(a) (2 marks) In one sentence, state what a reduction is.

• Answer: Using a supposed algorithm for one problem as a subroutine for an algorithm foranother problem.

(b) (2 marks) Give two purposes of reductions.

i.

ii.

60

• Answer:

i. Designing new algorithms

ii. Arguing that a problem is hard or easy.

iii. identifying equivalence classes of problems.

(c) (2 marks) What does it mean for a problem to be in NP (Non-Deterministic Polynomial Time)?

• Answer: Given an instance and a solution, it is easy to test its validity.

(d) (2 marks) What does it mean for a problem to be NP-Complete.

• Answer: It is as hard as any problem in NP. (and is in NP)

62. Greenhouses and plantsA botanical garden is planning to build n greenhouses, each of which is intended to represent a differentclimate zone. They have a list of m kinds of plants that they would like to grow there. Not every kindof plant can grow in every greenhouse, though usually one kind can grow in more than one climate; soassociated with every greenhouse i ∈ [1 . . . n] is a list Ci of plant varieties it can grow. In addition tothat, every greenhouse can host no more than 20 varieties of plants.

(a) Given the n, m and Ci for 1 ≤ i ≤ n, specify an algorithm to determine the maximal number ofdifferent varieties of plants that can be grown in these greenhouses (total over all greenhouses).Hint: design a flow network that represents this problem.

• Answer: This problem is similar to a bipartite matching problem. We need to construct a flownetwork representing a “matching” of plants to greenhouses. The flow network F = (G, s, t, c)for this problem will consist of the following:– The vertices of G will be s, t for source and sink, vertices

g1, . . . , gn representing greenhouses and vertices v1, . . . , vm

representing varieties of plants.

– Edges (s, gi) of capacity 20 for 1 ≤ i ≤ n. This will con-strain every greenhouse to have at most 20 varieties ofplants.

Varieties

ts 20

111

1

1

1

11

1

1 1

111

Greenhouses

20

20

v

3v

1v

2v

2g

1g

4

mvng

– Edges (gi, vj) of capacity 1, 1 ≤ i ≤ n, 1 ≤ j ≤ m for pairs (i, j) such that j ∈ Ci. Theseedges will match varieties of plants with greenhouses where they can grow. Since we wishto maximize the number of varieties of plants, we will not allow more than one plant ofthe same variety to grow in a greenhouse.

– Edges (vj , t) of capacity 1, where 1 ≤ j ≤ m. Restricting the capacity to 1 ensures thatevery variety will be grown in at most one greenhouse.

To find the maximal number of varieties of plants that can be grown in these greenhouses,we run the Ford-Fulkerson algorithm on this network. The value of the resulting flow is themaximal number of varieties that can be grown. Every greenhouse hosts no more than 20varieties of plants due to the capacity constraint on (s, gi) edges, and these varieties can begrown there because of the choice of (gi, vj) edges; therefore, this set of varieties can feasibly

61

be grown. To show that it is maximal we appeal to the fact that Ford-Fulkerson alwaysproduces the maximal flow. Suppose that there is a better assignment of varieties to plants,allowing for more varieties to be grown. We argue that every pair (i, j) where variety j isassigned to be grown in the greenhouse i corresponds to an augmenting path in the network;the path (s, gi, vj , t). Since no variety is chosen more than once, and no greenhouse hosts morethan 20 varieties, each pair (i, j) gives rise to an augmenting path satisfying the constraints.Therefore, there would be a flow on the network with value greater than the value of the flowproduced by Ford-Fulkerson; a contradiction. Thus, the value of the flow on this network isequal to the maximum number of varieties of plants that can be grown.

62

(b) How can you produce a list of chosen varieties for each greenhouse given the maximum flow onyour flow network? Is it uniquely determined by the original network?

• Answer: Take the resulting flow on the network, and consider a cut between (s, g1, . . . , gn)and (v1, . . . , vm, t). Since every cut has the same flow through it, and by the integralitytheorem if the capacities are integers then there is an integer-valued flow, there are exactly|f | edges in that cut that have flow of 1, and the rest have flow of 0. Now the list of chosenvarieties for a greenhouse i is the list of all j such there is an edge (gi, vj) and it has flow of1.The flow is not uniquely determined by the network, although its value is; the flow depends onthe order in which augmenting paths are chosen. For example, suppose there is one greenhouseand 21 varieties of plants that can all grow in it. Then the resulting network will have flowof 20, and 20 out of these 21 varieties will be chosen, but it is up to the implementation ofFord-Fulkerson to determine which subset of 20 varieties will be chosen.

End of Chapter Questions

63. Reductions

(a) What are the formal requirements for a problem to be in the class Non-Deterministic PolynomialTime?

• Answer: Given an instance and a potential solution, there is a poly time algorithm thatchecks if it is a valid solution

(b) What does it mean to say that Problem A reduces to Problem B? What does it say about theirrelative difficulties?

• Answer: There exists a polynomial time algorithm to map an instance of Problem A to aninstance of Problem B. Problem A can be no harder than Problem B.

64. Wrestlers

There is a group of wrestlers that must be separated into some number of teams for a competition.There are rivalries between some pairs of wrestlers. The goal is to split the wrestlers into teams so thatthere are no rivalries between any two members of the same team (or report that it is impossible).

(a) One of the keys to understanding algorithms is to be able to identify when two problems thatlook superficially different are the same at their core. The wrestling problem is in fact identicalto another problem you have seen in the very recent past. What is this problem? Explain howyou could transform the wrestler problem to this other problem to obtain a solution.

• Answer: This is the 2-colouring problem. Map wrestlers to nodes and rivalries to edges.The 2 colours correspond to the 2 teams.

(b) Suppose you are required to split the teams into exactly k groups. For what values of k will thisbe feasible? Please justify your answer.

• Answer: Since this is just 2-colouring, the problem is feasible for k = 2 and infeasible(NP-Complete) for k ≥ 3.

(c) Suppose that you have an algorithm to split the wrestlers into two teams that minimizes thewithin-team rivalries. Design an algorithm to set up the one-on-one matches for the competition.Each match pits a member of the first team against a member of the second team. Each wrestlercan fight in only one match. Your goal is to maximize the number of matches between rivals.Describe your algorithm in point-form English. Draw a simple figure to illustrate the computation.

• Answer: Construct the flow network below and run the Ford-Fulkerson algorithm to computea max flow network. Pair off wrestlers connected by non-zero flow, and pair off the remainingwrestlers arbitrarily.

63

65. Do question 1 in Micah’s problem set 4 from 2003. For 1b, try a bird-friend algorithm. This is a Cookreduction in that it calls the decision algorithm many times. You don’t need to worry about the 12steps.

www.cs.yorku.ca/ jeff/courses/6121/micah.ass/ps4.ps

• Answer:

(a) Suppose that P = NP . Hence, there is some poly-time algorithm AlgSAT for SAT . Considersome problem Pharder ∈ NP . Our goal is to prove that Pharder is NP-Complete. We alreadyknow that Pharder ∈ NP and that SAT is NP-Complete. Hence, it is sufficient to prove thatSAT ≤ Pharder, namely that if we have a poly-time alg for Pharder, then we have one forSAT . We this is true. Our poly-time algorithm for SAT , simply ignores our alg for Pharder

and run AlgSAT .

(b) Given an instance 〈S1, . . . , Sn, t〉, we ask the little bird whether Sn should be included inthe solution or not. If she says yes, we give the friend the instance 〈S1, . . . , Sn−1, t− Sn〉and return the friend’s answer with Sn included. If she says no, we give the friend theinstance 〈S1, . . . , Sn−1, t〉 and return the friend’s answer unchanged. The trick is to knowin poly-time what the bird will answer. This is done by asking the poly-time algorithm forthe decision problem to solve the instance 〈S1, . . . , Sn−1, t− Sn〉. If it says that there is asolution for this instance, then we assume the bird says to include Sn.

66. Reductions: The purpose of this question is to learn to appreciate why there are so many steps whendoing a reduction.

0) Poracle ∈ NP: Prove that problem Poracle is in NP.

5) InstanceMap: Define a polynomial time mapping Ioracle = InstanceMap(Ialg).

6) SolutionMap: Define a polynomial time algorithm mapping each potential solution Soracle forIoracle to a potential solution Salg = SolutionMap(Soracle) for Ialg .

7) Valid to Valid: Prove that if Soracle is a valid solution for Ioracle, then Salg =SolutionMap(Soracle) is a valid solution for Ialg.

8) ReverseSolutionMap: Define a polynomial time algorithm mapping each potential solution S′alg

for Ialg to a potential solution S′oracle = ReverseSolutionMap(S′

alg) for Ioracle.

9) Reverse Valid to Valid: Prove that if S′alg is a valid solution for Ialg , then S′

oracle =ReverseSolutionMap(S′

alg) is a valid solution for Ioracle.

Consider the problem Block whose instance 〈s, k〉 is a 01-string s and an integer k. Such an instanceis a yes instance if it has a solution consisting of a block si, si+1, . . . sj of all ones whose length is k.

(a) (2 marks) Is this Block problem in NP? If so, what would this mean?

• Answer: It can easily be solved in poly-time and hence is automatically in NP. If a fairy godmother wanted to help she could tell you where the block is, but this is not necessary. Thisis not very exciting.

(b) (2 marks) Is this Block problem is NP-Complete? What would it mean?

• Answer: It can be done in poly time. Hence, if it was NP-complete then all NP problemscould be done in poly time. This is unlikely.

(c) (2 marks) Your home work is to prove that Block is NP-complete by proving that 3-SAT ≤ploy

Block. You are to work in pairs. Your partner took on step 5 as follows. Palg is 3-SAT. Asan example, I3-SAT = (x or y or z) AND . . . AND (u or v or w) is an instance of 3-SAT. Givensuch an instance, he maps it to an instance IBlock = 〈s, k〉 where s = 000 . . .0 is the stringconsisting of n zeros and k is 5. Besides a few hints which he ignored, he noted that he did infact meet the formal requirements of step 5. Being a bit of a know it all, your partner thoughtsteps 8 and 9 are silly and hence decided to skip them. He left you the task of doing steps 6 and7. Carefully, review what these steps are and complete them if you can. Careful.

64

• Answer: No problem. The instance IBlock has no valid solutions because it is a no instance.Because there are none, it is easy to map every valid solution of it to a valid solution ofI3-SAT.

(d) Because your previous partner dropped out of school, you got a new one. Having learned, he tooka new approach. He decided to map an instance I3-SAT of 3-SAT to an instance IBlock = 〈s, k〉where s = 111 . . .1 is the string consisting of n ≥ 5 ones and k is 5. He went on to do steps 6and 7 as follows. Given a solution si, si+1, . . . sj to IBlock, he instructed how to construct a validsolution to I3-SAT as follows. For each of the m clauses (x or y or z) of I3-SAT choose one ofthe variables. If, like x, it is not negated in the clause then set the variable to true. If, like y,it is negated, then set it to false. Either way the clause is satisfied. Doing this for each clause,satisfies each clause. Hence, the assignment obtained is a valid assignment of the variables. Thisnew partner learned (from the guy who dropped out) the importance of doing steps 8 and 9 sohe left them for you to do.

i. (2 marks) Carefully, review what these steps are and complete them if you can. Careful.

• Answer: No problem. The instance IBlock has many valid solutions. One of then is thesubstring s1, s2, . . . , s5. Any valid solutions of I3-SAT (if there are any) can be mappedto it.

ii. (2 marks) Are you happy with your new partner? Did he follow steps 5, 6, and 7 correctly?Explain.

• Answer: He did 5 fine. 6 and 7 he did in one paragraph, which was a mistake. He shouldhave fully defined the assignment to the variables in step 6, before trying to prove it validin step 7. As is, he may have set some variable x to be true in order to satisfy one clausebut later set the same variable to false in order to satisfy another clause.

67. NP-Completeness

(a) The problem Clause-SAT is given a set of clauses where each clause is the OR of a set of literalsand each literal is either a variable or its negation. The goal is know whether you can set eachvariable to true or false so that each clause is satisfied.

The course notes (along with an exercise on 3-SAT), prove that the problems (circuit) SAT, 3-Colouring, Course Scheduling, Independent Set, and 3-SAT are NP-Complete. In a few sentences,explain how you know that Clause-SAT is NP-Complete.

• Answer: Clause-SAT is more general that 3-SAT. Hence, if you can solve any instance ofClause-SAT then you can definitely solve any of 3-SAT. Also being in NP, it follows thatClause-SAT is NP-Complete.

(b) The problem Card is defined by the following puzzle.You are given a box and a collection of cards as indicated in the fig-ure. The box and each card has r rows and two columns of positionsin the same places. The box contains Xs in these positions. In eachcard at each of these positions, there is either a hole punched out orthere is not. Each card must be placed in the box either face up orflipped over left to right. It has a notch at its top which must beat the top of the box. Consider some row in some card. Supposeits left position has a hole but its right position does not. Puttingthe card in face up covers the right X in the box of this row but notthe left. Putting the card in flipped over covers the left but not theright. The goal is to cover each and every X in the box. A solutionspecifies for each card whether to flip the card or not. A solution isvalid if it covers all the Xs.

X X

XX

X X

XX

XX

X X

XX

X X

XX

Box One side The other side

Prove that Card is NP-Complete by reducing it to Clause-SAT. Be sure to think about each ofthe 12 steps. As a huge hint, I do step 5 for you. You are to write up steps 0, 6, and 7. For eachof these have a clear paragraph with pictures.

65

0) Pcard ∈ NP:

5) InstanceMap: Given an instance IClause−SAT to Clause-SAT consisting of a set of clauses,we construct as follows an instance Icard = InstanceMap(IClause−SAT ) for the card gameconsisting of a set of cards. Our box and cards will have one row for each clause. We will havea card for each variable x. For each clause c, this card for x will have its left hole positionfor row c not punched out if x appears positively in c and will have its right hole position forc not punched out if x appears negatively in c. It will have all other hole positions punchedout. We will have one additional card that has all of its left hole positions punched out andnone of its right hole positions punched out.

6) SolutionMap:

7) Valid to Valid:

• Answer:

0) Pcard ∈ NP: Given an instance consisting of a set of cards and a solution consisting ofan orientation of each card, it is easy to determine if every hole position is blocked.

6) SolutionMap: Given a solution Scard for the instance Icard =InstanceMap(IClause−SAT ) consisting of a notflipped/flipped orientation of eachcard, we construct as follows a solution SClause−SAT for instance IClause−SAT consistingof a true/false setting of each variable. First, if in the solution Scard, the additional cardis flipped then we can flip the entire deck of cards over without changing whether all thehole positions are covered. Hence, without loss of generality assume that in Scard, theadditional card is not flipped. Then for each variable x, set it to true iff its associatedcard is notflipped in Scard.

7) Valid to Valid: Assume that Scard is a valid solution, i.e. every hole position is covered.Our goal is to prove that SClause−SAT is a valid solution, i.e. every clause is satisfied.Consider some clause c. Because the additional card is not flipped in Scard and everyhole position is covered, we know that there is some variable card x covering the left holeposition for clause c. If this card is not flipped then its left hole position for c must notbe punched. In this case, x is set to true in SClause−SAT and x appears positively inc. Hence, this clause is satisfied. Alternatively, if this card is flipped then its right holeposition for c must not be punched. In this case, x is set to false in SClause−SAT and xappears negatively in c. Again, this clause is satisfied. Hence, each clause in SClause−SAT

is satisfied.

(c) Suppose you work for an airplane manufacturing company. Given any detailed specification of aplane, right down to every curve and nut and bolt, your boss has a way of determining whetherit is a great plane or not. However, he has tried thousands of different specifications and none ofthem turn out to be great. He is completely frustrated.

Meanwhile back at home, your son didn’t finish high school, is socially awkward, and is living inyour basement at 31. You blame it on the fact that he is completely addicted to playing videogames. You feel this is only one step better to being a crack addict. One game he plays all thetime is the card flipping puzzle described above. He has a magical ability to instantly put any setof cards into the box so that all the Xs are covered.

You are are only able to do things described in the course notes (or hinted at in exercises). Howdo you get your son a job at work so that he quickly becomes a billionaire?

• Answer: Following your boss’ method, you write a JAVA program that takes as input thedescription of plane and outputs whether or not it is great. (Given a fixed number of bitsn to describe the input, it is not too hard to automatically compile this program into anAND-OR-NOT circuit that does the same thing. The notes describe a way to convert thiscircuit into a graph to be 3-Colored. The exercise in the notes hints at how to convert thisinto an instance of 3-SAT, which is itself an instance of Clause-SAT. Above we describe howto convert this into an instance of the Card game. You get your son to magically solve it. Youconvert this solution into a solution for the 3-Sat instance, which you convert into a solution

66

of the graph coloring instance, which you covert into a solution of the circuit problem anda valid input of the JAVA program. This you translate into a description of a great plane,which you give your boss. You and your son go on to solving all of the world’s optimizationproblems.

68. NP Complete

• 3-SAT ≤poly 3-Clique

• 3-SAT ≤poly CourseScheduling

(a) Are these true? Why?

• Answer: The first is not because 3-Clique can be solved in Θ(n3) time and hence is not likelyharder than 3-SAT which is believed to require exponential time.The second we proved is true in class.

(b) In one sentence explain what needs to be done to prove it.

• Answer: Given an oracle (a poly time algorithm) for CourseScheduling design an oracle for3-SAT .

(c) How are these useful to you when working as an algorithm designer?

• Answer: This second statement proves that CourseScheduling very likely requires expo-nential time, because if it could be solved quickly then so could 3-SAT , which is believed torequire exponential time.

69. Let Pgeneral and Prestricted be the same computational problem except that Prestricted is defined onlyon a subset of the instances on which Pgeneral is defined. For example, 3-SAT is the same problem asSAT except that each clause has at most three literals. Clique is given a graph G and an integer kand asks whether there is a clique in G of size k. 3-Clique is the same with k fixed to 3.

(a) Compare and discuss the meaning of Pgeneral ≤poly Prestricted vs Prestricted ≤poly Pgeneral. Whatare the intuitions as to why they may be true and why they may not be true. Which is easier toprove?

(b) Which of SAT ≤poly 3-SAT and 3-SAT ≤poly SAT are true. Give one sentence how you know.

(c) Which of Clique ≤poly 3-Clique and 3-Clique ≤poly Clique are true. Give one sentence how youknow.

(d) Do steps 5-9 to prove that 3-SAT ≤poly SAT . Hint: This REALLY is easy.

• Answer:

(a) Clearly a restricted problem is easier because there are few instances that might be the worstcase. Prestricted ≤poly Pgeneral is always true. Pgeneral ≤poly Prestricted will not be true if theinstances considered by the restricted problem are the easy ones and not the worst case ones.

(b) As said, 3-SAT ≤poly SAT is true because the first is a restriction of the second (See proofbelow). Surprisingly, SAT ≤poly 3-SAT is also true. One has to show that there are suffi-ciently difficult instances that have 3 literals per clause. The reduction shows that being ableto solve the 3 literal case is enough to be able to solve all of NP .

(c) Again 3-Clique ≤poly Clique is trivially true. Clique ≤poly 3-Clique is false (unless P = NP ),because Clique is NP-complete and 3-Clique can be solved in O(n3) time.

(d) 3-SAT ≤poly SAT .

5) InstanceMap: Given an instance I3-SAT to 3-SAT , InstanceMap(I3-SAT ) simply mapsto the same instance because this is also an instance of the more general SAT .

6&8) SolutionMaps: Given that the two instances I3-SAT and ISAT are the same, theyclearly have the same set of solution. We simply map solutions to themselves.

67

7&9) Valid to Valid: Being the same solution to the same instance clearly they are bothvalid or both not valid.

Part IV: AppendixChapter: 22: Existential and Universal Quantifiers

70. (Quantifiers) Let Works(P, A, I) be defined to be true if algorithm A halts and correctly solves problemP on input instance I.Let P = Halting be the Halting problem which takes a Java program I as input and says “yes” if Ihalts on the empty string. Otherwise, P must say ′′no′′.Let P = Sorting be the sorting problem which takes a list of numbers I as input and sorts them.Let Ainsertionsort be the sorting algorithm which we learned in class.Let Ayes be the algorithm that on input I ignores the input and simply halts and says “yes”.Similarly, let Ano be the algorithm that simply says “no”.Let A∞ be the algorithm that on input I ignores the input and simply runs for ever.For each part, explain the meaning of what you are doing and why you don’t do it another way.

(a) Express in first order logic that Sorting is computable.

• Answer: ∃A, ∀I, Works(Sorting, A, I). We know that there at least one algorithm, eg.A = mergesort, that works for every input instance I.

(b) Express in first order logic that Halting is not computable. Explain why you do it this way.

• Answer: ∀A, ∃I,¬Works(Halting, A, I) We know that opposite statement is true. Everyalgorithm fails to work for at least one input instance I.

(c) Express in first order logic that there are uncomputable problems.

• Answer: ∃P, ∀A, ∃I,¬Works(P, A, I)

(d) What does the following mean and either prove or disprove it: ∀I, ∃A, Works(Halting, A, I).(Not simply by saying the same in words “For all I, exists A, Works(Halting, A, I)”)

• Answer: It says that every input has some algorithm that happens to output the rightanswer. It is true. Consider an arbitrary instance I. If on instance I, Halting happens tosay yes, then let A be the algorithm that simply halts and says “yes”. Otherwise, let A bethe algorithm that simply halts and says “no”. Either way A “works” for this instance I.

(e) What does the following mean and either prove or disprove it ∀A, ∃P, ∀I, Works(P, A, I).

• Answer: It says that every algorithm correctly solves some problem. This is not truebecause some algorithm do not halt on some input instances. We prove the complement∃A, ∀P, ∃I,¬Works(P, A, I) as follows. Let A be an algorithm that runs for ever on someinstance I ′. Let P be an arbitrary problem. Let I be an instance I ′ on which A does nothalt. Note Works(P, A, I) is not true.

Chapter: 23: The Time (and Space) Complexity of an Algo-rithmChapter: 24: Logarithms and Exponentials

Chapter: 25: Asymptotic NotationSection: 25.1: Steps to Classify a Function

71. (4 marks) Suppose f(n) = abcnndloge(n). What values can the real numbers a, b, c, d, and e take onand have f(n) be considered in the course notes to have a “polynomial” growth rate? Give an example.

• Answer: a > 0, bc = 1, d > 0, and e anything. For example, f(n) = n2 is polynomial.

68

Section: 25.2: More about Asymptotic Notation

72. The formal definition of f(n) ∈ nΘ(1) is ∃c1, c2, n0, ∀n ≥ n0, nc1 ≤ f(n) ≤ nc2 . Formally prove thatif f(n) and g(n) ∈ nΘ(1), then f(n) · g(n) ∈ nΘ(1). What is this class of functions called?

• Answer: These functions are polynomials.It includes f(n) = n2logn because it is bounded between n2 and n3.Because f(n) and g(n) ∈ nΘ(1), we know that∃c〈f,1〉, c〈f,2〉, n〈f,0〉, ∀n ≥ n〈f,0〉, nc〈f,1〉 ≤ f(n) ≤ nc〈f,2〉

∃c〈g,1〉, c〈g,2〉, n〈g,0〉, ∀n ≥ n〈g,0〉, nc〈g,1〉 ≤ g(n) ≤ nc〈g,2〉

Consider some such constants c〈f,1〉, c〈f,2〉, n〈f,0〉, c〈g,1〉, c〈g,2〉, and n〈g,0〉.

Our goal is to now prove f(n) · g(n) ∈ nΘ(1), i.e.∃c1, c2, n0, ∀n ≥ n0, nc1 ≤ f(n) · g(n) ≤ nc2 .Let c1 = c〈f,1〉 + c〈g,1〉, c1 = c〈f,2〉 + c〈g,2〉, and n0 = max(n〈f,0〉, n〈g,0〉).Let n any value (given by an adversary) such that n ≥ n0 = max(n〈f,0〉, n〈g,0〉).Because n ≥ nf,0, we have that ncf,1 ≤ f(n) ≤ ncf,2

Because n ≥ ng,0, we have that ncg,1 ≤ g(n) ≤ ncg,2

Because these are all positive, we can multiply the above two lines together givingnc1 = nc〈f,1〉 · nc〈g,1〉 ≤ f(n) · g(n) ≤ nc〈f,2〉 · nc〈g,2〉 = nc2 .

73. We have seen three algorithms for the sorted matrix problem with running times T1(n, m) = n + m,T2(n, m) = n log(m

n ) and T3(n, m) = n log(m). We will be considering families of matrices. In aparticular family. the hight n can have any value and the width m = m(n) is some function of n. Therecursive T2(n, m) algorithm was a pain to write and implement. Hence, we only want to use it forfamilies of matrices where the running is really little oh of the running time of other two algorithms,i.e. not Theta. Use O, o, Θ, Ω, and ω, to bound for which functions m(n) we will use this algorithm.Start by trying different functions. Give extreme examples that work and others that do not work.

• Answer: Express the width as m(n) = n · c(n), where c(n) is some function yet to be bound.This givesT2(n, m) = n log(m

n ) = n · log(c(n)) andT1(n, m) = n + m = n · (c(n) + 1).If c(n) = Θ(1) is bounded by a constant, then T2(n, m) = Θ(T1(n, m)), meaning we should notbother with the recursive program. For example, even m(n) = 1000n is too small. On theother hand, if c(n) = ω(1) is not constant but grows with n, then c(n) + 1 = ω(log(c(n))) andlog(c(n)) = o(c(n) + 1), giving T2(n, m) = o(T1(n, m)), meaning we should use the recursiveprogram. For example, m(n) = n log log n grows fast enough.

Now express the width as m(n) = n · nc(n), where c(n) is some function yet to be bound. ThisgivesT2(n, m) = n log(m

n ) = n log(nc(n)) = c(n)n log(n) and

T3(n, m) = n log(m) = n log(n1+c(n)) = (1 + c(n))n log(n).If c(n) = Ω(1), meaning that it is either constant or grows with n, then T2(n, m) = Ω(T3(n, m)),meaning we should not bother with the recursive program. For example, even m(n) = n · √n andm(n) = n1.0001 are is too big. On the other hand, if c(n) = o(1) is not constant but shrinks withn, then c(n) = o(1+c(n)) and hence T2(n, m) = o(T3(n, m)), meaning we should use the recursive

program. For example, m(n) = n · n(1/√

log n) = n · 2√

log n and m(n) = n logn grow faster then nbut not too fast.

Chapter: 26: Adding Made Easy ApproximationsSection: 26.1: The TechniqueSection: 26.2: Some Proofs for the Adding Made Easy Tech-nique

69

Chapter: 27: Recurrence Relations

74. Asymptotic Analysis (20 marks)

(a) (3 marks) Briefly describe the structure of a typical recursive algorithm that has running timeT (n) = a · T (n

b ) + f(n).

• Answer: Work of f(n) in stack frame and a recursive calls, each of which receives an inputthat is reduced by a factor of 1

b .

(b) (6 marks) Depending on the values of a, b and c, what is the theta approximation of the recurrenceT (n) = a · T (n

b ) + nc? For each case, give a one sentence justification.

• Answer: If log alog b > c, then the time is dominated by the base cases and T (n) = Θ(n

log alog b ).

If they are equal, then all the levels have about the same time and T (n) = Θ(nc log n).If log a

log b < c, then the time is dominated by the top level and T (n) = Θ(nc).

(c) (4 marks) Give values for a, b, and c such that T (n) = a · T (nb ) + nc = Θ(n

43 ) and the time is

dominated by the base cases.

• Answer: If a = 24 = 16, b = 23 = 8, and c = 0, then T (n) = 16·T (n8 )+1 = Θ(n

log alog b ) = Θ(n

43 ).

(d) (4 marks) True or False? All logarithms are base 2. No justification is necessary.

i. 5n2 log n ∈ O(n2)

• Answer: False. It is a factor of log n too big.

ii. 48n ∈ O(84n)

• Answer: False: 48n = 216n, but 84n = 212n.

iii. 210 log n + 100(logn)11 ∈ O(n10)

• Answer: True: 210 log n = n10, 100(logn)11 ∈ O(n10).

iv. 2n2 log n + 3n2 ∈ Θ(n3)

• Answer: False: 2n2 log n + 3n2 ∈ O(n3), but 2n2 log n + 3n2 /∈ Ω(n3).

70

(e) (3 marks) For each of the following sums, give the Θ-approximation. No justification is necessary.

i. Σlog ni=1 i2

• Answer: Θ((last term) · (number of terms)), since the sum is arithmetic-like = Θ(log2 n ·log n) = Θ(log3 n)

ii. Σni=1i

2i!

• Answer: = Θ((last term)), since the terms in the sum grow faster than geometric-like

= Θ(n2n!

))

iii. Σni=1

2i3

• Answer: = Θ(1). −3 < −1, and hence the sum is bounded-tail.

75. Consider the following functions.

(a) f(n) = 5 · n5.5 + n2.5 log100(n)

(b) f(n) = 5 · 3√

n + 5 · 4√

n

(c) f(n) = 5 · 34n + n100

(d) f(n) = 5n1.01 + 7

n0.99

(e) f(n) = 5 + sin(n2)

Sort the functions by growth rate from fastest to slowest. While you are at it sort all the terms(3 marks). Now answer the following questions for each function, starting with the fastest growingfunction. Please report your answer for both questions for each function before moving on to the nextfunction.

• Classification: (2 marks per function) Provide a Θ-approximation, if possible classifying the

function into Θ(1), 2Θ(n), logΘ(1)(n), or nΘ(1). What is this class called?

• Summation: (2 marks per function) Approximate the sum Θ(∑n

i=1 f(i)). Which result did youuse and how did you use it? What type of sum is it?

• Answer:

c >> a >> b >> e >> d, i.e., 34n >> n5.5 >> n2.5 log100(n) >> 3√

n >> 4√

n >> 5+sin(n2) >>n−0.99 >> n−0.1.01

(a) Class: f(n) = 5 · n5.5 + n2.5 log100(n). The n2.5 log100(n) is smaller than n2.5001 which isclearly smaller that n5.5. Hence, f(n) ∈ Θ(n5.5) ⊆ nΘ(1). This is a polynomial function.Sum: ba = 1 and d = 5.5 > −1. Hence, the sum is arithmetic and half the terms areapproximately equal. Hence,

∑ni=1 f(i) = Θ(n · f(n)) = Θ(n6.5).

(b) Class: f(n) = 5 · 3√

n + 5 · 4√

n = 5 · n 13 + 5 · n 1

4 . Because 13 > 1

4 , the first term dominates.

Hence, f(n) ∈ Θ(n13 ) ⊆ nΘ(1) is an polynomial function. Though it is grows slower than n,

it is nd for d > 0.Sum: ba = 1 and d = 1

3 > −1. Hence, the sum is arithmetic and half the terms are

approximately equal. Hence,∑n

i=1 f(i) = Θ(n · f(n)) = Θ(n1 13 ).

(c) Class: f(n) = 5 · 34n + n100 is dominated by the exponential term not the polynomial term.f(n) ∈ Θ(34n) ⊆ 2Θ(n) is an exponential function.Sum:

∑ni=1 f(i) =

∑ni=1 Θ(34i) =

∑ni=1 Θ(81i). Since 81 > 1, the sum is geometric increas-

ing and dominated by the last term. Hence∑n

i=1 f(i) = Θ(f(n)) = Θ(34n).

(d) Class: f(n) = 5 · 18n = 5 = Θ(1) is constant.Sum: See (f)

71

(e) Class: f(n) = 5n1.01 + 7

n0.99 = 5 ·n−1.01 + 7 ·n−0.99. Because −0.99 > −1.01, the second termdominates and f(n) ∈ Θ( 1

n0.99 ). This decreases with n so is to small to even be consideredconstant, Θ(1).Sum: ba = 1 and d = −0.99 > −1. Hence, the sum is still arithmetic and half the terms areapproximately equal. Hence,

∑ni=1 f(i) = Θ(n · f(n)) = Θ(n0.001).

Sum: (not asked for) If f(n) = 5n1.01 , then ba = 1 and d = −1.01 < −1. Hence, the sum is

bounded tail and the sum is dominated by the first term. Hence,∑n

i=1 f(i) = Θ(1).

(f) Class: f(n) = 5 + sin(n2). The n2 was a trick because the sine is always between -1 and 1.Hence, f(n) ∈ Θ(1). Though it changes with n, the change is not considered significant andhence is said to be constant.Not asked for: This would be the same for f(n) = 1 if n is odd, 2 if n is even ∈ Θ(1).Not asked for: If f(n) = sin(n2), then f(n) would not be in Θ(1) because things in Θ(n)should be positive for big n.Sum: ba = 1 and d = 0 > −1. Hence, the sum is arithmetic and half the terms areapproximately equal. Hence,

∑ni=1 f(i) = Θ(n · f(n)) = Θ(n).

76. Consider the following functions.

(a) f(n) = 5 · n5

(b) f(n) = 5 · 3√

n

(c) f(n) = 5 · 34n

(d) f(n) = 5n2

(e) f(n) = 1 if n is odd, 2 if n is even

Sort the functions by growth rate from fastest to slowest. Then for each, do each of the followingthings.

(a) Give its Theta approximation. If possible classifying it into Θ(1), 2Θ(n), logΘ(1)(n), or nΘ(1).What is this class called?

(b) Approximate the sum Θ(∑n

i=1 f(n)). Which result do you use and how do you use it? What typeof sum is it?

(c) Describe an algorithm with a natural structure (like those found in the Appendix) which takes asinput a string containing n bits and prints Θ(f(n)) “Hi”s. Zero marks will be given for

loop i = 1 . . . f(n) print(“Hi”)

• Answer:

(a) Order: f(n) = 5 · 34n ∈ Θ(34n) ⊆ 2Θ(n) is an exponential function.Sum: As given in the table of Chapter 26.1, ba = 34 = 81 > 1. Hence, the sum is geometricincreasing and dominated by the last term. Hence Θ(

∑ni=1 f(n)) = Θ(f(n)) = Θ(34n).

Code: Note f(n) = 34n = 81n. The recurrence relation T (n) = 81T (n− 1) is similar to thatin the next assignment and evaluates to Θ(f(n)) = Θ(81

n1 ). Code which recurses 81 times

on inputs of size n− 1 and prints one “Hi” on each base case has this running time.

(b) Order: f(n) = 5 · n5 ∈ Θ(n5) ⊆ nΘ(1) is an polynomial function.Sum: ba = 1 and d = 5 > −1. Hence, the sum is arithmetic and half the terms areapproximately equal. Hence, Θ(

∑ni=1 f(n)) = Θ(n · f(n)) = Θ(n6).

Code: Code consisting of five nested loops, each iterating n times, with a print(“Hi”) state-ment in the middle will print n5 “Hi”s.

(c) Order: f(n) = 5 · 3√

n ∈ Θ(n13 ) ⊆ nΘ(1) is an polynomial function. Though is grows slower

than n, it is nd for d > 0.Sum: ba = 1 and d = 1

3 > −1. Hence, the sum is arithmetic and half the terms are

approximately equal. Hence, Θ(∑n

i=1 f(n)) = Θ(n · f(n)) = Θ(n1 13 ).

72

Code: The recurrence relation T (n) = 3T ( n27 ) from the next assignment evaluates to

Θ(f(n)) = Θ(n13 ). Code which recurses 3 times on inputs of size n

27 and prints one “Hi”on each base case has this running time.

(d) Order: f(n) = 1 if n is odd, 2 if n is even ∈ Θ(1). Though it changes with n, the changeis not considered significant and hence is said to be constant.Sum: ba = 1 and d = 0 > −1. Hence, the sum is arithmetic and half the terms areapproximately equal. Hence, Θ(

∑ni=1 f(n)) = Θ(n · f(n)) = Θ(n).

Code: Code which prints one “Hi” and another if n is even, prints this many “Hi”s.

(e) Order: f(n) = 5n2 ∈ Θ( 1

n2 ) decreases with n so is to small to even be considered constant,Θ(1).Sum: ba = 1 and d = −2 < −1. Hence, the sum is bounded tail and the sum is dominatedby the first term. Hence, Θ(

∑ni=1 f(n)) = Θ(1).

Code: It is impossible to print more than zero and less than one “Hi”.

77. Solve each of the following recurrence relations. Which result do you use and how do you use it?

(a) T (n) = 3T ( n27 ) + n2.

(b) T (n) = 3T ( n27 ).

(c) T (n) = 3T ( n27 ) + n3

(d) T (n) = 3T ( n27 ) + 2n

(e) T (n) = 3T (n− 4).

• Answer:

(a) T (n) = 3T ( n27 ) + n2: As in the first table of chapter 27.1, a = 3, b = 27, and c = 2.

We have log alog b =

log3 3log3 27 = 1

3 < 2 = c. Hence, the time is dominated by the top level and

T (n) = Θ(f(n)) = Θ(n2).

(b) T (n) = 3T ( n27 ): Same as above but, f(n) = 0. This is smaller than 1 = n0. Hence, we will

say c ≤ 0. This gives log alog b = 1

3 > c. Hence, the time is dominated by the base cases and

T (n) = Θ(nlog alog b ) = Θ(n

13 ).

(c) T (n) = 3T (n− 4). As in the second table of chapter 27.1, a = 3 > 1 and b = 4. Hence, thetime is dominated by the base cases and T (n) = Θ(f(n)) = Θ(a

n4 ).

Alternatively, just unwrap the recursion: T (n) = 3T (n − 4) = 3(3T ((n − 4) − 4)) = ... =3iT (n− 4i)... The base case occurs when n− 4i = 0→ i = n/4→ T (n) ∈ Θ(3n/4).

78. Math

(a) (5 marks) When f(n) has a polynomial growth rate, what is the theta approximation of the sum∑ni=1 f(i)? Give a one sentence justification.

• Answer: Half the terms are approximately the same giving that the sum is approximatelythe number of terms times the last term, i.e.

∑ni=1 f(i) = Θ(n · f(n)).

(b) (5 marks) Quickly describe the structure of an algorithm that has running time T (n) = a ·T (nb )+

f(n).

• Answer: I do work f(n) and have a friends each of which receives an input that is 1b the size

of my own.

(c) (5 marks) Give values for a, b, and c such that T (n) = a · T (nb ) + nc = Θ(n

43 ) and the time is

dominated by the base cases. Show the math.

• Answer: If a = 24 = 16, b = 23 = 8, and c = 0, then T (n) = 16·T (n8 )+1 = Θ(n

log alog b ) = Θ(n

43 ).

73

Chapter: 28: A Formal Proof of Correctness

Part Questions from Grad Class

79. Short Answer

(a) (2 marks) Fast Fourier Transforms (FFT) can be used for signal processing and many things. Theversion that we covered, what was the input and what was the output.

(b) (2 marks) As a TA, you have lots of experience marking assignments. You are given JAVA codewritten by first year students and you can always tell whether or not they halt. You are also givenmath statements like ∀x, ∃y, x+ y = 5 and you can always tell whether they are true or not. Yourteacher says something about the halting problem and the math truth both being undecidable,but you are convinced he is wrong. Debate this one way or the other.

• Answer: The TA may be smarter than the first year students, but let me give him a JAVAprogram or a math statement for him to understand. For every fixed algorithm for either thehalting problem or the math truth problem, there is an input for which these programs eithergive the wrong answer or fail to halt.

============================

In question not answer in notes.

Exercise 0.0.9 There is a collection of software packages S1, . . . , Sn which you are considering buying.These are partitioned into two groups. For those i ∈ N ⊆ [n], the costs of buying it out ways the benefitsand hence it effectively costs you a given amount bi ≥ 0 to buy it. For those j ∈ P ⊆ [n], the benefits outway the costs of buying it and hence it effectively costs you a given amount bj ≥ 0 to not buy it. Some ofthese packages rely on each other; if Si relies on Sj, then you will incur an additional cost of a〈i,j〉 ≥ 0if you buy Si but not Sj. Provide a polytime algorithm to decide the subset S ⊆ [n] of S1, . . . , Sn thatyou should buy. The cost of your solution is cost(S) =

∑i∈S∩N bi +

∑j∈S∩P bi +

∑i∈S,j∈S a〈i,j〉. Hint:

Do not design a new algorithm but do a reduction to min cut similar to that done for matching theboys and girls.

As said, this answer is not in the book. I will put it in the extended book for the professors.

• Answer: Lets start with some informal discussion. Given an instance ISoftware, the nodesof INetworkFlow = InstanceMap(ISoftware) will be the software packages S1, . . . , Sn and twoadditional nodes s and t. Our solution SSoftware = SolutionMap(SNetworkFlow) must specify foreach package Si whether or not to buy it. It is hard to get that information from SNetworkFlow

if we consider SNetworkFlow to be the max flow. However, if we consider SNetworkFlow to bethe min cut 〈U, V 〉, then we could either buy the software/nodes in U − s or we could buy thesoftware/nodes in V − t. If we buy Si but not Sj, then cost(SSoftware) acquires an additionalcost of a〈i,j〉. If i ∈ U and j ∈ V , then cost(SNetworkFlow) acquires an additional cost of c〈i,j〉.Hence, it is reasonable to buy the software/nodes in U − s and not those in V − t.

Lets now be more formal. Given an instance ISoftware, the network INetworkFlow =InstanceMap(ISoftware) will have an edge 〈i, j〉 with capacity c〈i,j〉 = a〈i,j〉 for each pair ofsoftware packages Si and Sj . For each i ∈ N , there will be an edge 〈i, t〉 with capacity c〈i,t〉 = bi.For each j ∈ P , there will be an edge 〈s, j〉 with capacity c〈s,j〉 = bj.

Given valid cut SNetworkFlow = 〈U, V 〉 for this network, The solution SSoftware =SolutionMap(SNetworkFlow) is that SSoftware = U − s, i.e. buy the software associated withthe nodes in U − s. This is a 1-1 mapping, because given a solution SSoftware one can map backto a SNetworkFlow = SolutionMap−1(SSoftware), where U = SSoftware +s and V = SSoftware+ t.

The final step is to prove that the costs of these two solutions are the same, i.e. cost(SSoftware) =cost(SNetworkFlow). By the definition of the cost of a cut 〈U, V 〉, we have that

74

cost(SNetworkFlow) =∑

〈u∈U,v∈V 〉c〈u,v〉

=∑

〈i∈U−s,j∈V −t〉c〈i,j〉 +

∑

〈i∈U∪N〉c〈i,t〉 +

∑

〈j∈V ∪P 〉c〈s,j〉

=∑

〈i∈SSoftware,j∈SSoftware〉a〈i,j〉 +

∑

〈i∈SSoftware∪N〉b〈i,t〉 +

∑

〈j∈SSoftware∪P〉b〈s,j〉

= cost(SSoftware)

This completes the proof.

80. There is a collection of software packages S1, . . . , Sn which you are considering buying. These arepartitioned into two groups. For those i ∈ N ⊆ [n], the costs of buying it out ways the benefits andhence it effectively costs you a given amount bi ≥ 0 to buy it. For those j ∈ P ⊆ [n], the benefitsout way the costs of buying it and hence it effectively costs you a given amount bj ≥ 0 to not buy it.Some of these packages rely on each other; if Si relies on Sj, then you will incur an additional cost ofa〈i,j〉 ≥ 0 if you buy Si but not Sj . The goal is to provide a polytime algorithm to decide the subsetS ⊆ [n] of S1, . . . , Sn that you should buy in order to minimize your total cost.

I will start with a major hints, otherwise the problem would be too hard. Hint: Do not design a newalgorithm but do a reduction to min cut similar to that done for matching the boys and girls. Formallya reduction is done as follows.

Instance Map: Recall the algorithm for matching the boys and girls. Given an instance Iboys&girls forthis problem, the algorithm constructed an instance INetworkFlow = InstanceMap(Iboys&girls) forthe network flow problem. Basically, the boys and girls in the instance Iboys&girls become nodesof the graph INetworkFlow with a directed edge from the boy to the girl if they are willing to marryeach other. In addition there is a node s with an edge to each boy and a node t with an edgefrom each girl. We will do a similar thing here.

In general, such an instance map is the first step of a reduction. Here we must to define apolynomial time algorithm InstanceMap(ISoftware) that, given an instance ISoftware of problemPSoftware, constructs an instance IMinCut of problem PMinCut that has “similar” sorts of solutions.(Recall that an instance of the min cut problem is exactly the same as that to the network flowproblem.) The nodes of IMinCut will be the software packages S1, . . . , Sn and two additional nodess and t. It will have an edge 〈i, j〉 with capacity c〈i,j〉 = a〈i,j〉 for each pair of software packagesSi and Sj . For each i ∈ N , there will be an edge 〈i, t〉 with capacity c〈i,t〉 = bi. For each j ∈ P ,there will be an edge 〈s, j〉 with capacity c〈s,j〉 = bj.

Solution Map: Again, recall the algorithm for matching the boys and girls. A solution SNetworkFlow

to the network flow problem specifies the flow along each edge of the graph. This is used toproduce a solution Sboys&girls for the boys and girls problem. Basically, a boy marries a girl ifthere is flow between them.

In general, such a solution map is the second step of a reduction. Here we must define a polynomialtime algorithm SolutionMap(SMinCut) giving a 1-1 mapping between any valid solution SMinCut

for the instance IMinCut = InstanceMap(ISoftware) you just constructed and a solution SSoftware

for instance ISoftware that was given as input.

Given valid cut SMinCut = 〈U, V 〉 for this network, The solution SSoftware =SolutionMap(SMinCut) is that SSoftware = U − s, i.e. buy the software associated with thenodes in U − s. This is a 1-1 mapping, because given a solution SSoftware one can map back toa SMinCut = SolutionMap−1(SSoftware), where U = SSoftware + s and V = SSoftware + t.

a) Cost Map: The next step is to prove that the costs of these two solutions are the same, i.e.cost(SSoftware) = cost(SMinCut). Prove this. (10 marks)

75

• Answer: By the definition of the cost of a cut 〈U, V 〉 is the sum of the capacities of theedges leaving the cut, ignoring those edges coming back into the cut. This gives

cost(SMinCut) =∑

〈u∈U,v∈V 〉c〈u,v〉

=∑

〈i∈U∪N〉c〈i,t〉 +

∑

〈j∈V ∪P 〉c〈s,j〉 +

∑

〈i∈U−s,j∈V −t〉c〈i,j〉

=∑

〈i∈SSoftware∪N〉bi +

∑

〈j∈SSoftware∪P〉bj +

∑

〈i∈SSoftware,j∈SSoftware〉a〈i,j〉

= cost(SSoftware)

By the definition of the software problem, the cost of the solution SSoftware is the sum of thecosts bi of buying the costly packages Si (i.e. i in SSoftware and in N); the sum of the costsbj of not buying the valuable packages Sj (i.e. j not in SSoftware but in P ); and the sum ofthe costs a〈i,j〉 incurred for buying Si but not Sj (i.e. i in SSoftware and j not in SSoftware).

b) Code: Give the algorithm/code that solves this Software problem. You may use the Network Flowsalgorithm in the text as a subroutine. (10 marks)

• Answer: This provides an algorithm for the problem PSoftware. Given in instanceISoftware, construct IMinCut = InstanceMap(ISoftware). Run the standard NetworkFlow algorithm on IMinCut to obtain a solution SMinCut. Finally, construct SSoftware =SolutionMap(SMinCut).

algorithm AlgSoftware(ISoftware)

〈pre−cond〉: ISoftware is an instance of the Software problem.

〈post−cond〉: Returns the optimal set SSoftware of packages to buy.

beginIMinCut = InstanceMap(ISoftware)

% IMinCut is an instance of both MaxFlow and of MinCut.〈SMaxFlow, SMinCut〉 = NetworkF low(IMinCut)

% SMinCut is the optimal solution for IMinCut.SSoftware = SolutionMap(SMinCut)return(SSoftware)

end algorithm

c) Proof ≤: Proof that if your program says that the minimum cost is cmin, then there is a solutionfor the instance that has this cost. (5 marks)

• Answer: We claim that this algorithm gives an optimal solution for our instance ISoftware.The solution SSoftware returned witness the fact that you can obtain a solution of valuecmin = cost(SSoftware).

d) Proof ≥: Proof that if your program says that the minimum cost is cmin, then there is not a bettersolution for the instance. (5 marks)

• Answer: If there was another solution S′Software that had a better cost cost(S′

Software), then

the inverse map S′MinCut = SolutionMap−1(S′

Software) would give a solution S′MinCut to

IMinCut whose cost cost(S′MinCut) = cost(S′

Software) is better than the cost cost(SMinCut) =cost(SSoftware). This contradicts the claim that the standard Network Flow algorithm ob-tained an optimal solution SMinCut for IMinCut.

e) Example: (20 marks) Solve the Software problem for the instance ISoftware consisting of foursoftware packages: S1, . . . , S4. S1 and S2 are in N and as such cost you b1 = 10 and b2 = 3 tobuy them. S3 and S4 are in P and as such cost you b3 = 9 and b4 = 5 to not buy them. Theadditional costs a〈i,j〉 for buying Si but not Sj are a〈1,2〉 = 5, a〈3,1〉 = 4, a〈3,2〉 = 4, a〈4,1〉 = 7,a〈4,2〉 = 3, and a〈4,3〉 = 2.

76

(a) (2 marks) Draw the input SNetworkFlow to network flow.

(b) (5 marks) Draw the max flow for this network and give its rate of flow.

(c) (5 marks) Draw the augmentation graph for this max flow.

(d) (5 marks) Draw the min cut for this network and give its value.

(e) (1 marks) Which software packages should you buy?

(f) (2 marks) What things cost you and what is the total cost?

• Answer:

4S

S11S

S4S4

1S

2S

S3

s t9

4

10

5

3

3

5

2

4

7

S1

4S

3/3

7/9 3/49/10

4/4

5/5

0/2

0/3

5/70/5 ts

3S

S2

Max Flow

2S

S3

s t9

4

10

5

3

3

5

2

4

7

Min CutAugmentation Graph

9

3

5ts

3S

S2

72

4

2

5

3

5

32

11

value of cut = 5+4+3=12rate = 7+5=12

Input Network

You should by S3 and S2. You pay 5 for not buying S4, 4 for buying S3 but not S1, and 3for buying S2.

77