Divide and conquer

• integer multiplication
• polynomial multiplication (FFT)
• Master Theorem and its limitations

Consider a recurrence relation like the following. \[ T(n) = a T(n / b) + O(n^k) \] It says your recursive solution to the problem has the structure of a tree with \( a \) branches per node, where each child corresponds to a subproblem of size \( n / b \), and where there is \( n^k \) work to be done at each node at this level of the tree.

Consider where the work is done in the tree (called the Master Theorem).

• At the bottom (bottom heavy): When \( a > b ^ k \). Here \( T(n) \) = number of leaf nodes = \( O(a ^ D) \), where \( D = \log_b n \) is the depth of the tree. Can be simplified to \( O(n ^ {\log a / \log b}) \).
• At the top (top heavy): When \( a < b ^ k \). Here \( T(n) = O(n ^ k) \).
• Steady state: When \( a = b ^ k \). Here \( T(n) = O(n ^ k \log_b n ) \).

Example

Suppose you have \( n = 2^k - 1 \) elements that you want to put in a heap. It will be a complete binary tree of depth \( k \). Can recursively solve in terms of subtrees of the heap (drawing on board). The recurrence relation is \( T(n) = 2 T( n / 2) + O(\log n) \). The Master Theorem cannot be directly applied because \( \log n \) is not a polynomial of \( n \).

So you can use a direct approach: The cost in level 0 is \( \log n \). In level 1 it is \( 2 (\log (n / 2)) \) which is \( 2 (\log n - 1) \). Next level is \( 4 (\log n - 2) \). Generally this is \( 2 ^i (\log n - i) \). Sum it for \( i = 0 \cdots \log n \). If you examine the series you find it converges to something \( O(n) \).

Another approach does use the Master Theorem. Realize \( 1 \leq \log n \leq \sqrt{n} \). Both sides of the bound lead to bottom-heavy trees, so by the Master Theorem they are \( O (n ^ {\log a / \log b}) \) which turns out to be \( O(n) \). Since the complexity is bounded between two \( O(n) \) complexities, it must be \( O(n) \).

Integer multiplication (continued from yesterday)

We got a recurrence of the form \[ T(n) = 4 T (n / 2) + O(n). \] So we have a complexity of \( O(n ^ {\log 4 / \log 2 }) = O(n ^2) \). So we are in the bottom-heavy case and don't beat the technique we learned in grade school. To get a faster technique, by the Master Theorem, we can try:

• Making \( a\) smaller, which means making fewer recursive calls.
• Making \( b \) bigger, which means each subproblem should be smaller.

Making \( a \) smaller

Consider replacing \( 2 ^ { n / 2 } \) with an abstract variable \( z \). So \[ P_{xy}(z) = x_h y_h z ^ 2 + (x_h y_l + x_l y_h) z + x_l y_l. \] To compute the values of the above polynomial, we only need to know its values at 3 points. Evaluate at \( 0, 1, -1 \). \[ P_{xy}(0) = x_l y_l \] \[ P_{xy}(1) = (x_h + x_l)(y_h + y_l) \] \[ P_{xy}(-1) = (-x_h + x_l)(-y_h + y_l) \]

The above involves 3 recursive calls. Once you have the polynomial at the three points, you can calculate the high, middle, and low order terms using simple operations:

• \( M = (P_{xy}(1) - P_{xy}(-1)) / 2 \)
• \( L = P_{xy}(0) \)
• \( H = P_{xy}(1) - M - L \)

The final result is \( H 2 ^ n + M 2 ^ {n / 2} + L \).

In the above, the recurrence relation is \[ T(n) = 3 T(n / 2) + O(n). \] We're still in the bottom-heavy case, so the complexity is \( O(n ^ { \log_2 3 }) \).

Making \( b \) bigger

Split \( x \) and \( y \) into three parts instead of two. For example, \( x = x_h 2 ^ {2n / 3} + x_ 2 ^ {n / 3} + x_l \). So now you're multiplying two quadratics to get a quartic. Solving the polyomial involves solving 5 subproblems of size \( n / 3 \). From the Master Theorem, you something of \( O (n ^ {\log_3 5}) \), which is slightly better than the previous bound.

Keep going

We got a slightly better bound by breaking \( x \) and \( y \) into 3 pieces instead of 2. Let's keep going and imagine breaking it into \( t \) pieces. We'd need to evaluate the polynomial at \( 2 t - 1 \) points. The recurrence would look like \[ T(n) = (2t - 1) T(n / t) + O(n). \] The above less than \[ T(n) = 2t T(n / t) + O(n). \] The above has complexity \( O(n ^ {\log 2t / \log t}) = O(n ^ {1 + 1 / \log t}) \).

When \( t = n \)

The above looks like you could get a linear algorithm by setting \( t = n \). But this isn't true, because \( t \) is a hidden constant in the above, and setting \( t = n \) makes it no longer a constant and the overall complexity ends up being bad.

Consider the previous scheme, with \( p_x(z) = \sum_{i = 0}^{n-1} x_i z^i \) and similarly for \( y \). The product of the polynomials is a polynomial with degree \( 2n - 2 \). We can determine it with its value at \( 2n - 1 \) points. We'll be a bit overkill and evaluate it at \( 2n \) points.

We're running out of time, but we're going to pursue the idea that we can determine the polynomial by evaluating it at any points we want. We can choose these points cleverly to minimize the cost of evaluating at them.

Consider \( z = 1 \). Polynomial is \( x_0 + x_1 + x_2 + \cdots + x_{n-1} \). Consider \( z = -1 \). Polynomial is \( x_0 - x_1 + x_2 + \cdots - x_{n-1} \). We can quickly evalute the above two polynomials:

• Add up the even positions.
• Add up the odd positions.
• Their sum is the first polynomial and their difference is the second.

To make the above trick for many \( z \)s we go to the roots of unity (imaginary numbers).