From backtracking:

• "Exhaustive case analysis"
  • Prune for: violating constraints, non-optimality, redundancy
• Improved exponential time alorithms for many hard optimization / search problems.

To dynamic programming (DP):

• Recursive procedures can make identical calls to sub-procedures.
• Store for reuse, don't recompute (memoization)
• Bottom-up order

Comment by Russell: Empirically, the most useful algorithm types for research are network flow and dynamic programming.

Graph 3-coloring

• Instance: \( G \), an undirected graph.
• Solution: \( X: V \rightarrow \{R, G, B\} \)
• Constraint: \( (u, v) \in E \Rightarrow X(u) \neq X(v) \)
• Objective: Decide if a solution exists.

Russell then worked an example with a straightforward backtracking technique, representing the node coloring process as the depth-first exploration of an exponentially large tree.

def L3C(G, L):
  if V not empty: return True
  if there exists u such that L(U) is empty: return False
  if |V| = 1: return True
  Pick x in V of smallest list size
  If size(L(x)) = 1: color this node the only way you can and update the other lists correspondingly
  Otherwise branch in the obvious way.

Recurrence is: \( T(n) = 2 T(n-1) + poly \). This implies \( O(2^n) \).

Interval scheduling with arbitrary profits and a single room

• Instance: \( I_i = (s_i, f_i, p_i) \)
• Solution: Set of non-intersecting intervals \( S \subseteq \{1, ..., n\} \)
• Objective: Maximize \( \sum_{i \in S} p_i \).

Assume events are sorted by start time.

def BTIS( I[1 ... n]):
  if n == 1: return p1
  if n == 0: return 0
  P0 <- BTIS(I[2 ... n]) // Don't schedule I1
  J <- 2, while sJ C F1 do J++ // Remove conflicting intervals
  P1 <- P1 + BTIS( I[J ... n])
  return max(P0, P1)

Recurrence: \( T(n) \leq 2 T(n - 1) + poly \). This is \( O(2 ^ n) \), though there are memoization opportunities.

Russell next showed how you can solve the above problem efficiently using memoization, memorizing the maximum profits you can get for I[1 ... n], I[2 ... n], etc. It simplifies the analysis if you order your computation such that you have always pre-computed the profits for your subproblems.