Warm up...

Fibonacci

What's the complexity? (in terms of n)

    def fibb(n: Int): Long =
      if(n < 2){ 1 }
      else { fibb(n-1) + fibb(n-2) }
  

Fibonacci

Test Hypothesis: $T(n) \in O(2^n)$

Divide and Conquer

Remember the Towers of Hanoi...

1. You can move $n$ blocks, if you know how to move $n-1$ blocks
2. You can move $n-1$ blocks, if you know how to move $n-2$ blocks
3. You can move $n-2$ blocks, if you know how to move $n-3$ blocks
4. You can move $n-3$ blocks, if you know how to move $n-4$ blocks

...

• You can always move $1$ block

Divide and Conquer

To solve the problem at $n$:

• Divide the problem into a problem at $n-1$
• Conquer the size $n-1$ problems
• Combine the size $n-1$ solutions

Merge Sort

Input: An array with elements in unknown order.

Output: An array with elements in sorted order.

Merge Sort

Merge Sort

Observation: Merging two sorted arrays can be done in $O(n)$.

Merge Sort

  def merge[A: Ordering](left: Seq[A], right: Seq[A]): Seq[A] = {
    val output = ArrayBuffer[A]()
  
    val leftItems = left.iterator.buffered
    val rightItems = right.iterator.buffered
  
    while(leftItems.hasNext || rightItems.hasNext) {
      if(!left.hasNext)          { output.append(right.next) }
      else if(!right.hasNext)    { output.append(left.next) }
      else if(Ordering[A].lt( left.head, right.head ))
                                 { output.append(left.next) }
      else                       { output.append(right.next) }
    }
    output.toSeq
  }
  

Merge Sort

Each time though loop advances either left or right.

Total Runtime: $\Theta(|\texttt{left}| + |\texttt{right}|)$

Merge Sort

Observation: Merging two sorted arrays can be done in $O(n)$.

Idea: Split the input in half, sort each half, and merge.

Merge Sort

Merge Sort

Merge Sort

Divide and Conquer

To solve the problem at $n$:

• Divide the problem into two problems of size $\frac{n}{2}$
• Conquer the size $\frac{n}{2}$ problems
• Combine the size $\frac{n}{2}$ solutions

Merge Sort

1. If the sequence has $1$ or $0$ values: Sorted!
2. If the sequence has $n > 1$ values:
   1. Sort values $1$ to $\left\lfloor\frac{n}{2}\right\rfloor-1$
   2. Sort values $\left\lfloor\frac{n}{2}\right\rfloor$ to $n$
   3. Merge sorted halves

Merge Sort

    def sort[A: Ordering](data: Seq[A]): Seq[A] = 
    {
      if(data.length <= 1) { return data }
      else {
        val (left, right) = data.splitAt(data.length / 2)
        return merge(
          sort(left),
          sort(right)
        )
      }
    }
  

Divide and Conquer

If we solve a problem of size $n$ by:

• Dividing it into $a$ sub-problems
• ...where each problem is of size $\frac{n}{b}$ (usually $b=a$)
• ...and stop recurring at $n \leq c$
• ...and the cost of dividing is $D(n)$
• ...and the cost of combining is $C(n)$

The total cost will be: $$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq c \\ a\cdot T(\frac{n}{b}) + D(n) + C(n) & \textbf{otherwise} \end{cases}$$

Merge Sort

Divide: Split the sequence in half

$D(n) = \Theta(n)$ (can do in $\Theta(1)$)

Conquer: Sort left and right halves

$a = 2$, $b = 2$, $c = 1$

Combine: Merge halves together

$C(n) = \Theta(n)$

Merge Sort

How can we find a closed-form hypothesis?

Idea: Draw out the cost of each level of recursion.

Merge Sort: Recursion Tree

Each node of the tree shows $D(n) + C(n)$

Merge Sort: Recursion Tree

At level $i$ there are $2^i$ tasks, each with runtime $\Theta(\frac{n}{2^i})$,
and there are $\log(n)$ levels.

Merge Sort: Recursion Tree

At level $i$ there are $2^i$ tasks, each with runtime $\Theta(\frac{n}{2^i})$,
and there are $\log(n)$ levels.

$\sum_{i=0}^{\log(n)}$ $\sum_{j=1}^{2^i}$ $\Theta(\frac{n}{2^i})$

Merge Sort: Recursion Tree

Merge Sort: Proof By Induction

Now use induction to prove that there is a $c, n_0$
such that $T(n) \leq c \cdot n\log(n)$ for any $n > n_0$

Merge Sort: Proof By Induction

Base Case: $T(1) \leq c \cdot 1$

$$c_0 \leq c$$

True for any $c > c_0$

Merge Sort: Proof By Induction

Assume: $T(\frac{n}{2}) \leq c \frac{n}{2} \log\left(\frac{n}{2}\right)$

Show: $T(n) \leq c n \log\left(n\right)$

Next time...

• Quick Sort
• Average Runtime