fullscreen
timer
qrcode
plickers
selector
audio
reset

Balanced Search Trees

COS 265 - Data Structures & Algorithms

audio_capture Nov 9 Nov 11 Nov 13

symbol table review

implementation search🞰 insert🞰 delete🞰 search† insert† delete† ordered ops on keys
seq search \(N\) \(N\) \(N\) \(N\) \(N\) \(N\) equals()
binary search \(\log N\) \(N\) \(N\) \(\log N\) \(N\) \(N\) ✓ compareTo()
BST \(N\) \(N\) \(N\) \(\log N\) \(\log N\) \(\sqrtN\) ✓ compareTo()
goal \(\log N\) \(\log N\) \(\log N\) \(\log N\) \(\log N\) \(\log N\) ✓ compareTo()

🞰guarantee, †average


Challenge: Guarantee performance

This lecture: 2-3 trees, left-leaning red-black BSTs, B-trees

balanced search trees

2-3 search trees

binary search trees

Binary search trees


Ordered, but cannot guarantee \(\log\) height unless we make one small tweak

2-3 tree

Search tree, but allow 1 or 2 keys per node

Symmetric order: Inorder traversal yields keys in ascending order

Perfect balance: Every path from root to null link has same length (how to maintain?)

2-3 tree demo

Search

2-3 tree: insertion

Insertion always goes into node at bottom of tree

Two cases: insert into a 2-node at bottom

2-3 tree: insertion

Insertion always goes into node at bottom of tree

Two cases: insert into a 3-node at bottom

2-3 tree: global properties

Invariants: Maintains symmetric order and perfect balance

Pf: Each transformation maintains symmetric order and perfect balance

2-3 tree: performance

Splitting a 4-node is a local transformation: constant number of operations

quiz: balanced search trees

What is the range of heights of a 2-3 tree with \(N\) keys?


    best case   worst case
A. \(\sim \log_4 N\) \(\sim \log_3 N\)
B. \(\sim \log_3 N\) \(\sim \log_2 N\)
C. \(\sim \log_3 N\) \(\sim 2 \log_2 N\)
D. \(\sim \log_3 N\) \(\sim N\)

2-3 tree: performance

Perfect balance: Every path from root to null link has same length

Tree height:

Bottom line: Guaranteed logarithmic performance for search and insert

ST implementation: summary

implementation search🞰 insert🞰 delete🞰 search† insert† delete† ordered ops on keys
seq search \(N\) \(N\) \(N\) \(N\) \(N\) \(N\) equals()
binary search \(\log N\) \(N\) \(N\) \(\log N\) \(N\) \(N\) ✓ compareTo()
BST \(N\) \(N\) \(N\) \(\log N\) \(\log N\) \(\sqrtN\) ✓ compareTo()
2-3 tree\(^\ddagger\) \(\log N\) \(\log N\) \(\log N\) \(\log N\) \(\log N\) \(\log N\) ✓ compareTo()

🞰guarantee, †average

\(^\ddagger\)but hidden constant \(c\) is large (depends upon implementation)

challenge: delete in 2-3 tree





Challenge: Delete from 2-3 tree while maintaining perfect balance?

2-3 tree: implementation??

Direct implementation is complicated, because

// fantasy code
public void put(Key key, Value val) {
    Node x = root;
    while(x.getTheCorrectChild(key) != null) {
        x = x.getTheCorrectChildKey();
        if(x.is4Node()) x.split();
    }
    if     (x.is2Node()) x.make3Node(key, val);
    else if(x.is3Node()) x.make4Node(key, val);
}

Bottom line: Could do it, but there's a better way

balanced search trees

red-black BSTs

implementing 2-3 trees as binary trees

Challenge: How to represent a 3-node as binary tree?






implementing 2-3 trees as binary trees

Challenge: How to represent a 3-node as binary tree?


Approach 1: Regular BST

implementing 2-3 trees as binary trees

Challenge: How to represent a 3-node as binary tree?


Approach 2: Regular BST with red "glue" nodes

implementing 2-3 trees as binary trees

Challenge: How to represent a 3-node as binary tree?


Approach 3: Regular BST with red "glue" links

left-leaning red-black BSTs

Guibas-Sedgewick 1979 and Sedgewick 2007

LLRB

llrb BSTs: 1-1 correspondence with 2-3 trees

A 2-3 tree and corresponding red-black BST

Key property: 1-1 correspondence between 2-3 and LLRB

an equivalent definition

A LLRB tree is a BST such that

search implementation for red-black BSTs

Observation: Search is the same as for elementary BST (ignore color), but runs faster because of better balance

public Value get(Key key) {
    Node x = root;
    while(x != null) {
        int cmp = key.compareTo(x.key);
        if     (cmp < 0) x = x.left;
        else if(cmp > 0) x = x.right;
        else             return x.val;
    }
    return null;
}

Remark: Most other ops (e.g., floor, iteration, selection) are also identical

red-black bst representation

Each node is pointed to by precisely one link (from its parent); can encode color of links in nodes

private static final boolean RED   = true;
private static final boolean BLACK = false;

private class Node {
    Key key;
    Value val;
    Node left, right;
    boolean color;  // color of parent link
}

private boolean isRed(Node x) {
    if(x == null) return false; // null links are black
    return x.color == RED;
}

red-black bst representation

root.left.color  == RED
root.right.color == BLACK

insertion into a LLRB tree: overview

Basic strategy: Maintain 1-1 correspondence with 2-3 trees

During internal operations, maintain:

How? Apply elementary red-black BST operations: rotation and color flip

elementary red-black BST operations

Left rotation (↺)
Orient a (temporarily) right-leaning red link to lean left
private node rotateLeft(Node h) {
    assert isRed(h.right);
    Node x = h.right;
    h.right = x.left;
    x.left = h;
    x.color = h.color;
    h.color = RED;
    return x;
}

Invariants: Maintains symmetric order and perfect black balance

elementary red-black BST operations

Right rotation (↻)
Orient a left-leaning red link to (temporarily) lean right
private node rotateRight(Node h) {
    assert isRed(h.left);
    Node x = h.left;
    h.left = x.right;
    x.right = h;
    x.color = h.color;
    h.color = RED;
    return x;
}

Invariants: Maintains symmetric order and perfect black balance

elementary red-black BST operations

Color flip (⇵)
Recolor to split a (temporary) 4-node
private void flipColors(Node h) {
    assert !isRed(h);
    assert isRed(h.left);
    assert isRed(h.right);
    h.color = RED;
    h.left.color = BLACK;
    h.right.color = BLACK;
}

Invariants: Maintains symmetric order and perfect black balance

insertion into a LLRB tree

Warmup: Insert into a 2-node at the bottom of tree with single node

  • search ends at left null link of root
  • red link to new node containing A converts 2-node to 3-node
  • search ends at right null link of root
  • red link to new node containing B (right-leaning)
  • rotate left to make a legal (left-leaning) 3-node

Note

Similar problem / fewer variables FTW!

insertion into a LLRB tree

General Case 1: Insert into a 2-node at the bottom of LLRB BST

insertion into a LLRB tree

Warmup: Insert into a 3-node at bottom of tree with only 2 nodes



  • search ends at right null link of root
  • attached new node with red link
  • colors flipped to black
  • search ends at leftmost null link
  • attached new node with red link
  • rotated right
  • colors flipped to black
  • search ends at left-right null link
  • attached new node with red link
  • rotated left
  • rotated right
  • colors flipped to black

insertion into a LLRB tree

General Case 2: Insert into a 3-node at the bottom

llrb bst insertion demo

Insert H into LLRB BST

  1. add new node left of R
    • red edge!
  1. add new node left of R
    • red edge!
  2. S has two red lefts in a row
    • rotate S right (↻)
  1. add new node left of R
    • red edge!
  2. S has two red lefts in a row
    • rotate S right (↻)
  3. R has two red children
    • flip R colors (⇵)
  1. add new node left of R
    • red edge!
  2. S has two red lefts in a row
    • rotate S right (↻)
  3. R has two red children
    • flip R colors (⇵)
  4. E has red right
    • rotate E left (↺)
  1. add new node left of R
    • red edge!
  2. S has two red lefts in a row
    • rotate S right (↻)
  3. R has two red children
    • flip R colors (⇵)
  4. E has red right
    • rotate E left (↺)
  5. now LLRB, so done!

llrb bst construction demo

insert(S)
insert(E)
insert(A)
insert(R)
insert(C)
insert(H)
insert(X)
insert(M)
insert(P)
insert(L)

insertion into LLRB: java implementation

Same code for all cases

private Node put(Node h, Key key, Value val) {
    if(h == null) {
        // insert at bottom and color it red
        return new Node(key, val, RED);
    }

    int cmp = key.compareTo(h.key);
    if     (cmp < 0) h.left = put(h.left, key, val);
    else if(cmp > 0) h.right = put(h.right, key, val);
    else             h.val = val;

    // only a few extra LoC provides near-perfect balance
    if(isRed(h.right) && !isRed(h.left))     h = rotateLeft(h);   // lean left
    if(isRed(h.left)  && isRed(h.left.left)) h = rotateRight(h);  // balance 4-node
    if(isRed(h.left)  && isred(h.right))     flipColors(h);       // split 4-node

    return h;
}

insertion into LLRB: visualization

255 insertions in ascending order

insertion into LLRB: visualization

255 insertions in descending order

insertion into LLRB: visualization

255 random insertions

quiz: balanced search trees

What is the height of an LLRB tree with \(N\) keys in the worst case?

  1. \(\sim \log_3 N\)

  2. \(\sim \log_2 N\)

  3. \(\sim 2 \log_2 N\)

  4. \(\sim N\)

balance in LLRB trees

Proposition: Height of tree is \(\leq 2 \lg N\) in the worst case

Pf:

Property: Height of tree is \(\sim 1.0 \lg N\) in typical applications

ST implementation: summary

implementation search🞰 insert🞰 delete🞰 search† insert† delete† ordered ops on keys
seq search \(N\) \(N\) \(N\) \(N\) \(N\) \(N\) equals()
binary search \(\log N\) \(N\) \(N\) \(\log N\) \(N\) \(N\) ✓ compareTo()
BST \(N\) \(N\) \(N\) \(\log N\) \(\log N\) \(\sqrtN\) ✓ compareTo()
2-3 tree‡ \(\log N\) \(\log N\) \(\log N\) \(\log N\) \(\log N\) \(\log N\) ✓ compareTo()
LLRB🞶 \(\log N\) \(\log N\) \(\log N\) \(\log N\) \(\log N\) \(\log N\) ✓ compareTo()

🞰guarantee, †average

‡hidden constant \(c\) is large (depends upon implementation)

🞶hidden constant \(c\) is small (at most \(2 \lg N\) compares)

war story: why red-black?

Xerox PARC innovations (1970s)

  • Alto
  • GUI
  • Ethernet
  • Smalltalk
  • InterPress
  • Laser printing
  • Bitmapped display
  • WYSIWYG text editor
  • ...

war story: red-black BSTs

Telephone company contracted with database provider to build real-time database to store customer information

Database implementation

war story: red-black BSTs

Telephone company contracted with database provider to build real-time database to store customer information

Extended telephone service outage

“

If implemented properly, the height of a red-black BST with \(N\) keys is at most \(2 \lg N\).
—expert witness

”

balanced search trees

B-trees

file system model

Page
contiguous block of data (e.g., a 4096-byte chunk)
Probe
first access to a page (e.g., from disk to memory)
slow
slow
fast
fast

Property: time required for a probe is much larger than time to access data within a page

Cost model: number of probes

Goal: access data using minimum number of probes

b-trees (bayer-mccreight, 1972)

B-tree: Generalize 2-3 trees by allowing up to \(M\) keys per node

search in a b-tree

Insertion in a B-tree

balance in b-tree

Proposition: A search or an insertion in a B-tree of order \(M\) with \(N\) keys requires between \(\sim \log_M N\) and \(\sim \log_{M/2} N\) probes.

Pf: All nodes (except possibly root) have between \(\left\lfloor M/2 \right\rfloor\) and \(M\) keys

In practice: Number of probes is at most \(4\) (when \(M=1024\), \(N = 62 \text{ billion}\), then \(\log_{M/2} N \leq 4\))

quiz 3: balanced search trees

Which of the following does the B in B-tree not mean?

  1. Bayer

  2. Balanced

  3. Binary

  4. Boeing

quiz 3: balanced search trees

Which of the following does the B in B-tree not mean?

  1. Bayer

B. Balanced

C. Binary

D. Boeing

E. Broad

F. Bushy

“

the more you think about what the B in B-trees could mean, the more you learn about B-trees and that is good.
–Ed McCreight

”



Note

Rudolf Bayer was working at Boeing Research Labs when he and Ed McCreight invented B-Trees

balanced trees in the wild

Red-Black trees are widely used as system symbol tables

B-tree cousins: B+ tree, B*tree, B# tree, ...

B-trees (and cousins) are widely used for file systems and DBs

×
start stop ×
delete_forever
undo
redo
gradient
close
draw
ink_eraser
edit_line
circle
rectangle
north_east
text_fields