Educational_Resources

Lecture 6: Big O

Created By: Yusuf Pisan

formatted to Github Markdown syntax by Ryan Peters

Be sure to check the other lectures out after you finish this one!

Previous lecture

Table of Contents

  1. Overview
  2. priority_queue - 1
  3. priority_queue - 2
  4. priority_queue - 3
  5. Improvements to Dijkstra
  6. STL dtatastructure for Dijkstra
  7. Dijkstra Applications
  8. Topological Sort
  9. Travelling Salesman Problem (TSP)
  10. xkcd: Travelling Salesman
  11. Spanning Trees
  12. Prim’s Algorithm
  13. Big O
  14. Big O (2)
  15. Big O (3)
  16. Big O (4)
  17. Big O (5)
  18. Big O (6)
  19. Big O (7)
  20. Big O (8)
  21. Big O (9)
  22. Big O (10)
  23. Big O (11)
  24. After Class

Overview

priority_queue - 1

void testPQ1() {
    priority_queue<int> pq;
    pq.push(27);
    pq.push(35);
    pq.push(5);
    // should get us 35 27 5
    while(!pq.empty()) {
        cout << pq.top() << " ";
        pq.pop();
    }
    cout << endl;
}

priority_queue - 2

void testPQ2() {
    // use greater rather than less,
    // sorted from smallest to largest
    priority_queue<string, vector<string>, greater<string>> pq;
    pq.push("apple");
    pq.push("orange");
    pq.push("banana");
    // should get us apple banana orange
    while(!pq.empty()) {
        cout << pq.top() << " ";
        pq.pop();
    }
    cout << endl;;
}

priority_queue - 3

// store a pair of objects in priority queue
typedef pair<string*, int> strPint;

// comparing strPint objects based on the int value
class StrPintCompare {
public:
    bool operator() (const strPint& lhs, const strPint&rhs) const {
        return lhs.second > rhs.second;
    }
};

void testPQ3() {
    priority_queue<strPint, vector<strPint>, StrPintCompare> pq;
    pq.push(make_pair(new string("apple"), 27));
    pq.push(make_pair(new string("orange"), 35));
    pq.push(make_pair(new string("banana"), 5));
    // should get us banana(5) apple(27) orange(35)
    // when StrPintCompare is >
    while(!pq.empty()) {
        string * strPtr = pq.top().first;
        cout << *strPtr << "(" << pq.top().second << ")" << " ";
        delete pq.top().first;
        pq.pop();
    }
    cout << endl;;
}

Improvements to Dijkstra

// finding shortest-paths from vertex 0
weight[v] = matrix[0][v] for all v
// weight[0] = "-", [1] = 8, [2] = -, [3] = 9, [4] = 4
add v (vertex 0) to vertexSet
while there are still vertices not in vertexset (do this n-2 times)
    Find v with smallest weight v
    Add v to vertexSet
    for all u not in vertexSet
        // if going through the newly selected v will make it shorter
        // set the new weight
        if (weight[u] > weight[v] + matrix[v][u])
            weight[u] = weight[v] + matrix[v][u])

STL dtatastructure for Dijkstra

// weight to node, use INT_MAX for not connected/infinity
// beware of overflow, don't att anything to INT_MAX
map<string, int> weight;     
// only one copy in vertexSet
set<string> vertexSet;
// pq of weights
priority_queue< pair<string, int>, vector<pair<string, int>>, LessWeight> pq;
// keeping path to vertex, best way to get to thsi vertex
map<string, string> previous

Dijkstra Applications

Looking for specific examples…

Topological Sort

Directed graph without cycles has a topological order.

x precedes y if there is a an edge from x to y

142 -> 143 -> 342 -> 343
143 -> 295
143 -> 340 -> 385
143 -> 342 -> 385
132 -> 133
142 -> 173
142 -> 305
142 -> 337
143 -> 340
301 -> 351
301 -> 370
342 -> 421
ENGL 182 -> 301

We cannot represent AND prerequisites (MATH 125 AND CSS 340)

We cannot represent “must be taken concurrently”

For scheduling, we want to generate a topological order

If all pairs of consecutive vertices in the sorted order are connected by edges, then the edges form a Hamiltonian path

Exercise: Draw a square without lifting pen, draw a house, …

Applications: optimizing triangle meshes in computer graphics, circuit design, package delivery problems, multiple vehicles servicing multiple customers, …

Hamiltonian Path: Visit each vertex

Euler Path: Travel each edge

Travelling Salesman Problem (TSP)

Given a set of cities and distance between every pair of cities, the problem is to find the shortest possible route that visits every city exactly once and returns to the starting point.

Paths: ABCDE, ABCED, ABDCE, ABDEC, .

Complexity: O(n!) based on (n - 1)! paths with a naive approach

100! would be 158 digits long, only 10^82^ atoms in universe

but we can do better with dynamic programming

For a set of size n, we consider n-2 subsets each of size n-1 such that all subsets don’t have nth in them.

There are at most O(n * 2^n^) subproblems, and each one takes linear time to solve

The total running time is therefore O(n^2^ * 2^n^)

https://www.geeksforgeeks.org/travelling-salesman-problem-set-1/

xkcd: Travelling Salesman

https://xkcd.com/399/

Spanning Trees

A tree is a psecial kind of undirected graph, connected but no cycles

Depth-First search spanning tree: Traverse the tree marking edges. When DFS ternminates, the marked edges make up a spanning tree

Breadth-First search spanning tree: Traverse the tree marking edges. When BFS ternminates, the marked edges make up a spanning tree

Minimum spanning tree: A spanning tree that where the sum of the edges is minimal - Ethernet line to all the outlets in your home with minimal cabling

Group Exercise: Construct DFS, BFS and minimum spanning tree (starting from e for DFS and BFS)

Prim’s Algorithm

Start at any vertex, r
mark r as visited
while (there are unvisited vertices)
    find the least-cost edge (v, u) from some visited vertex v to u
    mark u as visited
    Add the vertex u and the edge (v, u) to minimum spanning tree

Big O

O(n) represents upper bound. (Big-O)
Ω(n) represents lower bound. (Omega)
Θ(n) means tight bound. (Theta)

28 Examples from “Cracking The Coding Interview”

Big O (2)

How many questions will be asked?

Big O (3)

Two loops in a row:
for (i = 0; i < N; i++) {
    sequence of statements
}
for (j = 0; j < M; j++) {
    sequence of statements
}

A nested loop followed by a non-nested loop:
for (i = 0; i < N; i++) {
    for (j = 0; j < N; j++) {
        sequence of statements
    }
}
for (k = 0; k < N; k++) {
    sequence of statements
}
A nested loop in which the number of times the inner loop executes depends on the value of the outer loop index:
for (i = 0; i < N; i++) {
    for (j = N; j > i; j--) {
        sequence of statements
    }
}

// nested loop is n*n
for	(int	i	=	0;	i	<	n;	i++)	{
		for	(int	j	=	0;	j	<	n*n;	j++)	{
				cout	<<	“tricky!”	<<	endl;
		}
}

Big O (4)

f takes constant time, g takes time linear based on the value of its parameter.

Big O (5)

Dynamicc programming stores and reuses results

RE-writing fibonacci, so it is not recurive.

What is the complexity?

Big O (6)

Knapsack problem: Objects with given size and value, maximize value taht can be stored in capacity C

Big O (7)

Big O (8)

Shortest Path: Djikstra’s algorithm to find shortest path for all nodes

To find the shortest path from one vertex to all other vertices

// finding shortest-paths from vertex 0
weight[v] = matrix[0][v] for all v
// weight[0] = "-", [1] = 8, [2] = -, [3] = 9, [4] = 4
add v (vertex 0) to vertexSet
while there are still vertices not in vertexset (do this n-2 times)
    Find v with smallest weight v
    Add v to vertexSet
    for all u not in vertexSet
        // if going through the newly selected v will make it shorter
        // set the new weight
        if (weight[u] > weight[v] + matrix[v][u])
            weight[u] = weight[v] + matrix[v][u])

Big O (9)

Sort each string in an array and then sort the array (Example 8 from Cracking)

Big O (10)

Binary Search Tree: add, remove, search, traverse

Big O (11)

Binary Heap: add, remove, search, traverse

After Class