Greedy Algorithm-Dijkstra Optimization

introduction

The seventh line of the pseudo-code of Dijkstra algorithm: After each refresh of d(v), find the smallest d(v). It is not said what method is used in this place. Different methods are used to find the smallest method and involved. The time complexity of the data structure is also different.
The most direct and violent idea is to traverse a for loop to find the smallest d(v), but the time complexity is too high and all points need to be traversed. The total time complexity is $O(n^2)\; When$there are a lot of points, this algorithm is very slow.
Therefore, we need to optimize how to find the minimum value.

Dijkstra optimization

Let's take a look at what the data structure for storing d(v) needs to support. In the Dijkstra algorithm, it is necessary to support the ability to insert a value, extract the minimum value, upadte a value, and delete a value, and it is necessary to dynamically adjust the order of changes.

Priority Queue (PQ) can meet the above requirements.

Common priority queue application scenarios:
Dijkstra's shortest path algorithm
Prim's MST algorithm
Huffman coding
A∗ searching algorithm
HeapSort
…

The priority queue is used to store the d(v) value, that is, the PQ set (the priority is the d(v) value corresponding to the node, and then the key(v) is used uniformly). At the beginning, the insertion operation of the queue is executed n times to obtain the minimum Key(v) needs n times in the loop, and it takes m times to update key(v) in the queue (m is the number of edges). After key(v) is updated, the queue may need to be adjusted. (ExtractMin includes two operations: taking the minimum value and deleting from the queue)

The pseudo code is as follows:

Through analysis, we found that the main impact on the time complexity of the algorithm is the three operations of the queue, Insert, ExtractMin, and DecreaseKey. If we use an ordinary unordered array or linked list to form the priority queue, then each Insert operation The complexity is only O(1), but ExtractMin needs to be O(n) (each time you have to traverse the array, linked list), if it is an ordered array or linked list, Insert needs O(n) (traverse to find the appropriate insertion position), ExtractMin only uses O(1). Therefore, choosing different data structures to construct the queue will result in different algorithm time complexity. The following table lists four different data structure methods for implementing queues:

In general practical problems, the number of edges m>the number of points n

Minimal heap (binary heap)

The minimum heap is that the value of any node is always less than or equal to the value of its child nodes, and the heap must be a complete binary tree. Realization can be done in two ways: linked list or array.

Let's focus on the three operations involved in Dijkstra's algorithm. Why does the minimum heap reduce the time complexity of these three operations?

• Insert: Inserting an element into the heap includes 2 steps. The first is inserting (connecting the element to the end of the linked list or the end of the array), and the second is maintenance (adjusting the position of the element to form a new heap). The adjustment process can be found to be related to the number of comparison exchanges. If the newly inserted node is larger than the parent node, it does not need to be exchanged. If it is smaller than the parent node, then the two positions are exchanged. During the process, only the parent node needs to be compared, because if it is less than The parent node must be smaller than the other child node. It has been compared for at most logn times, so the complexity of Insert is O(logn).

Example: Insert (7)

• ExtractMin: As mentioned earlier, this involves two steps: one is to return to the top of the heap (that is, the smallest element), the other is to delete the top element of the heap (add it to the set of traversed points), and the complexity of returning directly to the top of the heap is O(1 ), the focus is on the time it takes to maintain the heap after deleting the top element of the heap. If we directly delete the top element of the heap, and then select the smaller one from its left and right child nodes as the parent node, the heap may not be formed in the end, which means that we have to reconstruct the heap once, but if we exchange the top and tail elements first, Then delete, you can keep the heap.

Example: ExtractMin()

• DecreaseKey: Update the Key value, compare a node, if the Key value is updated, it is similar to the insertion operation, compared with the parent node, the time complexity is still O(logn).

Example: DecreaseKey( ptr, 7)

Taken together, the time complexity of using the smallest heap to implement a priority queue is:
$$O(n\ast logn(ntimesInsert)+n\ast logn(ntimesExtractMin)+m\ast logn(m次DecreaseKey))=O\left(mlogn\right)$$

Binomial heap (binomial tree)

Binomial heap optimizes the time spent on merging heaps (from O(n) to O(logn)) based on the binary heap.
Binary heaps are stored in a single tree, while multiple trees are used in binary heaps.

Definition of Binomial Tree:

• A single node can be used as a binomial heap
• A binomial heap must consist of two binomial trees of the same size in accordance with the rules (connect the smaller root of the tree under the root node of the other tree)

Example: The picture below shows multiple binomial trees. They look crooked, but they are actually quite straight. When we look at each tree in order, it is binomial and Yanghui triangle, which is why it is called a binomial tree.
1
1 1
1 2 1
1 3 3 1

The binomial heap is composed of multiple binomial trees (forest), and the binomial tree containing only one node is called $B_0$, Put two $B_0$The combined binomial tree is called $B_1$, And so on. Connecting the root nodes of each binomial tree with a linked list is a binomial heap. (B is the level)

There cannot be two trees of the same level in a heap. If two trees of the same level appear in a heap, they will be merged. Therefore, if there are n trees in a binomial heap, there will be at most log 2 n + 1 log_2 n+1 after merging.$lo{g}_{2}n+1$ tree, the height of the highest tree is$lo{g}_{2}n$。

Let's briefly analyze the time complexity of the binomial heap:

• Union
  • No trees of the same size: put them together
  • There are trees of the same size: merge the same size until there is no the same size, two binomial trees become one cost O(1), and the total number of trees to merge the forest does not exceed O(logn)
• Insert
  • If there is no B 0 B_0 in the original binomial pile$B_0$, Put it directly
  • If yes , two $B_0$Composition $B_1$ , Continue to merge if there are still, cost O(logn)
• Extract Min
  • Get the minimum first, compare the root nodes from no more than logn trees, O(logn), after removing the roots, a binomial tree will be split into two binomial trees, which may be merged with other trees. The merge is still O(logn). ), total time O(logn)

It can be found that the binomial heap reduces the time complexity of the Union operation compared to the binary heap.

Binomial heap Insert and Union time is logn, in fact, we can further confirm that its time complexity can reach O(1).
Let's look at an example. After multiple merges took O(logn) in the last time, the next time it only takes O(1).

Assuming that it costs O(logn) at a certain time, the time consumption of continuously inserting elements after that is obviously O(1), O(2), O(1), O(3), O(1)... We found that, For a long period of time, the time consumed for insertion is constant. Therefore, we cannot simply assert that the time complexity is O(logn) based on a time consumption of O(logn) alone. We need to consider the overall expected time complexity of a series of operations. It can be proved that it is O(1) through amortized analysis.

to sum up

Speaking of priority queues, two new data structures have been introduced: Binary Heap and Binary Heap. Their design is another optimization in our algorithm design process, which optimizes the algorithm from the data structure.

• Binary heap
  can quickly obtain the minimum value of the queue, and has fast insertion and deletion speed
• Binary heap
  On the basis of binary heap, it reduces the time consumed by heap merge and insert operations

From the beginning, we used a linear linked list to implement a priority queue, and found that the minimum was too slow, changed to an ordered linked list, found that the insertion was too slow, and then became partially ordered, using a binary heap, and then changed from using only one tree to multiple The binomial heap of the tree gradually relaxes the restriction conditions, reduces many redundant calculations, and optimizes the problem.