Ten Classic Sorting Algorithms Analysis and Code Implementation

The sorting algorithm is one of the most basic algorithms. The sorting algorithm can be divided into internal sorting and external sorting . The internal sorting is to sort the data records in memory, while the external sorting is because the sorted data is very large and cannot accommodate all the sorting records at a time. , need to access external memory during the sorting process. Common internal sorting algorithms are: insertion sort, Hill sort, selection sort, bubble sort, merge sort, quick sort, heap sort, radix sort, etc.

Explanation of terms :

n : data size
k : the number of "buckets"
In-place : occupies constant memory, does not occupy additional memory
Out-place : takes up extra memory
Stability : the order of 2 equal key values after sorting is the same as their order before sorting

Regarding time complexity :

Square order (O(n2)) sorting - all kinds of simple sorting: direct insertion, direct selection and bubble sorting
Linear-logarithmic order (O(nlog2n)) sorting - quicksort, heapsort, and mergesort
O(n1+§)) sorting, where § is a constant between 0 and 1 - Hill sorting
Linear order (O(n)) sorting - radix sorting, in addition to bucket and box sorting

Regarding stability :

Stable sorting algorithms: bubble sort, insertion sort, merge sort, and radix sort
Not stable sorting algorithms: selection sort, quick sort, Hill sort, heap sort

1. Bubble sort

Bubble Sort (Bubble Sort) is also a simple and intuitive sorting algorithm. It iteratively walks through the array to be sorted, comparing two elements at a time, and swapping them if they are in the wrong order. The work of visiting the sequence is repeated until there is no need to exchange, that is to say, the sequence has been sorted. The name of this algorithm comes from the fact that the smaller elements will slowly "float" to the top of the sequence through exchange.

1.1 Algorithm steps

Compare adjacent elements. If the first is greater than the second, swap them both
Do the same for each pair of adjacent elements, from the first pair at the beginning to the last pair at the end. After this step is done, the last element will be the largest number
Repeat the above steps for all elements except the last
Continue repeating the above steps for fewer and fewer elements each time until there is no pair of numbers to compare

When is the fastest?

When the input data is already in positive order (it is already in positive order, what is the use of bubble sorting)

When is the slowest?

When the input data is in reverse order (it’s enough to write a for loop to output data in reverse order, why use your bubble sort, are you free)

1.2 Code implementation

Python：

def bubbleSort(arr):
    for i in range(1, len(arr)):
        for j in range(0, len(arr)-i):
            if arr[j] > arr[j+1]:
                arr[j], arr[j + 1] = arr[j + 1], arr[j]
    return arr

Java：

public class BubbleSort implements IArraySort {
    @Override
    public int[] sort(int[] sourceArray) throws Exception {
        // 对 arr 进行拷贝，不改变参数内容
        int[] arr = Arrays.copyOf(sourceArray, sourceArray.length);
        for (int i = 1; i < arr.length; i++) {
            // 设定一个标记，若为true，则表示此次循环没有进行交换，也就是待排序列已经有序，排序已经完成
            boolean flag = true;
            for (int j = 0; j < arr.length - i; j++) {
                if (arr[j] > arr[j + 1]) {
                    int tmp = arr[j];
                    arr[j] = arr[j + 1];
                    arr[j + 1] = tmp;
                    flag = false;
                }
            }
            if (flag) {
                break;
            }
        }
        return arr;
    }
}

2. Selection sort

Selection sorting is a simple and intuitive sorting algorithm. No matter what data enters, it has a time complexity of O(n²), so when using it, the smaller the data size, the better. The only advantage is that it does not take up additional memory space .

2.1 Algorithm steps

First find the smallest (largest) element in the unsorted sequence and store it at the beginning of the sorted sequence
Then continue to find the smallest (largest) element from the remaining unsorted elements, and then put it at the end of the sorted sequence
Repeat the second step until all elements are sorted

2.2 Code implementation

Python：

def selectionSort(arr):
    for i in range(len(arr) - 1):
        # 记录最小数的索引
        minIndex = i
        for j in range(i + 1, len(arr)):
            if arr[j] < arr[minIndex]:
                minIndex = j
        # i 不是最小数时，将 i 和最小数进行交换
        if i != minIndex:
            arr[i], arr[minIndex] = arr[minIndex], arr[i]
    return arr

Java：

public class SelectionSort implements IArraySort {
    @Override
    public int[] sort(int[] sourceArray) throws Exception {
        int[] arr = Arrays.copyOf(sourceArray, sourceArray.length);
        for (int i = 0; i < arr.length - 1; i++) {     // 总共要经过 N-1 轮比较
            int min = i;
            for (int j = i + 1; j < arr.length; j++) { // 每轮需要比较的次数 N-i
                if (arr[j] < arr[min]) {
                    min = j;// 记录目前能找到的最小值元素的下标
                }
            }
            if (i != min) { // 将找到的最小值和i位置所在的值进行交换
                int tmp = arr[i];
                arr[i] = arr[min];
                arr[min] = tmp;
            }
        }
        return arr;
    }
}

3. Insertion sort

Insertion sorting is the most simple and intuitive sorting algorithm. It works by constructing an ordered sequence. For unsorted data, scan from the back to the front in the sorted sequence, find the corresponding position and insert it.

3.1 Algorithm steps

Treat the first element of the first sequence to be sorted as an ordered sequence, and treat the second element to the last element as an unsorted sequence
Scan the unsorted sequence from the beginning to the end, insert each scanned element into the appropriate position of the ordered sequence (if the element to be inserted is equal to an element in the ordered sequence, insert the element to be inserted into the equal element later)

3.2 Code implementation

Python：

def insertionSort(arr):
    for i in range(len(arr)):
        preIndex = i-1
        current = arr[i]
        while preIndex >= 0 and arr[preIndex] > current:
            arr[preIndex+1] = arr[preIndex]
            preIndex-=1
        arr[preIndex+1] = current
    return arr

Java：

public class InsertSort implements IArraySort {
    @Override
    public int[] sort(int[] sourceArray) throws Exception {
        // 对 arr 进行拷贝，不改变参数内容
        int[] arr = Arrays.copyOf(sourceArray, sourceArray.length);
        // 从下标为1的元素开始选择合适的位置插入，因为下标为0的只有一个元素，默认是有序的
        for (int i = 1; i < arr.length; i++) {
            int tmp = arr[i]; // 记录要插入的数据
            int j = i; // 从已经排序的序列最右边的开始比较，找到比其小的数
            while (j > 0 && tmp < arr[j - 1]) {
                arr[j] = arr[j - 1];
                j--;
            }
            if (j != i) { // 存在比其小的数，插入
                arr[j] = tmp;
            }
        }
        return arr;
    }
}

4. Hill sort

Hill sorting, also known as decremental incremental sorting algorithm, is a more efficient improved version of insertion sorting , but Hill sorting is an unstable sorting algorithm. Hill sorting is an improved method based on the following two properties of insertion sorting:

Insertion sorting is efficient when operating on almost sorted data, that is, it can achieve the efficiency of linear sorting
But insertion sorting is generally inefficient because insertion sorting can only move data one bit at a time

The basic idea of Hill sorting is: first divide the entire record sequence to be sorted into several subsequences for direct insertion sorting, and when the records in the entire sequence are "basically in order", then perform direct insertion sorting for all records in turn.

4.1 Algorithm steps

Choose an incremental sequence t1, t2, ..., tk, where ti > tj, tk = 1;
According to the incremental sequence number k, sort the sequence k times;
For each sorting, according to the corresponding increment ti, the column to be sorted is divided into several subsequences of length m, and direct insertion sorting is performed on each sublist respectively. Only when the increment factor is 1, the entire sequence is treated as a table, and the length of the table is the length of the entire sequence.

4.2 Code implementation

Python：

def shellSort(arr):
    import math
    gap=1
    while(gap < len(arr)/3):
        gap = gap*3+1
    while gap > 0:
        for i in range(gap,len(arr)):
            temp = arr[i]
            j = i-gap
            while j >=0 and arr[j] > temp:
                arr[j+gap]=arr[j]
                j-=gap
            arr[j+gap] = temp
        gap = math.floor(gap/3)
    return arr
}

Java：

public class ShellSort implements IArraySort {
    @Override
    public int[] sort(int[] sourceArray) throws Exception {
        // 对 arr 进行拷贝，不改变参数内容
        int[] arr = Arrays.copyOf(sourceArray, sourceArray.length);
        int gap = 1;
        while (gap < arr.length/3) {
            gap = gap * 3 + 1;
        }
        while (gap > 0) {
            for (int i = gap; i < arr.length; i++) {
                int tmp = arr[i];
                int j = i - gap;
                while (j >= 0 && arr[j] > tmp) {
                    arr[j + gap] = arr[j];
                    j -= gap;
                }
                arr[j + gap] = tmp;
            }
            gap = (int) Math.floor(gap / 3);
        }
        return arr;
    }
}

C++：

void shellSort(vector<int>& arr) {
    int gap = 1;
    while (gap < (int)arr.size() / 3) {
    	gap = gap * 3 + 1;
    }
    for (; gap >= 1; gap /= 3) {
	for (int i = 0; i < gap; ++i) {
	    for (int j = i + gap; j < arr.size(); j += gap) {
		for (int k = j; k - gap >= 0 && arr[k] < arr[k - gap]; k -= gap) {
                    swap(arr[k], arr[k - gap]);
                }
            }
        }
    }
}

5. Merge sort

Merge sort (Merge sort) is an effective sorting algorithm based on the merge operation. This algorithm is a very typical application of Divide and Conquer.

As a typical algorithm application of divide and conquer, there are two ways to implement merge sort:

Top-down recursion (all recursive methods can be rewritten with iteration, so there is a second method)
bottom-up iteration

5.1 Algorithm steps

Apply for space so that its size is the sum of the two sorted sequences, which is used to store the merged sequence;
Set two pointers, the initial positions are respectively the starting positions of the two sorted sequences;
Compare the elements pointed by the two pointers, select the relatively small element and put it into the merge space, and move the pointer to the next position;
Repeat step 3 until a pointer reaches the end of the sequence;
Copies all remaining elements of another sequence directly to the end of the merged sequence.

5.2 Code implementation

Python：

def mergeSort(arr):
    import math
    if(len(arr)<2):
        return arr
    middle = math.floor(len(arr)/2)
    left, right = arr[0:middle], arr[middle:]
    return merge(mergeSort(left), mergeSort(right))

def merge(left,right):
    result = []
    while left and right:
        if left[0] <= right[0]:
            result.append(left.pop(0));
        else:
            result.append(right.pop(0));
    while left:
        result.append(left.pop(0));
    while right:
        result.append(right.pop(0));
    return result

Java：

public class MergeSort implements IArraySort {
    @Override
    public int[] sort(int[] sourceArray) throws Exception {
        // 对 arr 进行拷贝，不改变参数内容
        int[] arr = Arrays.copyOf(sourceArray, sourceArray.length);
        if (arr.length < 2) {
            return arr;
        }
        int middle = (int) Math.floor(arr.length / 2);
        int[] left = Arrays.copyOfRange(arr, 0, middle);
        int[] right = Arrays.copyOfRange(arr, middle, arr.length);
        return merge(sort(left), sort(right));
    }

    protected int[] merge(int[] left, int[] right) {
        int[] result = new int[left.length + right.length];
        int i = 0;
        while (left.length > 0 && right.length > 0) {
            if (left[0] <= right[0]) {
                result[i++] = left[0];
                left = Arrays.copyOfRange(left, 1, left.length);
            } else {
                result[i++] = right[0];
                right = Arrays.copyOfRange(right, 1, right.length);
            }
        }
        while (left.length > 0) {
            result[i++] = left[0];
            left = Arrays.copyOfRange(left, 1, left.length);
        }
        while (right.length > 0) {
            result[i++] = right[0];
            right = Arrays.copyOfRange(right, 1, right.length);
        }
        return result;
    }

}

C++：

void merge(vector<int>& arr, int l, int mid, int r) {
    int index = 0;
    int ptrL = l;
    int ptrR = mid;
    static vector<int>tempary;
    if (arr.size() > tempary.size()) {
        tempary.resize(arr.size());
    }
    while (ptrL != mid && ptrR != r) {
        if (arr[ptrL] < arr[ptrR]) {
            tempary[index++] = arr[ptrL++];
        } else {
            tempary[index++] = arr[ptrR++];
        }
    }
    while (ptrL != mid) {
        tempary[index++] = arr[ptrL++];
    }
    while (ptrR != r) {
        tempary[index++] = arr[ptrR++];
    }
    copy(tempary.begin(), tempary.begin() + index, arr.begin() + l);
}
void mergeSort(vector<int>& arr, int l, int r) { // sort the range [l, r) in arr
    if (r - l <= 1) {
        return;
    }
    int mid = (l + r) / 2;
    mergeSort(arr, l, mid);
    mergeSort(arr, mid, r);
    merge(arr, l, mid, r);
}

6. Quick Sort

Quicksort is a sorting algorithm developed by Tony Hall. On average, sorting n items requires O(nlogn) comparisons. In the worst case, Ο(n2) comparisons are required, but this is not common. In fact, quicksort is usually significantly faster than other Ο(nlogn) algorithms because its inner loop can be implemented efficiently on most architectures.

Quicksort uses a divide and conquer strategy to divide a list into two sub-lists. Quick sort is another typical application of the idea of divide and conquer in sorting algorithms. In essence, quick sort should be regarded as a recursive divide and conquer method based on bubble sort.

The name of quick sort is simple and rude, because when you hear this name, it is fast and efficient! It is one of the fastest sorting algorithms for big data. Although the time complexity of Worst Case reaches O(n²), they are excellent, and in most cases, they perform better than sorting algorithms with an average time complexity of O(n logn).

The worst running case of quick sort is O(n²), such as quick sort of sequential arrays. But its amortized expected time is O(nlogn), and the constant factor implied in the O(nlogn) notation is small, which is much smaller than the merge sort whose complexity is stable at O(nlogn). Therefore, for the vast majority of random number sequences with weaker order, quick sort is always better than merge sort.

6.1 Algorithm steps

Pick an element from the sequence, called "pivot" (pivot);
Reorder the sequence, all elements smaller than the reference value are placed in front of the reference value, and all elements larger than the reference value are placed behind the reference value (the same number can go to either side). After this partition exits, the benchmark is in the middle of the sequence. This is called a partition operation;
Recursively sort the sub-arrays of elements smaller than the reference value and the sub-arrays of elements greater than the reference value;
The bottom case of recursion is that the size of the sequence is zero or one, that is, it has always been sorted. Although it has been recursively going on, this algorithm will always exit, because in each iteration (iteration), it will put at least one element in its last position.

6.2 Code implementation

Python：

def quickSort(arr, left=None, right=None):
    left = 0 if not isinstance(left,(int, float)) else left
    right = len(arr)-1 if not isinstance(right,(int, float)) else right
    if left < right:
        partitionIndex = partition(arr, left, right)
        quickSort(arr, left, partitionIndex-1)
        quickSort(arr, partitionIndex+1, right)
    return arr

def partition(arr, left, right):
    pivot = left
    index = pivot+1
    i = index
    while  i <= right:
        if arr[i] < arr[pivot]:
            swap(arr, i, index)
            index+=1
        i+=1
    swap(arr,pivot,index-1)
    return index-1

def swap(arr, i, j):
    arr[i], arr[j] = arr[j], arr[i]

C++：

 //严蔚敏《数据结构》标准分割函数
 Paritition1(int A[], int low, int high) {
   int pivot = A[low];
   while (low < high) {
     while (low < high && A[high] >= pivot) {
       --high;
     }
     A[low] = A[high];
     while (low < high && A[low] <= pivot) {
       ++low;
     }
     A[high] = A[low];
   }
   A[low] = pivot;
   return low;
 }

 void QuickSort(int A[], int low, int high) //快排母函数
 {
   if (low < high) {
     int pivot = Paritition1(A, low, high);
     QuickSort(A, low, pivot - 1);
     QuickSort(A, pivot + 1, high);
   }
 }

7. Heap sort

Heapsort (Heapsort) refers to a sorting algorithm designed using the data structure of the heap. Stacking is a structure that approximates a complete binary tree, and at the same time satisfies the nature of stacking: that is, the key value or index of a child node is always smaller (or larger) than its parent node. Heap sort can be said to be a selection sort that uses the concept of heap to sort. Divided into two methods:

Large top heap: the value of each node is greater than or equal to the value of its child nodes, used in ascending order in the heap sorting algorithm;
Small top heap: the value of each node is less than or equal to the value of its child nodes, used in descending order in the heap sort algorithm;

The average time complexity of heap sort is O(nlogn).

7.1 Algorithm steps

Construct the sequence to be sorted into a heap H[0...n-1], and select a large top heap or a small top heap according to (ascending and descending order requirements);
Swap the heap head (maximum value) and heap tail;
Reduce the size of the heap by 1, and call shift_down(0), the purpose is to adjust the top data of the new array to the corresponding position;
Repeat step 2 until the size of the heap is 1.

7.2 Code implementation

Python：

def buildMaxHeap(arr):
    import math
    for i in range(math.floor(len(arr)/2),-1,-1):
        heapify(arr,i)

def heapify(arr, i):
    left = 2*i+1
    right = 2*i+2
    largest = i
    if left < arrLen and arr[left] > arr[largest]:
        largest = left
    if right < arrLen and arr[right] > arr[largest]:
        largest = right

    if largest != i:
        swap(arr, i, largest)
        heapify(arr, largest)

def swap(arr, i, j):
    arr[i], arr[j] = arr[j], arr[i]

def heapSort(arr):
    global arrLen
    arrLen = len(arr)
    buildMaxHeap(arr)
    for i in range(len(arr)-1,0,-1):
        swap(arr,0,i)
        arrLen -=1
        heapify(arr, 0)
    return arr

8. Counting sort

The core of counting sort is to convert the input data value into a key and store it in the additional array space. As a sort of linear time complexity, counting sort requires that the input data must be integers with a certain range.

8.1 Algorithm steps

8.2 Code implementation

Python：

def countingSort(arr, maxValue):
    bucketLen = maxValue+1
    bucket = [0]*bucketLen
    sortedIndex =0
    arrLen = len(arr)
    for i in range(arrLen):
        if not bucket[arr[i]]:
            bucket[arr[i]]=0
        bucket[arr[i]]+=1
    for j in range(bucketLen):
        while bucket[j]>0:
            arr[sortedIndex] = j
            sortedIndex+=1
            bucket[j]-=1
    return arr

9. Bucket Sort

Bucket sort is an upgraded version of counting sort. It makes use of the mapping relationship of functions, and the key to high efficiency lies in the determination of this mapping function. To make bucket sort efficient, we need to do two things:

In the case of sufficient extra space, try to increase the number of buckets
The mapping function used can evenly distribute the input N data into K buckets

At the same time, for the sorting of elements in buckets, the choice of a comparison sorting algorithm is crucial to performance.

When is the fastest? When the input data can be evenly distributed to each bucket.

When is the slowest? When the input data is allocated to the same bucket.

9.1 Code implementation

Java：

public class BucketSort implements IArraySort {
    private static final InsertSort insertSort = new InsertSort();
    @Override
    public int[] sort(int[] sourceArray) throws Exception {
        // 对 arr 进行拷贝，不改变参数内容
        int[] arr = Arrays.copyOf(sourceArray, sourceArray.length);
        return bucketSort(arr, 5);
    }

    private int[] bucketSort(int[] arr, int bucketSize) throws Exception {
        if (arr.length == 0) {
            return arr;
        }
        int minValue = arr[0];
        int maxValue = arr[0];
        for (int value : arr) {
            if (value < minValue) {
                minValue = value;
            } else if (value > maxValue) {
                maxValue = value;
            }
        }
        int bucketCount = (int) Math.floor((maxValue - minValue) / bucketSize) + 1;
        int[][] buckets = new int[bucketCount][0];
        // 利用映射函数将数据分配到各个桶中
        for (int i = 0; i < arr.length; i++) {
            int index = (int) Math.floor((arr[i] - minValue) / bucketSize);
            buckets[index] = arrAppend(buckets[index], arr[i]);
        }
        int arrIndex = 0;
        for (int[] bucket : buckets) {
            if (bucket.length <= 0) {
                continue;
            }
            // 对每个桶进行排序，这里使用了插入排序
            bucket = insertSort.sort(bucket);
            for (int value : bucket) {
                arr[arrIndex++] = value;
            }
        }

        return arr;
    }

    /**
     * 自动扩容，并保存数据
     *
     * @param arr
     * @param value
     */
    private int[] arrAppend(int[] arr, int value) {
        arr = Arrays.copyOf(arr, arr.length + 1);
        arr[arr.length - 1] = value;
        return arr;
    }

}

10. Radix sort

Radix sorting is a non-comparative integer sorting algorithm. Its principle is to cut the integer into different numbers according to the digits, and then compare each digit separately. Since integers can also represent strings (such as names or dates) and floating-point numbers in certain formats, radix sorting is not limited to integers.

10.1 Radix Sort vs Counting Sort vs Bucket Sort

Cardinality sorting: buckets are allocated according to each digit of the key value;
Counting sort: each bucket only stores a single key value;
Bucket sorting: each bucket stores a certain range of values;

10.2 Algorithm steps

10.3 Code implementation

Python：

def radix(arr):
    digit = 0
    max_digit = 1
    max_value = max(arr)
    #找出列表中最大的位数
    while 10**max_digit < max_value:
        max_digit = max_digit + 1
    while digit < max_digit:
        temp = [[] for i in range(10)]
        for i in arr:
            #求出每一个元素的个、十、百位的值
            t = int((i/10**digit)%10)
            temp[t].append(i)
        coll = []
        for bucket in temp:
            for i in bucket:
                coll.append(i)     
        arr = coll
        digit = digit + 1
    return arr

Reference: GitHub - hustcc/JS-Sorting-Algorithm: A GitBook online book "Ten Classic Sorting Algorithms" about sorting algorithms, implemented in multiple languages.