AlphaGo has another "little brother" joining!
Google DeepMind "rolled" the Alpha series to the sorting algorithm and launched AlphaDev.
It is like a "secret method of development". By using reinforcement learning AI to discover sorting algorithms and hashing algorithms, it is forced to speed up the algorithms designed by human programmers by about 70% and 30% respectively.
Once the research results are released, the software circle is instantly ignited! All of a sudden, the running speed of millions of software around the world has soared, directly surpassing the achievements of scientists and engineers for decades, and the LLVM standard C++ library that has not been updated for ten years has been updated.
(Source: Nature)
This is also a disruptive technology launched after the integration of Google's two AI departments. The paper was published in Nature under the title "Faster sorting algorithms discovered using deep reinforcement learning". DeepMind computer scientist Daniel Mankowitz is the lead author of the paper.
1. Evolution: the origin of the sorting algorithm
Sorting is a method of organizing many items into a specific order, for example, alphabetizing three letters, ordering five numbers from largest to smallest or sorting a database containing millions of records.
Sorting methods can be traced back to the second or third centuries and are still evolving. Initially, scholars alphabetized the thousands of books on the shelves of the Library of Alexandria by hand.
After the Industrial Revolution, self-sorting machines, tabulating machines that stored information on punched cards, were invented to collect the results of the 1890 U.S. census.
In the 1950s, commercial computers began to rise, and sorting algorithms were produced immediately. You feed a sequence of unsorted numbers into a sorting algorithm, and it outputs the numbers in sorted order.
Today, many different sorting techniques and algorithms are still used to organize large amounts of data online in codebases around the world.
These sorting algorithms have been developed over decades by computer scientists and programmers to become more efficient. However, significant challenges remain for its further improvement.
2. The highlight: How to use AlphaDev to generate a new sorting algorithm?
The purpose of researchers initially using AlphaDev to generate new algorithms is to efficiently complete a given task.
In this experiment, AlphaDev built a new algorithm from scratch, not based on the previous algorithm, which is original. In the process, it applies an intermediate language of assembly code. The language is closer to computer binary instructions and makes it easier for AlphaDev to create efficient algorithms.
Specifically, AlphaDev generates an instruction each time, then tests whether its output is correct or not, and also sets the requirements in the model to generate the shortest algorithm.
When asked to redesign the sorting algorithm, AlphaDev randomly generates a new sorting algorithm that is 70% faster than the existing algorithm, and can sort five data at the same time. It was also 1.7% faster than the best algorithm when sorting 250,000 data sets.
Since sorting algorithms are widely used in a variety of commonly used software, this innovation could have a major impact on global algorithms. DeepMind has made them open source and added them to the Libc++ common code library.
According to the description of DeepMind's researchers: "Due to the large number of instruction combinations, the seemingly simple research process is extremely difficult."
3. Origin: Find the best algorithm in playing games
Further, AlphaDev is based on a more advanced model produced by AlphaZero. AlphaZero was previously DeepMind's reinforcement learning model that beat world champions in Go, chess and other board games.
Through this experiment, the new model AlphaDev exerts its unique advantages in transferring from playing games to solving scientific problems, and from experimental simulation to real-world applications.
To train AlphaDev to discover new algorithms, the researchers simulated sorting as a one-player "assembly game." During each game round, AlphaDev can observe the generated algorithm and the information contained in the CPU, and then choose an instruction to add to the algorithm to make each move.
As mentioned in the paper, assembly games are very difficult, because AlphaDev must be able to efficiently search a large number of possible instruction combinations to obtain algorithms that can be sorted.
The number of instruction combinations is similar to the number of particles in the universe or the number of combinations of possible moves in chess (10,120 games) and Go (10,700 games), each wrong move will disable the entire algorithm.
The model then outputs an algorithm and compares it to the expected output, rewarding the agent based on the algorithm's correctness and latency.
When building an algorithm, each time an instruction is input, AlphaDev checks for correctness by comparing the output algorithm with the expected result (for a sorting algorithm, this means that after inputting unordered numbers, it can output correctly sorted numbers).
The model rewards AlphaDev for the correct ordering of numbers and for its efficiency. In the end AlphaDev won the game by discovering a more accurate, faster program.
4. Algorithm innovation: exchange movement and copy movement instruction sequence
AlphaDev not only generates faster algorithms, but also innovates two instruction sequences.
Specifically, the sorting algorithm it generates includes two new instruction sequences, swap-move and copy-move, saving one instruction each time it is used. The researchers call this "AlphaDev's swap and copy moves."
The novel approach is reminiscent of AlphaGo's "Move 37" - a "counter-intuitive" play that stunned onlookers and caused the defeat of a legendary chess player.
By swapping move and duplicating move instruction sequences, AlphaDev skipped a step to accomplish a goal in a way that looked like a bug but was actually a shortcut. This demonstrates AlphaDev's ability to discover original solutions and challenge how computer science algorithms can be improved.
5. Test: promote and improve the hash algorithm
After discovering a faster sorting algorithm, the researchers tested whether AlphaDev could generalize and improve another computer science algorithm: hashing.
A hash algorithm is a fundamental algorithm in computing that is used to retrieve, store, and compress data. Just like a librarian uses a classification system to locate a book, hashing algorithms help users know what they are looking for and where to find it.
These algorithms are able to take data for a specific key (e.g. username "Jane Doe") and sort it by hash - turning the raw data into a unique string (e.g. 1234ghty).
The computer uses the hash to quickly retrieve the data associated with the key, rather than searching through all the data.
The researchers applied AlphaDev to one of the most commonly used hashing algorithms in data structures in an attempt to discover a faster algorithm. When applied to the 9-16 byte range of the hash function, AlphaDev generated an algorithm that was 30% faster.
Earlier this year, the new hash algorithm generated by AlphaDev was released into the open-source Abseil library, which is available to millions of developers around the world and is now estimated to be used trillions of times a day.
6. Gain momentum: take the first step to develop AGI
By optimizing "sorting and hashing algorithms", AlphaDev demonstrated the ability to generate different useful new algorithms.
This is also AlphaDev's first step towards developing artificial general intelligence (AGI) tools, which can also help optimize the entire computing ecosystem and solve other problems that benefit society through similar AI tools.
While the ability to optimize algorithms in the low-level assembly instruction space is very powerful, it has limitations. Currently, the team is also exploring AlphaDev's ability to optimize algorithms directly in a high-level language (eg, C++), which is more useful to developers.
Together, it is hoped that these new findings will inspire developers to create new techniques and approaches, further optimize fundamental algorithms, and create a stronger and more sustainable computing ecosystem.
7. Open source: a milestone breakthrough in AI optimized code
Previously, sorting algorithms were used trillions of times per day. As computing demands grow, people place higher and higher performance requirements on algorithms. Although human engineers have discovered different sorting algorithms, after decades of optimization, it is difficult to make breakthroughs and cannot meet the growing needs.
Today, AlphaDev has discovered a faster sorting algorithm that sorts data.
The new ranking algorithm can do everything from ranking online search results and social posts to crunching data on computers and mobile phones.
To celebrate, the new sorting algorithm has been open-sourced in the main C++ library. It is now used by millions of developers and companies around the world for cloud computing, online shopping, supply chain management, and more.
In conclusion, optimizing algorithms using artificial intelligence tools will revolutionize traditional programming. This is the first time that the sorting library has been changed in more than ten years, and the algorithm designed by the reinforcement learning model has been added to the sorting library for the first time, so it has become a milestone breakthrough in using artificial intelligence to optimize code.
8. User: It may be just a gimmick
For the research results, users have mixed opinions, and the voices of praise on Twitter are mostly:
"This is awesome! A 70% speedup over the basic sorting task that programmers learn early on. Exciting to see AI being used to achieve major speedups in algorithms and libraries we all rely on."
"Soon, ordinary people can become advanced programmers."
"Interesting way to optimize from the assembly level"!
However, some programmers believe that this is just a gimmick, and DeepMind exaggerated the function of the algorithm.
First of all, from the perspective of efficiency, it only counts the delay of the algorithm, but does not really change the time complexity.
Also, it doesn't really change the ordering, which is common in various other codebases.