Introduction: Alibaba Cloud, as the first domestic vendor to launch Elasticsearch version 8.9, provides users with the best practices for enhanced AI and the hybrid search capabilities of ES on the basis of the Elasticsearch Relevance Engine™ (ESRE™) engine. More possibilities for innovation and exploration.
In recent years, the rapid development and widespread application of artificial intelligence have achieved remarkable results in various industries. In the field of search, Alibaba Cloud Elasticsearch, as a powerful search engine, has been providing enterprises with efficient and accurate search services. Now, Alibaba Cloud, as the first domestic vendor to launch Elasticsearch version 8.9, provides users with the best practices for enhancing AI and the hybrid search capabilities of ES on the basis of the Elasticsearch Relevance Engine™ (ESRE™) engine. More possibilities for innovation and exploration.
This new upgrade from version 8.5 to version 8.9 adds the following key features, which significantly improves Alibaba Cloud Elasticsearch's vector retrieval and hybrid search capabilities, greatly improving the accuracy and relevance of search results.
- Supports text and vector recall result rearrangement (RRF).
- The maximum vector dimension is increased to 2048.
- Brute force retrieval performance is improved.
- KNN query supports simultaneous query of multiple fields.
- Built-in ELSER model.
- Stable support for NLP distributed model scheduling management.
- ……
Vector retrieval---give wings to your search
As an important new capability of version 8.x, vector search breaks through the traditional keyword-based search and uses the power of machine learning and artificial intelligence to convert text content into vector representation, that is, each word in the text data is represented as A vector, and retrieval is achieved by calculating the distance between vectors to determine the similarity between texts, thereby achieving efficient retrieval and processing of texts. Compared with traditional text retrieval, by increasing the semantic relationship between words and documents, the relevance of search is significantly improved; at the same time, the processing objects are expanded from text to image, voice and other types, and the level of application scenarios has also been correspondingly improved. increase; of course, vector retrieval can also customize search results according to user preferences, providing users with a personalized search experience.
Vector retrieval technology has a wide range of application scenarios, including search engine optimization, image search, natural language processing, recommendation systems, sentiment analysis and other fields. Alibaba Cloud's Elasticsearch product now provides basic vector retrieval capabilities, and you can experience text-based image searches based on Elasticsearch vector retrieval .
Hybrid search RRF---double assistance of search results and performance
Hybrid search RRF (Reciprocal rank fusion) supports comprehensive re-ordering of multiple result sets recalled in multiple different ways, and returns the final sorting result. Elasticsearch has previously supported correlation sorting based on BM25 and recall sorting based on vector similarity. RRF can be used to comprehensively sort the results of the two, significantly improving the accuracy of sorting. Compared with single search technology, the advantages of hybrid search RRF are very obvious. Multiple search technologies can be combined arbitrarily and comprehensive search results can be obtained, which greatly improves the accuracy and relevance of search. From the perspective of scenario adaptability, enterprises can Design exclusive search solutions based on your own business, and the degree of freedom is also significantly improved. Regarding the experimental verification of hybrid search RRF's improvement in search result accuracy and relevance, Alibaba Cloud Elasticsearch uses RRF hybrid ranking to optimize semantic query results for testing and comparison, and the following results are obtained.
With the launch of the new version, Alibaba Cloud Elasticsearch once again demonstrates its continuous progress in the search field. It brings users a more intelligent and in-depth search experience. In the future, Alibaba Cloud Elasticsearch will continue to strive for innovation and bring more breakthroughs and possibilities in search technology to users.
Fined 200 yuan and more than 1 million yuan confiscated You Yuxi: The importance of high-quality Chinese documents Musk's hard-core migration server Solon for JDK 21, virtual threads are incredible! ! ! TCP congestion control saves the Internet Flutter for OpenHarmony is here The Linux kernel LTS period will be restored from 6 years to 2 years Go 1.22 will fix the for loop variable error Google celebrates its 25th anniversary Svelte has built a "new wheel" - runes