Recently, Milvus released version 2.2.6. After fixing some bugs, Milvus became more stable.
In fact, since Milvus was upgraded to version 2.X, we have been working hard to improve and optimize, such as batch importing data from files, disk-based approximate nearest neighbor (ANN) indexing algorithm, non-stop rolling upgrade, Coordinator HA New features such as high availability and resource groups also improve performance such as metadata storage and batch import.
This article reviews the changes and upgrades from Milvus 2.2 to Milvus 2.2.6 to give you an in-depth understanding of a more stable, smooth, and easy-to-use Milvus.
01. Milvus 2.2: Improve the stability, search speed and flexible expansion and contraction capabilities of vector search
Milvus 2.2 is a major version update. It not only introduces New Feature, but also makes many breakthrough improvements:
Batch import data from files: Milvus provides a series of new APIs to support more efficient batch import of data from files. Data in any JSON file stored on Minio or S3 can be directly imported into Milvus, saving time and effort for data import.
Search result paging: In order to prevent a single RPC session from returning too many search results, Milvus now supports reducing the number of returned search results by setting offsets and keywords.
Role Based Access Control (RBAC): Milvus now supports RBAC. Users can implement access control to the Milvus database by managing users, roles and permissions.
Quotas and limits: Quotas are a new mechanism used by Milvus to protect the system from out-of-memory (OOM) and crashes during traffic bursts. Through this mechanism, you can control data insertion, search rate, and memory usage.
Collection time to live (TTL): In previous versions, Milvus only supported setting cluster-level TTL. Milvus now supports setting the collection's TTL when creating or modifying the collection. Data in a collection with a TTL set expires after the TTL ends. With this feature, users can manage data more precisely.
Disk-based Approximate Neighbor Search Indexing (Beta): Milvus now supports DiskANN for disk-based indexing of large-scale datasets. DiskANN is an indexing algorithm based on Vamana images for approximate nearest neighbor search on datasets resident on solid-state disks. Using this algorithm saves about 10 times the memory usage.
Data backup tool (Beta): Milvus 2.2 provides a new data backup tool, and supports database backup and recovery using the command line and calling API.
In addition to the above new features, Milvus 2.2 fixes some bugs and makes many improvements to enhance the stability, observability and performance of Milvus.
Milvus 2.2 Release Notes ： https://milvus.io/docs/release_notes.md#v220
Click to read the article "Milvus 2.2 Released! "
02.Milvus 2.2.1 and 2.2.2: Minor version released, focusing on fixing problems
Both Milvus 2.2.1 and 2.2.2 are minor releases, which fix key issues in the old versions and add some new features.
Kite 2.2.1：
-
Support for Pulsar tenants and authentication
-
Support for Transport Layer Security (TLS) in etcd configuration
-
Search performance improved by more than 30%
-
Optimize the scheduler and increase the probability of task merging
-
Fixed multiple bugs, including: Index Node crash when filtering fails on indexed scalar fields, index creation fails
Kite 2.2.2：
-
Fixed the problem that the Proxy did not update the cache of the Shard leader
-
Fix the problem that the loaded data in the released collection/partition is not cleaned up
-
Fix the problem that the loading count was not cleared in time
Milvus 2.2.1 Release Notes：https://milvus.io/docs/release_notes.md#v221
Milvus 2.2.2 Release Notes：https://milvus.io/docs/release_notes.md#v222
03.Milvus 2.2.3：安全、稳定、升级友好
Milvus 2.2.3 主要增强了系统的安全性、稳定性和可用性，并推出了两个重要的特性。
-
不停机滚动升级：Milvus v2.2.3 版本上线了滚动升级能力。这意味着在升级过程中，用户的查询、搜索请求不会中断。
-
Coordinator 高可用（HA）：Milvus coordinator 能够以主备节点的模式工作，降低单点故障的风险。Coordinator 高可用可以大幅缩短系统恢复时间。如果意外发生灾难，系统只需最多30秒便可恢复。
除此以外，Milvus 2.2.3 版本还进行了改进和修复多个漏洞，包括：批量导入（bulk insert）性能大幅提升、Metrics 信息优化、查询性能提升、内存（memory）占用降低、元数据存储性能提升等。
Milvus 2.2.3 Release Notes：https://milvus.io/docs/release_notes.md#v223
点击可阅读文章《Milvus 新版本来啦！首席工程师带你划重点：安全、稳定、升级友好》
04.Milvus 2.2.4：资源隔离、多云、高效
Milvus 2.2.4 是 Milvus 2.2 的一次小版本升级。Milvus 2.2.4 推出了四个新特性，并提升了性能、可靠性和效率，降低资源消耗。
-
资源组：Milvus 支持将多个 Query Node 分到其他资源组中，从而完全隔离对不同资源组中物理资源的访问。
-
重命名集合（collection）：新增集合重命名 API，可用于更改集合名称，使得集合管理变得更灵活。
-
支持 Google Cloud Storage：Milvus 现在支持 Google Cloud Storage 作为对象存储，使用 Google Cloud 的用户可以无缝对接 GCP 的对象存储。
-
搜索（Search） 和查询（Query） API 的新选项：我们在查询参数中新增了跳过 Growing 数据的选项，启用此选项将跳过对所有 Growing 数据的搜索，在搜索插入场景下提供更好的搜索性能。
Milvus 2.2.4 Release Notes：https://milvus.io/docs/release_notes.md#v224
点击可阅读文章《又见版本升级！一句话概括 Milvus 2.2.4：资源隔离、多云、高效》
05.Milvus v2.2.6：稳定性升级
Milvus 2.2.6 版本修复了众多关于稳定性问题。
性能强化
当延迟不少于 5 秒时，为查询/搜索添加了慢日志性能。Milvus 默认日志级别是 info，查询链路并发大、性能敏感，所以查询链路的日志都是debug级别，在生产环境无法看到查询链路的日志。如果用户要统计慢查询就无从下手，MySQL的慢查询日志就是个很好的范例，所以 Milvus 也在日志中加入了慢查询记录。
解决的关键问题
-
修复了DataCoord GC失败的问题
-
修复了在创建 collection 时传递的索引参数将覆盖后续 create_index 操作中传递的参数的问题
-
解决了 RootCoord 中出现消息积压导致整个系统延迟增加的问题
-
修正了指标 RootCoord InsertChannelTimeTick 的准确性
-
解决 Proxy 报告时间戳可能在某些情况下停止工作的问题
-
修复 Coordinator 角色在重启过程中可能会 panic 的问题
-
修复由于 etcd 重新启动而导致垃圾回收 goroutine 异常退出，留下检查点等问题
Milvus 2.2.6 Release Notes:https://milvus.io/docs/release_notes.md#v226
From the latest version of Milvus 2.2 to 2.2.6, every version upgrade is inseparable from the support and suggestions of users. We always insist on providing users with cutting-edge solutions and providing support for applications in various fields. As a preview, Milvus 2.2.7 is already under intense preparation and will be released soon, so stay tuned!
This article is published by mdnice multi-platform