ModaHub Community: AI-native cloud vector database Zilliz Cloud imports data in batches

This article lists the common problems and corresponding solutions that may be encountered when importing data into Zilliz Cloud in batches.

How to bulk import data from Elasticsearch to Zilliz Cloud?

 

Please perform the following steps to bulk import data from Elasticsearch to Zilliz Cloud

1. Connect to Elasticsearch

print(fmt.format("start connecting to ElasticSearch"))

es = Elasticsearch(hosts=ELASTICSEARCH_HOST)`

1. Use Elasticsearch Scroll API to fetch data and import into Zilliz Cloud. The following sample code shows how to import Elasticsearch data into the hello_milvus Collection in Zilliz Cloud

print(fmt.format("Start inserting entities"))

rng = np.random.default_rng(seed=19530)

resp = es.search(index=ELASTICSEARCH_INDE