huggingface downloads and loads local dataset

For details, please see the official website Main classes

Download the data set to the specified path:

For example:

from datasets import load_dataset, load_from_disk
books = load_dataset("opus100", "en-zh")
books.save_to_disk('./opus100')  

Download to the opus100 folder on the current page

The downloaded file is as shown in the figure

Load local dataset:

ds = load_from_disk("./opus100")

 You can also change the type

如to_csv、to_pandas、to_dict、to_json etc