Azure solution: Azcopy and CRON usage scenarios

51CTO blog address: https://blog.51cto.com/14669127

Customer needs: As Azure ADMIN , you are review the following Azure architecture:

image.png

The estimated monthly cost of the architecture is shown in the following cost table:
image.png

Log files are generated from user activities to the Apache web server. The format of the log files is consistent. About 1GB of logs are generated every day . Microsoft Power BI is used to display weekly user activity reports. You need to provide a solution to maintain structural functions. While minimizing costs.

Solution analysis:

Azure Data Factory can use data flow activities to convert data in Azure Blob storage, and Azure is a command-line tool that can use simple commands to copy data to Azure Blob Storage , Azure File, or copy data from it. these commands are intended to achieve the best performance, while using Azcopy can store or copy data between the user and the file system storage between accounts, Azcopy can be used to copy data from the local storage to the account.

If you perform data replication regularly, you need to create a scheduled task for replication, such as creating a scheduled task or cron job for running Azcopy 's command script to identify new local data and upload it to cloud storage at specific intervals in.

Therefore, in order to maintain the structure and function while minimizing the cost, you can use the existing architecture to replace Azure Data Factory with the method of planning operations of Azure +CRON .

Relevant information description:

·       Synchronize with Azure Blob Storage by using Azcopy

·       Configure, Optimize and troubleshoot Azcopy

·       Copy and transform data in Azure Blob storage by using Azure Data Factory


Guess you like

Origin blog.51cto.com/14669127/2679014