In the process of database query, if the amount of data queried is very large, there will be query efficiency problems. In the process of querying the SQL Server database in PowerBI, two different methods are supported: data import method and DirectQuery method. Direct Query implements real-time data query, and the data will not be saved in PowerBI's data cache. However, if we use the data import method, the data for each data query will be saved in PowerBI. If the data cycle is very long, the problem of PowerBI file being too large will occur. PowerBI Pro provides users with 10G of data space. If it spans a long period of time, the data size will exceed 10G. To solve this problem, PowerBI provides incremental refresh function. PowerBI currently supports incremental refresh of relational databases. Databases such as SQL Server, MYSQL or Oracle can support corresponding incremental refresh. Compared with complete refresh, incremental refresh has the following advantages.
- Refreshing is faster - only the most recently changed data needs to be refreshed.
- Refreshing is more reliable - no long-term connections to unstable data sources are required. Queries against source data run faster, reducing the likelihood of disruption from network issues.
- Reduced resource consumption - The amount of data to be refreshed is reduced, thereby reducing the overall usage of memory and other resources in Power BI and data source systems.
- Allows for large data sets - Data sets may grow to contain billions of rows without requiring a full refresh of the entire data set each time a refresh operation is performed.
- Easy installation - Complete a few tasks to define an incremental refresh strategy in Power BI Desktop. After you publish policies, the service automatically applies them on every refresh.