Imagine that we are building a web application and need to implement a data report export function. This sounds simple, doesn't it? However, as we got further into development, we realized that the export process was much more complex and time-consuming than expected. Due to the huge amount of data reported, the backend takes quite a long time to generate the files. The original design was for the front-end to send a request to the back-end, then wait until the file generation is completed before returning it to the user. However, this method quickly showed its limitations: as the file generation time increased, the user's waiting time also increased, eventually causing the request to time out and the download task to fail.
This problem is not only a technical challenge, but also directly affects the user experience. Users may become frustrated by long waits and even doubt the reliability of the system. As a developer passionate about delivering a great user experience, we had to find a better solution to handle this problem.
Article directory
- Preface
- 1. Problem background
- 2. Technology selection and framework introduction
- 3. Front-end implementation
- 4. Back-end implementation
-
- 4.1 Back-end asynchronous implementation strategy
- 4.2 Back-end asynchronous practice
- Summarize
Preface
In modern web application development, the collaborative work of front-end and back-end has become the key to improving user experience and application performance. Especially when dealing with some time-consuming back-end tasks, how to manage these tasks gracefully not only affects the user's waiting time, but is also directly related to the stability and reliability of the system. This article will use a practical case to introduce how to implement an efficient solution in the front-end environment of Vue+ElementUI and the back-end environment of C#.NET Core. We will explore how to design an asynchronous task processing mechanism that allows the front end to initiate a download task and then receive an immediate response without waiting for the back end task to complete. At the same time, it can monitor the progress of the task in real time and notify the user to download when the task is completed. This approach not only optimizes the user's waiting experience, but also improves the performance and stability of the application.
1. Problem background
In today's rapidly developing web application field, users expect not only fully functional applications, but also a smooth and responsive user experience. Especially in data-intensive applications, back-end processing time has become one of the key factors affecting user experience. Long-time-consuming tasks, such as large-scale data processing and report generation, often become a challenge in front-end and back-end collaboration.
1.1 Problems with traditional synchronization processing
In the traditional web application model, after the front-end sends a request, it usually waits synchronously for the back-end to complete processing and return the result. This pattern works well when handling simple, fast-response requests. However, when it comes to time-consuming operations, such as report generation that requires complex calculations or processing large amounts of data, synchronization can be overwhelming. In this case, the request initiated by the user may time out due to the long backend processing time, thus affecting the overall user experience and application performance.
1.2 Specific case: file export function
Let's illustrate this with a concrete example. In an enterprise-level application, it is often necessary to provide the export function of data reports. It is assumed that these reports contain large amounts of data and require complex processing to produce the final files. If the traditional synchronization processing method is adopted, that is, after the user clicks the "Export" button, the front end needs to wait for the back end to complete the entire file generation process. This will not only lead to a long wait, but may also cause failure due to timeout. Not only is this a bad experience for the user, clicking the "Export" button multiple times also puts unnecessary stress on the backend server.
1.3 Requirements of modern web applications
The development of modern web applications is increasingly leaning towards a front-end and back-end separation architecture. In this architecture, the front end is responsible for presentation and user interaction, while the back end is responsible for data processing and business logic. This separation leads to better maintainability and scalability, but it also requires us to adopt a more efficient and user-friendly approach when dealing with long-time-consuming tasks. Therefore, finding a way to allow the front end to initiate task requests, while the back end processes these requests asynchronously and feeds back the progress to the front end in real time, has become an important requirement.
In the following content, we will explore in detail how to design and implement an efficient long-time task processing mechanism in an environment where the front-end using Vue and ElementUI works together with the C#.NET Core back-end, not only optimizing the user's waiting experience while improving the overall performance and stability of the system.
2. Technology selection and framework introduction
When building modern web applications, choosing the right technology stack is a critical step. It not only affects the development efficiency and future maintainability of the application, but is also directly related to the end-user experience. In our project, we chose Vue and ElementUI as the front-end framework, while the back-end used C#.NET Core. Below, we explore in detail the rationale for choosing these technologies as well as our data interaction and API design principles.
2.1 Front-end: Choice of Vue and ElementUI
2.1.1 Why choose Vue
Vue.js is a popular front-end JavaScript framework known for its simplicity, flexibility, and efficiency. The main advantages of Vue include:
- Responsiveness and componentization: Vue’s responsive system and componentization model provide powerful support for the development of complex single-page applications (SPA).
- Easy to learn: Compared with other front-end frameworks, Vue is easier to get started and helps speed up the development process.
- Flexibility: Vue is not only suitable for building large applications, but can also be introduced as part of a project.
2.1.2 Why choose ElementUI
ElementUI is a Vue-based component library designed for web application development. Its main features include:
- Rich components: Provides a wide range of reusable components, such as tables, dialog boxes, menus, etc., which greatly improves development efficiency.
- Easy to customize: Supports flexible theme customization, and you can easily adjust the appearance of components to match your brand style.
- Good documentation and community support: ElementUI has detailed documentation and an active community to facilitate developers to learn and solve problems.
2.2 Backend: Why choose C#.NET Core
C#.NET Core is a cross-platform, open source framework developed by Microsoft. It is suitable for building various types of applications, from cloud services to web applications to IoT applications. Its main advantages include:
- Performance: .NET Core is one of the frameworks with superior performance, especially suitable for handling complex and resource-intensive back-end operations.
- Cross-platform: Runs on multiple operating systems, including Windows, Linux and macOS.
- Strong Ecosystem: Extensive library and tool support, and a large developer community.
- Secure and mature: As one of Microsoft's core products, .NET Core has undergone rigorous testing in terms of security and stability.
2.3 Data interaction and API design principles
In an architecture where front-end and back-end are separated, the design of data interaction and API is crucial. Here are some design principles we follow:
2.3.1 RESTful API design
- Concise and clear URLs: Use URL paths that are easy to understand and reflect the hierarchy of resources.
- Standardized HTTP methods: Use standard HTTP methods (such as GET, POST, PUT, DELETE) to handle different operations.
- Stateless: Each request should be self-contained and not dependent on the context or state of the server.
2.3.2 Data format and error handling
- JSON data format: JSON is used as the main data format for front-end and back-end interaction because it is lightweight and easy to parse.
- Detailed error response: When an API error occurs, provide a clear and consistent error response, including error code and error message.
2.3.3 Security and performance
- Security Measures: Implement appropriate security measures such as HTTPS, authentication, and authorization.
- Performance Optimization: To improve response speed and reduce server load, take necessary performance optimization measures, such as paging and caching.
By carefully choosing a technology stack and adhering to these design principles, our apps not only provide powerful functionality but also ensure a great user experience and efficient performance. Next, we'll dive into the specific implementation details of the front-end and back-end.
3. Front-end implementation
When building modern web applications, the role of the front-end is not limited to displaying data and interfaces, but also involves dynamic interaction with back-end services, especially when dealing with long-term tasks. The front-end interface can initiate asynchronous tasks, monitor their progress, and provide feedback when the tasks are completed. We will demonstrate this process through a specific case - the implementation of the file export function.
3.1 Design of Vue components
Our goal is to create a user-friendly interface that allows users to initiate export tasks and monitor the progress of the tasks in real time. For this purpose, we designed a Vue component named ProgressDialog. This component is responsible for displaying the progress and status of each export task and providing the function of downloading the completed file.
3.1.1 ProgressDialog component
ProgressDialogThe component takes advantage of Vue's responsiveness and componentization capabilities to achieve the following key functions:
- Task monitoring: Monitor the status and progress of each task through polling API.
- User interaction: Display detailed information of each task, including task ID, file name, current status and progress bar.
- Download function: For completed tasks, provide a download button to allow users to download the generated file.
This component receivestaskIds (task ID array) and dialogVisible (Boolean value that controls the display of the dialog box) as props, making it flexibly embedded into different parent components.
3.1.2 Use of ElementUI components
InProgressDialog component, we use the following ElementUI components:
- el-dialog: A dialog box used to display progress information.
- el-row and el-col: used to layout task information and download buttons.
- el-progress: Used to display task progress.
- el-button: used to trigger the download operation.
The use of these components greatly simplifies the UI implementation process, while ensuring the beauty and user-friendliness of the interface.
3.2 Initiating asynchronous tasks and processing responses
3.2.1 Initiate a task
In our case, the user can initiate the export task in two ways:
- Select specific items to export: The user selects a specific data item and clicks the "Export" button to initiate the task.
- Export by condition: Users filter data based on specific conditions and then initiate an export task.
We implement both methods inabcManage the component. By calling the corresponding API (exportAbcManage and exportAbcManageByCondition), we can get the task ID and use the ProgressDialog component to display the task progress.
3.2.2 Processing responses
Once the task is initiated,ProgressDialog the component starts polling the backend service to query the current status and progress of each task. This is done by implementing a function in the component's methods. This function calls the backend API periodically to update the progress and status of each task. pollTaskProgress
When the task is completed, the component displays a download button. Clicking this button will trigger a download action, allowing the user to obtain the resulting file.
3.3 Code implementation
3.3.1 progressDialog.vue
<template>
<el-dialog title="下载进度"
:close-on-click-modal='false'
:visible.sync="localDialogVisible">
<!-- 遍历任务数组 -->
<div v-for="(task, index) in tasks"
:key="index">
<el-row>
<!-- 显示任务令牌或文件名 -->
<el-col :span="16">
<div v-if="task.status !== '执行完成'">{
{ '令牌：' + task.taskId }}</div>
<div v-if="task.status === '执行完成'">{
{ '文件：' + task.fileName }}</div>
</el-col>
<!-- 下载按钮 -->
<el-col :span="8">
<el-button v-if="task.status === '执行完成'"
type="primary"
size="mini"
@click="downloadFile(task.fileName)">
下载
</el-button>
</el-col>
</el-row>
<!-- 进度条 -->
<el-progress :percentage="task.progress"></el-progress>
<!-- 显示任务状态 -->
<div v-if="task.status !== '执行完成'">{
{ task.status }}</div>
<!-- 显示错误信息 -->
<div v-if="task.status === '执行失败'">错误信息: {
{ task.errorMsg }}</div>
</div>
<!-- 对话框底部按钮 -->
<div slot="footer"
class="dialog-footer">
<el-button @click="dialogClose">关 闭</el-button>
</div>
</el-dialog>
</template>
<script>
export default {
name: 'ProgressDialog',
// 接收父组件传递的属性
props: {
taskIds: {
type: Array,
required: true
},
dialogVisible: {
type: Boolean,
required: true
}
},
data () {
return {
localDialogVisible: this.dialogVisible, // 本地对话框显示状态
tasks: [] // 任务数组
}
},
// 监听器
watch: {
// 当任务ID数组变化时初始化任务和开始轮询
taskIds (val) {
if (val !== null && val !== undefined && val.length > 0) {
this.initializeTasks()
this.startPolling()
}
},
// 监听对话框可见性变化
dialogVisible (val) {
this.localDialogVisible = val
}
},
methods: {
// 初始化任务
initializeTasks () {
this.tasks = this.taskIds.map(id => ({
taskId: id,
progress: 0,
status: '等待执行',
errorMsg: '',
fileName: ''
}))
},
// 开始轮询每个任务的进度
startPolling () {
this.tasks.forEach(task => {
this.pollTaskProgress(task)
})
},
// 轮询特定任务的进度
pollTaskProgress (task) {
const polling = () => {
this.$api.download.getByTaskId(task.taskId).then(res => {
// 更新任务状态
task.status = res.status
task.errorMsg = res.errorMsg
task.currentCount = res.currentCount
task.totalCount = res.totalCount
task.fileName = res.fileName
// 计算进度
task.progress = task.totalCount > 0 ? parseFloat(Math.min(Math.max(
(task.currentCount / task.totalCount) * 100, 0), 100).toFixed(1)) : 0
// 如果任务未完成或失败，继续轮询
if (task.status !== '执行完成' && task.status !== '执行失败') {
setTimeout(polling, 1000)
}
})
}
polling()
},
// 下载文件
downloadFile (fileName) {
// 调用下载接口
...
this.$message.error("调用下载接口" + fileName)
},
// 关闭对话框
dialogClose () {
// 发出关闭对话框的事件
this.$emit('update:dialogVisible', false)
}
}
}
</script>
3.3.2 ProgressDialog component call
This code shows how to use the component within the parent component abcManage. ProgressDialog
In Vue templates,ProgressDialog components are embedded in the template of the parent component. It receives the necessary data and status through specific properties (taskIds and dialogVisible). In the script of the parent component, relevant data properties and methods are defined to control the display and data transfer of the ProgressDialog component.
<template>
<div class="container">
<el-card>
<!-- 其他内容，例如表格组件 -->
<my-table @exportExcel="exportExcel"
@exportByCondition="exportByCondition">
</my-table>
</el-card>
<!-- ProgressDialog组件的调用 -->
<progress-dialog ref="progressWindow"
:taskIds.sync="progressTaskIds"
:dialogVisible.sync="progressVisible"></progress-dialog>
</div>
</template>
<script>
import MyTable from '@/components/table/index'
import ProgressDialog from '@/components/progress/progressDialog'
export default {
name: 'abcManage',
data () {
return {
// ...其他数据属性
progressTaskIds: [], // 用于存储任务ID的数组
progressVisible: false // 控制ProgressDialog组件的显示
}
},
methods: {
// 显示ProgressDialog组件并传递任务ID
showProgressWindow (taskIds) {
this.progressVisible = true
this.progressTaskIds = taskIds
},
// 导出Excel的方法
exportExcel (items) {
// ...导出逻辑
// 调用showProgressWindow来显示进度对话框
this.$api.exportExcel.exportAbcManage(ids).then(res => {
this.showProgressWindow(res)
}).catch(_err => {
})
},
// 按条件导出的方法
exportByCondition (advanceSearch, searchData) {
// ...导出逻辑
// 调用showProgressWindow来显示进度对话框
this.$api.exportExcel.exportAbcManageByCondition(searchModel).then(res => {
this.showProgressWindow(res)
}).catch(_err => {
})
}
},
components: {
MyTable, ProgressDialog // 注册子组件
}
}
</script>
Through the combination of Vue and ElementUI, we successfully implemented a front-end interface that can not only initiate asynchronous tasks, but also monitor the progress of the tasks and interact with them in real time. This model not only improves the user experience, but also makes processing long-time-consuming tasks more efficient and reliable.
4. Back-end implementation
When building a responsive and efficient front-end interface, a powerful and reliable back-end implementation is indispensable. Especially when dealing with long-time-consuming tasks, such as file export, the backend needs to effectively manage these tasks while providing real-time status updates. In this part, we will discuss how to implement back-end logic in the C#.NET Core environment, including asynchronous programming, task management and status updates, as well as file generation and storage strategies.
4.1 Back-end asynchronous implementation strategy
4.1.1 DDD asynchronous programming ideas in C#.NET Core
When implementing asynchronous tasks using Domain Driven Design (DDD), we usually separate business logic and data persistence logic , to improve the maintainability and testability of the code. Here is an example of how to apply the above asynchronous file export logic to a DDD architecture:
Export request processing
In DDD, we will create an application service for processing export requests and implement business logic in it.
public class ExportService : IExportService
{
private readonly IExportTaskRepository _exportTaskRepository;
private readonly IExportProcessor _exportProcessor;
public ExportService(IExportTaskRepository exportTaskRepository, IExportProcessor exportProcessor)
{
_exportTaskRepository = exportTaskRepository;
_exportProcessor = exportProcessor;
}
public async Task<string> ScheduleExportAsync(ExportRequestModel request)
{
var exportTask = new ExportTask(Guid.NewGuid().ToString(), request);
await _exportTaskRepository.AddAsync(exportTask);
// 使用后台服务或队列来处理任务
_exportProcessor.ProcessInBackground(exportTask);
return exportTask.Id;
}
}
Export task entities
In DDD, we model the export task as a domain entity.
public class ExportTask
{
public string Id {
get; private set; }
public ExportRequestModel Request {
get; private set; }
// 其他属性如状态、进度等
public ExportTask(string id, ExportRequestModel request)
{
Id = id;
Request = request;
// 初始化状态等
}
// 更新状态的方法
public void UpdateStatus(/* ... */)
{
// ...
}
}
Export task storage interface
In DDD, we define a warehousing interface to encapsulate the persistence logic of the export task.
public interface IExportTaskRepository
{
Task AddAsync(ExportTask task);
Task<ExportTask> GetByIdAsync(string id);
// 其他持久化操作
}
Export processing logic
The export processing logic can be encapsulated in a separate service, which may use a background task or message queue to handle the actual export task.
public interface IExportProcessor
{
void ProcessInBackground(ExportTask task);
}
public class ExportProcessor : IExportProcessor
{
public void ProcessInBackground(ExportTask task)
{
Task.Run(async () =>
{
// 实际的导出逻辑
// ...
// 更新任务状态
// ...
});
}
}
In this way, we separate the business logic, data persistence, and actual export processing logic, making each part more focused on their responsibilities. This not only improves the maintainability of the code, but also makes the entire system easier to test and expand.
4.1.2 Task management and status update
When processing the back-end logic of web applications, especially when long-time operations such as file export are involved, task management and status updates have become the core of the asynchronous processing process. Effectively managing task status is not only related to the optimization of user experience, but also directly affects the overall performance of the system.
In our system, in order to track and manage long-running tasks, we created an entity class named DownloadCenterEntity to represent download center-related information. This class corresponds to a table in the database and is used to store status and progress information of various tasks.
Entity design
namespace AbcErp.Core.Entity.Download
{
/// <summary>
/// 下载中心
/// </summary>
[Table("erp_sys_download_center")]
[Index(nameof(TaskId), IsUnique = true)]
public class DownloadCenterEntity : IEntity
{
// ...属性定义省略
}
}
In the DownloadCenterEntity class, we define several key properties:
- TaskId: The unique identifier of the task, used to track each independent task.
- Status: Indicates the current status of the task, such as "waiting for execution", "executing", "execution completed" or "execution failed".
- CurrentCount and TotalCount: represent the current progress and total progress of the task respectively, used for calculation The completion percentage of the task.
- ErrorMsg: Stores the error message when the task fails.
- FileName: Generated file name for download operation.
The importance of task status management
By storing task status and progress information in the database, we can achieve several key functions:
- Real-time progress feedback: The front-end can query this information regularly to provide users with real-time task progress feedback.
- Error handling: If the task fails, the error message can help developers quickly locate the problem.
- Data consistency: Using the database to store status information ensures the consistency and integrity of task information in the event of multi-user access or service restart.
Asynchronous task processing flow
In the actual business process, when the user requests to generate a report or perform other long-time-consuming operations, the back-end service will create a new DownloadCenterEntity instance, set the initial state, and Save to database. Subsequently, the backend service starts an asynchronous task to handle the actual business logic, while regularly updating the task status and progress information in the database.
For example, for a file export operation, the backend service will first set the task status to "Waiting for Execution" and then start generating files. During the file generation process, the service updates the current progress of the task. Once completed, it updates the status to "Execution Completed" and saves the file name for downloading.
In the C#.NET Core environment, by combining entity class design and database operations, we can effectively manage and track long-time-consuming tasks on the backend. This approach not only improves the responsiveness and stability of the system, but also optimizes the user experience, allowing users to understand task progress and results in real time. With this approach, our applications remain efficient and stable even when faced with complex and resource-intensive tasks.
4.1.3 File generation and storage strategy
When dealing with time-consuming operations such as file export, the backend is not only responsible for generating the file, but also needs to consider the storage and transfer of the file. Here are a few key points to consider:
-
File generation: Generate corresponding files according to business needs, such as Excel, PDF, etc.
-
File storage: The generated files need to be stored on the server or in a cloud storage service.
-
File Access: Once the files are generated and stored, the backend needs to provide a mechanism to allow the frontend to download these files. This is typically accomplished by generating a pre-signed URL through which front-end users can securely download files.
Combining these strategies, the C#.NET Core backend can not only handle time-consuming tasks efficiently, but also provide necessary status information to the frontend to ensure the responsiveness and user experience of the entire system.
4.2 Back-end asynchronous practice
4.2.1 Web API interface design
Design the Web API interface in the C#.NET Core project to support efficient export task processing, specifically focusing on the two key interfaces defined in the ExportExcel class.
In our sample project AbcErp, the ExportExcel class plays a central role and provides the function of exporting data to Excel files. This class not only contains the necessary service dependency injection, but also defines two key Web API methods, which are used to handle export requests based on ID and export requests based on search conditions.
Service class dependency injection
ExportExcelThe class injects the IAbcExportService service via the constructor, which allows it to leverage the service to handle the actual export logic. This dependency injection method ensures a clear separation of class responsibilities, making maintenance and testing easier.
public ExportExcel(IAbcExportService abcExportExport)
{
_abcExportExport = abcExportExport;
}
Export data based on Id
ExportAbcManageToExcelThe method provides a POST interface that allows the front end to pass in a set of IDs to select specific data for export.
[HttpPost("exportAbcManage")]
public async Task<List<string>> ExportAbcManageToExcel(List<int> ids)
{
return await _abcExportExport.ExportAbcManageByIds(ids);
}
The implementation of this method is asynchronous, which means it can handle time-consuming operations without blocking other processes on the server. The method returns a list of strings, typically containing unique identifiers for the tasks, which can be used by the front end to poll the task's status and progress.
Export condition-based data
Another interface ExportAbcManageByConditionToExcel similarly provides data export functionality, but it allows users to filter the data that needs to be exported based on complex search criteria.
[HttpPost("exportAbcManageByCondition")]
public async Task<List<string>> ExportAbcManageByConditionToExcel(AbcManageAdvSrhDto searchDto)
{
return await _abcExportExport.ExportAbcManageByCondition(searchDto);
}
This flexibility is an important feature of modern web applications. Not only does it provide a user-friendly experience, it also allows the backend to handle large amounts of data more efficiently.
4.2.2 Service and warehousing design
We first definedIAbcExportService the interface, which is responsible for declaring business logic related to file export. Then, we implemented the specific logic of the interface and handled the specific tasks of file export in the AbcExportService class.
Service interface
public interface IAbcExportService
{
Task<List<string>> ExportAbcManageByIds(List<int> ids);
Task<List<string>> ExportAbcManageByCondition(AbcManageAdvSrhDto searchDto);
}
Service implementation
In the service implementation, we use dependency injection to introduce the required warehousing and other services, as shown below:
public class AbcExportService : IAbcExportService, IScoped
{
private readonly IRepository<DownloadCenterEntity> _downloadRepository;
private readonly IHttpContextAccessor _httpContext;
private readonly IServiceScopeFactory _scopeFactory;
private readonly OssUtil _ossService;
// 构造函数省略...
// 业务逻辑方法省略...
}
4.2.3 Asynchronous task processing
The core of asynchronous tasks is to not block the main thread while efficiently managing long-running operations. In our implementation, each export operation is treated as a separate task and handled by a background thread.
Asynchronous task start
We have defined methods in the service to start asynchronous tasks as follows:
public async Task<List<string>> ExportAbcManageByIds(List<int> ids)
{
// 任务初始化和启动逻辑
// ...
await Task.Run(() => PrepareAbcManageByIds(taskId, ids));
return new List<string> {
taskId };
}
Task processing logic
InPrepareAbcManageByIds method we handle the actual export logic. This includes getting data from the database, generating Excel files, uploading to OSS (Object Storage Service), and updating task status.
private async void PrepareAbcManageByIds(string taskId, List<int> ids)
{
using (var scope = _scopeFactory.CreateScope())
{
// 获取依赖服务
var repository = scope.ServiceProvider.GetRequiredService<IRepository<AbcProductEntity>>();
// 数据处理和文件生成逻辑
// ...
}
}
In .NET Core, it is a common practice to create a new service scope () using var scope = _scopeFactory.CreateScope(), especially when you need When using dependency injection in the context of background tasks or non-HTTP requests. The reasons and advantages of this approach are as follows:IServiceScope
1. Service life cycle and scope
Services (i.e. dependencies) in .NET Core can have different life cycles: Singleton (single case), Scoped (scope) ) and Transient. Among them, Scoped life cycle means that an instance of the service is created for each request scope. In the context of HTTP requests, this scope naturally corresponds to the life cycle of a request.
2. Service scope issues in background tasks
When running code in the context of a background task or non-HTTP request, such as in a newly opened thread or using Task.Run(), it is outside the request scope. In this case, using dependency-injected Scoped services directly would be problematic because these services are created for each HTTP request and no such request scope exists in the background task.
3. Use IServiceScopeFactory to create a new scope
To solve this problem, IServiceScopeFactory is used to create a new scope. In this new scope, a new instance of the Scoped service can be obtained. This means we can safely use these services in background tasks without worrying that they were created for a specific HTTP request.
In the sample code PrepareAbcManageByIds method, the reason for using IServiceScopeFactory to create a new scope is:
- Methods can run in a background thread, independent of the original HTTP request.
- By creating a new scope, Scoped services such as database warehousing (
IRepository<AbcProductEntity>) can be safely parsed and used.- This ensures that each background task has its own independent service instance, avoiding potential concurrency issues, while also following best practices for dependency injection and service lifecycle.
UsingIServiceScopeFactory Creating a new service scope is the standard practice for handling Scoped services in background tasks. It maintains consistency in the service life cycle and provides code runtime The flexibility and security you need.
4.2.4 File generation and upload
File generation and uploading are the most critical parts of the export task. We used the NPOI library to generate Excel files and used OSS services to store and share the generated files.
Excel file generation
InGenerateAbcManage method, we convert the queried data into an Excel file:
private async Task GenerateAbcManage(string taskId, List<AbcProductEntity> entities, ...)
{
// 使用NPOI生成Excel文件
// ...
}
Upload files to OSS
The generated Excel file is then uploaded to OSS so that users can download:
var uploadres = _ossService.UploadStreamToOss(ms, key);
// 错误处理和状态更新逻辑
// ...
In this way, we are able to efficiently handle export requests for large amounts of data while maintaining the responsiveness of the application. This asynchronous and task separation approach is particularly important when dealing with resource-intensive tasks, not only improving the efficiency of the system but also enhancing the user experience. In the environment of C#.NET Core, with the help of DDD principles and asynchronous programming, we can build a back-end service that is both powerful and flexible.
4.2.5 Some code snippet demonstrations
API interface
[Route("/abc/erp/ExportExcel")]
[ApiDescriptionSettings("ExportExcel", Tag = "ExportExcel", Version = "v0.0.1")]
[DynamicApiController]
public class ExportExcel: IExportExcel
{
// 其他代码 ...
private readonly IAbcExportService _abcExportExport;
public ExportExcel(
// 其他代码 ...
IAbcExportService abcExportExport
{
// 其他代码 ...
_abcExportExport = abcExportExport;
}
/// <summary>
/// 对账单（根据选中的id导出）
/// </summary>
[HttpPost("exportAbcManage")]
public async Task<List<string>> ExportAbcManageToExcel(List<int> ids)
{
List<string> str = await _abcExportExport.ExportAbcManageByIds(ids);
return str;
}
/// <summary>
/// 对账单（根据查询的结果导出）
/// </summary>
[HttpPost("exportAbcManageByCondition")]
public async Task<List<string>> ExportAbcManageByConditionToExcel(AbcManageAdvSrhDto searchDto)
{
List<string> str = await _abcExportExport.ExportAbcManageByCondition(searchDto);
return str;
}
// 其他代码 ...
In theAbcErp application, the design of the ExportExcel class embodies several key principles of modern Web API: clear separation of duties, asynchronous processing, and flexibility User input processing. Through such a design, we are able to provide an efficient and user-friendly export function, whether it is processing simple export requests based on ID, or meeting complex conditional filtering export requirements. This not only improves application performance, but also optimizes user experience, making data management and report generation more efficient and convenient.
asynchronous service
public class AbcExportService : IAbcExportService, IScoped
{
private readonly IRepository<DownloadCenterEntity> _downloadRepository;
private readonly IHttpContextAccessor _httpContext;
private readonly IServiceScopeFactory _scopeFactory;
private readonly OssUtil _ossService;
private ExcelStyle NopiCommon = new ExcelStyle();
/// <summary>
/// 构造函数
/// </summary>
public AbcExportService(
IRepository<DownloadCenterEntity> downloadRepository,
IHttpContextAccessor httpContext,
IServiceScopeFactory scopeFactory,
OssUtil ossService
)
{
_downloadRepository = downloadRepository; // 用于数据存储和检索的仓储
_httpContext = httpContext; // 用于获取当前HTTP上下文信息
_scopeFactory = scopeFactory; // 用于创建服务作用域
_ossService = ossService; // OSS服务工具类
}
public async Task<List<string>> ExportAbcManageByIds(List<int> ids)
{
List<string> str = new List<string>(); // 存储任务ID的列表
var userName = _httpContext.HttpContext.User.GetUserName(); // 获取当前用户名称
var taskId = Guid.NewGuid().ToString(); // 生成唯一任务标识
// 确保生成的任务ID是唯一的
while (true)
{
int iCount = await _downloadRepository.Where(x => x.TaskId.Equals(taskId)).CountAsync();
if (iCount == 0)
{
break;
}
else
{
taskId = Guid.NewGuid().ToString(); // 重新生成任务标识
}
}
// 创建并保存下载任务信息
DownloadCenterEntity downloadCenter = new DownloadCenterEntity();
// ... 设置任务的各种属性
downloadCenter.UserName = userName;
downloadCenter.TaskId = taskId;
downloadCenter.TaskName = "对账报表-导出勾选结果";
downloadCenter.SearchCondition = JsonConvert.SerializeObject(ids);
downloadCenter.InterfaceName = "ExportAbcManageByIds";
downloadCenter.Status = "等待执行";
downloadCenter.CurrentCount = 0;
downloadCenter.TotalCount = 100; // 防止除0
downloadCenter.CreateTime = DateTime.Now;
downloadCenter.CompleteTime = DateTime.Now;
await _downloadRepository.InsertNowAsync(downloadCenter);
await Task.Run(() => PrepareAbcManageByIds(taskId, ids)); // 在后台异步启动导出任务
str.Add(taskId);
return str;
}
public async Task<List<string>> ExportAbcManageByCondition(AbcManageAdvSrhDto searchDto)
{
// 方法类似
}
private async void PrepareAbcManageByIds(string taskId, List<int> ids)
{
using (var scope = _scopeFactory.CreateScope())
{
// 使用服务作用域获取必要的依赖服务
var repository = scope.ServiceProvider.GetRequiredService<IRepository<AbcProductEntity>>();
var downloadRepository = scope.ServiceProvider.GetRequiredService<IRepository<DownloadCenterEntity>>();
var mapper = scope.ServiceProvider.GetRequiredService<IMapper>();
DownloadCenterEntity downloadCenter = await downloadRepository.Where(x => x.TaskId.Equals(taskId)).FirstOrDefaultAsync();
if (downloadCenter == null)
{
// 如果找不到任务实体，直接返回
// 更新任务状态
return;
}
downloadCenter.Status = "准备数据"; // 更新任务状态
await downloadRepository.UpdateNowAsync(downloadCenter);
// 执行耗时操作：查询所有实体集合
var abcPrds = await repository.Where(x => ids.Contains(x.Id)).AsNoTracking().ToListAsync();
// ... 处理查询结果和错误情况
await downloadRepository.UpdateNowAsync(downloadCenter);
await GenerateAbcManage(taskId, abcPrds, downloadRepository, mapper);
}
}
private async void PrepareAbcManageByCondition(string taskId, AbcManageAdvSrhDto searchDto)
{
// 方法类似
}
private async Task GenerateAbcManage(string taskId, List<AbcProductEntity> abcPrds, IRepository<DownloadCenterEntity> downloadRepository, IMapper mapper)
{
// ... 获取任务信息和准备数据
try
{
// 渲染任务并生成Excel文件
// ... Excel文件生成逻辑
// 渲染任务 长耗时任务
// 10行更新一次记录
if (z % 10 == 0)
{
downloadCenter.CurrentCount = z;
await downloadRepository.UpdateNowAsync(downloadCenter);
}
// 上传文件到OSS
MemoryStream ms = new MemoryStream();
workbook.Write(ms);
ms.Flush();
var uploadres = _ossService.UploadStreamToOss(ms, key);
if (!uploadres.Res)
{
// 如果上传失败，更新任务状态为执行失败
downloadCenter.Status = "执行失败";
downloadCenter.ErrorMsg = "导出文件失败！";
downloadCenter.CompleteTime = DateTime.Now;
await downloadRepository.UpdateNowAsync(downloadCenter);
return;
}
// 更新任务完成状态
downloadCenter.CurrentCount = abcMans.Count();
downloadCenter.FileName = fileName;
downloadCenter.Status = "执行完成";
downloadCenter.CompleteTime = DateTime.Now;
await downloadRepository.UpdateNowAsync(downloadCenter);
}
catch (Exception ex)
{
// 出现异常时，更新任务状态为执行失败
// ...
downloadCenter.Status = "执行失败";
downloadCenter.ErrorMsg = ex.Message;
downloadCenter.CompleteTime = DateTime.Now;
await downloadRepository.UpdateNowAsync(downloadCenter);
}
finally
{
// 最终处理
}
}
}
Summarize
1. Project Challenges and Solutions
When developing the data report export function of a web application, we faced a significant technical challenge: how to efficiently process large amounts of data and generate files while providing a good user experience. The initial way synchronous was handled resulted in long waits and request timeouts, which negatively impacted the user experience. In order to solve this problem, we have adopted a series of technical strategies and best practices, including front-end and back-end separation, asynchronous processing, service optimization, and API design.
2. Collaboration between front-end and back-end
Front-end implementation
We chose Vue and ElementUI as the front-end framework and UI component library to improve development efficiency with their simplicity and flexibility. In particular, the design of the ProgressDialog component enables the front end to display the progress and status of asynchronous tasks in a user-friendly way.
Backend architecture
C#.NET Core was chosen for the backend not only because of its cross-platform and high-performance features, but also because of its powerful ability to handle asynchronous operations and large amounts of data. We use domain-driven design (DDD) to organize the code structure and improve the maintainability and testability of the code.
3. Key technical points
-
Asynchronous programming: On the backend, we took advantage of the asynchronous programming features of .NET Core to ensure that long-running tasks will not block the main thread, thus improving the application efficiency. Responsiveness and throughput.
-
Task Management: By tracking the status and progress of each task in the database, we are able to provide users with real-time feedback while maintaining data consistency and integrity.
-
RESTful API design: We carefully designed the API to support flexible data export needs while ensuring the ease of use and security of the interface.
-
File processing strategy: In terms of generating and storing files, we adopt efficient methods, including using the NPOI library to generate Excel files and uploading files to Alibaba Cloud OSS for storage and distribution.
4. Results and reflections
Through these technical practices, we not only optimized the performance of the data export function, but also improved the user experience. This project demonstrates the importance of front-end and back-end collaboration in modern web application development, and the necessity of flexible application of technical solutions in the face of complex business needs. In the future, we can also explore more optimization methods, such as further performance tuning, code refactoring, and the introduction of new technology stacks to cope with the growing data processing needs.