Large amounts of data concurrency .NET Solution
Large amounts of data concurrency .NET Solution
Large amount of concurrent large data requests typically several conditions:
- A large number of simultaneous users of the system
不同功能页面were查找、更新操作- A large number of concurrent users on the system
同一个页面，同一个表a large amount of data to be查询操作- A large number of simultaneous users of the system
同一个页面，同一个表were更新操作
First category: a large number of simultaneous users of the system 不同功能页面 were 查找、更新操作
First, the processing of the server level
1. The adjusting IIS 7 application pool queue length
1000 was changed from the original default 65535.
IIS Manager > ApplicationPools > Advanced Settings
Queue Length : 65535
2. Adjust the IIS 7 appConcurrentRequestLimit settings
The default was changed to 5000 from the original 100,000.
c:\windows\system32\inetsrv\appcmd.exe set config /section:serverRuntime /appConcurrentRequestLimit:100000
In the%systemroot%\System32\inetsrv\config\applicationHost.configmiddle you can view this setting:<serverRuntime appConcurrentRequestLimit="100000" />
3. Adjust in machine.config processModel> requestQueueLimit provided
The default was changed to 5000 from the original 100,000.
-
<configuration> -
<system.web> -
<processModel requestQueueLimit="100000"/>
4. Modify the registry, to adjust the IIS 7 support simultaneous connections TCPIP
The default was changed to 5000 from the original 100,000.
reg add HKLM\System\CurrentControlSet\Services\HTTP\Parameteris /v MaxConnections /t REG_DWORD /d 100000
4 provided the above, the basic support 100,000 simultaneous requests.
If visits to more than 100,000, you can consider 程序 and 数据库 divided by function module, deployed to multiple servers to share access pressure.
Also it can be considered 软硬件负载均衡 .
- Hardware load balancing can be directly
智能交换机realized, processing capability, and has nothing to do with the system, but expensive, difficult to configure, the system can not distinguish between state practice and application. So hardware load balancing applies to一大堆设备，大访问量，简单应用. - Load balancing is based on system software and application, be able to live better distribute the load according to the state system and applications. Cost-effective. PCL load-balancing software, LVS software under Linux.
Second, the process of database-level
- When two users simultaneously access a page, a user may update another user has deleted records.
- In the time between a user loads the page and click the Delete button in his other users to modify the contents of this record.
So it is necessary to consider数据库锁the problem.
There are the following three in the concurrency control strategies available:
- Do nothing - if concurrent users modify the same record, so that the final results submitted by the entry into force (the default behavior).
- Optimistic concurrency (Optimistic Concurrency) - assumes that concurrency conflicts occur only occasionally, most of the time does not appear; then, when a conflict occurs, just simply inform the user, he made changes can not be saved, because other users We have modified the same record.
- Pessimistic concurrency (Pessimistic Concurrency) - assumes that concurrency conflicts occur frequently, and users can not tolerate being told their modification can not be saved due to the concurrent behavior of others; then, when a user starts editing a record, the record locking to prevent other users edit or delete the record, until he had finished and submit your changes.
When you try to modify data at the same time, the need to establish control mechanisms to prevent adverse effects to modify a user's modifications to other users operating simultaneously made. Systems deal with this situation is called "concurrency control."
The type of concurrency control
In general, the management database concurrency There are three common methods:
- Pessimistic concurrency control - during this time until the recording acquired from the database update, the line is unavailable to users.
- Optimistic concurrency control - only when the actual data update, the line is unavailable to other users fishes. Updates will check the line and determine whether any changes in the database. If you try to update records that have changed, it will cause concurrency conflicts.
- The last update to take effect - only when the actual update data, the bank fishes are not available to other users. However, the update does not compare with the initial recording; but only written records, which might rewrite the record changed since the last refresh performed by other users.
1. Pessimistic concurrency
Pessimistic concurrency commonly used for two purposes.
First, in some cases, a large number of records with the same contention. Placed on the data 锁 cost is less than the fee 发生并发冲突时回滚更改 costs charges.
In 事务过程 the case of the change should not be recorded, 保守式并发 but also very useful. Inventory application is a good example.
Assuming a company representative is a potential customer to check stock.
You usually want to lock the record until generate orders so far, that this will usually be marked as "Order" status and removed from the available stock.
If the order is not generated, then the lock is released, so that the user check the inventory of other available accurate inventory counts.
However, 断开the structure can not be pessimistic concurrency control. 连接打开Just enough time to read data or update data, and therefore 不能长时间地保持锁. In 长时间保留锁的应用程序将无法进行伸缩addition, .
2. Optimistic Concurrency
In the optimistic concurrency, the only 访问数据库only 设置并保持锁.
These locks will prevent other users from updating records at the same time. In addition to updating this exact moment, the data is always available.
For more information, see 开放式并发.
When trying
更新time, changed the initial version of the line will be compared to a database of existing lines.
If they are different, the update will fail and cause concurrency errors.
At this point, you use the business logic will be created to coordinate these two lines.
3. The last update to take effect
When the "last update to take effect," it will not 初始数据be checked, but just 更新written to the database.
Obviously, the following may occur:
A user obtain a record from the database.
User B can obtain the same record from the database, modify it, and then write the updated record back to the database.
User A modifies the "old" record and write it back to the database.
In the above case, the user A will never see the changes made by the user B's. If you plan to use the "last updated the entry into force" method of concurrency control, make sure that this situation is acceptable.
Three, ADO.NET, and Visual Studio .NET concurrency control
Since the data structure is based on 断开the data, ADO.NETand Visual Studio .NET use 开放式并发.
Therefore, you need to add business logic to use optimistic concurrency problem.
If you choose to use 开放式并发, it can determine whether a change has occurred in two conventional methods: 版本号方法(actual version number or date time stamp) and 保存所有值方法.
1. Method version number
In process version, to update the record must contain a date column having a version number or a time stamp.
When reading the record date time stamp or version number will be stored in the client.
Then, the value of the partial update.
A method of processing concurrency is only WHEREupdated if the value matches the value of the recording clause.
SQL representation of this method are:
-
UPDATE Table1 SET Column1 = @newvalue1, Column2 = @newvalue2 -
WHERE DateTimeStamp = @origDateTimeStamp
Alternatively, you can use the version number for comparison:
-
UPDATE Table1 SET Column1 = @newvalue1, Column2 = @newvalue2 -
WHERE RowVersion = @origRowVersionValue
If the date or time stamp version numbers match, the record indicates that the data storage area has not been altered, and can safely use the new values in the data set the record to be updated;
if you do not match, an error is returned.
You can write code to achieve this form of concurrency checks in Visual Studio .NET. You must also write code to respond to any update conflicts.
To ensure the accuracy of the date time stamp or version number, you need to 表on the set 触发器, so that when changes occur on the line, on the date or time stamp to update the version number.
2. Save all value method
Date stamp using alternative or version number is recorded at the time of reading 获取所有字段的副本.
The ADO.NET DataSet object to maintain two versions of each modified record:
初始版本(Version initially read from a data source), and修改版本(indicating that the user update).
When you try to write the record back to the data source, the initial value of the data row will be compared with the record in the data source.
If they match, it indicates that the database record has not been altered after they are read.
In this case, the value of the data set has been changed successfully written to the database.
For four of the data adapter the command ( DELETE、INSERT、SELECT 和 UPDATE), it has a set of parameters for each command. Each command has a parameter initial value and the current value (or a modified value).
The second type processing conditions:
Because it is 大并发请求, can be employed in the processing method of the first case,
further, as is 大数据量retrieved, it is necessary to consider 查询效率the question:
- Table indexed by search criteria
- To query optimization
- Consider using cached query data
Third act processing:
Processing method can be adopted in the first case,
addition is because the refresh operation on the same table, consider using the following approach:
- First the data stored in the cache, when the data reaches a certain amount, and then update the database
- The table by
索引dividing (分表，分区), for example: a table for storing information of people across the country, this amount of data is great, if by province is divided into multiple tables, the people in the country by province information stored in the corresponding table, then according to the respective provinces and queries and updates, such a large amount of concurrency and large data problems will be much reduced