Connection recovery and command interception of entity framework in MVC program
This is a translation of Microsoft's official tutorial Getting Started with Entity Framework 6 Code First using MVC 5 series. Here is the fourth article: Connection Recovery and Command Interception of Entity Framework in MVC Program
So far, the application can already run normally on your local machine. But if you want to publish it on the Internet for more people to use, you need to deploy the program to the WEB server and the database to the database server.
In this tutorial, you will learn two valuable features when deploying an Entity Framework to a cloud environment: connection reply (automatic retry of transient errors) and command interception (capture all SQL query statements sent to the database in order to Record them in the log or change them).
Note: This tutorial is optional. If you skip this section, we will make some minor adjustments in subsequent tutorials.
Enable connection recovery
When you deploy an application to Windows Azure, you deploy the database to a Windows Azure SQL database—a cloud database service. Compared with connecting your web server and database directly in the same data center, connecting to a cloud database service is more likely to encounter transient connection errors. Even if the cloud Web server and the cloud database server are in the same data center computer room, when there is a large number of data connections between them, it is easy to have various problems, such as load balancing.
In addition, the cloud server is usually shared by other users, which means that it may be affected by other users. Your access rights to the database may be restricted, and you may also encounter SLA-based bandwidth restrictions when you try to access the database server frequently. Most connection problems occur instantaneously when you connect to the cloud server, and they will try to solve the problem automatically in a short time. So when you try to connect to the database and encounter an error, the error is likely to be transient. When you repeat the attempt, the error may no longer exist. You can use automatic transient error retry to improve your customer experience. The connection recovery in Entity Framework 6 can automatically retry the wrong SQL query.
The connection recovery function is only available after the correct configuration of a specific database service:
- You must know that those exceptions may be temporary. You want to retry the error caused by the network connection, not the programming bug.
- You must wait for the appropriate time in the interval between failed operations. Online users may have to wait for a long time to get a response when retrying in batches.
- You need to set an appropriate number of retries. In an online application, you may retry multiple times.
You can manually configure these settings for any database environment supported by the Entity Framework provider, but Entity Framework has already made the default configuration for online applications that use Windows Azure SQL Database. Next we will implement these configurations in Contoso University.
If you want to enable connection recovery, you need to create a class derived from DbConfiguration in your assembly, which will be used to configure the SQL database execution strategy, which contains the connection recovery retry strategy.
In the DAL folder, add a new class called SchoolConfiguration.cs.
Replace the ones in the class with the following code:
using System.Data.Entity;
using System.Data.Entity.SqlServer;
namespace ContosoUniversity.DAL
{
public class SchoolConfiguration : DbConfiguration
{
public SchoolConfiguration()
{
SetExecutionStrategy("System.Data.SqlClient", () => new SqlAzureExecutionStrategy());
}
}
}The Entity Framework will automatically run the code found in the class derived from the DbConfiguration class. You can also use the Dbconfiguration class to configure in web.config. For details, see EntityFramework Code-Based Configuration.
In the student controller, add a reference:
using System.Data.Entity.Infrastructure;Change all exception code blocks that catch DataException, use RetryLimitExcededException:
catch (RetryLimitExceededException /* dex */)
{
//Log the error (uncomment dex variable name and add a line here to write a log.
ModelState.AddModelError("", "Unable to save changes. Try again, and if the problem persists see your system administrator.");
}Before, you used DataException. This will try to find the exception that may contain a transient error, and then return to the user a friendly retry prompt message, but now that you have turned on the automatic retry strategy, the error that still fails after multiple retry will be wrapped in RetryLimitExceededException .
For details, see Entity Framework Connection Resiliency / Retry Logic.
Enable command interception
You have now turned on the retry strategy, but how do you test to verify that it is working as expected? Forcing a transient error is not easy, especially if you are running locally. And transient errors are difficult to integrate into automated unit tests. If you want to test the connection recovery function, you need a method that can intercept the query sent by the Entity Framework to the SQL database and replace the SQL database to return the response.
You can also follow the best practice on a cloud application: log the latency and success or failure of all calls to external services to implement query interception. Entity Framework 6 provides a dedicated logging API to make it easy to log. But in this tutorial, you will learn how to directly use the Entity Framework's interception feature, including logging and simulating transient errors.
Create a logging interface and class
Best practice for logging is to call System.Diagnostice.Trace or logging classes through an interface instead of using hard coding. This can make it easier to change the logging mechanism later when needed. So in this section, we will create an interface and implement it.
Create a folder in the project and name it Logging.
In the Logging folder, create an interface class named ILogger.cs and replace the automatically generated one with the following code:
using System;
namespace ContosoUniversity.Logging
{
public interface ILogger
{
void Information(string message);
void Information(string fmt, params object[] vars);
void Information(Exception exception, string fmt, params object[] vars);
void Warning(string message);
void Warning(string fmt, params object[] vars);
void Warning(Exception exception, string fmt, params object[] vars);
void Error(string message);
void Error(string fmt, params object[] vars);
void Error(Exception exception, string fmt, params object[] vars);
void TraceApi(string componentName, string method, TimeSpan timespan);
void TraceApi(string componentName, string method, TimeSpan timespan, string properties);
void TraceApi(string componentName, string method, TimeSpan timespan, string fmt, params object[] vars);
}
}The interface provides three trace levels to indicate the relative importance of logs, and is designed to provide latency information for external service calls (such as database queries). The log method provides overloads that allow you to pass exceptions. In this way, exception information can be included in the stack and internal exceptions can be reliably recorded by the class implemented by the interface, rather than relying on each log method from the application to call and record.
The TraceAPI method enables you to trace the delay time of each call to an external service (such as SQL Server). In the Logging folder, create a class named Logger.cs and replace the automatically generated one with the following code:
using System;
using System.Diagnostics;
using System.Text;
namespace ContosoUniversity.Logging
{
public class Logger : ILogger
{
public void Information(string message)
{
Trace.TraceInformation(message);
}
public void Information(string fmt, params object[] vars)
{
Trace.TraceInformation(fmt, vars);
}
public void Information(Exception exception, string fmt, params object[] vars)
{
Trace.TraceInformation(FormatExceptionMessage(exception, fmt, vars));
}
public void Warning(string message)
{
Trace.TraceWarning(message);
}
public void Warning(string fmt, params object[] vars)
{
Trace.TraceWarning(fmt, vars);
}
public void Warning(Exception exception, string fmt, params object[] vars)
{
Trace.TraceWarning(FormatExceptionMessage(exception, fmt, vars));
}
public void Error(string message)
{
Trace.TraceError(message);
}
public void Error(string fmt, params object[] vars)
{
Trace.TraceError(fmt, vars);
}
public void Error(Exception exception, string fmt, params object[] vars)
{
Trace.TraceError(FormatExceptionMessage(exception, fmt, vars));
}
public void TraceApi(string componentName, string method, TimeSpan timespan)
{
TraceApi(componentName, method, timespan, "");
}
public void TraceApi(string componentName, string method, TimeSpan timespan, string fmt, params object[] vars)
{
TraceApi(componentName, method, timespan, string.Format(fmt, vars));
}
public void TraceApi(string componentName, string method, TimeSpan timespan, string properties)
{
string message = String.Concat("Component:", componentName, ";Method:", method, ";Timespan:", timespan.ToString(), ";Properties:", properties);
Trace.TraceInformation(message);
}
private static string FormatExceptionMessage(Exception exception, string fmt, object[] vars)
{
// Simple exception formatting: for a more comprehensive version see
// http://code.msdn.microsoft.com/windowsazure/Fix-It-app-for-Building-cdd80df4
var sb = new StringBuilder();
sb.Append(string.Format(fmt, vars));
sb.Append(" Exception: ");
sb.Append(exception.ToString());
return sb.ToString();
}
}
}We used System.Diagnostics for tracking. This is a built-in feature of .Net that makes it easy to generate and use tracking information. You can use various listeners of System.Diagnostics to track and write to the log file. For example, store them in blob storage or store in Windows Azure. You can find more options and related information in Troubleshooting Windows Azure Web Sites in Visual Studio. In this tutorial you will only see the log in the VS output window. In a production environment you may want to use the tracking package instead of System.Diagnostics, and when you need it, the ILogger interface makes it relatively easy to switch to a different tracking mechanism.
Create interceptor class
Next, you will create several classes that will be called each time the Entity Framework queries the database. One of them simulates transient errors and the other logs. These interceptor classes must be derived from the DbCommandInterceptor class. You need to rewrite the method so that it will be called automatically when the query is executed. In these methods, you can check or record the queries sent to the database, and you can modify the queries before sending them to the database, or even return them to the entity framework without sending them to the database for query.
Create a class named SchoolInterceptorLogging.cs in the DAL folder and replace the automatically generated one with the following code:
using System;
using System.Data.Common;
using System.Data.Entity;
using System.Data.Entity.Infrastructure.Interception;
using System.Data.Entity.SqlServer;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Reflection;
using System.Linq;
using ContosoUniversity.Logging;
namespace ContosoUniversity.DAL
{
public class SchoolInterceptorLogging : DbCommandInterceptor
{
private ILogger _logger = new Logger();
private readonly Stopwatch _stopwatch = new Stopwatch();
public override void ScalarExecuting(DbCommand command, DbCommandInterceptionContext<object> interceptionContext)
{
base.ScalarExecuting(command, interceptionContext);
_stopwatch.Restart();
}
public override void ScalarExecuted(DbCommand command, DbCommandInterceptionContext<object> interceptionContext)
{
_stopwatch.Stop();
if (interceptionContext.Exception != null)
{
_logger.Error(interceptionContext.Exception, "Error executing command: {0}", command.CommandText);
}
else
{
_logger.TraceApi("SQL Database", "SchoolInterceptor.ScalarExecuted", _stopwatch.Elapsed, "Command: {0}: ", command.CommandText);
}
base.ScalarExecuted(command, interceptionContext);
}
public override void NonQueryExecuting(DbCommand command, DbCommandInterceptionContext<int> interceptionContext)
{
base.NonQueryExecuting(command, interceptionContext);
_stopwatch.Restart();
}
public override void NonQueryExecuted(DbCommand command, DbCommandInterceptionContext<int> interceptionContext)
{
_stopwatch.Stop();
if (interceptionContext.Exception != null)
{
_logger.Error(interceptionContext.Exception, "Error executing command: {0}", command.CommandText);
}
else
{
_logger.TraceApi("SQL Database", "SchoolInterceptor.NonQueryExecuted", _stopwatch.Elapsed, "Command: {0}: ", command.CommandText);
}
base.NonQueryExecuted(command, interceptionContext);
}
public override void ReaderExecuting(DbCommand command, DbCommandInterceptionContext<DbDataReader> interceptionContext)
{
base.ReaderExecuting(command, interceptionContext);
_stopwatch.Restart();
}
public override void ReaderExecuted(DbCommand command, DbCommandInterceptionContext<DbDataReader> interceptionContext)
{
_stopwatch.Stop();
if (interceptionContext.Exception != null)
{
_logger.Error(interceptionContext.Exception, "Error executing command: {0}", command.CommandText);
}
else
{
_logger.TraceApi("SQL Database", "SchoolInterceptor.ReaderExecuted", _stopwatch.Elapsed, "Command: {0}: ", command.CommandText);
}
base.ReaderExecuted(command, interceptionContext);
}
}
}For commands that are successfully queried, this code writes relevant information and delay information to the log. For exceptions, it will create an error log.
Create a class named SchoolInterceptorTransientErrors.cs in the DAL folder. This class generates virtual transient errors when you enter "Throw" into the search box and make a query. Replace the automatically generated one with the following code:
using System;
using System.Data.Common;
using System.Data.Entity;
using System.Data.Entity.Infrastructure.Interception;
using System.Data.Entity.SqlServer;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Reflection;
using System.Linq;
using ContosoUniversity.Logging;
namespace ContosoUniversity.DAL
{
public class SchoolInterceptorTransientErrors : DbCommandInterceptor
{
private int _counter = 0;
private ILogger _logger = new Logger();
public override void ReaderExecuting(DbCommand command, DbCommandInterceptionContext<DbDataReader> interceptionContext)
{
bool throwTransientErrors = false;
if (command.Parameters.Count > 0 && command.Parameters[0].Value.ToString() == "%Throw%")
{
throwTransientErrors = true;
command.Parameters[0].Value = "%an%";
command.Parameters[1].Value = "%an%";
}
if (throwTransientErrors && _counter < 4)
{
_logger.Information("Returning transient error for command: {0}", command.CommandText);
_counter++;
interceptionContext.Exception = CreateDummySqlException();
}
}
private SqlException CreateDummySqlException()
{
// The instance of SQL Server you attempted to connect to does not support encryption
var sqlErrorNumber = 20;
var sqlErrorCtor = typeof(SqlError).GetConstructors(BindingFlags.Instance | BindingFlags.NonPublic).Where(c => c.GetParameters().Count() == 7).Single();
var sqlError = sqlErrorCtor.Invoke(new object[] { sqlErrorNumber, (byte)0, (byte)0, "", "", "", 1 });
var errorCollection = Activator.CreateInstance(typeof(SqlErrorCollection), true);
var addMethod = typeof(SqlErrorCollection).GetMethod("Add", BindingFlags.Instance | BindingFlags.NonPublic);
addMethod.Invoke(errorCollection, new[] { sqlError });
var sqlExceptionCtor = typeof(SqlException).GetConstructors(BindingFlags.Instance | BindingFlags.NonPublic).Where(c => c.GetParameters().Count() == 4).Single();
var sqlException = (SqlException)sqlExceptionCtor.Invoke(new object[] { "Dummy", errorCollection, null, Guid.NewGuid() });
return sqlException;
}
}
}This code only rewrites the ReaderExcuting method used to return multiple rows of query result data. If you want to check other types of connection recovery, you can override methods such as NonQueryExecuting and ScalarExecuting just as you do in the log interceptor. When you run the student page and enter "Throw" as the search string, the code will create a virtual SQL database with error number 20, which is treated as a transient error type. The currently recognized transient error numbers are 64,233,10053,10060,10928,10929,40197,40501 and 40613, etc. You can check the new version of the SQL database to confirm this information.
This code returns an exception to the Entity Framework instead of running the query and returning the query result. The transient exception will return 4 times and then the code will run normally and return the query result. Since we have all the log records, you can see that the Entity Framework made 4 queries before it was successfully executed, and in the application, the only difference is that the events it takes to render the page become longer. The number of retries of the Entity Framework is configurable. In this code, we set 4 because this is the default value of the SQL database execution strategy. If you change the execution strategy, you also need to change the existing code to specify the number of transient errors generated. You can also change the code to generate more exceptions to cause RetryLimitExceededException of Entity Framework. The values you enter in the search box will be saved in command.Parameters [0] and command.Parameters [1] (one for the last name and the other for the first name). When the input value is found to be "Throw", the parameter is replaced with "an" to query some students and return. This is just a way to test the connection recovery through the application's UI. You can also write code for updates to generate transient errors.
In Global.asax, add the following using statement:
using ContosoUniversity.DAL;
using System.Data.Entity.Infrastructure.Interception;Add the highlighted line to the Application_Start method:
protected void Application_Start()
{
AreaRegistration.RegisterAllAreas();
FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
RouteConfig.RegisterRoutes(RouteTable.Routes);
BundleConfig.RegisterBundles(BundleTable.Bundles);
DbInterception.Add(new SchoolInterceptorTransientErrors());
DbInterception.Add(new SchoolInterceptorLogging());
}
These codes start the interceptor when the entity framework sends the query to the database. Please note that because you separately created the transient errors and log records of the interceptor class, you can independently disable and enable them.
You can use the DbInterception.Add method anywhere in the application to add an interceptor, it does not have to be done in Applicetion_Start. Another option is to put this code into the DbConfiguration class where you created the execution strategy before.
public class SchoolConfiguration : DbConfiguration
{
public SchoolConfiguration()
{
SetExecutionStrategy("System.Data.SqlClient", () => new SqlAzureExecutionStrategy());
DbInterception.Add(new SchoolInterceptorTransientErrors());
DbInterception.Add(new SchoolInterceptorLogging());
}
}No matter where you put the code, be careful not to exceed it once. Performing DbInterception.Add for the same interceptor may cause you to get additional interceptor instances. For example, if you add a logging interceptor twice, you will see that the query is recorded in both logs. The interceptor is executed in the order in which the Add method is registered. Depending on what you want to do, the order may be important. For example, the first interceptor may change the CommandText property, and the next interceptor will get the changed property.
You have written code that simulates transient errors, and you can now test by entering a different value in the user interface. As an alternative, you can write code that directly generates transient errors without checking specific parameter values in the interceptor. Remember to add the interceptor only when you want to test transient errors.
Test logging and connection recovery
Press F5 to run the program in debug mode, and then click the Student tab.
Check the VS output window to view the trace output, you may want to scroll up the window content.
You can see the SQL query that was actually sent to the database.
In the reference page, enter "Throw" to query.
You will notice that the browser will hang for a few seconds, obviously the Entity Framework is retrying the query. The first retry occurs quickly, and then each retry query will add a little wait event. When the page execution is complete, check the output window, you will see that the same query was tried 5 times, and the first 4 times all returned a transient error exception. For each transient error, you see abnormal information in the log.
The query of raw data is parameterized:
SELECT TOP (3)
[Project1].[ID] AS [ID],
[Project1].[LastName] AS [LastName],
[Project1].[FirstMidName] AS [FirstMidName],
[Project1].[EnrollmentDate] AS [EnrollmentDate]
FROM ( SELECT [Project1].[ID] AS [ID], [Project1].[LastName] AS [LastName], [Project1].[FirstMidName] AS [FirstMidName], [Project1].[EnrollmentDate] AS [EnrollmentDate], row_number() OVER (ORDER BY [Project1].[LastName] ASC) AS [row_number]
FROM ( SELECT
[Extent1].[ID] AS [ID],
[Extent1].[LastName] AS [LastName],
[Extent1].[FirstMidName] AS [FirstMidName],
[Extent1].[EnrollmentDate] AS [EnrollmentDate]
FROM [dbo].[Student] AS [Extent1]
WHERE ([Extent1].[LastName] LIKE @p__linq__0 ESCAPE N'~') OR ([Extent1].[FirstMidName] LIKE @p__linq__1 ESCAPE N'~')
) AS [Project1]
) AS [Project1]
WHERE [Project1].[row_number] > 0
ORDER BY [Project1].[LastName] ASC:You have no parameters to record values in the log, of course, you can also choose to record. You can get the property value from the parameter property of the DbCommand object in the interceptor method.
Please note that you cannot repeat this test unless you stop the entire application and restart it. If you want to be able to perform multiple tests in the running of a single application, you can write code to reset the error counter in SchoolInterceptorTransientErrors. To see the difference in execution strategy, comment out SchoolConfiguration.cs, then close the application and restart debugging, run the student index page and enter "Throw" to search.
public SchoolConfiguration()
{
//SetExecutionStrategy("System.Data.SqlClient", () => new SqlAzureExecutionStrategy());
}This time when the first query is attempted, the debugger will immediately stop with an exception.
Uncomment and try again to understand the difference.
to sum up
In this section you saw how to enable connection recovery for Entity Framework and record the SQL query commands sent to the database. In the next section you will use Code First Migrations to deploy it to the Internet.
author information
Tom Dykstra -Tom Dykstra is a senior programmer and writer on the Microsoft Web Platform and Tools team.
