Automatic differentiation is easy to use where an explicit expression for the cost function is available. Usually programs need to interact with external programs or data. In this chapter, we will consider several different approaches.
To this end, we will consider the problem of finding the parameters θ and t to solve the following optimization problem in the form of a distorted 2D affine transformation:
min ∑ i ∥ yi − f ( ∥ qi ∥ 2 ) qi ∥ 2 such that qi = R ( θ ) xi + t \begin{split}\min & \quad \sum_i \left \|y_i - f\left (\|q_{i}\|^2\right) q_i \right \|^2\\ \text{such that} & \quad q_i = R(\theta) x_i + t\end{split}minsuch thati∑
yi−f(∥qi∥2)qi
2qi=R ( i ) xi+t
Here, R is a 2D rotation matrix parameterized by the angle θ, and t is a 2D vector. f is an external distortion function.
Let's first consider the case where we have a templated function TemplatedComputeDistortion that computes a function f. The implementation of the corresponding residual functor is then straightforward, as follows:
template <typename T> T TemplatedComputeDistortion(const T r2) {
const double k1 = 0.0082;
const double k2 = 0.000023;
return 1.0 + k1 * r2 + k2 * r2 * r2;
}
struct Affine2DWithDistortion {
Affine2DWithDistortion(const double x_in[2], const double y_in[2]) {
x[0] = x_in[0];
x[1] = x_in[1];
y[0] = y_in[0];
y[1] = y_in[1];
}
template <typename T>
bool operator()(const T* theta,
const T* t,
T* residuals) const {
const T q_0 = cos(theta[0]) * x[0] - sin(theta[0]) * x[1] + t[0];
const T q_1 = sin(theta[0]) * x[0] + cos(theta[0]) * x[1] + t[1];
const T f = TemplatedComputeDistortion(q_0 * q_0 + q_1 * q_1);
residuals[0] = y[0] - f * q_0;
residuals[1] = y[1] - f * q_1;
return true;
}
double x[2];
double y[2];
};
So far so good, but now let's consider three ways of defining f that cannot be used directly for automatic differentiation:
- 1. A non-template function that computes its value.
- 2. Find the function of its value and derivative.
- 3. Defined as a function of the table of values to be interpolated.
We will consider them in turn below.
1、A function that returns its value
Suppose we are given a function ComputeDistortionValue declared as follows:
double ComputeDistortionValue(double r2);
Computes the value of f. The actual implementation of the function doesn't matter. There are three steps to interface this function with Affine2DWithDistortion:
- 1.
ComputeDistortionValueEncapsulate it into a functionComputeDistortionValueFunctor. - 2.
ComputeDistortionValueFunctorNumerically differentiate the NumericDiffCostFunction to create a CostFunction - 3. Use CostFunctionToFunctor to encapsulate CostFunction . Wrapped to get a
operator()function with templated operators. This operator function method can turn the Jacobian matrix calculated by NumericDiffCostFunctionJetinto an object.
The implementation of the above three steps is as follows:
struct ComputeDistortionValueFunctor {
bool operator()(const double* r2, double* value) const {
*value = ComputeDistortionValue(r2[0]);
return true;
}
};
struct Affine2DWithDistortion {
Affine2DWithDistortion(const double x_in[2], const double y_in[2]) {
x[0] = x_in[0];
x[1] = x_in[1];
y[0] = y_in[0];
y[1] = y_in[1];
compute_distortion.reset(new ceres::CostFunctionToFunctor<1, 1>(
new ceres::NumericDiffCostFunction<ComputeDistortionValueFunctor,
ceres::CENTRAL,
1,
1>(
new ComputeDistortionValueFunctor)));
}
template <typename T>
bool operator()(const T* theta, const T* t, T* residuals) const {
const T q_0 = cos(theta[0]) * x[0] - sin(theta[0]) * x[1] + t[0];
const T q_1 = sin(theta[0]) * x[0] + cos(theta[0]) * x[1] + t[1];
const T r2 = q_0 * q_0 + q_1 * q_1;
T f;
(*compute_distortion)(&r2, &f);
residuals[0] = y[0] - f * q_0;
residuals[1] = y[1] - f * q_1;
return true;
}
double x[2];
double y[2];
std::unique_ptr<ceres::CostFunctionToFunctor<1, 1> > compute_distortion;
};
2、A function that returns its value and derivative
Now suppose we have a function ComputeDistortionValue, we can get its value, and we can get its Jacobian matrix if we want. Its function declaration is as follows:
void ComputeDistortionValueAndJacobian(double r2,
double* value,
double* jacobian);
Again, the actual implementation of the function doesn't matter. Interfacing this function with Affine2DWithDistortiondocking requires two steps:
- 1.
ComputeDistortionValueAndJacobianEncapsulate it into a CostFunction object. This CostFunction object we callComputeDistortionFunction. - 2. Encapsulate the ComputeDistortionFunction object just obtained with CostFunctionToFunctor. Gets a Functor with a template operator
operator()method that turns the Jacobian computed by the NumericDiffCostFunction into a fitJetobject.
code show as below:
class ComputeDistortionFunction : public ceres::SizedCostFunction<1, 1> {
public:
virtual bool Evaluate(double const* const* parameters,
double* residuals,
double** jacobians) const {
if (!jacobians) {
ComputeDistortionValueAndJacobian(parameters[0][0], residuals, nullptr);
} else {
ComputeDistortionValueAndJacobian(parameters[0][0], residuals, jacobians[0]);
}
return true;
}
};
struct Affine2DWithDistortion {
Affine2DWithDistortion(const double x_in[2], const double y_in[2]) {
x[0] = x_in[0];
x[1] = x_in[1];
y[0] = y_in[0];
y[1] = y_in[1];
compute_distortion.reset(
new ceres::CostFunctionToFunctor<1, 1>(new ComputeDistortionFunction));
}
template <typename T>
bool operator()(const T* theta,
const T* t,
T* residuals) const {
const T q_0 = cos(theta[0]) * x[0] - sin(theta[0]) * x[1] + t[0];
const T q_1 = sin(theta[0]) * x[0] + cos(theta[0]) * x[1] + t[1];
const T r2 = q_0 * q_0 + q_1 * q_1;
T f;
(*compute_distortion)(&r2, &f);
residuals[0] = y[0] - f * q_0;
residuals[1] = y[1] - f * q_1;
return true;
}
double x[2];
double y[2];
std::unique_ptr<ceres::CostFunctionToFunctor<1, 1> > compute_distortion;
};
3、A function that is defined as a table of values
The third and final case we will consider is when the function f is defined as a table of values on the interval [0,100], with one value for each integer.
vector<double> distortion_values;
There are many ways to insert a table of values. Perhaps the simplest and most common method is linear interpolation. But using linear interpolation is not a good idea, because the interpolation function is not differentiable at the sampling points.
Another simple but excellent differentiable interpolation method is Cubic Hermite Spline (Chinese Hermite interpolation). Ceres provides the entire process of Cubic and Bi-Cubic interpolation, and can easily apply automatic differentiation algorithms.
To use Cubic interpolation, you need to first construct a Grid1D object to encapsulate the value table, and then use it to construct a CubicInterpolator object.
The resulting code looks like this:
struct Affine2DWithDistortion {
Affine2DWithDistortion(const double x_in[2],
const double y_in[2],
const std::vector<double>& distortion_values) {
x[0] = x_in[0];
x[1] = x_in[1];
y[0] = y_in[0];
y[1] = y_in[1];
grid.reset(new ceres::Grid1D<double, 1>(
&distortion_values[0], 0, distortion_values.size()));
compute_distortion.reset(
new ceres::CubicInterpolator<ceres::Grid1D<double, 1> >(*grid));
}
template <typename T>
bool operator()(const T* theta,
const T* t,
T* residuals) const {
const T q_0 = cos(theta[0]) * x[0] - sin(theta[0]) * x[1] + t[0];
const T q_1 = sin(theta[0]) * x[0] + cos(theta[0]) * x[1] + t[1];
const T r2 = q_0 * q_0 + q_1 * q_1;
T f;
compute_distortion->Evaluate(r2, &f);
residuals[0] = y[0] - f * q_0;
residuals[1] = y[1] - f * q_1;
return true;
}
double x[2];
double y[2];
std::unique_ptr<ceres::Grid1D<double, 1> > grid;
std::unique_ptr<ceres::CubicInterpolator<ceres::Grid1D<double, 1> > > compute_distortion;
};
In the above example, we used Grid1D and CubicInterpolator to interpolate a one-dimensional value table. Grid2D is used in conjunction with CubicInterpolator to allow users to insert two-dimensional tables of values. Note that neither Grid1D nor Grid2D are limited to scalar-valued functions, they can also use vector-valued functions.