12 monitoring metrics to ensure your API strategy is successful

The open source China community team made its first live broadcast, telling the story behind the open source China community in the name of sharing."

Original author: Andrew Stiefel of F5

Original link: 12 monitoring indicators to ensure a successful API strategy

Reprint source: NGINX open source community

The only official Chinese community of NGINX, all at nginx.org.cn

As companies adopt API-first design practices to build modern applications, measuring the operational performance and value of these APIs has become imperative. Building a framework that clearly defines API metrics and ties them to key performance indicators (KPIs) is one of the most important steps to ensuring a successful API strategy.

Typically, KPIs are closely tied to specific goals. They have clear time frames and match what the API strategy needs to deliver. In contrast, API metrics are important data points. Not all metrics are KPIs, but every KPI is based on metrics.

So how should you go about it? First, you need to clarify the goal of your API strategy from the beginning, and then choose the metrics that match that goal. Remember that each team needs to measure and track different metrics based on what’s important to them and necessary to the business.

Broadly speaking, there are three broad categories of API metrics that companies can track, and different categories of metrics reflect different issues:

Operational Metrics – Does the API provide the stability, reliability, and performance you require?
Adoption Metrics – Are developers adopting your API?
Product Metrics – How is the API supporting your business goals?

You can think of these three broad categories of indicators as a pyramid. Operational metrics at the bottom measure the tactical performance of individual APIs and their supporting infrastructure. The top product metrics measure the business value created by the API. The two are tied together through adoption metrics, which track the growth of API programs among end users (developers). Generally speaking, product metrics and adoption metrics relate to the business results you need to measure, while operational metrics relate to the technical standards you need to maintain.

This article breaks down 12 specific metrics that must be measured, describes how they support infrastructure and application teams, and explains how these metrics relate to KPIs.

Operation and maintenance indicators

In the initial stages, operational metrics are often the first metrics to be measured. They are tactical metrics that provide insight into how your API is performing. Operational indicators are usually not KPIs. But they can help you measure the quality and performance of the software your team builds and provide early indicators of new problems, or help you drill down and uncover issues that may impact key KPIs.

You can track different operational metrics based on team and responsibility.

Infrastructure team

The Platform Operations team maintains, connects, and secures the infrastructure and technology portfolio used by different teams to deliver applications. In the case of API programs, this typically includes API gateways and API developer portals.

Key metrics for infrastructure teams such as platform operations include:

Uptime – Even one of the most basic metrics, uptime is the gold standard for measuring service availability. It is often associated with a Service Level Agreement (SLA).
CPU and Memory Usage - Tracking the resource usage of your API Gateway is critical to determining when you may need to scale up your instance. It can also serve as an early warning indicator of an impending failure or a spike in usage due to errors.
Overall pass and error rates - Measuring how often an API triggers an HTTP error status code (not 200) can help you understand how error-prone your API is. This composite metric provides information that helps you judge the overall quality of the APIs your team puts into production.

application team

The application team consists of API developers and service owners, who are responsible for building and operating each service or application. These services or applications can be used as part of a larger product, integrated with partner products, or used to provide APIs as a service to developers.

Application teams need to measure the following metrics:

Requests per minute – This performance metric measures the number of requests your API handles per minute. While it changes over time, you generally want to limit the number of requests per minute to ensure the best experience for your API users.
Average and Maximum Latency – It’s critical to track the average time it takes for your API to receive a request and return a response. Slow APIs can have a negative impact on user experience, which can adversely affect your business.
Errors per minute - The truth is often the same, there is no perfect API. Mistakes are inevitable, sooner or later. You need to monitor errors and implement a well-planned course of action to fix them as quickly as possible before they suddenly escalate.

See Chapters 3-5 of the O'Reilly e-book Mastering API Architecture by NGINX for a deeper understanding of API operations and the KPIs and metrics that are critical to your business.

adoption metrics

API-first businesses should look beyond engineering metrics to understand how developers are interacting with your APIs. You also need to measure and monitor API developer experience to ensure developers are adopting your API and getting value from it.

Here are a few examples of using indicators:

Unique API Consumers – This metric typically measures how many developers are using your API via the number of monthly active users. Ideally, this metric will grow as more developers integrate your API into their applications.
API Usage Growth - This metric also measures API adoption and is often the preferred metric for measuring API adoption. Ideally, API traffic will grow month by month as more applications and developers use the API.
Time to First Call – This metric measures the time it takes for a developer to create an account, generate API credentials, and run the first API call. Getting developers up and running as quickly as possible is a high priority, so this metric is the most important metric for overall API developer experience.

Note: We recommend having at least one KPI measuring API adoption. This helps you estimate the overall growth of your API program. For example, you could set up a KPI to track the growth in the number of developers using your API to create continuous integrations or apps.

Product indicators

API product metrics play an important role in helping understand the value of your API. While only a small subset of APIs can bring direct revenue contribution, each API needs to create value for the business.

Key product metrics to measure include:

Direct and indirect revenue – These metrics measure the different ways in which APIs contribute to revenue. Some of these APIs are directly monetizable, others support integration with business partner products, or third-party integrations that are critical to customers. Like tracking API adoption, tracking indirect revenue helps developers build revenue-generating applications for partners.
Number of applications per API – APIs need to be reusable. This metric measures how many applications are integrated with the API to understand which APIs create the most value.
Number of partners – APIs often enable building business relationships. Tracking a partner's number of API integrations can help drive adoption and demonstrate value to other business units.

Note: These product indicators are closely related to business impact. You can choose to convert some product indicators into KPIs based on your business goals. For example, if the business goal of your API strategy is to reach more customers through a third-party provider, you can track the number of partners using your API and the indirect revenue generated through these integrations.

Please refer to Chapters 3-5 of "Mastering API Architecture" for a deeper understanding of the life cycle of API products, including business and operation KPIs.

Conclusion

Correlating API metrics with business KPIs is one of the primary ways businesses can make data-driven decisions and ensure their API strategy delivers the value they need. In addition, enabling API visibility helps infrastructure and application teams measure the operational metrics that matter most to each.

NGINX enables the visualization of dozens of important API indicators. You can view real-time and historical metrics and easily export them to your preferred application performance monitoring (or APM) or data analytics solution.

The only official Chinese community of NGINX, all at nginx.org.cn

More NGINX-related technical information, interactive Q&A, series of courses, and event resources: Open Source Community Official Website | WeChat Official Account