Technology is integrated into every aspect of the modern workplace. Operating costs, security, communications, employee satisfaction, and customer base all depend on technology. Savvy CIOs know that there is a direct correlation between a high-performing IT organization and a high-performing business.
As a technical leader, you need to be able to measure how fast your team is progressing and whether they are headed in the right direction. If you don't measure, you can't improve.
1. Lessons learned from the shortcomings of previous measurement methods
Trying to measure how well a tech team is delivering is tricky. A team is a collection of individuals. In the case of an IT organization, these individuals perform different and complex tasks. Over the years, managers of software development teams have tried many ways to measure productivity, most of which suffer from these two fundamental flaws:
1. Focus on outputs rather than results.
2. Emphasize individuals rather than teams.
These flawed approaches produced several anti-patterns that not only failed to provide meaningful productivity measures, but also contributed to low team morale.
lines of code
Perhaps the most well-known and most hated measure of developer productivity is counting lines of code. There is little correlation between the number of lines of code a developer writes and the total value the developer delivers to the organization.
In fact, rewarding developers for lines of code written leads to code bloat and ultimately higher maintenance costs.
speed
Given the popularity of agile methods in the software development world, some agile coaches may recommend using velocity as a way to measure team productivity. Team velocity rather than individual contributor velocity is a useful metric for planning workloads.
However, as a measure of productivity, speed is not satisfactory. Equating velocity with productivity will only lead to inflated estimates by developers, which not only misrepresents the effectiveness of the team, but may also eliminate the usefulness of this metric in capacity planning.
Utilization
In many consulting organizations, developer utilization (i.e., the time they spend coding) is used as a proxy for productivity. This is doubly flawed because we all know effort doesn't always mean results, since this kind of measurement incentivizes project managers to keep developers at 100% utilization.
In mathematics, queue theory tells us that when utilization reaches 100%, delivery times approach infinity. This is due to the fact that resources that are 100% utilized have no capacity to innovate, improve or change.
2. Take a Data-Driven Approach to Measuring Software Delivery
In 2018, Nicole Forsgren, Jez Humble, and Gene Kim published the book Accelerate, which included a cluster analysis of more than 23,000 responses from more than 2,000 different organizations. They found four common characteristics in the data that help classify software development teams as high, medium, or low performers:
- Lead time for changes: How long does it take from committing code to running it in production?
- Deployment frequency: How often does your team deliver software updates to your live customer base?
- Mean time to recovery: How long does it take your team to restore service after a failure is detected in production?
- Change Failure Rate: What percentage of changes to the production environment subsequently required remediation?
3. Consider Other Factors Affecting Team Performance
In addition to strictly code-based metrics, there are several cultural factors that help evaluate software team performance.
- Team members actively seek information.
- There will be no penalty for delivering bad news.
- Responsibility is shared.
- Collaboration across functions is rewarded.
- Failure is seen as an opportunity for improvement.
- New ideas are always welcome.
4. Allow time to evaluate performance data
Once you know what metrics indicate how your team is performing, you as a CIO must set aside the time and resources to build a dashboard to measure it. The data you need will most likely not come from a single place, so you will need to capture and transform data from multiple sources and then present it using a custom visualization tool like Tableau or PowerBI.
It is best to start simple and gradually expand to where you can get the most value. You can usually get most of the quantitative data you need from version control systems and APIs on code pipelines. For more qualitative measures, consider quarterly surveys.
5. Implement changes based on performance data
At the end of the day, collecting data and metrics (even a fraction) is a waste of effort if the organization is not continually reviewing the data and using it to correct the direction of the business.
As an organization, setting aside time to regularly review metrics, gather valuable information, and implement changes based on data is the fastest path to becoming a high-performing IT business.
Original title: 5 tips for measuring developer productivity , Author: William J. Francis