Datawhale dry goods
Translation: Duan Qiuyang Datawhale member
Research background
Inbal Shani, chief product officer at GitHub, said:
Today's developers do more than just write and ship code; they need to be proficient at applying a variety of tools, environments, and technologies, including the new field of generative AI coding tools. But it's not story points or deployment speed that matter most to developers. This is the Developer Experience (DevEx) , which defines how effectively and productively developers can go beyond the norm, into a state of flow, and make an impact .
I'm not only saying this as GitHub's Chief Product Officer, but as a long-time developer with experience working in various parts of the entire tech stack. Decades ago , when I got my master's degree in mechanical engineering, I became one of the first technologists to apply artificial intelligence in the lab. Back then, it took our model five days to process our large dataset—shocking given the speed of today's AI models. I long for tools that increase efficiency and reduce production time. That's why I'm passionate about developer experience and make it a focus for GitHub's Chief Product Officer.
In the rapid development of generative AI, we wanted to gain a better understanding from developers of how new tools and current workflows impact the overall developer experience. As a starting point, we looked at some of the biggest components of the developer experience: developer productivity, team collaboration, artificial intelligence, and how developers think they can best drive impact in enterprise environments .
To do this, we partnered with Wakefield Research to interview 500 developers at enterprise companies at companies with more than 1,000 employees in the US .
We'll show how organizations can remove barriers to help enterprise engineering teams drive innovation and impact in this new era of software development. Ultimately, the way to innovate at scale is to empower developers by increasing their productivity, increasing their satisfaction, and enabling them to do their best work every day. After all, there can be no progress without developers empowered to drive impact .
Main conclusions of the survey
Artificial intelligence has arrived and is being used on a massive scale. 92% of U.S. developers already use AI coding tools on and off work.
Waiting to build and test is still an issue . Despite the industry-wide investment in DevOps, developers still say that besides writing code, the most time-consuming thing they do on the job is wait for builds and tests.
The developers want more collaboration. The average developer in an enterprise environment works with 21 other engineers and expects collaboration to be the number one metric for performance evaluation.
They think artificial intelligence will help. More than four in five developers expect AI coding tools to make their teams more collaborative.
Developers also see huge benefits in AI. 70% said AI coding tools would give them an edge at work, citing better code quality, time to completion and resolution of incidents as some of the most important benefits AI will bring.
At GitHub, we realize that there is often a huge gap between the day-to-day of most developers and the "conversations about ' what developers want '".
Through this survey, we hope to gain a better understanding of the typical developer experience and identify key ways companies can support their developers and be more successful.
C = Collaboration, a multiplier for the entire developer experience.
Developer Experience (DevEx) is a formula that takes into account:
How easy and fast -- or rather productive -- it is for developers to implement changes on the code.
How smooth it is to go from idea to production to impact.
How work environment, workflow, and tools can positively or negatively affect developer satisfaction.
For leaders, developer experience is about creating a collaborative environment where developers can be most productive , impactful and satisfied at work . For developers, collaboration is one of the most important parts of the equation.
Specific results of the research
1. The current assessment indicators have not met the expectations of the developers
In many cases, developers value and want to do more, conflicting with how their performance is measured .
Learning new skills and designing solutions to new problems have the most positive impact on a developer's workday, but most of their time is spent waiting for code reviews, builds, or tests.
Developers also believe that their ability to work with others and the quality of their code , rather than the quantity and efficiency of their output, should be the primary performance indicators by which they are measured.
2. The developer said that the assessment indicators did not meet expectations
The way developers are currently evaluated doesn't align with how they think their performance should be measured.
For example, developers we surveyed said they are currently measured by the number of incidents they resolve. But developers believe that how they deal with these bugs and problems is a more important performance indicator. This is consistent with the belief that code quality trumps code quantity, that code quality should remain the most important metric.
Developers also believe that collaboration and communication should be as important as code quality when it comes to performance metrics. Their ability to collaborate and communicate with others is critical to their work, yet only 33% of developers say their companies use it as a performance indicator.
The metrics currently used to measure performance are compared to the metrics the developer believes should be used to measure their performance.
The highest-ranked responses, developers said their teams spent the most effort on tasks including writing code and finding and fixing security vulnerabilities.
3. Developers Want More Opportunities to Upskill and Increase Impact
When developers were asked what positively impacted their workday, learning new skills (43%), getting feedback from end users (39%), and automating testing (38%) and designing new issues were the top choices solution (36%).
Tasks that developers think have the most impact on their work
But developers say they spend most of their time writing code and testing, and then waiting for code to be reviewed or builds and tests to be executed.
On a typical day, the enterprise developers we surveyed reported that their teams were busy with a variety of tasks , including writing code, fixing security vulnerabilities, and getting feedback from end users. The developers also said they spent about the same amount of time on these tasks , suggesting they were stretched throughout the day .
What developers say they spend the most time on each day
Notably, developers say they spend the same amount of time waiting to build and test as they spend writing new code .
This shows that despite investments in DevOps tooling over the past decade, build and test wait times remain a chronic problem.
Developers also continue to face barriers such as waiting for code reviews, builds, and test runs, which can hinder their ability to learn new skills and design solutions to new problems, factors our research shows have an impact on their overall satisfaction maximum impact.
4. Developers want feedback from end users, but face challenges
Developers say getting feedback from end users (39%) is the second most important thing for positively impacting their workday – but for development teams, getting this feedback directly is often challenging.
Product managers and marketing teams often act as middlemen, making it difficult for developers to receive direct feedback from end users .
Ideally, developers receive feedback from automated and validation tests to improve their work, but sometimes these tests are sent to other teams before being handed off to engineering.
The top two daily tasks for development teams include writing code (32%) and finding and fixing security vulnerabilities (31%).
This shows how developers are taking security more seriously and highlights how companies are making security a priority.
It also demonstrates the critical role that corporate development teams play in meeting policy and board directives on security.
In our survey of enterprise engineers, developers said they collaborate with an average of 21 other developers on a typical project , and 52% reported collaborating with other teams on a daily or weekly basis. Notably, they ranked regular contact as the most important factor for effective cooperation .
A developer in an enterprise environment typically works with an average of 21 other developers on a daily or weekly basis.
But developers also have a holistic view of collaboration - not just defined as talking and meeting with others, but also uninterrupted working hours, access to fully configured developer environments, and formal mentor-mentee relationships:
The absence of team communication gives developers time and space to write code and work on team goals.
Access to a fully configured developer environment promotes consistency throughout the development process. It also helps developers collaborate faster and avoid the infamous " but it works on my machine ".
A mentorship system can help developers upskill and build interpersonal skills, which are essential in a collaborative work environment.
Our survey shows that the factors most important to effective collaboration are so critical that when they are not done effectively, they can have a noticeably negative impact on developers' work.
The tasks that developers say most often negatively impact their workday experience.
Ineffective meetings can distract developers instead of helping them.
6. We want to learn more about how developers collaborate
So we got some answers from our followers on Twitter. We asked developers what tips they have for collaborating effectively. Here's what one developer had to say:
Demonstrates synergistic behavior by working with peers and incorporating their ideas.
Share information and resources to foster teamwork.
Schedule regular meetings to brainstorm, set goals, and align roles.
Use collaborative chat tools like Slack or Teams to keep communication open.
Encourage face-to-face or video conferencing to communicate more clearly and avoid misunderstandings.
Go "side by side" by sharing documents and collaborating in real time.
Resolve conflicts quickly and professionally to maintain a healthy team environment.
Lead by example and invite co-creation and open communication.
Encourage team members to seek help when they need help or support.
Set clear, measurable goals to track collaboration progress and improvement.
We also asked what makes meetings productive and valuable:
Don't have a meeting if it can be resolved asynchronously.
Announce the meeting agenda in advance.
Set a reasonable time frame.
Clearly define clear goals after the meeting.
Driven by conflict (we waste too much time on topics everyone agrees on), we should focus on the things on which there is no consensus.
Send topics and readings ahead of time.
Record/share notes.
Clear actions and ownership after the meeting
7. Collaborate effectively to improve code quality
As the developer experience continues to be defined, so will successful developer collaboration. Too many tweets and messages can disrupt workflow, but there is still a need to stay connected .
In our survey, developers say effective collaboration improves test coverage and writes code faster, cleaner, and more safely—a best practice for any development team. This suggests that when developers collaborate effectively with others , they believe they build better, more secure software.
Developers generally agree that effective collaboration helps improve what they publish and how often they publish.
The developers we surveyed felt that collaboration and communication—along with code quality—should be the top priority of the assessment.
From DevOps to agile methodologies, developers and the larger business world have long talked about the importance of collaboration.
But developers are still not measured at this point.
We asked developers to share their thoughts on measuring their collaboration. Here's one developer's take: Companies and engineering managers should encourage regular team communication and set times to check in -- especially in remote environments, but respect developers' work needs and focus.
Developers believe that effective and regular contact with colleagues is critical to effective teamwork.
8. 4 Tips for Engineering Managers to Improve Collaboration
At GitHub, our researchers, developers, product teams, and analysts are dedicated to researching and improving developer productivity and satisfaction. Here's their advice to engineering leaders looking to improve collaboration among developers:
Make cooperation a goal in performance goals. This creates spaces and expectations for people to collaborate. This can take the form of lunch and learns, joint projects, etc.
Define and frame collaboration situations in your organization. Let people know when they are told about something versus when they are asked about something. A matrix outlining roles and responsibilities helps define everyone's role, which is what GitHub teams already implement.
Give developers time to talk and get to know each other. In particular, remote or hybrid organizations need to dedicate a portion of a developer's time and virtual space to building relationships.
Identify principal and distinguished engineers. Academic research supports the positive impact of change agents in organizations—and how they should be exceptionally good at collaborating. It's a matter of identifying your brilliant engineers and elevating them to a place where they can model the desired behavior.
9. Developers are already using AI coding tools at work
Ninety-two percent of developers working for large U.S. companies report using AI coding tools at work or in their personal time, and 70 percent say they see significant benefits from using these tools.
AI is here to stay—it's already changing the way developers approach their day-to-day work. This makes it critical for business and engineering leaders to adopt enterprise-grade AI tools to keep their developers from using unsanctioned applications. Companies should also establish governance standards for the use of AI tools to ensure their use is ethical and effective.
Almost all developers already use AI coding tools at work and outside of work.
10. Developers believe AI coding tools will improve their performance
Since most developers are experimenting with AI tools in the workplace, our findings suggest that it's not just interest that leads developers to use AI. Rather, it's an endorsement that AI coding tools will help them meet performance standards.
In our survey, developers indicated that AI coding tools could help them meet existing performance standards, improve code quality, speed up output, and reduce production-level incidents. They also believe that these metrics should be used to measure their performance beyond code count.
It's worth noting that the amount of code a developer produces doesn't necessarily match its business value.
stay smart. With the increasing use of AI tools in software development (which often increases code size), engineering leaders need to ask whether measuring code size is still the best way to measure productivity and output.
11. Developers think AI coding tools will lead to better teamwork
In addition to improving individual performance, more than four in five developers surveyed ( 81% ) said AI coding tools would help enhance collaboration within their teams and organizations.
In fact, security reviews, planning, and pair programming are the most important points of collaboration and tasks that development teams should undertake with the help of AI coding tools. It also suggests that code and security reviews will remain important as developers increase their use of AI coding tools in their work.
Developers believe their teams will need to become more collaborative as they start using AI coding tools.
Sometimes, developers can do the same thing with one or more lines of code . Still, a third of developers in our survey said their managers measure their performance based on the amount of code they write .
Notably, developers believe AI coding tools will allow them to spend more time focusing on scheme design. This has direct organizational benefits, meaning developers trust they will spend more time designing new features and products with AI, rather than writing boilerplate code.
Developers are already using generative AI coding tools to automate parts of their workflows, freeing up time for more collaborative projects, such as security reviews, planning, and pair programming.
12. Developers think AI can boost productivity and prevent burnout
Developers believe that AI coding tools will help them focus on more valuable problem solving.
According to our research, AI coding tools can not only help improve overall productivity, but also provide upskilling opportunities to help create a smarter workforce.
57% of developers believe that AI coding tools can help them improve their coding language skills—the top benefit they see. In addition to being a skill-enhancing aid, developers say AI coding tools can help reduce cognitive work, and since mental capacity and time are limited resources, 41% of developers believe AI coding tools can help prevent burnout.
In a previous study we conducted, 87% of developers reported that GitHub Copilot, an AI coding tool, helped them keep mental work while completing more repetitive tasks. This suggests that AI coding tools allow developers to reserve cognitive effort and focus on more challenging and innovative aspects of software development or R&D.
AI coding tools help developers improve their skills while working. In our surveys, developers consistently rank learning new skills as the number one contributor to a positive workday. But 30 percent also said L&D negatively impacted their overall workday, suggesting some developers felt L&D would add to their workday. Notably, developers say the biggest benefit of AI coding tools is learning new skills—these tools help developers learn while they're on the job, rather than making learning and development an extra task.
Developers are already using generative AI coding tools to automate parts of their workflows, freeing up time for more collaborative projects, such as security reviews, planning, and pair programming.
Outlook
Developer satisfaction, productivity, and organizational impact can all benefit from AI coding tools—which will have a significant impact on the overall developer experience.
92% of developers already say they use AI coding tools at work and in their personal time, clearly showing that AI is here to stay. Of the developers we surveyed, 70% said they have seen significant benefits from using AI coding tools, and 81% of the developers we surveyed expect AI coding tools to make their teams more collaborative—this is important for A net benefit for companies looking to improve both developer productivity and developer experience.
Notably, 57% of developers believe that AI can help them improve their skills and has the potential to integrate learning and development into their daily workflow. With all of this in mind, technology leaders should start exploring AI as a solution to improve satisfaction, productivity, and the overall developer experience.
In addition to exploring AI tools, engineering and business leaders should consider the following three key points to improve the developer experience:
Get your developers into a state of flow with the tools, processes, and practices that help them be more productive, drive impact, and do creative and meaningful work.
Facilitates collaboration by breaking down organizational silos and giving developers the opportunity to communicate effectively.
Make room for upskilling in developer workflows with critical investments in AI to help your organization experiment and innovate for the future.
References:
https://github.blog/2023-06-13-survey-reveals-ais-impact-on-the-developer-experience/
It's not easy to organize, so I like it three times ↓