Uncategorized Measure Outcomes, Not Outputs: Software Development in Today's Remote Work World – InfoQ.com
InfoQ Live Feb 22: How do traditional security approaches scale in Cloud Native architectures? Register Now
Facilitating the spread of knowledge and innovation in professional software development
In this podcast summary Thomas Betts, Wes Reisz, Shane Hastie, Charles Humble, Srini Penchikala, and Daniel Bryant discuss what they have seen in 2021 and speculate a little on what they hope to see in 2022. Topics explored included: hybrid working and the importance of ethics and sustainability within technology.
What is the single best API technology you should always use? Thomas Betts moderated the discussion, with the goal to understand some of the high-level features and capabilities of three popular technologies for implementing APIs. The discussion covers some of the pros and cons of GraphQL and gRPC, and why you might use them instead of a RESTful API.
In this article, author Juan Pan discusses the data sharding architecture patterns in a distributed database system. She explains how Apache ShardingSphere project solves the data sharding challenges. Also discussed are two practical examples of how to create a distributed database and an encrypted table with DistSQL.
Psychological safety is a work climate where employees feel free to express their questions, concerns, ideas and mistakes. We cannot have high-performing teams without psychological safety. In this article, you will learn practical ideas, interesting stories, and powerful approaches to boost psychological safety in your team.
At QCon Plus, Mathias Schwarz, a software engineer at Uber, presented safe and fast deploys at planet scale. Uber is a big business and has several different products. They are, in most cases, deployed to dozens or hundreds of markets all over the world.
How do traditional security approaches scale in Cloud Native architectures? Register Now!
Learn from practitioners driving innovation and change in software. Attend in-person on April 4-6, 2022.
Uncover emerging trends and practices from software leaders. Attend online on May 10-20, 2022.
Your monthly guide to all the topics, technologies and techniques that every professional needs to know about. Subscribe for free.
InfoQ Homepage Articles Measure Outcomes, Not Outputs: Software Development in Today’s Remote Work World
Dec 17, 2021 10 min read
by
reviewed by
The expression ‘time is money’ is particularly suitable for software startups who are under immense pressure to develop innovative products faster than their competitors. In order to build products more quickly, one of the metrics that startups need to track is the elapsed time from the point a feature is defined to when it is rolled out to customers. This duration is known as Cycle Time. In many cases, company revenues are often directly impacted by the cycle time for product feature delivery. Cycle time depends on how fast developers can write code that can be moved to production. But speed is not the only consideration. The quality of code is equally important. Code quality decides whether product features would work as envisioned and also influences the ease or difficulty of testing and maintaining the product.
Therefore, it is paramount that software startups optimize developer productivity to get both the speed and quality of code necessary to move features into production quickly. This is easier said than done. Even before the COVID-19 pandemic forced most developers to leave the office and write code from home, there was not a standard, agreed upon metric for measuring developer productivity. Now that developers are working remotely in separate locations from product owners, the need for an objective way to measure the productivity and quality of code has become glaringly apparent.
Although there is no standard metric for measuring developer productivity, there are two approaches commonly used to measure software development efficiency. One method is to count the lines of code (LOC) or source lines of code (SLOC) a developer writes. This measurement is based on the premise that the size of the software project and amount of source code for a particular system directly relates to the technical complexity of the system. SLOC is typically used as a way to predict the amount of time and number of people required to develop a software program. Once the project begins, the LOC metric is also often used to estimate programming productivity.
Uncover emerging trends and practices from domain experts. Attend in-person QCon London (April 4-6, 2022) or online QCon Plus (May 10-20, 2022).
But the issue with using LOC as a metric for developer productivity, is that this measurement prioritizes quantity over quality and can indirectly encourage developers to write code that is inefficient and can create a huge technical debt over time. The LOC metric only captures the code written by the developers, but not what really went to production. It is possible that a developer writes a bunch of inefficient code that gets refactored or deleted during code review and feature testing. In fact, with the LOC metric, the more code the developer writes and deletes, the higher the productivity number. So it’s important to look at what part of the code written was actually useful.
The second approach is to measure the number of ‘story points’ completed. A story point is "a metric used in agile project management and development to determine (or estimate) the difficulty of implementing a given story. Elements considered in assigning a story point include the complexity of the story, the number of unknown factors and the potential effort required to implement it."
The issue with story points is that they are arbitrary and measuring them is subjective. What one person considers as one story point, another may consider two. For example, a development team may assign 1 story point to the feature based on the complexity, but another team might consider the same feature to be 2 story points based on their understanding. There is no defined formula or measurement that people can use to determine story points – it is all subjective. Therefore, this form of measurement could be inexact and can lead a developer or a development team to think they are doing a good job, when in reality, they are highly inefficient.
I have worked with many engineering teams in my role as CTO at a software development company that has built technology products for more than 150 startups across the globe. Over the years, my experience has taught me that the code released to production (outcome) is more important than the amount of code each developer writes/ commits (output) every day. In an ideal situation, both should be the same, but multiple factors affect the outcomes.
The most objective way to measure developer efficiency is to analyze the code developers write based on three factors:
These metrics not only reveal how skilled the developers are, but also how well they understand requirements – which is much more challenging these days with the majority of developers working remotely and working on the right priorities.
Measuring the useful lines of code is an important aspect of understanding a software developer’s productivity. Usually, developers write code to contribute to a feature.
A typical development process would involve:
By using lines of code committed as a metric, a developer who builds features with efficient code, minimal iterations and good design will seem less productive as compared to someone who copy pastes code or commits generated files. Therefore, it’s important to measure the ‘Useful or productive lines of code’ (PLOC). PLOC is defined as the lines of code released to production after code review, unit testing and integration testing.
One could argue that even after code is moved into production, the code may need to be cleanly refactored to remove bloat, technical debt and/or to remove features that are no longer useful to reduce the size of the product, which in turn means lower PLOC. Refactoring is good and it does lead to reduction of code. However, if we consider the productivity trend of 6 months and see that the developer has spent most of the time in refactoring rather than building new features, then there is likely an issue with the quality of code delivered and it might make sense to rewrite the feature from scratch rather than spend time fixing it.
PLOC can be calculated by looking at all the commits to the release branch after identifying and removing anomalies such as large one time commits (typically open source code), duplicate code, generated files etc. and applying a churn factor. Anomalies can be automatically detected by using machine learning algorithms. Churn factor can be calculated by looking at the additions / deletions to the file or repository.
PLOC ensures that developers do not game the system by committing extra unnecessary code. The measure also helps identify highly productive developers who not only write well designed, good quality code but also do so by understanding the requirements, so things get done right the first time.
Lower productivity does not always mean that the developer lacks skills and is therefore inefficient. Comparing how much code was written to how much was moved into production provides some key insights.
The first insight is whether or not the developer was working on features that are important to the business. Suppose the development team wrote a lot of code, but only a small amount made it to production. In such a scenario, it could mean they weren’t working on the right features because someone misunderstood the business priorities or spent a lot of time on prototyping.
Secondly, it is possible that the product owner did not fully define the requirement and kept on changing it, resulting in code churn.
Code churn measures the amount of code that was re-written for a feature to be done right. Code churn can happen because of a) inexperienced developers writing bad code, b) the developer’s poor understanding of the product requirements, or c) the product owner not defining the feature well leading to scope changes, or d) the prioritization of features not done right by the product owner.
Churn is not necessarily bad, especially for early-stage startups that want to validate features through quick iterations. Measuring churn gives you a mirror to see which of these four causes is the underlying problem, so you can work on them to become more efficient. It lets teams know whether they need to focus on prioritization of features, defining features and explaining them better to developers, improving the design, or spending time helping inexperienced developers.
Understanding code quality includes examining the code design and code structure. The design is an indicator of how good the underlying code is. Ensuring the code is well designed and modular makes it easier to test as well as maintain.
Tools like SonarQube do static code analysis to reveal code design and structure for all the code in a particular module or feature. However, these tools do not tell you the code quality for an individual developer, which is required to improve developer efficiency.
SonarQube identifies a list of issues that need to be looked into and fixed. It’s important to figure out which ones should be fixed first to optimize developer time, impact and resources. Metrics such as coupling, age, fragmentation and churn can help in identifying the impact a particular file or a component would have on the upcoming releases.
Once the issues are prioritized, it is important to identify the right developer who can fix these issues by looking at who contributed most code to the file/component – Code Owner, her productivity and quality.
Assessing the value of that code (whether it is good or bad) will indicate which developer should have ownership of that module for things like changes or iterations. To assess the value, one needs to look at the contribution, churn, and quality. For example, if a particular developer contributed a lot of code for a feature and the quality is good, you know the developer is good. If the code quality is bad in that same case, it is necessary to make another developer the owner to improve the quality.
Focusing on outcomes (code in production, code churn, and code quality) rather than outputs (lines of code) makes it possible to measure software developer productivity objectively. Software development teams that take an outcome-focused approach can accelerate product development, enhance the quality of products and improve the team overall.
In addition to measuring productivity and quality, analyzing code in production can provide a myriad of other useful insights such as understanding a developer’s skill matrix, early signs of developer burnout, top and bottom performers, and quality of product owners. Analyzing a developer’s code quality and commits provides a knowledge base of the skill matrix for a developer. One can detect signs of burnout by looking at the productivity trend of the developer and evaluating whether their productivity has started to drop steadily.
The overall business is also positively impacted by measuring developer outcomes. Managers can reduce delivery risks by estimating features more precisely. If there is a production issue, this approach helps identify the right developer who can fix it without issues getting regressed. Customer satisfaction improves as product managers are focused on features that are required in production, rather than on proof-of-concepts.
Most importantly, by embracing an outcome-focused approach to development, the cycle time for product delivery can be reduced, which in turn accelerates product time-to-market, increasing revenue and can establish a competitive advantage, improve profitability or help secure financing for the business.
Manjusha Madabushi is the Co-Founder and CTO of Talentica Software, the global leader in outsourced product development services for early and growth-stage technology businesses. She has mentored over 70 start-ups in building successful products across domains like FinTech, Media & Ads, EdTech, and Healthcare in a career spanning more than 3 decades. Manjusha is an avid hiker and an IIT Mumbai & Northwestern University alumnus. Follow Manjusha on LinkedIn.
A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example
We protect your privacy.
You need to Register an InfoQ account or Login or login to post comments. But there’s so much more behind being registered.
Get the most out of the InfoQ experience.
Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p
Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p
Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p
A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example
We protect your privacy.
QCon, the international software development conference, is returning (in-person and online) in 2022.
QCon brings together the world’s most innovative senior software engineers across multiple domains to share their real-world implementation of emerging trends and practices.
Find practical inspiration (not product pitches) from software leaders deep in the trenches creating software, scaling architectures and fine-tuning their technical leadership to help you make the right decisions. Save your spot now!
InfoQ.com and all content copyright © 2006-2022 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we’ve ever worked with.
Privacy Notice, Terms And Conditions, Cookie Policy
