I had the privilege of overlapping with Kent Beck at Gusto for a couple of years. He changed how I think about a lot of things, and importantly how I think about Engineering velocity (see Outcome Over Output: Also Impact and Effort). In my experience, velocity is usually thought of as the amount of stuff an Engineering team produces within a given timeframe; it's output-focused. Output is interesting, it's something we're in more control over, but it's not what matters.
Realized customer value is what matters.
Output that just sits there, undiscovered and unused, is waste. Output that our customers find, understand, and use is valuable. And this is how Middesk Engineering thinks about velocity; it's the rate at which we create realized value for our customers.
Measurement.
The problem with our new definition is it’s hard to measure. At least, it’s harder to measure than the old-school definition, where all we need to do is look at something like PR throughput or commits over time.
How do we quantify “realized customer value” and attribute it to an Engineer’s work?
One way to do it is with a cascading OKR system, which we use here at Middesk. First, the company starts with top-level objectives and key results. Many of those are related to business impact, which we think of as the value customers return to us and is almost always in the form of $.
Second, teams establish OKRs that relate to the top-level. These usually take the form of a hypothesis: If we unlock Value X for our customers, they will eventually and indirectly reciprocate with Impact Y. Engineers introduce telemetry to measure Value X, and we insert faith that it'll eventually influence Impact Y, our top-level OKR.
Yep, "faith." (I first saw this word used in this context in the seminal Intercom post Outcomes vs. Features; I'm an acolyte).
Realized value is a leading indicator of business impact. If we accept the relationship as correlative, and if we accept that trying to prove causation is (usually) frivolous, we find freedom and speed.
So we measure realized value now, and we expect to see correlative movement with our top-level OKRs later. Examples of realized-value measurements at Middesk are:
- Identity-report volume
- Turn around time for Identity reports
- Agent transfer volume
- % of Agent orders fulfilled without the need for additional information from the customer.
If these move in a positive direction, we assume influence on business impact.
Non-coding incentives.
Holding ourselves accountable to realized value creates an interesting set of non-coding incentives for Engineering. We have to spend time knowing the customer to understand what matters to them.
This means participating in discovery (user research), listening to sales calls (Gong's rad here), understanding what themes our go-to-market teams have picked up, talking to Operations about ways to bolster fulfillment efficiency, etc.
Product Management and Design do these things too and have strong, informed opinions about what's valuable. Middesk Engineers help set cycle priorities with their PM and Design counterparts, and importantly they make micro-decisions about product behavior every day, and these add up. They can't do this effectively without understanding the customer and what's valuable to her.
This changes how Engineers frame their work.
Here’s an update from a Middesk Engineer highlighting a feature he and a Designer quickly shipped in response to customer feedback that had just come in and matched a broader usability problem the team had been pondering:
With the help of Design we were able to ship this very quickly. We informed [the customer] of the change this morning and they were so stoked to see the change they requested yesterday, live today! Quick feedback loop, quick team effort, and a happy customer.
Another Engineer, after shipping a feature she and the team thought would make it easier for customers to access liens data when making underwriting decisions, shared:
Customers are now able to get links to primary liens data and when possible a link directly to the UCC lien. Some customers occasionally look through UCC lien to extract several fields, including collateral notes, from the UCC lien in order to fulfill their compliance/risk evaluations so this change will help them quickly gather the information they need!
Our #shipped channel in Slack is filled with customer-focused updates like these. Indeed, putting the customer first is something we love about our culture.
Is it really that clean?
No, it's not that clean. There are 100's of micro-updates we deploy every week that don't make the #shipped channel and can't be framed in terms of realized customer value (at least in a non-coerced way). Yes, we sometimes fall back to the old-school velocity metric, output/time, to get a sense of volume for work like that.
Output matters. Output is always happening.
But output isn't intrinsically valuable. We've built an Engineering culture that values what customers do with what we ship over the act of shipping itself.
I'm excited to see what new problems we'll help our customers solve.
