Avoiding the DevRel ROI Trap with Better Strategic Alignment
“How do you measure the success of DevRel?”, “Can you quantify the impact of a developer advocate’s work?”, “What’s the ROI on DevRel?”, “How does your work help us?”. If these questions sound familiar, chances are so does the imminent urge to leave the room when they’re raised. In my experience, good answers to such questions do exist and they don’t always involve numbers and metrics, however counterintuitive it may seem. Intrigued? Read on!
The endless search for the holy metric
Numerous studies, tweets, and conferences indicate that the battle for the measurability of DevRel is far from over. DevRel professionals still spend their time looking for the killer DevRel KPI — a golden metric that would lead to salvation and justification of their’s team existence once and for all.
Simple metrics often seem not to capture the essence of what we’re trying to achieve or don’t cover it in its entirety and therefore get neglected in favor of more complicated ones.
And when we think we’ve finally nailed it — when we think we’ve found a metric that is not obscure and that managers and other non-DevRel folks could possibly understand, a metric that focuses on outcomes rather than outputs, we realize we’re trying to measure the immeasurable. We’ve probably come up with something so complex that it’d take six months just to implement and another half a year to collect enough actionable data.
I did no different — once, I came up with a so-called “community engagement funnel”. 42 brilliant ideas on how to measure the value of our community across various channels carefully spread out across a 6x7-cell sheet. It would take an army of BI engineers to implement such a complex report, not to mention the necessity of close collaboration of three other departments. Looking back, I’m glad the idea never really left the internal wiki page. I leave that problem to specialized companies such as Orbit, who can afford to dedicate much more (if not all) resources to solve it.
The inability to come up with a solid metric on their own leads people to frenetically generating and copying ideas from each other without further thought and without taking into account their current situation — the maturity of their organization, the market, the environment, and finally, the company strategy….and ultimately, missing the point again.
We travel a downward spiral that leads to the utmost despair. Now, don’t take me wrong — I think brainstorming and sharing ideas with others are immanent essences of progression, however, the ideas need to be applied with care, otherwise, it’s just stabbing in the dark.
What if KPIs are not *always* the answer?
I’m not saying KPIs are not important, I’m just saying that we shouldn’t solely rely on them and that we shouldn’t spend our youth defining them.
They say “If you can’t measure it, you can’t improve it.”…well yeah, to some extent perhaps. But a nice colorful report won’t do you any good if, in the meantime, your competitor is able to improve their DX with a new SDK, release a new version of API, and organize three virtual meetups to educate their community about it. Now, quantify that!
So I wouldn’t be so literal about the earlier quote and rather take it with a teeny-tiny grain of salt. In fact, I believe some things are not worth measuring at all (at least with the existing tools). Take brand awareness for instance — you can do social listening, surveys, track mentions, SoV, followers, traffic, engagement, and yada yada yada. It soon gets very complex and the truth is that unless every human has a brain implant, it’s unreliable and impractical to measure. I’m of the same belief as Steve Pousty in thinking that not everything of value can be measured (a brilliant talk btw).
Now, take a deep breath and think again. Do you really need KPIs? What’s the real problem? The real problem is to prove the business value of DevRel. KPIs are just one of the methods to do that, sometimes they work, sometimes they don’t. And sometimes, we unrightfully put too much effort into them. So, how to get out of this pickle?
When it comes to measuring the impact of DevRel, there is one approach that can be equally good to show the value but is often neglected. In fact, it proved tremendously effective in our case.
It has much to do with the activities we choose to do as DevRel and how well we listen to the world around us. I am talking about the strategy and its ALIGNMENT with the broader company strategy.
I noticed a couple of people who publicly emphasize the importance of strategic alignment. To mention some, Phil Leggetter in his AAARRRP strategy framework, Mary Thengvall in her recent article:
In order to successfully prove the value of our work in a way that the company understands and sees as beneficial, we need to attach the main goals of the Developer Relations department to the goals that are shared by the company as a whole.
Source: https://www.marythengvall.com/blog/2020/12/14/first-understand-the-company-goals
Or, if you give a thought to Max Katz’s 3-part framework for measuring success in DevRel, you’ll realize that two parts are actually a subset of the third — Measuring something that’s tied to business goals.
To me, it was Ashley Smith who hit the nail on the head:
The first rule of measuring DevRel ROI is tying DevRel goals to company goals. This may sound like common sense, but you wouldn’t believe how often people get this wrong. If your DevRel goals are not aligned with larger company goals, people will be hard pressed to recognize the value of DevRel activities. Without context, they won’t be able to connect the dots between what you’re doing and ultimate business outcomes. This makes it hard for other functions to get behind DevRel. And if the DevRel team doesn’t understand the importance of their role within the organization, they can get off course pretty quickly.
Source: https://openviewpartners.com/blog/measuring-the-impact-of-your-developer-relations-team/
@Ashley, if, by any chance you are reading this — Thank you!
So can our ability to adapt to company strategy be our “KPI” and talk for us? I’ll show you how we’ve done it and what tools we’ve used.
A recipe for a perfect strategy
To put things in a broader context, allow me a little detour and let me share a simplistic model of what I consider a good strategy:
To succeed, I need to collect the company’s management’s expectations, understand the target audience’s needs, and know my team well. None of these is an easy task — to start with, I wouldn’t expect the management team to provide me with anything more than the target segment + the stage of the customer’s journey or lifecycle to focus on, second, developers can be a very loud and demanding bunch and the demands tend to vary quite often, and finally, learning my team members’ superpowers requires some time working together. But all this research is soooo worth it and it saves so much trouble in the future!
My favorite quote from Cristiano Betta captures my feelings perfectly:
One of the big dangers to any developer relations program is that it is so easy to try and do it all. The problem though with trying to do it all is that you end up being busy and doing nothing well.
Source: https://devrel.net/dev-rel/meet-cristiano-betta
What happens if you choose a wrong strategy? Well, you end up:
- doing someone else’s job, or
- worrying about being unable to explain your actions, or
- not doing DevRel work at all.
In either case, your motivation vaporizes very quickly. Now let’s get back to strategic alignment and our approach to it in Kentico.
Kentico is a software vendor focusing on content management. I lead a team of awesome Developer Advocates for Kentico Kontent — our cloud-based CMS offering.
Strategic alignment tools
If you do a quick search through the web, you’ll stumble upon more than one methodology for strategic alignment. At Kentico, we were lucky enough that our management team decided to adopt OKRs (Objectives and Key Results) across the whole company. This enlightened decision came more than 2 years ago and I think we’ve made good progress since then and learned to avoid the biggest traps through time.
This framework helped us realize what’s important and what’s not. If it weren’t for OKRs, our team would probably be still stuck trying to drive community engagement through forums and gamification platforms without realizing that we have a more important mission and that the community is, in our case, just a positive side-effect. We’d probably be trying to control unimportant details and micromanaging, instead of steering the bigger picture and innovating. We’d be building one SDK after another instead of influencing the product strategy and connecting the dots between the product, documentation, and SDKs, and driving developer experience in the company. Now, I’m not saying that what’s good for us has to be good for you, I’m just saying that we’ve learned to pick our battles.
OKRs helped us pick our battles and become active participants in the broader company strategy.
The relation between KPIs and OKRs
Let me get one thing straight — OKRs and KPIs are not contradictory concepts, neither are they replacements for one another. Au contraire, they can play together nicely and complement each other. But it’s necessary to understand the difference and learn when to apply one or the other.
KPIs serve as gauges indicating the health or success of an existing ongoing activity or process. They’re rather long-term and can take the form of both leading and lagging metrics.
OKRs is a means of improving (not only) your KPIs. The framework strictly differentiates between Objectives (the direction you’re heading towards), Key Results (the lagging indicators of success — work outcome), and Initiatives (the leading indicators of the invested effort — work output).
OKRs challenge your way of working. They force you to think out of the box, innovate, and change processes. When creating OKRs, you’ll often find yourself asking questions such as “How can we change our WoW to improve KPI X without hurting KPI Y?”
If you want to learn more about how OKRs and KPIs compare and work together, or just see some examples, head on to Perdoo’s superb explanation or to their awesome OKR guide.
Real-world implementation of OKRs for DevRel
Planning
Every year, we’re presented with a couple of very high-level annual objectives by the management team. These, usually 3–5, objectives change slightly every quarter but the core goals typically remain the same.
We hold a planning session every 1 to 2 months — we figured that a quarterly cycle is too long and rigid for us. We take a look at the high-level objectives and try to see where we (as in DevRel), given our competencies and skillsets, can be most helpful. Although our team sits under Customer Success, we don’t limit ourselves to customer retention-related activities, we help product management, customer education, engineering, marketing…whatever is the most important at any given moment. If the objective is, for instance, to increase the exposure of our brand, we can certainly relate to it and focus on the developer part of the audience — that’s going to be our (sub-)Objective.
We employ the aforementioned “perfect strategy recipe” and our expertise to figure out what will be the pillars of our strategy and decide what we’d consider a success. Plus, we put the target a bit out of our comfort zones to make it rather aspirational 😀.
In the earlier example of brand exposure— based on our target segment, we might consider it a success if certain media, influencers, or technology partners talk about our product — that’s what our company needs, and that’s going to be our Key Result.
The last part is to figure out the specific initiatives — aka the tactics. This is a combination of responding to developers’ needs and utilizing the best of our skills.
We’ll pick a topic that is relevant to our audience — e.g. securing Jamstack e-commerce apps — and find the best way to deliver it to the right audience — e.g. a set of best practices, a handy cheat-sheet + a deep-dive video stream.
The order of steps during the planning session is essential. We first take into consideration the company strategy, and only then we try to map our skills and our knowledge about developers to it in order to come up with the right mix of activities.
Tracking
Once we’re happy with the definition of our OKRs, we track progress on a weekly basis. This consists of two components — first, we check-in the progress to Perdoo (the OKR management software we use), and second, we assemble a short report for the weekly department newsletter. Since we all contribute to the same high-level objectives, there is always something relevant and meaningful to report. These bullet-ins are usually not very number-heavy — they feature product updates, customer success stories, influencer highlights, partnerships, etc. If our colleagues want to dive into details, they can go to Perdoo and browse through the hierarchy of objectives and sub-objectives, check how we are progressing in terms of numbers.
Every month, there is a department-wide meeting and a company-wide newsletter where all teams reflect on selected OKRs. We make sure these updates are as brief and understandable as possible. This way it doesn’t waste anybody’s time and keeps everybody in the loop. People can ask questions, challenge decisions being made, suggest improvements, or just praise their colleagues. At the end of the year, we zoom out and repeat the process from a higher perspective.
That’s a lot of updates, you say? The level of exposure and the frequency of these updates put us under a constant spotlight. Not only they give our work audibility (guess what car make I’m into 😀) and visibility but they help us keep pace with what’s going on in the company and respond to it promptly → leading to even better alignment.
Lessons learned
There are a few things we learned along the way that make the DevRel team more efficient and credible when it really comes to metrics and measurability:
- We learned to settle for less-than-ideal metrics, sometimes for metrics influenced indirectly. We favor simplicity over excessive accuracy.
- We learned not to depend on other teams when it comes to collecting data for reporting. Often, people like customer success managers or sales reps have the data you’re looking for but they’re also often swamped with work so they forget to report it.
- We learned to pay attention to how people within the company understand our work and goals. If we notice, e.g. during a retrospective meeting, that there is a lack of information or understanding somewhere, we don’t hesitate and set up an OKR to change that. Hint: a good way to solve this issue is to collaborate on a project with the given person or team.
- We learned to introduce new KPIs only when we see a repetitive pattern that we can measure in the long-term and that gives us actionable info. We revise our KPIs regularly and drop metrics that are no longer relevant. We monitor things like API and SDK adoption, Developer NPS, adoption of 3rd party tools, and some essential community metrics.
How strategic alignment helped our DevRel
Strategic alignment, and especially OKRs, significantly helped change the perception of DevRel in our company. OKRs testify about our ability to adapt to the company strategy. And the fact that we are aligned with the company strategy means that we have (success) stories to tell every week — stories that everybody actually understands and can relate to. This tremendously helped establish the trust of both peers and stakeholders and prove value to them. Ultimately, it means less stress for all involved parties.
My hope is that I was able to offer an alternative, yet viable perspective on how to manage the perceived value of DevRel in your company and give fellow DevRel managers and advocates some hope.
This article is meant to lay the foundation of how I see DevRel. Let me know if you’d like me to expand on any of what I mentioned down in the comments. I’ll make sure to write about it in the future.
