- Bastien Vélitchkine
- Posts
- The right takeaway from a failed experiment
The right takeaway from a failed experiment
Making sure to learn the right lessons from a failed growth experiment.
This post is a reedition. recruitivity is no longer being maintained even though it’s still available and free.
After 3 monts spent revamping recruitivity, we've lost users in the hours following the release 🤡
All because we'd removed a tiny, and yet, very important feature from the extension.
Let's not bring the wrong takeaway back home though.
Knowing what we knew at the time, it was the right decision.
What happened
Chrome extension revamping
But first, let's throw back a few months and see what happened.
The goal wasn't to add nor remove major features.
We mostly wanted:
To use Node packages
A neutral - yet sleek - design system
(minor features might've changed in the process though 🌝)
Marketing 80/20
The revamping alone took 3 months. The release was long overdue, so we aimed for efficiency.
And in my opinion, the 80/20 of any marketing will always be to focus on copywriting, because:
It's at the root of everything—videos and emails alike
It involves nothing more than a sheet and a pen
The goals of the campaign
So we did focus on copywriting, starting with an email campaign (with videos inside) for all the users we had collected hitherto. The goals were to:
Boost the hype on upcoming changes
Warn them the extension's name had changed
Get them accustomed to the names and faces of the team (Gauvain and myself)
Visuals for the chrome store
Then we moved on to design all of the chrome store material:
A description
A product video demo
A slide deck
It had to be visually appealing, but copy remained the crux of the matter.
We couldn’t wait
In less than a week, everything was ready.
We couldn't wait to release.
The chrome store's approval was the only thing missing.
So, when we received this mail, it wasn't long until we hit publish.
And the results have been...
Underwhelming 🌝
Overview of the email campaign sent to 250 people.
Good open-rate, but lots of unsubscribes, low click rates on the videos and even lower response rates.
We crippled the thing they cared about most
However, it's still thanks to this half-ass campaign that we understood our blunder.
We'd crippled the one thing they cared about most by removing the extension's automatic opening on LinkedIn profiles:
We'd crippled their productivity.
We’d do it again though
We'll obviously put the feature back in the next release.
However, should we make the first release again, with the same info that we had at the time, we'd take the same bet.
Let’s not bring the wrong takeaway home.
Takeaway
The wrong takeaway
The wrong takeaway from all this would be: "never change a thing users have never complained about".
Yes, we did lose users' goodwill. But we gained invaluable intelligence: the certainty that it's better for the extension to open automatically.
So, in hindsight, it's a win.
The right takeaway
1. Take risky bets
Imagine the opposite situation. Imagine if the extension did not open automatically and if users had never complained about it.
By not taking the bet, we'd have missed an opportunity to delight them.
2. The earlier the better
The earlier the experiment, the better because there are less users involved (which makes the failed A/B test less expensive).
Last words
"If I had to live my life again, I'd make the same mistakes, only sooner."
Reply