The right takeaway from a failed experiment

Making sure to learn the right lessons from a failed growth experiment.

This post is a reedition. recruitivity is no longer being maintained even though it’s still available and free.

After 3 monts spent revamping recruitivity, we've lost users in the hours following the release 🤡

All because we'd removed a tiny, and yet, very important feature from the extension.

Let's not bring the wrong takeaway back home though.

Knowing what we knew at the time, it was the right decision.

What happened

Chrome extension revamping

But first, let's throw back a few months and see what happened.

The goal wasn't to add nor remove major features.

We mostly wanted:

(minor features might've changed in the process though 🌝)

Marketing 80/20

The revamping alone took 3 months. The release was long overdue, so we aimed for efficiency.

And in my opinion, the 80/20 of any marketing will always be to focus on copywriting, because:

  1. It's at the root of everything—videos and emails alike

  2. It involves nothing more than a sheet and a pen

The goals of the campaign

So we did focus on copywriting, starting with an email campaign (with videos inside) for all the users we had collected hitherto. The goals were to:

  • Boost the hype on upcoming changes

  • Warn them the extension's name had changed

  • Get them accustomed to the names and faces of the team (Gauvain and myself)

Visuals for the chrome store

Then we moved on to design all of the chrome store material:

  • A description

  • A product video demo

  • A slide deck

It had to be visually appealing, but copy remained the crux of the matter.

We couldn’t wait

In less than a week, everything was ready.

We couldn't wait to release.

The chrome store's approval was the only thing missing.

So, when we received this mail, it wasn't long until we hit publish.

And the results have been...

Underwhelming 🌝

Overview of the email campaign sent to 250 people.

Good open-rate, but lots of unsubscribes, low click rates on the videos and even lower response rates.

We crippled the thing they cared about most

However, it's still thanks to this half-ass campaign that we understood our blunder.

We'd crippled the one thing they cared about most by removing the extension's automatic opening on LinkedIn profiles:

We'd crippled their productivity.

We’d do it again though

We'll obviously put the feature back in the next release.

However, should we make the first release again, with the same info that we had at the time, we'd take the same bet.

Let’s not bring the wrong takeaway home.

Takeaway

The wrong takeaway

The wrong takeaway from all this would be: "never change a thing users have never complained about".

Yes, we did lose users' goodwill. But we gained invaluable intelligence: the certainty that it's better for the extension to open automatically.

So, in hindsight, it's a win.

The right takeaway

1. Take risky bets

Imagine the opposite situation. Imagine if the extension did not open automatically and if users had never complained about it.

By not taking the bet, we'd have missed an opportunity to delight them.

2. The earlier the better

The earlier the experiment, the better because there are less users involved (which makes the failed A/B test less expensive).

Last words

"If I had to live my life again, I'd make the same mistakes, only sooner."

Tallulah Bankhead

Reply

or to participate.