The era of Google Optimize is drawing to a close – Google has already announced that they will be winding down the service in its original form by September 30th. However, there's some good news (hallelujah) – they've introduced support for third-party split-testing applications to integrate with GA4. That's where my journey began.

I'm Alex Ignatenko, and I'm deeply immersed in marketing analytics. I consult on data-driven decision-making, offer guidance on setup, and also lend a hand with training, developing, and implementing custom attribution models and A/B experiments. On top of all that, I curate a Telegram channel focused on marketing analytics, appropriately titled "Attribution Model".

Over the past month, I've been delving into a slew of different software products, putting them through their paces with demos, trials, and thorough documentation analysis. My goal was to find the cream of the crop, the ones that could stand in for Google Optimize without leaving me high and dry.

This article is a curated list of applications for experiment conduct and analysis – a lineup that could serve as viable alternatives to Google Optimize once it's sunsetting. So, let's dive in.

Introduction – Key Criteria

In the spirit of Oscar Wilde's philosophy, I kept it elegantly simple. All features i need are:

  • Smooth and seamless integration with GA4.
  • Ability to set up and manage via GTM.
  • Precision targeting through dataLayer variables.
  • Flexibility to configure custom JS events (via GTM) as experiment objectives.
  • A hint of a visual editor wouldn't hurt.
  • Responsive and helpful support would be a godsend.
  • Capacity to handle 1 million MAU without causing a financial crisis.

8th Place – Eppo

These folks are a bit peculiar. I submitted a demo request three times, and each time, it seemed to vanish into thin air. Eventually, they put me on their mailing list (thank goodness for that). When I responded to their email requesting a demo, they sent me a scheduling link. I picked a time, joined the call as scheduled, and waited for a while. Cross my heart, I did everything I could.

7th Place – AB Tasty

An approximate estimate of the cost (based on traffic-related queries) ended up around $100k per year. Frankly, that could have wrapped things up. The service seemed to offer everything I was after and more, but it left the crucial question unanswered – why the hefty price tag?

6th Place – Optimizely

Here's another product squarely targeting enterprises. During our initial call, a somewhat whimsical "demo" session, the lion's share of our time was consumed by me answering qualifying questions. Once the spotlight shifted to my inquiries, they assured me a detailed email response would follow. Fair enough.

And indeed, that email landed in my inbox, containing remarkably comprehensive answers. Virtually all of my modest checklist was addressed. Except, perhaps, for the price tag – a steep $45k per year (I might be a tad off with the precise number, but you get the idea – my apologies, Optimizely team). Furthermore, due to the lack of a trial option (which isn't part of the plan), I found myself nudging it to the lower rungs of my ranking.

5th Place - Kameleoon

A fascinating service. It comes at a slightly lower price – according to my requirements, around €35k per year – and it includes a bunch of intriguing (though not entirely necessary for me) personalization and AI features. The managers claim it has the fastest response time in the industry and the least impact on the flickering effect.

Polite and friendly ladies took a bit of a pause when I asked for a trial access. Unfortunately, that pause is still ongoing. So, it takes the 5th place.

4th Place - ABsmartly

A young but quite impressive service from people who previously worked at Booking.com. The demo call was with co-founder Jonas Alves (ex-Product Owner Experimentation @ Booking), and it was the first conversation during my research where the other person didn't focus too much on pre-qualifying questions but instead delved into technical and statistical intricacies.

I was drawn in by how they've organized the experiment log – with participant comments, history, and other elements – kind of blend between a Slack channel and a Confluence document.

Also, I like their emphasis on sequential testing. This lets you run multiple experiments concurrently and get reasonably quick and meaningful results.

Unfortunately, AB Smartly doesn't offer a visual editor and seamless GA4 integration. So, we didn't get to the point of discussing costs.

3rd Place - VWO

The top three contenders had all the features I needed and the possibility of trial access.

Visual Website Optimizer – an Indian service with a long history. It has a pretty user-friendly dashboard and detailed documentation. The support channels are quite responsive. I had to tinker a bit with GA4 integration, but nothing critical. They don't recommend installing through GTM.

For analysis, they offer frequency-based and Bayesian methods, various types of metrics, and such. Everything seemed good, except the price – based on my limitations, I would have had to pay $3750 per month, which seemed a bit steep for a series of simple experiments.

2nd Place - GrowthBook (YC W22)

The first and only service where the pricing isn't dependent on traffic. There's a fixed annual fee of $10k per account. It includes 10 million API calls per month, and additional ones cost $100 for 10 million. For simplicity's sake, you can consider one request equal to one session. Also, there are mechanisms to cache requests, so there's potential to save somewhere.

Growthbook integrates with your Google Cloud project and fetches stats from the BigQuery project, which should be linked to your GA4 resource. Data consistency is maintained by sending an extra event to GA4 when initializing the Growthbook script with the appropriate parameters.

They have their visual editor, some specifics for SPAs, and overall, the application looks quite impressive in terms of value for the price.

The downsides are the relatively high entry bar (you can't publish a new GTM container version and forget) and additional costs for BigQuery. Aside from that, Growthbook is really cool.

1st Place - Convert.com

Think of it like Google Optimize on steroids. Simple, easy to understand, and lightweight. Can be set up through GTM, integrates well with GA4 – separate audiences are created for each test. The structure of segments, events, and other entities is intuitive. You can use multiple statistical criteria and work with variant previews.

The monthly cost for 1 million MAU is $1100 with an annual contract. They provide professional support – the staff can offer guidance and even help tweak scripts if necessary.

So, if you're in search of a comprehensive replacement for Google Optimize that's reasonably priced, definitely take a look at Convert.

Out of the Contest

Confidence by Spotify – a fresh service in beta format, which I haven't had a chance to try yet.

GTM Custom HTML Template. If you're not bothered by the flickering effect and are okay with handling conclusions on your own, you can develop your own or enhance an existing Google Tag Manager tag template. It would be really cost-effective (basically free) and perhaps not too complicated.

I haven't considered Russian-language services. Nevertheless, my favorite remains Sigma by expf

Conclusion

This was a sketch of my detailed and not-so-detailed impressions from my encounters with A/B experimentation services. There might be changes in pricing and app functionalities after publication – I apologize for that and am always ready to update the content.

Russian-spoken subscribers can

Russian-spoken subscribers can read more of my thoughts on marketing analytics in the "Attribution Model" Telegram channel.

If you enjoyed this post, please consider leaving a like and sharing it with friends – it means a lot to me.