The Background
I love Hotjar. It’s one of my favourite tools.
If you haven’t used it yet, go and sign up. They have just launched a great new feature called Incoming Feedback where you can get instant feedback from website users, complete with screenshots.
If you’re on our website right now, try it out by clicking this guy in the bottom left corner .
I undertake lots of small experiments with Hotjar.
Like this one that I launched just before Christmas:
This is a pop-up that appears on our blog after users have been browsing for a while.
I made a couple of crucial mistakes when launching this pop-up experiment.
If you are a HotJar user, I hope you haven’t made these mistakes. Keep reading, and I’ll show you how to check if you have.
In late January, I received this email from Jon (our Web Analyst).
It turns out that Jon was right, the low bounce rate was a useful idea for a blog post.
Unfortunately, for all the wrong reasons.
I had actually spotted the extremely low bounce rate the night before Jon’s email. I knew it wasn’t going to be good news.
Here’s what I saw in our Google Analytics account:
It’s very rare you see bounce rate drop like this day over day. This is almost always a sign that there’s been a configuration issue and it’s not a positive change.
Jon’s optimism filled me with guilt and nudged me to do some investigation with a little more urgency.
I was certain that something had messed up, but I had no idea what it could be. I work pretty closely with Sean (our Marketing Manager) on our website, and I couldn’t recall doing anything that would influence bounce rate in this way.
I spent about an hour looking at:
- Google Analytics reports
- Google Analytics change history
- Google Tag Manager versions and notes
- Google Analytics annotations
- I even went through my calendar, Sean’s calendar and both of our email accounts on that day to see what we had been up to on that day
I found nothing.
(Turns out we don’t do much in the lead up to Christmas…)
Then a lightbulb went off in my head.
I remembered that December 20th was the last day I was in the office before the Christmas break and I had wanted to set up a Hotjar survey to collect responses over the holiday period.
So, what was the issue?
My hypothesis: Clicks on the Hotjar pop-up were triggering Event interactions on blogs posts, and this was causing an artificially low bounce rate.
Sure enough, I jumped to the Google Analytics Event report and saw the following day over day increase in (not set) data recorded for our Outbound Link Click Event Labels.
I was now even more confident that I had found the problem. Though I still couldn’t be 100% sure.
In hindsight, the Events report probably should have been the first place I looked, and I could have saved myself an hour. It was late; I forgave myself.
How did we solve the issue?
As it was getting late in the day, the first thing I did was turn off the Hotjar pop-up.
I figured that if the bounce rate increased the following day, I could be certain that this was the culprit.
Sure enough, our bounce rate increased the following day.
Now all I needed to know was why was this happening?
What you should know about Hotjar if you use Google Tag Manager
We use Google Tag Manager to track a number of different Events. One of the Events that we have configured is to track Outbound Clicks (a pretty common Event configuration).
One of the first things I knew I needed to review was our Outbound Click configuration.
I opened our Google Tag Manager container in Preview and Debug mode, loaded up our website and then waited for the fatal pop-up to show.
I closed the Hotjar pop-up, and BAM and Outbound Link Click fired in the Preview and Debug console.
Now that I knew what was causing the issue, all I needed to do was check exactly how our Outbound Link Clicks had been configured in Google Tag Manager.
This was our trigger criteria for Outbound Link Clicks:
We don’t have too many pop-ups, opt-ins etc. on our site, so up until this point, we had only used the most basic configuration to track outbound link clicks by excluding anything that didn’t contain our website URL or our mailto links.
When I had started to add Hotjar pop-ups, I hadn’t considered how this would impact our Event tracking.
In this instance, I added the following trigger criteria to prevent an outbound click from firing when users closed the Hotjar survey:
I tested this new configuration in Preview and Debug mode and the outbound click Event no longer fired.
I could finally go home for the evening.
Conclusion: Lessons Learned from our Hotjar/Google Tag Manager Issues
- Annotate everything in Google Analytics.
Remember to annotate EVERYTHING in Google Analytics. On this one occasion that I forgot to annotate a change, a problem occurred, and I wasted a good couple of hours of everyone’s time trying to figure out what had gone wrong.
Annotate everything. Remind your colleagues to annotate everything. Remind your clients to annotate everything.
If they ask why, send them this post.
- When you’re experiencing ridiculously low bounce rates, always check the Events report first.
False positives can often be caused because of issues relating to Google Analytics Events being triggered by pop-ups, carousels, etc.
- Use Class IDs or Class Elements to trigger Outbound Click Events if you’re a more advanced GTM user.
Our catch-all configuration for Outbound Clicks was fine… until it wasn’t.
If you’re tracking a lot of click interactions on your site, the most accurate method is to use Click IDs, or failing that, Click Classes.
Are you a GTM User?
If so, you might want to download our handy Have you checked this? Checklist for when Event tracking goes wrong.
Or if you haven’t yet got started with GTM you might want to check out our posts:
If you have any questions or queries about this post or infact anything to do with your Google Analytics account, get in touch using the form below, the team will be happy to help out 🙂