Paul Rouke
Founder & Director of Optimisation
View posts

  • User Research

7th Mar 2018

20 min

“People are probably more important than the tool & technology”

About David…

Formerly a consultant and in-house specialist on anything related to growth and optimisation design, David Darmanin is the founder and CEO of Hotjar, established in 2014, growing sales to $10m per year and now running hundreds of tests with some of the worlds biggest brands.

David formed Hotjar realising growth was critical, wanting to understand what type of people, process and systems would best facilitate and maximise that growth.

In 2016 I brought together people with years of experience working at the coal face of the conversion industry and growth and as a result, I asked David to become one of the 17 thought leaders contributing to the book “Growth Strategy That’s Being Ignored“.

David joined the likes of Bryan Eisenberg, Angie Shottmuller, Craig Sullivan, Roger Dooley, Talia Wolf  and Chris Goward in contributing to the book. I’m delighted David took time out to contribute and share his thoughts with me through the interview below.

 

Here’s a few of the many insights from my interview with David…

 

“I think in terms of the industry, we’re just at the beginning.”

What would you say if you had to just give a summary of the conversion optimisation industry today and how would you describe the kind of maturity of where businesses are at the moment?

I think there’s a big appetite for understanding how to grow businesses. But I think there is also a slight misalignment between some of the more hardcore CRO specialist and potentially other areas within the business. The perception of who should be doing what. The thing is, conversion optimisation growth or whatever we call it when done well typically overlaps onto many areas within the company.

What I’ve seen is that there’s been quite a lot of resistance from certain teams within firms. But at the same time as I said they think in terms of management or owners, there’s a huge appetite to kind of be doing this better. I think in terms of the industry, we’re just at the beginning. I think there is a little bit of a perception that it might be too complicated. The name itself doesn’t help. Conversion rate optimisation. I think the first thing that needs to be optimized is the name we’ve given it.

“…the most evolved are the ones that are actually bringing in some form of qualitative data”

You say that the acronym CRO [conversion rate optimisation] itself is potentially damaging. What would you feel currently is the perception of what we do on a daily basis?

I think depending on the maturity or the stage or anatomy of evolution. I look back on my own evolution. I tend to see most companies or teams I speak to at different stages of it. Typically the first stage, I think anyone who has an experience with CRO recognises this stage. Its where you’ve got the tools in place and you’re firing on all barrels and trying to hit. You’ve trying to hit a win and typically this is the stage where there is no research and it’s typically where the tech and tools are there. Teams and companies are just changing staff in the hope that they’re going to have an impact.

The second stage is where you have the product teams involved and it’s typically a little bit more sophisticated because there is some element of research, of looking at some numbers, analysis, and seeing where people are dropping off or what they are using and what not. There is some quantitative data available.

By far the minority which are probably the most advanced, the most evolved are the ones that are actually bringing in some form of qualitative data. The ones that I’m interviewing, their users are doing some surveys or a usability and user testing. Basically, they are probably doing less tests. But they’re having their win rate and the amount of wins they have is much higher. It’s a pyramid structure. The first thing I mentioned is the bottom of the pyramid where you have the most and not surprisingly the ones that are more sophisticated, are the smaller in quantity.

 

Brian Eisenberg says conversion rate optimisation should not be a tactic but embedded into the culture of a business. Do you have any thoughts on that or any kind of experiences where you’ve seen a business transition from?

Newer smaller firms that have started with this culture at the onset clearly have a big advantage over the bigger organisations that are trying to introduce this. It’s what I mentioned before, it’s purely because some teams and some people see this as a challenge or a threat, let say, to their way of doing things and to the organisation.

Because essentially the mindset is one where you say, “Listen, we need to look at how we’re building the product, how we are doing our marketing and rethink it, and split this and change it.” It has an impact on multiple parts of the organization. I think that’s going to actually have quite a big impact on which companies will succeed going forward. The younger ones that were born in a CRO generation do have a big advantage because there is that cultural mindset.

That’s the problem, right? When a team doesn’t truly embrace CRO as a culture, so what do we mean by that? So it’s just a way of thinking, which is when you’re thinking of an improvement or a change or a way of doing things, you build your testing plan around that, the way you build and evolve your site is built around this model of hypotheses, go out there measure and then come back full circle and keep on going.

And when it’s a tactic, it’s less about the process, it’s less about the way of doing it and again the bottom of the pyramid, the tactic is usually, “What did that guy do?” and “Who got that win?” and “This site is doing that?” It’s this random search for the silver bullet, this hopefully quick wins that will get you somewhere.

But the problem is that any organisation that takes that route is deemed for failure. Because essentially you’re just trying to emulate what other people are doing but you don’t have the strategy nor the culture that is actually building something unique based on this way of doing things. It’s a big difference.

“I highly recommend doing that, working with experts so they can bring in some of these knowledge into the team.”

 

Have you experience with any businesses what they’ve done to change that mindset within their business from the traditional fixed mindset?

Typically what happens is— what I’ve seen is that usually there’s success when these organisations managed to bring in an internal champion and they put that champion in a senior enough position to allow them to succeed. If you have a VP in an organisation that is just spearheading growth. That’s what I did in a company. I joined a software company when there were 30 people and when we were 50, I was put in a position that I was VP of optimisation and design. In that position I built out teams competencies and basically trained people internally to think in a different way. So I think when you put leaders in this position, then organisations can pivot quite quickly in the way that they do things.

Usually you see failure when an organisation for example brings in a consultant but then does not match internally, does not have for example a leader internally to make sure that those ideas can actually be executed. And actually on the note of consultants, I’ve also seen a lot of firms that have worked successfully with consultants allowing internal teams to absorb a lot of thought leadership and this whole leadership thing that we talked about from consultants so I highly recommend doing that, working with experts so they can bring in some of these knowledge into the team.

“…our point of view we see the adoption of our tooland the amount of our users talking about CRO is just skyrocketing”

What would you say to senior decision-makers, who maybe haven’t yet really started to harness conversion optimisation for their business?

Yes, as I said, I think whether it’s five or ten years, like how quickly it will happen, I guess will depend a lot on the industry and your competitors. But I would say, my message to them would be beware. If you’re ignoring this, it’s definitely not going to go away, at least from our point of view we see the adoption of our tool and the amount of our users talking about CRO is just skyrocketing. In a way, the best way to put it is you cannot afford not to look into this.

“You cannot afford not to have some form of testing, experimentation culture within your firm”

You cannot afford not to have some form of testing, experimentation culture within your firm, purely because other players, your competitors, new entrants, disruptors, they are going to be doing it and the reality is what we are playing is a game, that’s what business is all about. Once you get a— let’s call it a race nearly— if you are driving a big, slow, clunky car that is being optimized in big phases as opposed to your competitors that are constantly tweaking and tuning their vehicles, then it’s going to be difficult to win that race in the long term.

“People are probably more important than the tool & technology”

We often find that many businesses have been testing but with no real impact and no real earnings. It’s not really caught on within their business. What would you say for businesses that are at that very low stage of maturity? What would you recommend should be some of their first steps, the foundational things that they look to get in place in order for them to build on from that?

I’d say, most definitely, people are probably more important than the tool, technology and all that stuff, which is still, obviously, great to have. I’d say, people is the most important aspect of all within that equation. And then, obviously, process and technology are key once you have the people. But time and time again, like you mentioned the examples of failure are usually down to not having the right people on board. And this is a big challenge. Honestly, CRO hasn’t taken off isn’t that big, but there’s this huge pool of people out there available.

My point would be— and here what I say by people is not necessarily really experienced people but we are looking for really clever creatives, people that can learn. Obviously, if you can find very skilled people that also have experience, that’s awesome, but if not, then invest into clever creatives, that is the most important thing. Stop investing into managers or analysts- some people would kill me for saying this- but essentially it’s more about clever creative people that are willing to question and design, and design not the graphics but come up with solutions to how to address problems and impact the bottom line.

If your team has potential but they don’t have experience – it’s not exactly very easy to go out there and learn CRO, so put them in a position to fail, to make mistakes and be comfortable doing that because like any experimentation, any lab which is just trying to reach the end result, and failure is part of it, so you have to embrace failure, and give them the resources they need, get people resources. We talked about consulting, get them to work with external teams, companies that have delivered successes to other companies because I’ve found at least throughout my career working with multiple firms and consulting throughout the years, that was a big learning curve for me.

Again, the last thing you want is to invest in all of this and have a set-up where everything is dependent on one person being the hero. I’m a firm believer in a team as the hero, and in a way, this is actually one of the biggest reasons that we mentioned in our vision as well internally of why we built Hotjar. We built a tool that is supposed to be the weapon.

We believe that data research is the weapon at the end of the day that you can use against the one-person hero or the person who wants to be the one making these decisions. Ideally, you’re building up your team with the right people and the right tools, that basically decisions are made based on the insights that you’re gathering, and it’s a team efforts to get to the end result. I think, this is probably one of the most important things when it comes to setting up a CRO team and a culture within the whole company.

“Once you have the people & technologies in place and you have some momentum, the key thing becomes process.”

 

Do you have any advice from experiences what could potentially take more mature businesses to the next level?

I think once you’ve had some wins, and you’re on the right track, the key actually then becomes optimising the optimisation. I think, you use the word there which is key in this case. Which is, once you have the people, technologies in place, you have some momentum, the key thing becomes process. I think it’s all about measuring your velocity as a team in terms of how you ship tests, the win rate you have and starting to measure that over time so that you’re essentially optimizing the optimization, optimizing those rates.

What I’ve found in an organisation where I was for around six years, and owning the CRO piece was that in the later stages when we reached the maturity, that was the key then, investing into that.

 

What do you think are the key metrics around things like the percentage success rates or the average up-lift per test, would you regard them as potentially more quality metrics about refining the outputs of your optimization? 

Yes, I would say, you definitely need metrics which are very close to the team, such as the number of tests, deploy and the win rates, so these are metrics, they don’t indicate performance in themselves. Then you’ll always going to need the KPI. My advice is always to have a KPI for this activity which is ideally not too close to the team. My point is, there’s always a way to change the metrics and inflate the numbers. The metrics are important to understand what the team is delivering, number of tests, the win rate, the uplift, blah blah.

But then, the hope is that obviously by doing the tests the team is moving a number. It’s very common for me to see organisations lose trust in CRO and the whole way of doing things purely because the team is maybe not using statistical confidence in the right way or not setting the tests in the right way, so they are reporting wins, but no one in the company is seeing any numbers move. So, it’s really important to have metrics to overview the process, but then a KPI- or maybe they should be tied to certain KPIs which are then critical to measure, “Listen, this is the ROI of the team at the end of the day.

 

You talked before about whether it’s going to be five or ten years away until businesses really mature and move through this cycle that you’ve described earlier, what are you looking to try to do yourself to try and help develop and progress that maturity?

“You have to be asking questions, speaking to your users”

What we are doing is I’m trying to build a tool for David of ten years ago. It took me very long to get out of that first layer, the bottom layer of the pyramid and move up. Basically, I’ve built a tool for me back then which gives you the visibility and ability to listen to your users— because let’s face it, we live in the technical digital age. It’s so easy to jump into conversion rate optimisation and you sit behind your screen and you look at the data, we all tend to be a little bit introverted in this space, and you just sit there and you design your tests and then you monitor the results.

But the reality is, to be really successful, you have to be out there in the field, you have to be asking questions speaking to your users. In a way, Hotjar is a tool that addresses this by giving you the ability to see what people are doing and why they are doing it. So collecting the feedback. It is our vision in fact so even though obviously we are a business, and there obviously are business considerations, our vision at Hotjar is to change the way the web is built and improve. And we are doing that by democratising user research, that is our vision.

You mentioned before the affordability piece and the fact that we’re free is all tied to the fact that we’d love for our students or beginners or people who are just doing this because they want to transition from their current role to be able to just jump in to this data and experiment and see for themselves how visitors are actually using a site, that’s our vision.

“CRO needs to become more a term which describes the way the organisation is run”

Just like to get your thoughts on, finally, the future of the conversion and optimisation industry. Do you feel there a positive or negative outlook and whether or not you feel there’s lots of noise out there and how you feel this will all play out over the next five years?

I had two startups before Hotjar which were very focused on CRO, on general and the big problem is actually CRO is very niche. If we look at demand and keywords being searched using Google Trends, it’s very easy to realise that CRO in itself is actually quite small. Tends to happen when things are relatively more complex. The fact that we’re saying you have a methodology, come up with a hypotheses and then you translate that into a test and then you measure the statistical— it’s a lot to chew on and not many organizations are used to doing it this way.

So it is still niche and as you said, the fact that it’s become more granular with more terms is I think the by-product of CRO having such a complicated name and rep. But it doesn’t help. The more it fragments itself with other terms like you said transformation, growth hacking and whatnot, the more it obviously becomes more broken down. So that makes it less of a movement, actually that’s one of the reasons why at Hotjar, we intentionally don’t speak too much about conversion optimisation when we’re trying to generate demand for the tool.

Instead we speak more to the value, it’s much better to say, “See how exactly how your visitors are using your site,” as opposed to, “Increase your conversion rates.” Because one, with most teams the second case of talking about conversion rates, it just doesn’t connect with the outcome especially if teams have never experienced that. And two, because as I said it’s just the term which can be a little bit difficult to process.

I think we are probably moving slower, so things are expanding slower than I would expect in terms of the name and if we had to give it a formal shape to call it CRO. But I think the disciplines of CRO are actually growing at a very very fast rate. It just feels that no one yet has packaged, given it a name or packaged like the processor whatnot. And everyone is trying to do that, which makes me think that that’s probably not going to be the future. It’s not going to be someone who gives this one name and one label and everyone is doing it the same.

I think we need to start looking more at CRO as a group of disciplines and a way of doing things and then firms can adopt them to their needs depending on what they need. Another example of that would be to say, “You’re lean in the way you build products.” Probably, I see the future being more that CRO is less a title of a person doing that in a company. It needs to become more a term which describes the way the organisation is run. If instead of CRO, the name was more an adjective similar to lean then I think we would have seen much bigger adoption. I think there’s a resistance to new roles but people are definitely open to doing things in a different way. Does that make sense?

 

What are your initial thoughts on what impact or influence you feel that Google optimized 360 will have?

To be honest, I already signed up for the beta and I spoke to our contact at Google to get on to test it out. It’s all going to depend on how well they’ve executed on that. For us, that’s great news. We haven’t built testing in to Hotjar purely because we think it’s not our objective right now. It’s always great to see Google building that out. I think it’s going to have a big impact if it’s done in an experienced and in a way which is truly simple and not complicated to understand. Originally, what it was called Content Experiment. It’s an after thought for them, it felt like at least. I think it was a little bit too complicated. I think it’s all about how they’re going to execute on it. Yes, I think it could have a big impact.

Sure, especially if CRO is generally new to you and if you were in my shoes which is more towards the bottom of the pyramid. If you associate more yourself with trying to hit the jackpot by firing in all directions then what I would suggest you do is search for Hotjar Action Plan. Basically what this is, is I’ve put together a plan like a series of things that I think any organisation should be doing in terms of understanding their user so that they can feed their list of what to test or change next.

It is very deep-rooted in user-research. It’s what got me to the higher level of that pyramid and it’s not tied down to Hotjar in anyway. You can go and use any other tool if you want. I hope that I have shared some valuable tips in there of what you should be looking at.

 

Other Similar Links

Interview with Industry Pioneer Peep Laja on the Past, Present & Future of the CRO Industry

“90% of A/B Test Results are a Lie” Interview with Andre Morys from WebArts AG

 

Useful Links