Naked Wines: Don’t Debate, Test

How do you, as a leader, think about this testing culture? Why do you think that most leaders find it so hard to follow such a similar test and learn culture?

Very early on, we developed a ‘don’t debate, test’. It’s one of our key founding principles. The reason we did that was, as the organization grew and we brought people in, especially marketing people, what we found is, they placed a lot of emotional store on trying to be right; trying to be a water diviner. ‘I think the answer is over there.’ We kept going, ‘we don’t care what you think; we don’t care what I think, either’. We just want to know where the water is. What we really care about is, are you going to do a really good job of finding the water. Once you’ve found it, are we going to be certain that’s all the water?

So what had started as being something which was just instinctive, as the organization grew and we bought more people in, from the outside, we realized we needed to codify it. What that boiled down to was three things. First of all, anybody who’s got any idea, we’re not interested in the idea, unless you can turn it into a test. I think there are a lot of companies who ask people for ideas and ideas are cheap. If you really believe in your idea, you have to be prepared to put the energy into figuring out how we are going to prototype it. We want ideas, but only if you’re prepared to show that you believe in it enough that you can develop it to a prototype.

The second thing was, we found a way of just answering questions really quickly. A lot of people would build big, complex supply chains, around potential new business ideas. Everyone gets really absorbed in the minutiae of building a new business – it’s fun, it’s exciting – without really knowing if there were ever any legs in it. We developed something we called paint a door testing. This was, if you want to know if an idea for a new nightclub is going to work, don’t open a nightclub. Just find a wall, paint the door in it and if you see lots of people trying to open the door, you know it’s a great idea. Then build the nightclub. But if everyone just walks by, don’t bother building the nightclub as it’s just a waste of time.

The third thing is, we don’t care if you can guess right. We only care that you’re thorough. We used the example of an oil prospector. Oil prospectors don’t pay people to go, I think it’s over there, I think it’s over there. They pay people to map out the landscape and do survey testing and drill lots of cheap holes. So that, number one, if there is oil, you’re going to find it. Number two, once you’ve found the oil, that’s all the oil. There isn’t another pot of oil somewhere. In a way, the best thing about testing is celebrating negative results. It just means, right, we can stop talking about that. We can stop diverting our attention. We can stop challenging and questioning ourselves. We’ve tested it, we know our customers don’t want that; don’t waste another second thinking about it.

It’s like the data wins and the data is all we care about and is important, but the key corollary to that is, you celebrate negative test results as much as you celebrate positive ones, because they are just as valid.