Stitcher for Podcasts

Get the App Open App
Bummer! You're not a
Stitcher Premium subscriber yet.
Learn More
Start Free Trial
$4.99/Month after free trial

Show Info

Episode Info

Episode Info:

Apptimize: Mobile Strategy, UX and Growth

This is the inaugural Apptimize podcast! Our CEO, Nancy Hua, speaks with Vinayak Ranade, Director of Engineering for Mobile at KAYAK about how to identify what to test. Listen to the podcast and/or read the transcript below.

Nancy Hua: Hi everyone! I’m Nancy, the CEO of Apptimize. I’m here with an old MIT friend, Vinayak, the Director of Engineering for Mobile at KAYAK. Vinayak’s been at KAYAK for three years leading KAYAK's development of iOS, Android, and Windows apps. He’s had a lot of success with optimizing KAYAK’s mobile app through A/B testing. In this interview, he’s going to share some of what he’s learned with us. Welcome to the Apptimize podcast, Vinayak. Thank you for coming. What’s the most interesting A/B test examples you’ve done?

Vinayak Ranade: One of the most interesting A/B tests that we’ve done recently is something that confirmed one of our design decisions. And it was related to user security on the app. So one of the things that users are worried about today in mobile is paying for stuff. And one of the reasons they’re worried about it is because they don’t know if it’s secure or not. What if you’re going through a tunnel and your connection goes down? Or what if there’s someone who’s snooping on your wifi? For some reason people don’t seem to have the same sense of security that they have on their desktop as they do on mobile. And to be fair, they didn’t also have this security when they started out on desktops. So, to kind of help users through this process and to make them feel comfortable when they’re actually trying to buy something and charge their credit card, at KAYAK we decided to add a small message at the bottom that just reassured the user that everything’s going to be okay if they buy something. And the idea behind this was to tell the user it’s really okay if you make this payment.

There’s a different company that tried this and they actually A/B tested it, and what they found was that removing this messaging actually improved their conversions, which was very puzzling. And the theory that they had about it was that if you remove this then you are no longer reminding the user about the security issue in the first place. So their thesis was that users don’t generally think about this and if you put it there, you’re reminding them that oh, this could have been insecure. That’s why users then tend to shy away from clicking and actually spending money on your app.

So we decided to do our own experiment about this and we actually found the opposite that when we removed the messaging, people tended to book less. So this was an interesting thing for us because we expected that our original design decision may not have been the right one but it turned out to be the right one. And so this was an interesting experiment because it was related to user behavior, it was something that we weren’t super sure about but it definitely confirmed our design decisions that we made originally.

NH: That makes a lot of sense. Just because something worked for someone else doesn’t mean it will work for all apps. You have to try it out yourself.

VR: Absolutely.

NH: So with everything you could be A/B testing, how do you prioritize what you test so you can work on something that will have a big impact?

VR: So normally, our test requests come from many different sources. Sometimes they’re from executives, sometimes they’re from engineers, sometimes they’re from salespeople who are trying out an idea that a client suggested, and sometimes they’re just changes that we’re going to do and we just want to measure them. So we try to keep a balance between kind of being a little bit bold and reckless and also making small and incremental changes just to the app. Generally what happens is that one or two people, including myself, collect all of the test requests and we go through them and we try to figure out which on...

Read more »

Discover more stories like this.

Like Stitcher On Facebook


Show Info

Episode Options

Listen Whenever

Similar Episodes

Related Episodes