15 Nov Interview: Mobile WebRTC development
WebRTC is not just about web browsers, more and more applications are deploying WebRTC on mobile devices too. I had a chance recently to talk with Emilio Cueto, CTO of LiveNinja about their experiences building WebRTC based applications on iOS devices.
A transcript of our conversation is below the video.
Arin Sime (Editor, RealTimeWeekly): Thanks for joining us today! We’re talking with Emilio Cueto, Co-founder and CTO of LiveNinja, in Miami, Florida. Thanks for joining us Emilio!
You’ve been with LiveNinja since 2011, providing customer service focused tools based on WebRTC. Tell me more about LiveNinja.
Emilio Cueto (Co-founder & CTO, LiveNinja): We started in 2011, and it was originally a marketplace to buy and sell expertise. It was WebRTC from the start and a browser based experience. At that time, WebRTC was still very new – you had to set a flag in Chrome to use it. But the golden egg for us that we thought was so cool was that you didn’t need to download any applications, you just talked to somebody right away.
In doing that, we would take a percentage of each session that happened. So if I wanted to learn Spanish from you, I would pay whatever the session was, we would have a video chat over WebRTC, and LiveNinja would take a cut. But those transactions were only $40 or $50, we would take 10%.
We got a lot more interest in people wanting to use LiveNinja on their websites and pay for it that way. And as a startup, you obviously want to go where there is more of a chance of success, so we pivoted into what you know now as LiveNinja. We are focused on customer service, for brands to communicate with their customers, as frictionless as possible. Basically just click a button on someone’s site, and you can speak to someone from that brand who can serve you.
Arin: I should mention that we are doing this conversation today over LiveNinja, so I went to a link to get into a direct conversation with you. One of the things about the UI that I really liked is you have the guide, and her video is there and she’s telling me that I need to turn on my camera. It’s a little bit more than the typical interactions you see, and I presume some of your users are actually using WebRTC in the browser for the first time.
Emilio: Yes, that still happens quite often. WebRTC, in and of itself, is really just one component of the experience. It’s like driving a car, whether you are driving a Hyundai or a Formula 1 car, you want to make the experience as nice as possible.
The video guide that you saw was the default, but it’s customizable and could be someone from your company. You can do it in any language and color scheme that you want so it fits into your site. The WebRTC part should be transparent to users.
Arin: Are there particular segments that you focus on?
Emilio: Fashion is a vertical that we have a lot of traction with. Makeup and stuff like that as well. People use it for sales calls, and just recently we launched a kiosk application on iOS to have a presence in brick & mortar stores. I think we are the first company to do that, to have mobile, web, and brick & mortar.
Arin: That’s very cool, so the iPad just loads your application and anybody can walk up to it?
Emilio: Yes, the kiosk is a single app deployment where the only thing you can access is this application. Keep It Simple Stupid (KISS), with one button, lets you speak to somebody from this branch, and magic happens!
You can speak to the best available agent at that time, which is a huge component of what we do, the routing to the agent. For instance, if you have 5 agents serving calls, as a default we will get the person available with the least amount of talk time that day. Otherwise, we can just blast everybody, or any sort of scenario that you need.
Arin: You started out on web only, were you using a particular platform or did you build your own WebRTC application?
Emilio: At the very very beginning, we used OpenTok. There were only 3 of us, and we just had to get something together. OpenTok was great, but we realized we were reliant on others as we grew and that might limit you.
As the company grew, we decided we needed to this in house. That’s not to say anything against OpenTok at all, we love them.
What we do now, for the kiosk for example, we use SIP as our signaling layer. WebRTC is already supported in Chrome, Firefox and Opera. We do use one 3rd party plugin to support Safari and we will support IE soon also.
As far as SIP goes, I specifically chose that because at the time, four years ago, it was much harder than it is today to support WebRTC on iOS. It’s not a walk in the park today either, but with SIP there are a lot of libraries already iOS compliant. So to me it just made a lot more sense to use a component we could license and wrap our business logic around it. And like I said, to the end user, they don’t care if it’s WebRTC or not, they just want to see you!
Arin: Things in WebRTC are changing monthly, still even now. Looking back, are there things you would do differently? Any different technical choices?
Emilio: We went too far into the do-it-yourself route. We setup our own site, with freeswitch, with MCU’s in the middle, and it was a huge undertaking. We bit off more than we could chew, and so we offloaded that and partnered with OnSIP. They’ve been absolutely superb and we use their SIPjs library in the browser. We channel everything through OnSIP’s services right now. All of the application logic that we have is obviously separate.
Looking back, I would have not tried to build the whole stack ourselves.
Arin: Things are pretty solid on the web side of things, but let’s start to talk about mobile. The kiosk application was your first move into iOS?
Emilio: That is our first move into iOS. On Android devices, the application works fine on any browser, but this was our first chance to dedicate resources to iOS.
The kiosk has been something we’ve received more interest in than the web app, which is exciting.
With video chat, people are always thinking about if they are ready to go on camera, but when you walk into a store, you’re already prepared to meet people so it’s very easy to just click a button and talk to somebody.
Even though it’s easy on the web, it’s still a little bit shocking to people on the web that they are about to go on camera. “Wait! I’m not ready yet!”
The in-store kiosk might even be a bridge to make the web side better, who knows?
Arin: So everyone on the representative side is connected through a Chrome browser?
Emilio: They can connect to LiveNinja via Chrome, Firefox, Opera, but we also have a desktop application. We also offer the ability to select different cameras in the session easier on desktop.
For example, if I wanted to change my camera during this call and I click the camera icon in the browser, I have to restart the call. That’s a problem, especially on our system, because when you restart the call, you might not get the same agent again! With a desktop application, we have our own virtual driver that allows you to access any camera connected to the system so it’s seamless.
Arin: That’s a lot of extra work to maintain a desktop as well as the web application!
Emilio: I agree, but at the same time it’s worth it. At the end of the day, we’re trying to allow you to have the best experience possible on both sides of the call. It’s very easy for us to deploy the desktop application to you. It’s basically a wrapped version of Chrome, so it’s not that much overhead.
Arin: Do you see any difference in reliability or quality on the desktop versus web?
Emilio: No, since it’s Chromium so it’s basically a web browser. But it’s Chromium Embedded Framework, not Google Chrome, so it is a separate community and product and there have been some differences we had to account for, but nothing major.
Electron is another good one to consider for building a desktop app wrapped around the browser. I think Slack uses that one.
Arin: How did you implement WebRTC on the iPad?
Emilio: We are using an open source framework, it’s basically a wrapped version of Chrome, and it supports VP8 and Opus. Essentially we are using the same two codecs that the agent side already supports, and we are using SIP as the signaling protocol. So when that negotiation happens, you’re using the same codecs, and it’s golden. That’s why I chose SIP long ago. I was thinking ahead that I didn’t want to deal with all that.
If I had to do it again today, there’s other SDK’s that exist, like OpenTok and PubNub and Twilio. But some of them are audio only, and again, then I’m limited to a third party.
But I think I might have still done it this way, because it also adds the flexibility of adding a desktop. A lot of people hate SIP because it’s old and cumbersome, and I will say it’s cumbersome. But it’s also been bullet tested and 90% of all VOIP traffic uses SIP.
Emilio: My background – that’s a whole different discussion! I was a musician before this, maybe that’s why my ideas are what they are!
When you’re on the road as a musician, you work for 2 hours and then you’re on the road. The rest of that time I was building website for my manager and their artists. Nothing application driven, but I was fairly good at it. So when I settled down, got married, and moved back to Miami, development was what I chose to do.
In 2012 we went full force on LiveNinja, and I’ve grown immensely since then. I learn every day, and I think the key to that is not pretending you know it all. Maybe a CTO shouldn’t say that, but I promote it. Surround yourself with people from other skillsets and learn more everyday.
Arin: How many developers do you keep in house?
Emilio: In the company total, we’re about 16. For developers, we have about 8, including myself because I code every day.
Arin: In the mobile development space, do you have to keep pace with the browser changes? How is that matching up?
Emilio: Not yet, since the underlying signaling is SIP and we’re supporting VP8 and Opus. So we really haven’t run into any issues yet.
Arin: You mentioned supporting the Edge browser soon…
Emilio: Edge is interesting! Right now, you can get an audio call running with Edge, but they are only supporting as of now a proprietary fork of H264. There is no public announcement of when they will support the open H264 or VP9, any of that. So for us, we are still waiting to hear what they will support. Do we want our product to be only supported for audio? I don’t think so.
I don’t want to limit our product just so we can support Edge. Internet Explorer is different, it doesn’t support anything. But the same 3rd party plugin provider we use to support Safari, we may use for IE in the future.
Arin: Any other lessons learned you would share with WebRTC developers?
Emilio: It’s still not even a standard, so don’t ever think that you know it all. Just talk to people, if there’s a Meetup in town, go to it. Make sure you are conversing with other WebRTC developers and learn everything you can. The fact that it’s not a standard yet means even the people behind this are learning every day what’s best.
Arin: Anything else you want to plug about LiveNinja?
Emilio: We are awesome! Go to LiveNinja.com and check out our site. We are working on our next version of the product based on the year or two of feedback selling the current version 1.0. I think that’s going to be pretty exciting and hopefully completely disrupt messaging in general!