Tavus CEO Hassaan Raza: Teaching AI to Converse Like Humans | The AI Rabbit Hole 2025 | DVC

Hey everyone, I'm Hassan. I'm the co-founder and CEO of Tavus. It's so nice to see everyone. So, I want everyone to take a second and really think about the last great conversation you had. Like a conversation that left you feeling good. For me, it was a conversation I recently had with one of my best friends of Spawn. And the conversation left me feeling really, really warm. And it felt like there was this openness, this connectedness. I don't see him often, but it felt like we could talk about practically anything. I want you to hold on to your memory of a great conversation and let's talk about what made it so good. What was it that left you thinking about this conversation? It made you think about it even now. Well, the first thing that makes a great conversation is one that flows smoothly. So great conversations are often one where there's a feeling of connectedness.

For me in Osman, it was we were almost like playing off each other, like we were almost finishing each other's sentences. It felt like we were connected. And it's been studied that humans that really connect will respond to each other in a fraction of a second, 250 milliseconds. And this level of connection is what makes us build trust. I'd also wager that the conversation you're remembering was not over a phone call, but it was probably a FaceTime call or even in person. For me, this conversation with Aman was me visiting him in Texas. And the reason for this is that humans are face-to-face communicators. We evolutionally designed to communicate face to face because so much of our communication is non-verbal. I mean the expressions on your face like this emotional intelligence and I'd also bet that the conversation that you're remembering was not with AI and I promise you my best friend Isman is not an AI.

And this is interesting. It's because, well, machines don't understand us. Computers aren't built like us. They were actually built in our opposite. They're logical, unambiguous, binary communicators versus humans are lossy, ambiguous communicators that that speak as much with their words as they do with their expressions. There's so much nuance to humans. So, computers just don't get us. And AI was supposed to be the step forward, and it has been, but we're still relegated to shitty chat bots and audio bots that still don't quite get us. And this is unfortunate because we all grew up reading and watching the same sci-fi, whether it was, you know, data from Star Trek, Rachel from Bladeunner, or Cortana from Halo. These AIs that felt human, they felt empathetic, they felt like they got us. And if we believe in this future, whether it's an AI therapist that understands the nuance of words or it's an AI tutor that can sense curiosity or confusion, if we believe in this future, then the interfaces in which we communicate with machines has to change.

Machines have to learn to communicate the way we do face to face without compromise. So how do we teach machines the art of how to be human? how to communicate with us face to face. Well, again, the first thing we need to teach them is how to have the rhythm of conversation, how to know when to speak up and when to stay silent, the art of turn taking. And this feels really simple, right? It's like ah like you just have to listen for silence. And and speed is key here. We talked about how humans that connect best, they respond back in 250 milliseconds. But speed isn't everything. I'll give you an example. Last year at Tavis, we launched the conversational video interface, CVI, and the team was incredibly proud that we were able to get it to respond in as low as 500 milliseconds. But, you know, the first feedback we got from some of the enterprises was that it was too fast to slow it down.

And it wasn't that it was too fast, it was that it was unintelligently fast. You would take a pause to think and it would respond. But humans are thoughtful. They're considering what you're saying and how you're saying. They use that to determine whether it's their turn or not. And we have to teach machines the same thing. So if you build models for lexical and semantic analysis to determine the context of situation and use visual analysis to figure out if someone has really stopped talking or you know should I talk, should I interrupt? You can have the rhythm of conversation that makes it flow naturally. Okay. So you've taught it when to take its turn. But again, human conversations are majority non-verbal. And if you want to have machines that have empathy and emotional intelligence, you actually have to teach them to see the way that humans do in real time.

And humans use something called contextual perception, which is that we use all of the context that we're seeing, whether it's where you are, the expression of your face, what you're holding up, and we use all these things, and then we bring them together to figure out how we should use them in the conversation. So, questions like, is the person in bed right now or in their are they in their car? Is that smile on their face due to nervousness or are they actually happy? Are they holding up something and referring to something? Or are they actually listening to us? Are they not paying attention? All of these are questions that are essential to answering the key elements of a conversation. But you can't have them without perception. So if you want to teach a machine to have empathy and emotional intelligence and to be able to participate in a conversation with rich visual context, you have to give them real time contextual perception, the ability to see.

Okay. So now the AI can it knows when to take its turn and it knows it can see the way a human does, but you actually have to be able to interpret those signals and know when to use them and how to use them. And this is where you need conversational reasoning, something that LLMs are quite good at. So using a fine-tuned LLM that can interpret these signals and and use them to determine what to say at the right moment is really important. So, you want it to answer empathetically, efficiently, and intelligently. You want it to respond the right way at the right time. You want it to respond empathetically to someone frowning, for example. But you don't want it saying something like, "I see that you're frowning, and that really shows the wrinkles on your face, even if that's what the computer sees." So, conversational reasoning gives you the ability to use the right context at the right time to achieve your conversational goals.

Okay. So now it can see like us, it can hear like us, you know, it has great conversational abilities, but we're still only achieving 50% of the face-to-face equation that humans are evolutionally designed for. And so, you know, if you want AI that feels natural to talk to, it has to look the part, too. And I want everyone to remember the last time they were in a Zoom call and the other person has their camera off and you have it on. It feels a little sketchy, right? Well, it's because it kind of is. You're missing a whole modality because, you know, you don't know, are they listening to you? Are they even there? They're probably not. It's really tough, right? You're missing a whole piece of the equation. And so, if you want to have AI that feels natural, that feels empathetic, that feels face to face.

You have to give it a face that feels human, builds trust, it reacts to your happy moments, it reacts to your sad moments, it gives you the confidence that it's there. This is how trust and openness is built. So whether it's a slight smile, a raise of the brow, just nodding to show you that's listening to you. All of these are key elements that come together to build for an expressive, emotional, empathetic experience. And if you have all these things, then you can have a great conversation. And great conversations unlock so much for us humans. It's how we build relationships, how we build trust, how we learn, how we teach. And if you teach AI these things, if you teach AI to see, hear, look, perceive like a human does, then you can unlock a future where you have AI therapists that really understand us.

AI tutors that can help every student uniquely, AI interviewers that can read between the lines, or even AI companions that get us, really get us, or it can even be our friends. And maybe it can even have as good of a conversation as I did with my best friend. At Tavis, we're building foundational models to do exactly this. We build models, teach machines to see, hear, look, and respond like a human does, all in real time, so that you can build this future. And we can't wait to see what you build with it. If you haven't already, come check us out at our booth or talk to Charlie on our website. I know he would love to talk to you. and it's probably going to be the best conversation you've had with an AI at least. Thank you everyone.