Enterprise AI at Scale: Google Cloud, SAP, Koyeb & GetGenAI | The AI Rabbit Hole 2025 | DVC

the most comfortable one. Good. Welcome everyone. We're going to be discussing today I hope the most pressing topic that has at least the largest checks for startups on how do we adopt AI at scale at enterprise. And here I have industry leaders who have that who sit actually at the front seat to the real world scaling of AI across the enterprise. So if you don't mind just introduce yourself and after the introduction maybe let's kick it off with this simple question of how does AI at scale look at your org and what have changed in the last six or 12 months. Yeah. So I guess I can get started. I'm Yann. I'm I'm the co-founder and CEO of a company called Koyeb. So we are providing serverless infrastructure and we are serving customers around inferencing workload finetuning and we are also serve more classical a high performance APIs.

and for the introduction I think that's that's it. So we see a large variety of workloads from small companies to larger businesses who are trying to enter this field. and I guess I'm I'm going to hand it over to you. Thank you. Yeah, nice to see you all. Yeah, my name is Raghu. I'm with SAP for a little over two decades in different roles. I wrote a couple of books on how we are implementing AI at at SAP with our customers. So currently I'm leading u one of our AI strategy about how do we transform our customers with a huge depth I think they have a wide variety of complex landscapes and how do we get into the newer world and there is a lot of buzz about AI and generative AI and obviously these are the questions coming up. So generally it's kind of going from the curiosity phase into the capability phase.

What can be done? What is the business value that I can attain with this? So those are some of the things. Over to you Kevin. Thank you. I'm Kevin Sai. I'm in the team called applied engineering which is part of cloud AI. So if you've heard of the products like Vertex or agent space, if you heard of the recent framework called ADK or agentto agent A2A protocol, that's that's that's cloud AI. So my my team focuses on working with customers. but we're trying to find patterns as opposed to just driving. We're not really sales team, right? We want to find patterns of failure, patterns that work. and then we build certain solutions to help customers adopt cloud and we also are the feedback channel back to product as well on how they could improve their products. so if I look across scale right the question was around scale what does scale look like for our customers I can't I think I will kind of break it down into maybe two or three categories.

One would be the traditional enterprises, right? So traditional enterprises, they deal with a lot of very messy backends. There are a lot of systems, right? And they have a lot of I wouldn't say smaller problems, but they have a lot of problems like range from smaller to bigger ones, right? So scale for them probably would be, you know, how could they have a you know a practice that they could adopt AI more quickly across the organization? Whereas on the other end, you have digital natives. These are the people who you know they don't have as much legacy right you know and they have usually have much bigger but fewer problems so scale for them is probably you know actual like being able to scale in tokens right the the the speed right so that's kind of like how we look at scale for you know the two different groups of customers I really like this definition right there are different parts of the scale depending on the angle so happy to share a bit More about myself.

I'm a CEO and co-founder of Get Genai and we help regulated industries like healthcare, pharma, finance to automate content compliance and our mission is to give creativity creative and legal teams speed and freedom to do what they need to do without compromising on the policy. so Raghu I was really interested to hear a bit more. You say that you observe this shift from curiosity to the capabilities the implementation. So what what do you see within this shift? What did not work at the curiosity stage and what is providing insights and values at the capability stage? So yeah thanks for that question. And so when we've been working when we are working with the customers for the last four five years a lot of things have happened right I think if if I go back a bit more in the mid of 2017 2018 that was the time we built the first set of use cases how do we increase intelligence in a particular end toend process whether it is procure to pay order to cash or something like that and then later on we started building robot robotic bots and then we started kind of interfacing and putting a lot of conversational interfaces on top of that.

But now every every few months we kind of get some new technology and then we say that hey we have to implement this. So the customer questions us like what is happening here? I think I have my end process. I have this particular scenario. How can it make me kind of increase the productivity or kind of address the operations at scale? So this is where we have now being transitioned from a normal deep learning or machine learning phase into an agentic AI space where we are trying to connect all these different tasks and how I can kind of make it streamlined or or or I I would say like how can I make these workflows better with these different contexts or different concepts that we have and we are we are in that process. And for us the customers have their complex complex or hybrid landscapes out there.

How do we convert that or how do we transform them into these new businesses? So that is where I think a a lot of questions would come up. What are my business capabilities and what are the solutions that are available here to address these business capabilities? What are the technological challenges that I have? am I able to kind of leverage the current technology at hand and then finally it comes to the question what is the business value I'm attaining so that is where we are and when that is when that is what when I said that from a curiosity phase about some new technology coming in to what are the capabilities that I'm trying to address am I really making some sense out of it so that's where we are got it so and how long does it usually take for a client to switch from curiosity to capabilities and then to implementation.

So this is a great question. I think you might have noticed I think depending on the scale of the customer that you have from a small and medium business customer to a medium or a large customer and depending on you have operations at a head head office to all these different plants or locations at different locations different countries. So depending on that it can it can go anywhere from few weeks or few months to years too but then you have to plan this in a phased out approach and how how can I take this from one maybe I'll start with u one particular location or one particular country or and then maybe kind of go to the other locations so to answer that question it is depends on the current business scenario that they want to first transform to what we have currently. Got it.

Makes perfect sense. Thank you so much. So Kevin, question to you. so we all know that Google has seen the rapid adoption of AI. and it's not just within one or but it's across all the industries. So I'll be interested to hear what are the most significant shifts in how enterprise clients are implementing AI. What were you observing? Yeah. So if you kind of roll back to maybe 2023 right there's a lot of experimentation right and people are experimenting with prompts on you know models and then we started having more evolution right then there's rag and there's all these other capabilities that that are coming in I think one of the things that really has shifted for enterprises I'm going to talk about just you know as opposed to you know if you're a startup you know for enterprises we see that there are more actually startups, right, and companies giving the giving these enterprises a choice instead of building on their own to buy instead, right?

So two years ago, you probably have to build a rag pipeline, but today is a rag pipeline you're differentiating value of your company. Do you want to your engineers to spend time building rag pipelines to do search or would you want to go and use agent space or glean or some other product where you can buy? Yeah, you're paying money, but you buy and then you you keep your engineering resources to focus on the layer on top of that, which probably will be some kind of more powerful agentic workflows that are customized for your business that could actually make you differentiate, right? So, and if we have this conversation two years from now, those agents will probably be more commoditized as well, right? So, what we're seeing is this ladder where there's comm commoditization going up this ladder, right? so now I would say in the past 12 months customers have much more of an option of buy versus build conversation as opposed to I always have to build that is probably one of the and also you kind of think about the the connectors right you know enterprises have a lot of backend systems do you want to build connectors or do you want to like just let somebody else do it and then you kind of you know take advantage of that right so I think that that's where the enterprise story is today these are definitely good news for startups jobs, right?

selling to Google. that is a good thing to have when you're an early stage startup. and I actually observe it within our company as a get AI. We made our first enterprise sale to a US top 10 company in the heavy regulated space. and it happened after six months after our release of the product. So I really believe that a lot of enterprise are making this decision of buy versus build which is amazing. So yeah I understand that you're on a different side so I'll be really interested to hear from you about the application areas and value. So from the in infrastructure side, which use cases like co-pilots, generative models or predictive analytics are you seeing deployed most aggressively? Yeah. So on our end we see mostly generative models. that's like our main focus and we seen basically the shift from like one year ago where the conversation was still around like LMS mostly.

we we do have a lot of like custom and high performance deployment of LLM. One of our key focus is like realtime inference. So we have people who are latency sensitive usually and have actually like production workloads. and because there is no point in a way I mean you can get started with us but it's we give value when you switch to production and when you can customize your stack. and so we've seen this shift from LLMs to the rise of VLMs, vision language models. and there is also like there has been like a significant shift with flux on the vision models too. And now since the I think like the beginning of the year we are seeing like all the conversation being around like agents and the question shifting actually from on the infrastructure side needing like GPUs or any kind of accelerators to getting to something which is more traditional infrastructure which is actually running on CPU to run agents and where the need is more traditional in a way it's probably around like getting sandboxed execution.

and so this is what we're seeing right now. most I mean like newcomers are asking about like agentic workflows. So you still need the LM, the VLM, the or potentially a vision model, but it's part of a complete infrastructure where you use this multiple models to to run your end application. So Jana, I just learned at our backstage conversation that you live in Paris. Your company is located in Europe as well and you spend a lot of time here. I'm sure you have a lot of clients coming from the US as well. What do you observe? What are the differences in adopting AI in enterprises in Europe versus US? Yeah. So we have we have a platform which is which we operate across like 10 locations. So most of our locations are actually in the US. we have four location in the US some in in Europe and some in Asia.

And in terms of adoption like the the what we observe is is like it was 10 years ago which is basically US adopts first. people tend to move faster. so we do have an edge in our local ecosystem. but there is probably more fear. I'd say it's like the the the way I see it's like people are more yeah they are more careful basically in the way they adopt technology and they tend to wait longer before they adopt they tend to let people make mistake before which means they are not moving necessarily fast in Europe. So that's why one of our main market is actually the US. and also even if you go from our infrastructural perspective like the silicone layer and the hardware layer is happening here in the Bay Area. So most of our strategic partners around technology are in the Bay Area.

yeah so and we have like u users which are happy to in Europe because we're a European company but people the main question is what is the best product in in general like I think like that's what people still care about in the way they adopt today then in enterprises it's a it's a bit different and they are really like careful I think it's the right way to put it like they they think a lot about like what's going to happen. Yeah, that must be really interesting to be in a position right to observe these differences between different continents. So on this topic question to all of you what do you believe are the top blocker to AI scale today? We we were just kind of talking about you know how to define that right but again I think blockers again it depends on the type of organizations you have right you know for some more traditional enterprises it could be bureaucracy right it could be you know just the culture shift right and then for some that are you know especially the startups who are really in the cutting edge of technology right then they run into more technical blockers so I I think it kind of depends on you know what what what type of client or customer that you're talking to.

Yeah. I think like one of the feeling I have is also skill set. is probably still like a key question like in startups like you have people who are here to work on this technology but like in larger enterprise it's just like keeping up to date with the the pace of change is is a real like topic like on our even on our side like we're below everything so we see all the workloads like running and so we have to try to understand what everyone is doing and I guess it's the same for you and just like keeping up is difficult for us but it's our business. so if it's not your business, I think it's a it's a real challenge like to to to keep up and to figure out like what to do. Actually, that's a really good point because we we've been talking to customers about moving to Gemini 2.0, right?

And then we announced 2.5 and the customer, well, I don't know about 2.0 anymore. Let me wait. Let me wait. Right. And then so so innovation itself, right, is so fast that it's it's kind of overloading the customers. Yeah, that's a really good point. Maybe I'll add a few more things. I think initially I thought like I'll come up with infrastructure and cost as one of the things but then I would not go there. The reason being I think most of those challenges are being addressed in some way or the other whether it is the infrastructure or the cost or the technology because there is some kind of I think now that every company I think they have some kind of a budget but like you mentioning I think yeah I'm mentioning I think change management is one of the biggest things that is where you have different cultures different things that's that's happening.

The second important thing is like I was talking with Kevin backstage about u deployment. Deployment is going to be a huge things I think you will have to plan well because deploying and making it available enterprise scale. always delivering something in a prototype or a P kind of a thing is easier but when you go and deploy large scale I think those are one of the things and this is where your budgets and everything gets affected. I think that's the other thing I would say there's also an expectation right so for example we worked with a couple of rating agencies on rating cos right collaterized loan obligations and these are like really complex documentations docu do documents and it varies depending on the document and you want to extract out you know certain very critical elements right for an analyst to do that it'll take like weeks right so we want to so this is actually something that we could do but then you you're the then they oh it's not 100% accurate it's not 100% accurate are your analysts 100% accurate right so what is that what should that bar be right so sometimes they they have this bar of 100% right when in reality right that's even 90% is better than what they're getting yeah because you you couldn't measure the analytics accuracy before right so you don't have like a a comparison to make absolutely so as a founder of get genai What our mission is is to solve the friction between different teams, right?

So, we're solving this friction between marketing team and legal team because if you're a regulated industry, you cannot just write your blog post, right? You need first to go to your legal and check on all these hundreds of rules. and that's why they value us because we are reducing all the friction in the major workflow for them. So that's why I'm wondering like where do you see the biggest opportunity for AI to reduce friction between teams within your company or within your clients that you advise? Sure I can can start on that I think so the first and foremost thing is that you need to have the proper alignment between the business and the tech the tech teams. So that is one thing I think you'll have to address it. The second most important thing is there are a lot of teams a lot of tools out there.

but how do you get this into perspective? So rather than using this there are different tools how do you make teamwork more priority? How can you make these teams work together? Not not about working within teams but how do you work with cross functional teams. So that's one thing I think you'll have to address. Of course, this goes hand inhand with understanding between the business and technology too. So it's pretty much like speeding up the process, right? Yes. Yes. Yeah. I mean I was I was just thinking that one of the key thing we see recently especially with like the rise of agentic workflow is probably around business process automation in general. So and so I I feel like there is a lot of potential around gigal but then like there is a lot of resistance. I mean like legal is a resistant field in terms of like change especially since like the when you you look into law films like the way u they are compensated is by hour.

So if you reduce the amount of hours they have to do you're reducing like their u compensation. So it's a complete field which can be radically changed and I think it it affects like a lot of internal processes like typically when you need I mean what you were just mentioning is like so if you're able to apply especially because it's a lot of text it's one of the key application but they enter you also enter this resistance about accuracy that you were mentioning where they keep claiming yeah but we cannot use it because it's not accurate. but I think like still it's like one of the area where you you do have a lot of like potential today 100%. I'm I'm very aware aware of like the value that your company, you know, geta provides. You know, we work with a lot of Can we sell it to Google Mark?

What can we sell it to Google? Well, wait, you're not we're not a customer yet. Okay. Okay. later because because we we work with a lot of advertisers, right? You know, so that is a very big problem for them. Absolutely. You know, but I think I would I would consider that as kind of like a very core process offering, right? you know but then there are also other there are also other AI capabilities which are more general probably less value but applies to more people. So, for example, think about, you know, audio transcription, you know, in Google Meets or Teams, right? Microsoft Teams. I mean, you don't have to bang away, you know, like taking notes anymore. You could really be present. And I I noticed that a lot when I'm interviewing candidates. I obviously I ask, you know, for permission, can we transcribe this?

And if if we could do that, I could be really present, right? You know, or tools like Notebook LM, right? you know maybe if I'm a product manager I I want to maybe you know understand some of the regulation we're just talking about the regulations from different countries right with Rau yeah so then I could be more informed before I even go to my legal team right for example so there are those kind of more horizontal tools right as well so I think there are a lot of opportunities for startups in this world yeah the horizontal tools definitely a great example I mean I use Google meet transcript and I I feel it myself how I can be more present on the interviews. So, thank you for this product. I know we're out of time, but we're going to do the last rapid question. so one sentence only.

If you had to name one AI trend that will dominate boardrooms conversations over the next year, what would it be? I can get started. So, I think like it's I mean definitely agent close like that's the thing which we see like coming up everywhere. So I guess like the question your your B is should is asking like is like how do you use agent workflow or AI agents? how does it apply to you? Yeah, I would like to add on top of that like AI agent orchestration or multi- aent orchestration right between how how can different teams work together different? Yeah, I' I'd probably say it is when digital workers we we used to have LLMs as just tools, right? But when digital workers actually come in, how how should you restructure organization, right? What's your management structure going to be like? Should you even change your business processes and business, you know, what what business model actually, right?

So there are some of those you know existential type of questions that they probably need to start answering because if another competitor knows how to integrate digital workers better that competitor is going to eat your lunch. Yeah. So to summarize the overall agentic future and the connecting tissue for it in between different business units and overall how does it impact your competitive advantages and your overall business model. So that's the future for the next 12 months for all of us to think of. thank you so much. Thank you for all your thoughtful answers. Thank you.