The Future of AI Infrastructure: DVC's Nick Davidov in Conversation with Nebius' Roman Chernin

Okay, welcome Roman and thank you so much for supporting us three years in a row.

I was Is cookie real?

It's real. You you can have it. and it's free. Well, I I think Nebus paid for it anyway. So,

okay, we paid for it.

Okay. So, guys, Nebus is the hottest info company in the world. I think it's not trying to build data centers in space. It's doing way more practical things. A lot of things amaze me like your Finnish data center uses recaptures all of the heat and uses it to heat up villages around it. It's all like not consuming too much water. It's it's like really really cool technology. But on top of that, I've read that you guys save about 20% on hardware and then pass it to tokens per what? How do you guys do that?

It's kind of be cool called hottest here in San Francisco in any category. take it as a credit. Actually the answer on many things like how we do it is like why we do it is kind of because we can and here you need to go back to how company was created and a lot of people know maybe someone doesn't know that the core team went out of Yandex and Yandex was a R like large Russian whatever hyperscaler we were super lucky just to start with the mix of expertise with the people who know how to build across the full stack. So we had people who built and operated all the infrastructure of Yandex. people who built servers for many years because Yandex had like the long history of working with all the vendors with Nvidia, with Intel, with AMD and we had like few generations of Rex and obviously we have a team of software engineering that in the past built cloud and not only cloud large services and we have a YI expertise.

So like that's we were lucky to start with we kind of got a feeling from our past maybe or experience you can call it intuition that building the full stack will benefit a lot even though most of the industry at that moment of so-called neo clouds were not building the full stack people rented data centers not building nobody builds hardware a lot of people actually don't build software or full stack software And yeah, that was the bet to build this full stack kind of cloud approach, full stack hyperscaler approach. yeah and and and now it gives it benefits.

So you operate everything starting from like plumbing and wiring to all the way to now a gentic search. Can you tell us a little bit about what types of customers buy what and why do you go like across the whole thing?

Yeah, it's it's a good question and we need to look from the customer's perspective. You're you're absolutely right and each layer of what we build we build to address the next or another type of the customers and another archetype of the customers. Most of the media talking about this super large compute deals that we also have we have meta we have Microsoft

27 billion from Meta 20 billion from Microsoft if I'm not mistaken.

Yeah, approximately 7 17 and 12 I think or seven whatever. so it's a big deals it's a big it's it's actually a big headers but what you need to remember when you work only with those customers that first of all there are maybe like 10 of customers of this type all the super labs and hyperscalers and second at least we view it that they don't need us for the long term why Microsoft buys from us why metab buys for us from us because they just need to move fast they cannot build physical world fast enough just yet but eventually we should probably assume that they will be able to build fast enough they don't need too much of services so those largest deals they come to so-called bare metal so it's still very interesting engineering task to build data center equip it with I don't know few hundreds of thousands GPUs like interconnect and make it work eventually like they will probably do it themselves.

So and this is the basic layer that serves only this population of the customers. Then if you think about the next kind of layer we have here probably also we have a lot of AI labs or AI researchdriven companies that still need like val like scalable compute but they would wouldn't go to the bare metal if they could. They would prefer to work with the managed infrastructure when they don't need to deal with all the hardware issues and so on and they would benefit from us being kind of providing them managed managed services and this is our cloud business multi-tenant cloud business. and we think that it's difficult to serve hundreds of thousands of customers on this layer if you don't have real kind of software platform robust services and so on. But then what we noticed is that a lot of people build the products when they even don't think in terms of GPUs or GPU hours or infrastructure and that's how we came to the managed inference platform.

You have the product called neighbor stock token factory which a lot of people know fireworks based and this is kind of this product from this category when you can come and build and consume in terms of tokens you don't need to choose the best GPU to run this particular model you can switch from model to model and we will take care of optimization for you and so on so forth so it's like managed service and probably that's also not the end probably the next is when people will even not think in terms of tokens but think in terms of agents outcomes and now everybody are the best specialists in the world of what what is the best quen minimax or entropic of version X for some workload but eventually we'll end up in the world where a lot of developers will not think in terms of kind of types of the models and types of the tokens and how much tokens needed to execute this task.

they will just have something working and they will need the the the environment when they can deploy it and run like at scale and reliable and so this is the logic how we view. So at each layer eventually we sell compute. So in the simple words each customer comes for the scaled compute bare metal and pay for megawatts cloud pay for GPU hours tokens token factory pays for tokens agentic runs will run for something that we even don't know the the name yet but eventually it will be converted in the compute but the way how different customers consume is different and I think what we do differently from many people in this new AI infrastructure world is that we try to follow these different customer profiles and different builders to kind of introduce the next layers of abstraction to simplify the way to consume. So this is how we think about the product in general.

So eventually as Vanessa just told us tokens are going down in price dramatically and I think she like the report says it's 280 times since last year. Does that affect your margins or the way you see the business or it just like makes you build higher?

You mean do we affect it by the the tokens become cheaper?

Yeah.

I I thought we closed this topic when Deepseek was released and like at one week market went down 40% but then everybody understood that a good deep

it was a good Yeah. But no, talking seriously, every time we see the new more efficient, way to produce kind of intelligence, if you want, new cheaper, faster, better tokens. We see that just it unlocks the new the new growth and the new way like the new the new tasks that we can solve. And no like the cheaper the cheaper the better the cheaper we consume the the more we consume and not we builders and users eventually it feels like we still so far from solve most of the tasks the and feels like the limitations to solve a lot of the tasks are sitting both in how much intelligence we can get for the given budget and like some plumbing part of the work that we do like can we build applications and projects like products really kind of production grade and reliable

co-founder Oratti said that there are four C's needed to run your kind of businesses capacity capital chips and customers

product

and product so yeah so so four C's and a and a P okay it's it's it's good it's not just two C's and a but which ones are now a bottleneck?

Yeah, we we we were laughing before the section. I I said that I'm happy not to be on the supply side of the business. actually the team that is building data centers and delivering like capacity, they live in hell because it's it's all the complications of the physical world. And like people ask sometimes what is the like main bottleneck in supply? Is it chips? Is it like energy? Is it I don't know memory? What is it? And being not professional, I'm asking that it's not none of that. it's actually the the the the real world. So when you need to build like gigawatt factory, gigawatt AI factory in the real world in a in the middle of nowhere like from the ground up and equip it with the very expens like one gigawatt compute is like 40 billion capex I think something like that. So you need to deploy it there and it's so many kind of project problems complex project problems in one data center before we launched it we it was in Israel so it's a maybe like outlier but we had war we had fire we have water fall and like everything in 6 months and people people talking about like oh that project was delayed two months or 3 months and so on.

But in reality, it's such a compressed work on so many real physical kind of limitations. As an anecdote, in one of data centers, our team decided to build cluster and walls in parallel. So they had like they literally didn't when the when the building will be ready and they deployed they deployed kind of the the the the they started assembling the hardware on the floor and even before the the walls were constructed. So it's just you you try to cut every corner and like to accelerate it but you deal with all the physical world. So this is the the real challenge. The rest is actually easy and demand now is obviously easiest part. in again what difficult in our game is also capital because we are small company like we we're competing with the biggest with the companies with the biggest balance sheets in the world.

And this year we we will deploy like 2025 billion dollars in in in bu in in what we build. And for comparants and it's not easy for us for comparants our like hyperscalers deploy like 150 180 billion the challenge for us next year how we can grow from 20 25 billion financing to 50 billion or 60 billion and so this is also very kind of creative work as you can imagine supply for everyone and capital for us to move faster that we would want. But if you zoom out and take a look at the market roughly how much more demand there is now than there's supply like is is it like a 2:1 I

1.5 to one

the the fair answer I don't know I can quote what we said publicly on in in our case in in in a in a recent earning calls we said that we have four customers for each GPU that we kind of deploy. so we have like few times more demand that like actually we can build in time. I think in general in the market we are pretty much short and you can think about it like you have entropic that I don't know what margin they do but looks like they have a healthy like gross margin right and they told it publicly that the only limitation they have is like how much kind of commit to compute right so if the market would be more flexible in like not not forcing people to the long-term reservations and so on that like imply a lot of financial risk to the buyer, we would grow much faster.

But this this comes to the financing because like we cannot finance too much and take all the risk on the supply side.

That makes sense. Do you think compute ever we we had a really interesting talk just offstage. Do you think compute ever is going to become a commodity that is like tradable like oil? I think there is two complications with that. One is in reality computer is not real commodity because the quality of the quality of what you can buy is quite different and it's really difficult to to do the right kind of discounts you know for the quality but what is maybe and it's very speculative thing I'm far from being professional in like commodity markets but what we see now in the market is it's very much dependent on few big buyers and their appetite to to buy and I I I'm just not sure that you can build really kind of predictable market when the price on the market is very much can be moved by few big kind of players like literally like we have five seven players that can push the price significantly for entire market because of their demand.

And geographically is is is the word kind of already split into like Chinese companies with Chinese compute and then the western world with western compute or is there some middle ground?

We don't work with China. A lot of Chinese companies buying compute in like outside of China. So it's not fully divided world I would say geographically. It's also difficult to say that it's fully divided world when most of the products today are built on Chinese open source models. Right? So it's not really divided yet. You can divide the workloads in the world in three categories from being constrained by location. You have large training jobs like big labs training that is regulatory like more or less regulatory constrained to be in mainland US.

Mhm.

And there is a huge demand for the big clusters in US. Then you have inference workloads that partially also location constraint because of latency but even more often because of data. And then the third you have rest of training and rest of inference that is actually pretty global and we see that like we we serve customers from very from not obvious locations just because compute is available there. So it's a mix. So there there is a big part of demand that is location constraint but also a big portion that you can build wherever you can get electricity and infrastructure and and you will not be like latency constraint or data constraint.

Thank you. When we met two years ago on our first rabbit hole I I remember you were like planning things in hundreds of millions. now it's in tens of billions. the company has grown a lot. What is the biggest learning for you? Like what what what do you know now that you would love?

If you don't sleep, you die.

other than that, no for for me the most interesting thing and like the challenging thing we have we are fairly distributed company and multinational company. Our initial team was was very much like Russian speakers biased. Now we have I don't know the last report was I I I I forget like 60 nations or whatever and we have few very large hubs like with a time difference of 10 hours different kind of subcultures like our GTM team is mostly here in like is leadership is here in US and a lot of big part here in US and this is one culture we have like Russian engineers, original Yandex engineers and like the culture around them. We have Israeli finance gurus who finance all this party and this is something else. how this all work together given that like in two years we grew like many times both in people and like whatever scale and we barely have time to meet together and like speak together.

So this is something that like I didn't understand how should work at all. It's insane to be honest. I mean you if someone if two years ago you would ask me like what I think we will be in two years it it wouldn't be this it would be this conference but wouldn't be the rest and and I think that it's a lot about market it's it's much more about market than us like it wouldn't exist if the market wouldn't be so crazy but it's also about few very important decisions that we made this building this full stack product. focusing on diversified customer portfolio, not just going the easier way of serving like handful of super large customers, betting on software like betting on software engineering a lot. Those were like few decisions that were initially made and they defined a lot of things that happened after that.

you built a very practical company and very riskaverse company that does not like try to do breakneck things. Is there anything that still keeps you awake at night

other than jet lag and

no everything it's a very executiondriven business like everything we discuss like it might disappear like if the execution will not be in place on all the layers supply customer success product like we launch new products they don't work and like you you need to go all the cycle from the very beginning and It's it's just the it's just the execution. I think I just like maybe few months ago stopped to panically check all the Slack channels with the customers every morning because there are some bots now that check it. and also because we like have this kind of more sustainable system with people and so on. It's the execution and again I'm even in a better place than like for example folks that work on supply and they have all these fires and what's not

and wars and

and wars and truck was told with the computers or like anything.

Oh my god this this is insane man. Thank you so much for joining us and thank you. I I think we might we can we can maybe take one question from the audience like anyone is eager to ask Roman about the future of compute. Are

are you guys using any c custom accelerators like the the TPUs or the XPUs from Broadcom or any of the like Marvel the world?

we in production we have Nvidia only. and I can explain like why in R&D we have everything and we work with like I think we work with every vendor like not like there are a lot of start new startups but even from startups we work with with few we test everything we try to kind of be aware what's going on but we look at this very practically as Nick said like we we track track the bottlenecks And now bottlenecks are in like data like data center capacity first of all. If you would have more data center capacity, we would grow faster. And then if you if your limitations is data center, you need to decide like what to put there to maximize your business. And in reality we don't have any single reason just yet maybe to put other like something other than Nvidia because everything that we can build on a physical level we can sell with Nvidia with a healthy margins but also we don't need to build like multi- vendor operations and we can partner with Nvidia on the like very closely and so on so forth.

So at the moment when we'll come to the place when either customers will knock the door and every thought customer will ask us something else or we will be able to build and finance more than we can supply with Nvidia that will be a very practical kind of question should we diversify but it's not we are not yet there maybe just because we're small

big pleasure hearing your understanding of how you guys building your business layer after layer I very much enjoy your openness do I understand you correctly that your point is like what you do for Microsoft and for matter in the short term is a fantastic deal for you because you have a huge scale you can grow fastly but you are trying to be as smart as possible by understanding that once that capacity might not be needed by them because they might build their own you will already put more more layers on top of that and instead of trying to run after all things at once you do it methodically. layer after layer because I remember our first conversations with you. I tried to pitch your Gentic AI stuff a couple years ago and now I see how methodically you are and do it correctly. That's how you do things.

Yeah. Yes. So,

no there are so many good points there. So, first of all, yes, those deals actually are very important for us because they help us to grow and finance the rest of the business. yes, we don't want to stuck there and we want to like have more diversified portfolio not only because we're curious but also because actually it's much better business to serve many customers rather than few and yes we have I call it the like the advantage of second like second mover advantage we are in a quite good place to see what's going on on the layers above and actually like we're trying to be in right time where we need to be. We are slow though we we need to move faster and like much faster but we don't think we need to build too much things. We try to focus on part of the stack that is benefiting from this vertical integration of compute.

So every single service that that we bring we should answer the question why we will do it better than others and mostly we answer through the qu like through the vertical integration that we will do like better total cost of ownership better performance better scalability whatever we don't need to build all the tools we we better partner with people we don't need to build like all the I don't know we don't need to build our own link chain to help people build agents But when people build agents and they bring them in production and they need to run on scale, that's where we can create the value because we can do like the runtime that works kind of more efficient. And so we try to kind of be very focused on what we can do better than like people who are not sitting on vertically integrated infrastructure.

Okay, no worries. Thank you so much, Roman.