Future of AI Software Development — GitHub, OpenAI, LangChain & Zencoder | AI Rabbit Hole 2025

All right, hope everyone had a good lunch. excited to be back on the stage. earlier today I had a great conversation with Eric about cloud code and now I'm super thrilled to dive even deeper into how AI is shaping the next generation of software development with this amazing panel. initially we planned to do this panel the safe way carefully going through our grid upon topics but sitting here today I noticed something is that most of our speakers avoided making future predictions as a wise and responsible person would do given how fast things change but here's the thing our panel is literally called the future of software development so if someone needs to make bold predictions is going to be this panel even at the risk of looking maybe funny or silly in a few month. so here's how we're going to kick things off. so each panelist will give a quick self- introduction and immediately giving your spicy or contrarian prediction about the future of software development.

And here's the twist. We're going to involve all of you our audience. You will be our contrarian meter. So after each prediction, we're going to do a show of hands. If you agree, raise your hand. If no hands, that's P contrarian. Exactly the fun we're aiming for. All right. So ready, let's dive in. Let's start. And hey everybody, u Andrew Filev here. I'm founder CEO of company called Zencoder. we built awesome AI agents. Prior to this, I built company called Wrike, sold it for two and a quarter billion. I'm generally very reasonable guy. So I don't know if my predictions are going to be contrarian for for any of you, but sometimes I like to take the first principle thinking to its extreme. So, for me, I think less contrarian prediction. I think there there's going to be a 10x more software written. I I personally being in this day-to-day, I do not see this as contrarian.

So, but it's kind of funny to follow that trail down the rabbit hole. what this means is actually that 10x more software is going to be ran somewhere and that's going to be ran by hyperscalers. So, house always wins. So, that's my spicy prediction. not contrarian but but spicy. We're we're going to engineers are going to be 10 times more productive. We're going to create 10 times more software. Software agents and Frontier Labs are going to help you do that. But but the funny thing is that hyperscalers are going to take their cut from all of that. Thank you. Thank you, Andrew. So 10x more software is going to be written. Raise your hand if you agree. Okay. Not a contrarian but insightful. All right, Idan. hey everyone. I'm Fouad. I'm on the agency research team at OpenAI and let's see my contrarian take. I think the future of software development will look more like software management where you'll have many agents running in parallel and and where today we're usually triggering these agents and we're having to put more in than we're necessarily sometimes getting out.

I think we'll find that we will be struggling to keep up with the agents and the pace at which they're working. So suppose from development to management. Cool. Raise your hand if you agree. Okay, more contrarian. Awesome. hi, I'm Nunu, founding engineer at Lang Chain. my take, I don't know how spicy this is, but I think, going to be one of the main skills to be a really good software engineer. because prompt engineering is really nothing more than specifying the problem well and as I'm sure we all know that's you know big part of writing good software is knowing what you're writing it about. All right, let's do a quick show of show of hands. Think even more contrarian. Great. Now our last speaker I I have to be more contrarian. Okay. Okay. Hi, my name is Idan. and I lead GitHub Next, which is, GitHub's long bets team.

and we're the people that created original Copilot before Microsoft turned it into a whole brand. GitHub C-Pilot, the one that wrote software. as well as all sort of these follow-up things. My contrarian take is that where the next 40 years of the tool that people use to author software is not going to look like the thing that we're currently using. Like where we are today, it's like when we invented the graphical display and we were using Vim in a window and now we're doing the same thing, but we're like slapping a chat sidebar on the side of it. And I don't think that's what the next 40 years of software development looks like. I think we need new UX's to make all that parallelism happen, right? So, and all that prompting easier. So, that's my contrar contrarian tape. All right. New form factor in four years.

Raise your hand. Yes. Fewest hands. Yes. Awesome. Amazing. Well, thank you so much for sharing all those brave and spicy predictions. I think we got the most construm from Idan. So, I would love to start our discussion there. Idan, can you elaborate a bit more on what you mean by what's what do you see as the form new form factor in the next four years and then we can get some reactions from other other panelists? I'll be honest up front. I don't know. This is exactly sort of the area of research right now. we were just talking in the green room about how, we say generative AI and people think that means that it's about generating code because that's what we've been using this magic box to do for the last few years, right? It's, you know, I type in a prompt or I just wait there and then it suggests some code for me.

So, it's doing some typing on my behalf. But as a developer, typing is not where I spend my time. And all the research on software development says that most of the time is spent understanding a codebase. Where is this stuff in the codebase? How does this code work? If I need to change it, what do I need to touch in order to change it? Right? So, I think these interfaces are going to have to help us less with the typing, this magical like the piano that plays itself kind of feeling, and more with the understanding, the reasoning, these tasks that are actually the bulk of what I do as a software developer. So, from typing to vibing, full stack vibing. Yeah, I think to build on that I think we'll see this evolution and even when you are using a lot of these agents often it's less even about to some degree even the prompting as much as it is the QA that you end up having to do at the end of the process making sure that expectations are meeting reality and I think what we'll see is a lot of and part of what we're focusing on on our team is how do we bring a lot of the things that we know about in training into test time kind of production inference and then also very quickly iterate and bring what we're seeing from how people are using these agents.

Often it's you know some combination of the model might generate some new web page and you need to go click around to make sure it works. Ideally the model could do that as well. So using computer use to kind of complete that loop. and I think taking a lot of those kind of QA workflows where you will still always have to just keep increasing your bar of what satisfactory completion looks like, but bringing more of those into the actual development workflow where previously your IDE would just stop at just editing the code or the agents stop at editing the code. Now they actually need a build test run. one thing I would add to build on that is obviously these models write in the same period of time. they write a bunch more code than we could. so I think kind of a downstream consequence of that is there will be a much more code for someone to review whether that's another model but very likely at least part of that is going to be down to you know the the human that prompted the model to write that code in the first place will probably want to take a look and make sure it's doing what it should do.

and I don't think the current, you know, IDs are really designed for code review. They're designed for code editing. I don't know what it looks like, but that that's probably something that will be part of it. Amazing. I' I've got a little bit of a contrarian view. so so one, I I generally think that long-term predictions with exponentials do not make a lot of sense, right? So prefer to focus on kind of a short term, right? And in the short term I think as humans u there's what you you would know better is like 20 million professional software engineers right now right and we've been kind of growing that number steadily over the last what four decades so so there's a lot of biological agents figuring out what's the best way to do things and how to best scale them from their you know old days of IBM and whatnot and so I feel whatever stack we have today in terms of tools processes should work to some degree and so in the near future I don't necessarily think we got to reinvent the wheel.

I think in general AI is not necessarily trying to reinvent thinking it's trying to mimic the data that's fed to it by by humans right and so from that it's logical to envision that we don't necessarily have to reinvent the software development life cycle the software tooling we just have to support it better and help people be more productive and so in that sense one of my sort of practical predictions is I think if you remember some some of you remember kind of that agile process tenants and kind of agile revolution how moved from very silo functional teams into cross functional team where you have a team that can take something from A to Z. You got the product owner there, you got engineers there, testers there, like everybody in one unit. And I think we might get to the point where there's like crossunctional individuals if you will, right?

So, so instead of like now you you have to have a team with PO and engineer and QA and whatnot, you can have an individual which enabled by AI can actually do a lot of those things. And the reason why in the first place scrum moved from this very silent organization to cross functional team is because by bringing things together you remove the communication overhead and to your point you know right to today there's a lot of time spent in software teams on that overhead not necessarily on coding. So if you can collapse it to one person who is enabled with this AI agents to do their work so much better faster and across different areas then not only you're doing that but you're cutting a lot of that bureaucracy and communication overhead. So I think part of the future of software development in the near term is kind of that those cross functional individuals who are generally professional across a lot of different areas AI enables them to be much more so and then maybe they they kind of in certain areas they're super good and again they're since they're still working in some company with a team of other people if they're not so good in any other area they can pat somebody on the shoulder and kind of ask them to help them out 100%.

And I think in the last generation we are seeing you know PM as engineers are two separate functions and they have pretty clear boundary but recently I think we're seeing the rise of product engineers whereas PM have more engineering capability using web coding tool and also engineers has better sense of product and then contributing to product design part. so definitely resonate with that. I think one point Nuno and Fad mentioned is ides are not doing a lot of things for us just yet right for example like code reviews deployment and also like QA testing and we're trying to leverage computer agent I would love to dive into what's the re what's still stubbornly hard right what's the remaining hot part of software engineering that AI is not help solving us just yet and then what are opportunities there what one thing that I think is hard is especially if you think of a large codebase that you've been working on for months or years.

there's going to be like a lot of knowledge about that codebase that is in your head. and that when you ask some pointed question like a you know one sentence to the LLM model to write out something for you when you ask that question internally you have all that context that's in your head but obviously the model doesn't have that context because that you know all the all the things you know about that codebase the files you learn to ignore because they're like deprecated things that shouldn't be there anymore or all the little mistakes you know that you along the way, but you didn't remove. All that stuff is implicit to you, but it's not ma being made available to the models that are working on it. So, I don't know exactly what that looks like, but I think this is probably like the, you know, last final task that's going to take the longest to to port over to model assisted.

I mean I' I'd ask is that not just you know a form of context selection you know like I'm I mean like you know it's not there it's not yeah it doesn't exist right now I mean it's not just that context selection needs to happen it's how do we clue the models into where they can find things and you know in the database world we talk about indexes I'm trading eager compute and storage in return for the ability to produce a result to a query more quickly at runtime time and so what kind of indexes do we need to be building on top of code bases in order for us to deliver these kinds of experiences that we want to do without the human sitting there being like okay now another spinner for five minutes you know whatever which sucks I hate that so yeah I think there's definitely this I think there's two sides of this one is the tribal knowledge that you're describing where just by talking you can ask someone and get a response in a minute that would otherwise take you you know maybe like an hour or two to read the code I think And then there are, you know, areas of memory and, you know, context stuffing that are very important in selection so you don't kind of overpop populate the window.

I think there's another side of it that's been pretty interesting to see with our latest reasoning models that actually genuinely surprised me when we started initially using 03 internally. and you kind of see it with both 04 mini and 03 that you didn't really see with 03 mini, which is pretty interesting that it's able to actually pretty intelligently decide what is relevant and use because we're including more trading tasks that actually teach it that that's actually an important thing to consider, not just go off and, you know, make the change as as instructed, but to actually understand. Now, there's a trade-off there similar to what you're describing where it can sometimes spend more tokens learning about the context before it makes changes. but some of the issues that we saw previously where it would just do exactly as you asked, without a second thought and that typically isn't exactly what you want.

I want to be able to ask an ambiguous question and have it disambiguate and kind of get a little bit deeper, based on the codebase. But it's actually been pretty surprising even with agents like we released Codeex CLI last week. having a model that has just even access to a terminal that can run code, you know, run run commands to search your codebase, it can actually very quickly understand, oh, I probably don't need to be looking in the node modules folder or maybe I should go look at the git history. really giving the model that kind of longer leash. And I think the longer those leashes become and the more indistribution the training tasks are, I think we'll see models that can overcome some of those kind of gaps around tribal knowledge. Well, one metaphor that I I use in my own head, I don't know if I can communicate it well.

It's there there somebody said about cultures, right? So cultures in the companies, it's not what sort of makes them look like each other, what makes them different, right? And so so it's like the quirky parts of your company culture define who who the company is. And so I I feel that applies to kind of the role of human in this future where the role of AI is to be kind of the common denominator, the common wisdom, right? That the principles that would work for everybody, their techniques that were and and then the role of human is actually to to bring that sort of quirkiness, that specialty that actually makes it stand out and win in whatever competitive task it's used to because economy is a competitive environment, right? where my company competes with yours or somebody else's. And so again, AI will help level up the playing field for everybody, but then ultimately one of those companies got to win, right?

And that's going to be what what human brings to to the table with their own quirkiness and out of distribution thinking that and again like startup worlds, you know, 10,000 will fail and one one will succeed. So you know a lot of those companies might fail but then once in a while that quirkiness kind of leads to that evolutionary progress that that creates better better product better company and whatnot. Yeah, I think that quality of to frame it as a taste where it's really the distinction between you know a given company given members within a team that you see it in coding style in terms of how you would prefer to kind of name your functions how you prefer to organize them but then also in terms of the final output the taste of an organization where you I think there's two versions that can can manifest one is where a lot of software generated by AI looks very similar to each other I think that's a less exciting world than one where we're able to bring back some of the things we saw in the early 90s where people were just trying things.

They're, you know, really kind of kind of accentuating that kind of quirkiness or that kind of unique taste. And I'm really excited and you know, we really try to push agents in that direction where people to encode and identify and understand the taste of both an existing codebase and kind of amplify it but then also show different paths. And I think that's one of the maybe going back one step to this question around the IDE and kind of what needs to happen in the software development life cycle and what's different. I think if agents are able to generate you know many different attempts for a given issue I think typically we will have one prompt and then one output. I think what we'll expect to see is you know you define something like a task maybe it's in your task tracker or some other system and you have many agents at different levels of you know competency many different tastes that they're all attempting the same task and then you get to pick either through you know your team's taste function or having your users actually decide through AB tests as people do typically today with humanuilt features.

I think that's definitely a direction where I'm I'm optimistic that we can actually see this almost a Cambrian explosion of people's tastes really coming through in their software. Do you think there's space for another agent to make that choice? Yo, dog, I heard you like agents. How deep does the how deep does the rabbit hole goes? As long as you're using 03. But no, I think that there's definitely a space for agents to review. I and the way we have currently thought about it so far is these kind of code executing agents where the main thing we want to make sure that they understand is how to actually test their work so that they're not producing you know buggy code you know code that can't be merged. So step one is just completeness and correctness. and then the kind of next frontier beyond that that we're thinking very actively about in our reasoning models beyond even just coding is how can they actually understand the taste of both you know some inherent taste of the model which I think gets more to Andrew's point around this kind of homogenization and kind of raising the floor in some ways of we want the default output to to look good and to be usable to be accessible all the things that we like in good software but then also make sure that I can give people options and I think there's a bunch of coding tools that have been amazing for people to try many different takes all in parallel and then pick the one that feels best for their use case.

Yeah, that the taste really reminds me a recent story with my front end engineer because personally I'm a machine mostly writing machine learning and backend code and then with the help of all the amazing models and products now I can contribute to my front end code base and my engineers hated it. basically they told me stop please stop contributing you know to our codebase you are now generating you know code that really adheres to the best practices. so I was wondering you know how would you say you know how can we better leverage you know wipe coding or AI code that can really you know be meaningful contribution to a very large you know mature production level codebase. what are some best practices that can make that happen? I can maybe do an initial step. I think part of it is making sure that the base models are representative of this starts all the way down to pre-training and making sure that we're starting with the best possible foundation.

then I think at every level of our stack and as we're you know scrutinizing kind of what goes into our training runs, making sure that we are able to both respect kind of user preferences. so it's not just homogenizing, but then also making sure that we're teaching it how to work in these very complex large code bases. But then the rest of it, I think, is in the software harness and how are you prompting it correctly. But either of you going to say something I I can elaborate on the next layer. So there's their model layer, right? And then there's kind of what application layer on top. So one context is the king. and luckily their new generation of models also pick the style better from the context. so you have to do less kind of wrangling or prompting to to to force that they start picking up on clues and comply with their with that better.

But I say kind of again now now I'll go a little bit from their super short term to still short term but like six months ahead whatever 12 months ahead at some point you will start stop giving a crack about the style. right so what you will care more about just verification that you know you don't have new cyber security issues introduced right that they're that it's scalable performant that you you don't break anything in regression so because again one when it's 90% human 10% AI yes style matters when it's 10% human 90% AI and you're moving at 10x pace you you don't care about style you care about the functional requirement by functional I mean broadly functional right require requirements. Does the patch meet it or does it not? And for that you need regression testing which again going back to my previous point that software industry have worked for 40 years to develop the best practices.

Some of our customers have 30,000 regression tests, right? So like it's a crime not to use that to to verify the work that AI does. And as long as it meets all the criteria, it looks good, it feels good, it doesn't have any security issues, and it complies with the regression test and new end to end test, why do you care what style is it written? So, so I think that's kind of mental transition that at some point will happen where people will stop caring about their details of the style and just hey, if it works, it works. I don't care. so what is vibe coding for, right? Like we we talk about it now. We're all excited about this new capability that we didn't have two, three years ago, right? Of like, you know, I just I just I just vibe with it, man.

I hit tab and everything's fine. but the reality is is that it it doesn't adhere as people experience to the semantic style of the codebase. And one day the models will all get bigger. They'll all have 10 million token windows and we'll be able to shove the entire repository into the window. And maybe it'll take longer, but you know, it will adhere to the thing. But even then, you know, like what's actually stopping me from shipping software? Most of the time it's not something about the writing of the software. it is I need to go persuade an executive that this is the right thing to make and in that context vibe coding is a fantastic tool because I don't need to make something for the ages I need to build something I can show somebody else like you know people talk about sketching as an important tool for communicating ideas not because it's art or it's beautiful drawing it's because it enables me to get an idea out of my head and into yours Not in a philosophical way, but like here use it.

What do you think? Right? And then if we like it, then we can do the expensive step of actually writing it for real. But the hard part is the earlier phase of like we could do this 18 ways. Which one is the right one? And for that purpose, vibe coding fantastic. So it doesn't need to be one or the other. It can be like I'm iterating here on a branch and don't worry, this is not going to stay in production. This is for the purpose of showing you what's in my head. and maybe one day it grows up into actually being able to produce production ready code, but I have a hard time estimating how far away that is. Maybe you have a better sense for that. I mean, on the production ready side, I think it really comes down to the prompting. I think maybe this is my most controversial opinion is I don't really like the term vibe coding.

And it's not because I don't think it makes sense as a premise. I think especially for nontechnical audiences being able to you know just kind of communicate their almost like stream of consciousness ideas to it and have that in code. But maybe part of it's because I I I work in it all day and I take it so seriously that it feels almost like it trivializes the kind of the kind of level of technical requirements to get it really good. And I think that when you are prompting these models or how you actually so just briefly on like the context window item I actually find that cutting down and kind of clearing the context window to be the most effective whether I'm using chatbt as a consumer or I'm using one of these coding agents is actually less context is better because it's less likely to confuse the model for irrelevant content and that actually the kind of self-distillation of having it first explain the codebase read it and then write its own kind of description of what actually matters or having it evaluate what is the style guide of this codebase as it understands it and I think that's where and we do plan on kind of releasing more of these kinds of techniques of how we found it to be successful.

but I think there that is kind of in my mind the distinction is it's still a code executing agent that you're working with there is still a vibing component to it. but I think the amount of workflow steps just like in any software development life cycle where you really treat it as a professional production ready process. we use codeex extensively internally and there was a lot of things we had to do as you can imagine to get it right before we could even ship it externally let alone really kind of maximize it across the organization. But yeah great we have covered some deep ground and before we wrap up I want to do some quick rounds of u rapid fire questions. all right let's first one is one skill every developer should master next year QA QA context management context management just use AI like just use it like you know everybody's just like how do I get the most value out of this and you just use it you you're not going to learn how to ride a bike by reading a book you need to ride the bike until you ride it so I still stick by that one Just use it on all three.

Awesome. Next up, biggest dev hype you are skeptical about right now. That you can vibe code to production right now. I mean, you can v code to an awesome prototype. My my friends do it all the time, but not to production today. Cursor. Okay. Okay. That's my job. any cursor engineers in the crowd. further to this thing of of of like you know the the future of the IDE not looking like the last 40 years of the ID like you know I think it's a great product. VS Code's a great product. Windsurf's a great product. I don't think any of them are what the next 40 years look like. So that's my spicy. I'd say the shift away from the terminal. I think the reason why we've bet so hard on the terminal is that it's an existing interface where users are already used to managing many programs in parallel.

And I think that what we'll see is a kind of resurgence of terminal focused applications that can run whether it's on your local machine or it's on a remote server. But I think bringing back the terminal is my bet. I think I would say multi- aent. I think people make their AI things very complicated without first trying out the simple approaches that are probably going to work better. Awesome. last question. so we have four pioneers building dev tools for developers. What is your most exciting or looking forward to feature they're going to release soon? if you can share. Who wants to go first? I can't share. I I'll go. so we we launched a product last week Codex CLI. we're we're very excited about what it can do for for folks. I think one of the distinctions that we've heard from people is that it's not as useful for the vibe coder audience because it lives within your terminal.

there's definitely improvements we want to make there but I think this focus on professional software developers you can see and I think we actually announced some some amount of these but we'll be adding support for MCPs and a bunch of other features but I think just briefly on MCPS I think seeing more people develop CLIs is the primary way that they provide tools to models is I think a really important thing that people are not accounting for so very excited to support more there. I I'll share a little bit. I think the next three three months we'll see much more done not just on the coding side but on the testing side and how those two work together. I think that's going to be new horizon. Verification is king. Yeah. one feature that we've launched recently that I'm quite excited about is GenUI. which I think the idea of colllocating your UI code with your agent code is is very interesting and can let you make new things.

we just handed off like one of the well two of the large projects which was copilot workspace and spark and so now we're at the top of our cycle and I don't know yet because we're still in sort of like mess around and find out mode but a lot of our energies now are going into this notion of code understanding code navigation and some of the early things that we have cooking in the lab we're not ready to talk about them yet but it's really exciting as a developer to be like where is this I need I need to add a forgot password flow. What files do I need to touch? Where is it? Show me a few different approaches. You know, the fact that that's possible now and it's not just like here's a suggestion, autocomplete ghost text, you know, but like getting out of that and into like a like a pair programmer for thinking.

that's definitely the kind of stuff that we're really excited about. So, all right, with that, thank you very much to our four amazing panelists for putting their reputation on the line and making those boat predictions. Thank you very much.