If more than one person is responsible for feeding the cat, the cat starves. My guest today used that line for a question a lot of teams are dodging right now. Who owns AI-generated code once it's running in production? His company surveyed engineering teams. Almost half already run AI-generated code in production. And about a third write it but won't ship it. He says, Code review is only act two.
The bigger bill comes later, when somebody has to understand code that nobody really wrote. And the leaders who come out ahead won't be the no squad. They'll be the yes squad, with guardrails. I'm Brian Teller from Teller's Tech, and this is Ship It Weekly. Welcome back to Ship It Weekly, where I filter the noise and focus on what matters when you're the one running infrastructure and owning reliability.
Most weeks, it's a quick DevOps, SRE, platform, cloud, and security news recap. In between those, I do conversation episodes with people building and operating these systems. Ship It Weekly turns one this Friday. To celebrate through mid-December, you are getting two conversations a week, Mondays and Wednesdays, with the news every Friday. Then... We're back to our usual rhythm.
Today, I'm joined by Ted Julian, founder and CEO of Flux, an engineering intelligence platform. Ted has been building security and infrastructure companies for more than 20 years, including Resilient Systems, which IBM acquired, Arbor Networks, and @stake. Earlier this year, Flux put out a report on the gap between AI-generated code getting written and teams actually trusting what it changes.
We talk about why so much AI -generated code gets written but never shipped, why code review is the first thing to break, a simple way to measure what AI is actually doing for your team, who should own AI-generated code in production, and why he thinks this is the best career opportunity engineers will ever get. Alright, let's jump in. Ted, thank you for joining me. It's a pleasure. Thanks for the invitation.
So I've read the report. I'm interested. Give me your thesis. AI-generated code is clearly being used now, but what are teams still not ready for? I think the surprise to me was to see the the gap between the code generation and the code actually in production. Okay.
And then what the survey users did tell us, like, it's like every survey, you kind of want to do it again, because you get answers to questions, and then you want to know, oh, wow, it'd be great to know more about that. But anyway, they did give us some examples of like, what was holding them back in terms of security concerns and other things that I'm sure we'll talk about.
Okay, so high level, what are some concerns that you've seen? For sure, it's There's a lack of trust, especially in more enterprise class, more regulated environments, about taking this generated code that maybe was for a prototype and it went super quick and very excited, legitimately excited about that.
But when you pause to think about putting it in production, if it's like a HIPAA environment or if it's a highly sensitive, like every minute costs us some huge amount of dollars environment around the reliability of that code broadly defined and then they did enumerate some specific concerns whether it was complexity that might be spiraling out of control and could lead to an outage or the introduction of a new
Security vulnerability based on a library that was put in you know a pull request to to sort of generate this prototype that we're now going to make real um or other concerns around regulated data?
And has this code really been scrutinized sufficiently for how it's handling this data that looked great in the prototype? But again, we're a regulated healthcare company, and now this is winding up in production. Yeah, for sure. Your report says 44.7% have AI-generated code in production, and 35% are writing it but not shipping it. What does that tell you?
This is an example where I'd love to do the survey, to like really double click on that, because I just talked about some of the hesitations, but I sure would love to know more about them. And there's some things we didn't ask that I suspect honestly could be behind that too.
So for example, if you think about the time in which the survey was done, there were probably a lot of people that were still in the early era of experimentation, right? So it literally could have been like some throwaway something, but you know what? It was a chance for us to, Try Claude because we haven't done that yet. And, you know, it was just a little prototype. Maybe you turn into something, maybe not.
So honestly, like just things like that, just tests, proof of concepts could easily explain some percentage of that. But we do know based on what they did tell us that it that is was not all that was going on here. You know, there were those other concerns that we alluded to that were the things that were giving them pause.
And then also just highlighted a general visibility need, I think it's very safe to say, around what's happening in our teams as they're embracing these tools that would be required to sort of close that gap and increase that percentage. Yeah, that's for sure. So do you think there's like a code quality problem? Or do you think it's more like operational model problem? There is a code quality problem.
I mean, we could use something like the Amazon outage from it was probably six months, eight months ago now. Right. And that was kind of a shot heard around the world because that was the one where there's the memo that leaked that basically said you're going to sign off on all PRs, which I don't know how that's going to work in practice with the team that big, but they're smarter than me.
But anyway, like we're joking about it, but an outage at a place like Amazon is a really big deal. And for somebody that is so focused on moving fast like Amazon is to put that kind of countermeasure in place goes to show you that this is not theoretical, that it is happening, but it's early.
I think in some respects you could say that's one of the early canaries in the coal mine, and it's probably a function of leaning in hard and leaning in early. But it's a cautionary tale for the rest of us that the models are the worst they will ever be. In other words, they're going to get better. And so maybe this problem isn't quite as bad for those that are coming along behind it.
But that the more you lean in and the more that you are regulated, care about... Five nines and so on, then the more that the smart leaders are going to be putting guardrails, looking to get that visibility to avoid those kinds of things happening to them. Yeah. For me, I think of the review side of things. So if AI makes code faster to produce, like what breaks first in the review and release process, right? 100%.
We see this at all our customers and we're starting to build kind of a journey map for organizations as they go through their AI transformation. And without a doubt, that's kind of act two. And it's like any good play. Act one, you're introducing characters and there's all sorts of promise and things start to look really interesting. But act two, that's when some things happen.
There's a villain or there's a tragedy or something like that because it makes it a great story. It'd be boring otherwise. But I'd say this is maybe the beginning of act two where, yeah, code review becomes... A real problem. And what I think is interesting about that is it gets into the people dynamics, right? Where this isn't just a technical bottleneck.
And so we're going to throw more computer storage or something at it. And I think in this case, it's also not as easy as just using AI to do code review, which is probably a very, very good idea. But if you use Amazon as an example, whether or not that's the right conclusion, I think there still has to be humans in the loop when we're talking about really sensitive environments.
So it isn't a simple, it isn't just More tech, different tech. And so now there's a different way of... It's more important to look at the team dynamics and measure those team dynamics because progressive improvement is going to require solving that problem around a people.
And whether that's more people or different people or swapping them around or whatever, until you solve that problem, that continuous improvement will be gated on that. How do we measure the... Efficiency gains of AI? Is it bigger PRs, more frequent PRs? I mean, that's what everyone likes to use as metrics.
There is a once in a career hack available to you now, which is to just do, this is what we see smart customers do. They sort of start to measure, they find a way to measure features versus maintenance versus bug fixes, this type of thing. Flux does this, but there's lots of ways you could do this. You could tag it in Jira. You could tag things in GitHub. You could use some other tool, some way or another.
Try to find a more programmatic way of getting this measurement than... What maybe people have done in the past, which often is for R&D tax credits, and it's sort of at the end of the quarter, and you look at the team, and the leader will be like, yeah, looking at GitHub, I'd say 30 % were features, and so that's what we'll put in the spreadsheet, and that determines R&D tax credits.
Something that is a little bit more real-time to try to get that measurement. Anyway, so you backfill, and you define it as before, right? So as of, let's just say, the beginning of this calendar year. We've backfilled our PRs and we know that we believe that our features as a percentage of all engineering was 10%.
Now, if you're even remotely like leaning into code gen and maybe you're migrating from GitHub Copilot to Claude or you're rolling it out across more teams, I mean, that should grow quite a bit. Because if you simply say that the number of the amount of code that we generate, let's just keep the number simple, goes up by 10x. Well, then probably the features go up by 10x or something like this. Right.
And so now you've got this before and you got this after, which not only shows velocity improving, but it shows features in particular growing by, let's say, 5x. Keep it that simple. I can tell you based on our customers, like that one customer that became a board chart, like their Q1 board meeting. That was a chart in the board meeting. And I know somebody in the meeting and the board was thrilled. They're like.
We were pushing you guys so hard last year to lean in the AI. We thought the team was kind of stagnant. They went from 9% to 43% by the end of last year. So they were like, this is tremendous. So anyway, I don't care how you do it. This is kind of a once-in-a-lifetime career hack. We're not going to have the curve like it is now, right? It's only going to flatten a little over time.
So you're kind of crazy not to avail yourself if you are leaning in the code gen to document this spike in feature delivery to both give the team the credit that they're due for driving this transformation because let's face it, like it's a little scary and it's a new tech. And so people have to feel comfortable about that, what that means for their careers, their jobs, that kind of thing.
So you absolutely want to ground this in data and to give them the well-deserved pat on the back. But at the same time, you'd be crazy not to leverage that with senior management to show like, hey, we're doing it. We're doing exactly what you asked us to do. And look at these results. They're tremendous. Okay, so shifting gears a little bit. Let's say we figured out how our AI efficiency gains are.
We've nailed all that down. We've nailed down the FinOps side of it. How do we deal with the security side? Your report calls out security dependency changes and performance impact as some of the hardest week-to-week changes to detect. Those are exactly the things that become incidents, at least in what I've been through. Why are those so easy to miss?
I mean, we'll start with the less scary example, which is simply you got teams leaning really hard into generated code because there's an enormous amount of pressure and they want to do the right thing. And so they're prototyping stuff and it's going great. And it is so easy to just keep, you know, tab, tab, tab, you know, or accept, accept, accept.
And if you're not careful, like there's a new library and that new library has a vulnerability. Thing works. It doesn't create a functional issue. And that winds up into production. I mean, this is absolutely a very real concern without some guardrails. It's just human nature. So I don't think it's sufficient to say, oh, we're just going to have really experienced developers and they're going to know better.
I mean, of course, you want to have that as part of the best practice. But it's belt and suspenders time. I mean, there have to be some other guardrails on this process or else you're just asking for trouble. Now that all assumes the traditional dev team, you know, but operating in a new way. But you and I know, like, that's not the new reality. The new reality is everybody's coding, right?
Marketing coding, sales is coding, supports coding. And it's actually great, you know, managed properly. So then the last thing you as an engineering team was be the no squad and like shutting all this down. I mean, you want to be the yes squad, but...
You know, if you think it's easy for a developer to miss that, like a marketing person with all the best intentions in the world trying to do the right thing, like forget it. So it's you can't have one without the other, you know, like you can't say we're going to lean really hard into this organization wide. It's not just the engineering team.
We're encouraging everybody to lean into this opportunity to maximize productivity gain and to innovate and to have fun and all those fabulous things while not acknowledging that these are absolutely side effects that come with that. You can't have the one without the other.
And so the smart leaders are the ones that are going to be the, not the no squad, they're going to be the yes squad, but they're going to find ways to put those guardrails in to architect processes and so forth that help protect the organization. Yeah. Where do you think those performance regressions hide? When changes are coming faster?
We talked about code review, where I think we're early is in, it's basically dealing with the tech debt and, and dealing with the complexity. And I think it's only the early adopters that are living this on a day-to-day basis. I think this is a bill that's going to come due for maybe, like, the rest of us over the next year, 18 months, depending on where you are in your journey.
Um, but this is inevitable, and I do think it's, it's going to be, uh, very much an eye-opener, like the code review problem or that bottleneck will look like a small problem than when this Code complexity, tech debt issue starts to rear its head because that will be the first time that we really have to figure the code out, if I'm honest, right? And there's going to be no choice.
It's not going to be just click accept that looks reasonably okay. Somebody's going to have to understand that code in a detailed fashion and nobody really wrote it. It was generated. And God knows, like, it was outsourced and we dropped them. We're on another thing. Yeah, Sally was amazing, but she found a great opportunity. She's not here anymore. So, Joe, good luck. Yeah, and it's not documented anywhere.
No, no, no. And again, the tooling's all going to get better. None of this is cast in stone, right? And so, yeah, there will be AI-driven improvements in these various areas that are probably going to make it better. But I don't think at the end of the day, it's not a smart career decision to say that it'll be fine.
I think, again, the smart leaders are going to be the ones that are putting some guardrails, thinking about process that will avert these problems is the one side of the coin. And I guess the other side of the coin is proactively identifying the bottlenecks and getting out ahead of them so that they continue to go through the various phases of their journey and they're unlocking improvements at every step of the way.
OK, we need to take this seriously, especially as we're moving quicker. Right. It matters even more. Right. Please correct me because I'm coming. I bring a cyber background to this and a cyber point of view. And so what I'm about to say may be wrong, may be offensive. So I'd encourage to be challenged. But this to me is very much a security mindset.
I mean, security for effectively forever has had this dilemma of we're the no squad. Right. You name it. Commercialization of the Internet, mobile, cloud, whatever, every step of the way, security has to walk this tightrope between, you know, managing risk, but also you cannot be the no squad. Like if you just are saying no all the time, like they're going to find somebody else who has a way of saying, totally.
That's all that stuff. Yes. Well put. Right. And I just, I'm just not sure engineering's quite been in this position before to the same level. And maybe to me, this is, It's part of this unprecedented territory we're in right now. I mean, engineering is being upended completely for the first time ever. It's never happened. It happened with sales, like with Salesforce.
It happened with marketing, with HubSpot, where those processes digitized. And that was really painful and shook things up, but it also was an enormous opportunity. And I think that's at a very, very high level. That's kind of where we are with engineering right now. This whole thing is being completely upended. And yes, it's absolutely fabulous, but it's also scary and risky.
And it's like I said, you can't have one without the other. Yeah. Okay, so let's stay, I guess, on that thread a little bit with security and the compliance legal side of it or engineering leadership side. One thing that stood out to me is that this is not just an engineering concern anymore. Security, compliance, CTO, CIO and legal all showing concern. What changed? I mean, we've seen this movie before.
This is what always happens. Again, I kind of just talked about the big transformations, right? Like this happened with the commercialization of the Internet, right? Like, just get the website up. For God's sakes, you know, X, Y, Z competitors are all doing it. We need a website. And everybody in IT or in the security department is like, oh, hold on a second.
Like, you're taking a network that has never been connected to anything outside. You know, it's just been completely internal. And now you're connecting it literally to the entire world. Like, there's some things we need to do to be smart about how we do this. Ditto cloud, ditto mobile.
So I think that's the same thing is that the organizations have leaned so hard into this AI transformation, but eventually these other functions realize that it is their job to force the organization to be smart about these kinds of things.
And so that's why you kind of go through these waves of like this initial elation, but then things cool a little bit as all these other functions weigh in and figure out how to do this in a responsible way. So that's like we've seen this movie before.
And this is maybe, again, the early part of act two, where these teams sort of go from latent concern to tooling up and figuring out like, this is how we believe the organization needs to tackle this, what the procedures are, they discover maybe some tooling to help. So who owns the behavior of AI generated code and production then? Is it the engineers? Is it the... CIO? Is it the AI director? What level?
I mean, I guess everybody does to an extent, but like at a traditional company, who owns that? I don't know that that's been discerned. I can't say that, like, I don't think we can canonically say, like, the universe has voted, 62% say. But, like, it's who feeds the cat, right? If multiple people feed the cat, the cat starves. So it's got to be one person that's responsible for feeding the cat.
And I think it's got to be the head of engineering and ultimately the CIO. I mean, it's the code. Let me ask it a different way then. What does governance look like without crushing velocity? That is the $60 million question. No, I mean, it always is. Yeah. And I don't think there's any easy answer because I think every organization is different.
And it's a function of how regulated they are and how sensitive their industry, their business is, how important reliability is, where they are in their own maturity for how they handle these things. I guess what I would say is this isn't the first rodeo for anybody. And so you just got to have to look back to these past transformations to figure out what your map is for going forward.
It's just it's in a very different domain. And so the controls are different and maybe even not completely understood. But that it's not like if you've it's not like most modern organizations of any size haven't seen this before. They've seen it before. It's just figuring out how to make that muscle work in a new way, you know, where these new exercises or something. I don't know.
OK, let's switch gears to safeguards. So the report says teams are buying code quality analysts automated code review training, which safeguards actually move the needle and which ones can become checkbox theater? What should block a release versus what should create a tracked risk?
I really do think, no, I mean, I think that's an individual, I mean, that determines like how regulated you are or how important uptime is or what have you. And again, I really think that we're still figuring out like gross best practices. Right. Like I think if we just use code review as an example, what we see all the time was organizations are they get really dynamic as they're leading in the code gen. Right.
People who were just front end developers, back end developers, whatever, sort of becoming full stack developers because they kind of can. And so and code review needs are, you know, skyrocketing and seniors are just like jumping into the breach, which in a way is amazing, right? That Sally sort of maybe done this part time, but now there's just so much need for it. And she loves doing it.
And she's super good at it, that that's where she's spending the bulk of her time. And it's a fabulous thing. She's doing great. She's by the way, by virtue of all these code reviews, she's mentoring like all these more juniors and making them better at what they're doing. I honestly have goosebumps just talking with you about this, Brian. This dynamic is incredible to the development of an engineering team.
And people are certainly adding in tech there too, like using CodeRabbit for code reviews, things of this nature, any part of the CI/CD pipeline. But per the Amazon comment earlier, you're never going to take humans out of the loop. I don't think it would be smart. And I also just don't think it would even fly regulatory wise, right?
Like you need to have a trusted independent party like reviewing this stuff and a human should be involved at some point in the process. But I guess what I was getting at is I think that these people dynamics are under discussed and critically important to success. And so much of this is shadow work that has never been in a ticket, really.
I mean, it's the kind of thing that it comes up because you're doing something else related to a pull request and the person just spends some time on it and then the pull request gets done. But what this transformation is unveiling is there's examples of this that are not trivial, like code review.
And if you aren't looking at like developer participation ratios around commits versus code reviews or that kind of thing, then you're going to be blind to a very important dynamic to the success of the engineering team in maximizing this transformation. But just as importantly, like the team dynamics that were underneath that, right?
Like this opportunity for Sally to shine in a way that like she didn't even see coming and she's loving it. And the impact that that's having on the rest of the team. I guess the reason I get goosebumps talking about it is like, this is what any manager just, it's what makes the job awesome. Like truly awesome. It makes you like never want to stop doing it.
Like I'll take that dopamine hit and the team dynamics like all day, every single day. So anyway, that opportunity is there, but you have to be tooled to look for it and to find it. Yeah. It just makes me think like, where will we be in five years even? You know, just five years, which isn't a long time, but is in the current climate. It's happening everywhere.
Like, in our prep, we were talking a little bit about, so we can maybe save this for later, but it's like, what do teams look like in the future? And I was just thinking about how, you know, we're going back to, like, MRDs and PRDs in a way because that's what we've learned is like the next unlock.
Um, and here too, what does that mean for the people you need to have on the team and for what their skill sets are as a leader what does it mean for How you're tracking, you know, people's contributions and how are you sort of measuring them? And based on that, how are you fostering continuous improvement and development of the team? And here again is where it's like unprecedented territory that we're in.
Like these are not things that were part of like just day in, day out blocking and tackling engineering team conversations. They're things you talk about once a quarter, once a year, when you're thinking about your teams and how they're developing. But that is not nearly fast enough right now if you want to be the best of the best.
Yeah, I even see companies iterating over their OKRs continually now, which is interesting. Okay, so I guess let's talk about that. Regarding the report, what did you glean as far as the future of AI? Development? Yeah, I mean, to me, it felt like we're in the first inning, you know, these are just like, honestly, such basic concerns.
That if you want to think about that future, where we're just verbosely prescriptive, and we're in like this continuous loops of development, and so on and so forth, like that ain't possible. Right? Unless we have done a lot of spade work on all of these guardrails and controls, because Here again, you can't have one without the other.
You can't say you love loops and what that does for your velocity without understanding that there's a dark side. It will accelerate just as much the bad as it does the good, so you better have some guardrails around that, or else you'll find yourself worse off. That, I suppose, is maybe the cautionary tale for engineering leaders, is that there's an exacerbating effect here that...
If you lean really hard into this stuff without thinking carefully about the guardrails or about what the processes and controls are, it can spiral out of control really quickly. There's going to be some smoking craters here. The Amazon thing, I think, will seem small. I've read the Medium articles on the Ralph Wiggum loops. I've read Claude's program where they had the internal vending machine that went rogue.
There's just countless examples of people just open claw, even just what people were doing with open claw early on. There's just so many examples of, yes, if you let AI just loose, there are going to be some interesting consequences that you couldn't have even fathomed really before letting that go. So guardrails, obviously, just goes back to that. So important.
It seems like the boring side of the conversation, but like... It's paramount to everything that comes after that. Yeah. I mean, it's an engineering process. So every time you'd make something better, that's going to create a new bottleneck and you're going to have to figure out how to tackle that bottleneck.
I mean, just thinking back on this question, I think it's safe to say that in each of the discrete functions, there will emerge progressively better. AI driven solutions for that thing, whatever it is, code review, generating documentation, finding and fixing bugs, you name it. And so of course, you're going to want to be tracking all of that to try to maximize your productivity.
But every time you do that, there's going to be a change in process and that's going to require considering what the controls are, right? Like does this, now that we've improved the way that we're finding and fixing bugs, does that Does the guardrails we had or the human control that we had, is that still appropriate? Does that need to change? Does that need to be updated?
I think that's the nature of the work that'll need to happen. And I think it does also suggest that there's some sort of a control plane that's going to have to sit above all of this. Because I do think what this will bring to a head is the stovepiped nature of the way a lot of these things happen today, right? A lot of these things like...
In your environment you'd know far better than me like it's probably people looking at like network stuff and application level stuff and cloud level stuff and you're trying to get some signal from all that noise to like figure out what's going on in terms of performance of reliability outages all the rest but that's just as true in all of these other domains And at the end of the day, the engineering leader is going to need to have some visibility across all of this in order to be effective.
If them being effective is going to require them to look discreetly at all these different tools or to manually... Sorry, but this is really very funny. I mean, right now, these are pretty static dashboards is our best attempt at doing this, right? So how the hell are you going to do that when we move into loop world? It's just not going to work.
So you're going to need to have, you know, some triggers or controls and this sort of management plane that in real time is kind of looking at what's happening that ultimately gives that engineering leader like that insight, because it's probably often going to be blended, you know, or that's that would be the ideal scenario, right, where security is a little tricky.
So maybe we'll talk about there's a there's some churn happening in a portion of the code base and that's slowing your velocity for that that team and that part of the application oh and by the way we think that's because there was um A new library with a vulnerability and the team is figuring out like how to solve for this while preserving the functionality of the application or something like this.
That's my best way off the top of my head I can think of to talk about. Like there's multiple things that are involved in understanding that problem. And if you're in loop world where the code is just writing itself and you're trying to like... After it's already into production, try to figure this out with a weekly snapshot or something like that. It's just not going to fly.
We're going to need agentic systems is what I think I'm using a lot of words to say. And those agentic systems are going to have to be multidisciplinary in order to be effective and to support this vision that we're talking about of like kind of autonomous, what are they calling it? Dark factories, you know, just cranking the out. For sure. Okay.
So wrapping up, your conclusion says the code isn't the hard part anymore. Visibility and controls are. What are high performing teams doing differently? It's really interesting. What we're seeing is a lot of people coding their own stuff, actually, to try to solve this problem. And to be honest, I think that's a function of there's a lot of legacy tooling that people have used in the past.
And it's all been based on like derivatives of work, right? You're, you've got to Ticketing system, and that's great.
I'm not saying get rid of that or anything like that, but you have this ticketing system and people are using it a certain way, and so things are being tagged a certain way, and maybe you've got similar things happening in GitHub, and then you're building a dashboard maybe off of all of that to try to get the visibility that you want.
And the problem, of course, is that there's no way that can keep up as we move to loops and as we move to dark factories. Or why would you? You're just going to have the AI working as hard to keep the tickets up to date as it is creating the code. That seems silly. So there's got to be a better way. And this does get to what we do, but forget whether it's Flux or anything else.
We have to move to a code-based approach. And we can move to a code-based approach because we have AI now, and AI understands code. But we need a better mousetrap, I guess is what I'm trying to say. Whether it's Flux or anything else, we need a better mousetrap that's better suited to the pace of development that we've now...
Unleashed in the organization, to provide that, that, to close that gap, to close that visibility gap. And the first step that we're seeing people do because they can, because it's easy to try, is coding their own thing. Um, because they're finding those, like, it's like easier to do that than it is to retool your legacy solution, to be honest, to get the visibility that you desire in this new AI environment.
So I think we'll see where that goes. Like, is that sufficient? Maybe. But this, I think, is the the canonical question for, like, coding your own anything. Um, it's easy to try it, and I'm not saying it can't work, but like, that argument writ large means you're writing all your own software for everything, and that just on its face doesn't make any sense.
So people are thinking we'll have to choose, like, where that's good enough and where, you know what, like, this is important enough, there's enough change, like, we don't need to own this, like we're better off making this somebody else's problem. So how can Flux help? What does Flux offer?
I mean, the whole reason for starting the company was seeing that starting with code generation, that there was a complete upending of engineering and that to manage that process would require a completely different approach than everything that we've tried so far.
And so that's why we built Flux, is to do a code-first approach to how you're understanding how your code is changing and how your teams are changing as you go through your AI transformation.
In a nutshell, That's what we're here to do is to help make it easy for engineering leaders to prove the efficacy of their AI transformation and to continuously improve it going forward with a code-first approach because it's more accurate. It's better scalable. Oh, and by the way, it removes all the toil of all those legacy systems.
So you get the benefits of this approach without having to put more burden on your teams to change how you're doing ticketing or change how things are happening in GitHub or any of that. Very cool. Awesome. Any last thoughts you'd like to leave our audience with, Ted? I guess it's mainly like, what a fun time this is. Yeah, no, I agree.
Right on, you know, I mean, the transformation that's happening in engineering has never happened before and it never will happen again. And so that's both scary, but it's also what an incredible opportunity for everybody that's in this field and their careers, wherever you sit in the stack. When I mentor...
People, like in college, this, what I have said forever is a tech disruption, that is your best opportunity to advance your career, right? Is to go deep on that, to become an expert in that, because in the early days there are no experts. And that is, that has always been the, like the silver bullet for really accelerating the growth in your career. And that's exactly where we are with engineering.
And so I try to focus on the upside of that because I think there is enormous upside there for people who do lean in and who do become the experts. And so that's, I guess, what I would take away is wherever you're in the stack, step back and just realize this ain't gonna happen again. There'll be other stuff. It'll be interesting. It'll be fascinating. But the first time's the first time.
And that's what we're in for engineering with code. Absolutely. Well, where can people find more about Flux and you? Askflux.ai. We've completely revamped the website. We have the research survey on there. We have a great case study about effective AI transformations for engineering leaders that I think would be a fun read for anybody to get sort of a case study.
That's what it is for like how you could do it or maybe see where you're at in your journey and what you can learn from it. Very cool. Ted, thank you so much for your time. Thank you for coming on. Really appreciate it. Thanks a ton, Brian. It was a pleasure. That was my conversation with Ted Julian from Flux. The biggest takeaway for me is that writing the code isn't the hard part anymore.
Seeing and trusting what it changes is. When Ted's survey asked what was holding teams back, the answers were about trust. Complexity that spirals into an outage. A new library with a vulnerability. Regulated data nobody has really scrutinized. Code review breaks first. Ted calls it act two. And it's a people problem as much as a tooling problem. AI review helps, but he doesn't think you take humans out of the loop.
And the bigger bill comes later. When someone has to understand code nobody really wrote. His once-in-a-career hack is to start measuring features versus maintenance versus bug fixes now, backfill it, and show the before and after. One customer went from 9% features to 43%. And it ended up as a chart in their board meeting. On ownership, it's who feeds the cat.
Ted says it has to be the head of engineering and ultimately the CIO. And for your career, tech disruption is the best chance you'll get to move up. Because in the early days, there are no experts. You can find Ted and Flux at askflux.ai, along with the full report and a case study on AI transformations for engineering leaders through the links in the show notes.
Follow or subscribe to Ship It Weekly wherever you listen, and find previous episodes at shipitweekly.fm. Thanks for listening, and I'll see you Friday for the news.