Ship It Weekly Transcripts

For the last decade, the default infrastructure answer has basically been cloud first, Kubernetes probably, and then good luck to whoever has to operate the thing after the architecture diagram gets turned into real life. And to be fair, the cloud solved a lot of real problems.

Nobody wants to go back to waiting on hardware, filing tickets for VLANs, or discovering that the one person who understood the SAN is on vacation. But there is another side of this now. Cloud bills are getting harder to explain. Kubernetes platforms are getting harder to maintain. Data sovereignty is becoming a real business requirement.

And a lot of teams are quietly looking at all of the layers that they have built and asking a pretty reasonable question. Is this actually simpler?

Because sometimes the thing that was supposed to reduce operational burden just moves the complexity somewhere else into Terraform into Helm charts into networking into managed service glue into a platform team that is now responsible for making 15 different abstractions feel like one coherent system and that is really what this conversation is about not cloud is bad not bare metal is back everybody grab a screwdriver more like What happens when teams want more control, more predictable cost, better performance, and less platform sprawl without going back to the old-school pain of managing infrastructure by hand?

I'm Brian Teller from Teller's Tech, and this is Ship It Weekly. Welcome back to Ship It Weekly, where I filter the noise and focus on what actually matters when you are the one running infrastructure and owning reliability. Most weeks, it's a quick news recap.

In between those, I do conversation episodes with people who are building platforms, running infrastructure, organizing events, and thinking through where this industry is actually headed. Today is one of those conversations. I'm joined by Jake Warner, founder and CEO of Cycle.io.

Cycle is an infrastructure platform that lets teams run containers and virtual machines across bare metal, cloud, private cloud, and hybrid environments without trying to turn every company into a full-time platform engineering shop. And I like this conversation because it gets into a topic that a lot of people are thinking about right now, even if they are not always saying it out loud.

A lot of teams are tired, not lazy, not anti-cloud, not anti-Kubernetes, just tired of the amount of complexity that has piled up around modern infrastructure. You start with a simple goal. Run the application. Scale it. Deploy safely. Keep it reliable. Don't spend a fortune.

Then suddenly, you have Kubernetes clusters, node groups, disruption budgets, autoscalers, Terraform modules, managed services, IAM policies, GitOps controllers, observability agents, service meshes, secrets systems, and a Slack channel where somebody asks why the platform is blocking delivery. And somewhere in the middle of that, people start wondering whether there is another way to think about infrastructure.

In this conversation, Jake and I talk about why some teams are moving back toward private cloud and bare metal, but not in the nostalgic racking and stacking servers was awesome kind of way. More because of cost, performance, data sovereignty. And wanting more ownership over the stack.

We also get into what people still misunderstand about bare metal, why some teams want VMs and containers living together, where Kubernetes is still the right answer, and where an opinionated platform might be a better fit than giving every team every possible knob to turn.

There's also a good thread in here around failover versus active-active systems, stateful workloads, why application -level replication often beats platform-level magic, and what it really means to make raw infrastructure feel like a cloud-like resource.

And towards the end, we talk a bit about AI workloads, GPUs, hype cycles, and why the most bleeding-edge teams are not always the same teams that want an opinionated platform.

So if you work around DevOps, SRE, platform engineering, cloud infrastructure, Kubernetes, private cloud, or you are just starting to wonder whether your modern platform has become a very expensive junk drawer, this one should be worth your time. All right, let's jump in. Today, I'm joined by Jake Warner. He's CEO and founder of Cycle.io.

We're going to be talking about private cloud, bare metal, and why a lot of teams are quietly exhausted by platform complexity. Jake, thanks for joining me. Thanks for having me. Give me your thesis. Why are teams pulling back toward private cloud and bare metal again? So, you know, for most of the people watching this podcast, we all knew that bare metal was really sexy a decade ago, right? 15 years ago.

The cloud made it super easy to move away from bare metal. It solved the big complexity that came into, you know, I mean. We all know how hard it was to get a bare metal server online, get it configured, do it at scale. In the early days of Cycle, I'd always say, you know, anyone can deploy one or two servers. But when you need to start automating 100 servers, that's when the problems get really complex, right?

I'm guessing most people listening to this podcast are completely aware of that. With what we built with Cycle, our goal was to simplify that process so that way you could have companies that were able to own their bare metal, et cetera, and still provision it like a cloud-like resource. And the reason why we kind of did that and why we see companies coming back to bare metal is, number one, cost.

Hyperscalers ended up, you know, they made it really easy. But we've all seen hyperscalers just continue to increase costs. And once you get locked into that ecosystem, your costs kind of only go up. So that's number one. Number two is about data sovereignty concerns.

I think especially with a lot of the geopolitical issues that we're seeing in the world right now, most of the companies that we've been working with are companies that are saying whether it's compliance reasons or they have customer demands, maybe is the right term for it. We have companies that are coming to us and saying. We cannot be on a US-owned hyperscaler.

Even if geographically that infrastructure is sitting over in Europe, we need to, because of the US Cloud Act, we have companies that are saying we cannot be on US -owned infrastructure. So I think that in terms of private cloud, in terms of bare metal, it all kind of comes back to number one, cost, number two, data sovereignty and compliance. And then the third item there would be performance.

And for a long time, I think people kind of... Gave up the performance that bare metal would give because of the ease of the cloud outweighed the performance you got from bare metal. But fast forward to today where you have platforms like what we've built that can make bare metal a cloud -like experience on top of bare metal.

It allows those organizations to go back toward performance, lower their costs and own more of their stack in the process. So along those same lines, is there like a common misconception you see that people have regarding? Bare metal or on-prem in general? Oh, absolutely. So, you know, as recently as, I mean, it happens all the time, but we were doing a demo back in November.

And this was with a large company over in Denmark. And they were one of the early companies to adopt AWS. And this company, they were looking at getting away from Kubernetes. They were using EKS. And the main reason they were coming to us was data sovereignty issues. They had customers that were saying that they needed to not be on AWS to be able to continue to be customers of that company.

And during that demo, one of the requirements, because I kind of posed the question, like, are you interested in just getting off AWS? Are you also interested in adopting bare metal as part of this. And the company was like, no, no, no, we don't want to talk bare metal. Like we don't, we don't want to be responsible for maintaining hardware.

We don't want to be responsible for setting up all the networks and all of those things. And you could tell, again, very technical team, but they had spent so much of the last decade plus in the cloud that they were just kind of used to that. And we did the demo and three quarters of the way through, I was like, you know, hey, we're talking about all these DevOps terms.

We're talking about containers, infrastructure, et cetera. What if I do the rest of this demo and do it on bare metal? But not tell them until afterwards. And it was kind of like a game to me. Like I had someone kind of challenge my thoughts on, you know, the ease of bare metal.

And so we got to the end of the demo and I showed my cards and I was like, oh, by the way, like the infrastructure we just deployed and the containers we just deployed, that was all bare metal. And the company immediately did a 180. And they're like, okay. You've sold us. Let's talk about bare metal now.

Because they realized it was one of those things where, you know, kind of to the three points that I just made about cost, data sovereignty and performance. The main item for them was data sovereignty. But when they realized that now they could have that cost and performance conversation at the same time, and it wasn't necessarily exclusive, it was kind of eye opening.

So yes, back to the question that you asked, which is what is the big misconception? There's a lot of people that have spent much of their careers in the cloud. I mean, again, it's been the main way we've been deploying for a decade now. That because of that, they've kind of, what's the whole cliche?

Going through the forest or the mountains or you're missing, I don't remember the cliche, but the point is that when you're so focused on something, you kind of miss some of the advancements that are happening in the process. Yeah, for sure. I mean, so I am one of those people where I was early days, I worked at a colo facility, racking and stacking, you know, servers.

Very, very different world now than it was back in the late 90s, early 2000s. Yeah, very different as far as management of those colo facilities. Completely different now. With those colo facilities, it's really interesting because so many of the people that I encounter that are bare metal friendly and like ready to return to it are the people that were racking and stacking back at that time.

And, you know, the early cPanel days and they're like, I want to get back to that because they kind of missed that. Like it means something to them. But then you have this whole generation kind of in the middle where cloud was the beginning of their career. So as you talk about racking and stacking, like that is dear to me as well. Yeah.

So when you say turn any infra into a private cloud, what does that mean for a DevOps team day two? So with Cycle, the way we kind of approach things, from a philosophical standpoint, I'm kind of anti-infrastructure as code. Like in Cycle, we call it environments as code. The idea is that you have a thin but defined line between infrastructure and everything that's going to run on top of that.

And so in Cycle, we have what we call environments as code, which is... Called stacks. And that's where you define your load balancers, containers, everything that you are going to be deploying over and over and over. But then infrastructure, we treat as just a pool of resources.

So I guess maybe that's kind of a throwback to the racking and stacking days that we're just talking about where you're taking cPanel and trying to pack these hosts with as many things as you can and so with Cycle the idea is that you know you bring bare metal you bring vms whether it's in the cloud or we have we have some companies that are running Cycle on top of vmware the idea is you bring raw compute a cycle will automatically join it into a mesh and then that becomes kind of your private cloud so it can be hybrid infrastructure multi-cloud fully on-prem or a mixture of all of the above And then applications that you're building, your networks, et cetera, just overlay on top of that.

So again, you have this thin but very defined line between infrastructure on one side and your applications on the other. So one of the things that I believe I read on, I believe it was on your website, you say no DevOps army required. What does that mean in practice? So I've been writing code for, I don't know, probably 22 years now.

I guess there's the common saying that the best DevOps engineer will automate themselves out of a job, whether that's true or not. Up for continuous debate, but I've been writing code for a long time and it seemed like any time it came to actually deploying that, whether it was back in the cPanel days or with Docker or Kubernetes, et cetera, that was always the thing that slowed me down the most.

And while I always loved working with infrastructure, I didn't really like maintaining it. That was always a thing like, I mean, sure, getting it up and running is fun, right? I mean, it's why I think most of us get into it. The first days of setting up infrastructure and, you know, placing an order to buy a new bare metal machine. That's exciting, right?

But, you know, three months later, no one wants to maintain it anymore. And so the goal cycle was how can we how can we build how can we build a platform that allows developers to do the things that typically you would need a DevOps engineer to do? And I'm not saying that, you know, to fully replace DevOps engineers, but in many cases, companies that are coming to us are companies where it's very engineering heavy.

So, you know, like 10 developers to every DevOps engineer or a greater ratio. We have some companies that are 25 to one, et cetera. And those seem to be the places where we do best. And one of my favorite components of that is when we have developers that are deploying things on top of the platform that aren't really DevOps engineers at all. And they can't tell you how they did something, but it works.

And, you know, that's one of the things that is nice about that. So no DevOps army is required is just that philosophy. Like, how do we empower developers to do what they need to do without having to become DevOps experts as part of that? So talking about mixed workloads, can you give me like a real world workload mix? Like, why do teams want VMs and containers living together? Why would they want that?

There's probably... Hundreds of different potential reasons. One company that I was helping earlier this morning are running all of their different microservices in containers. That's just how they built their platform. But then due to a whole bunch of legacy, their company that's been around for. I think, 18 years.

They have a number of applications that are built on top of .NET and require Microsoft SQL and things like that. For them, all of their newer microservices are running inside of containers. But for them being able to have those legacy applications running inside of Windows VMs sitting on the same infrastructure, on the same network where to the containers, they don't know they're talking to a VM and vice versa.

It's just fully abstracted. So everything just sees network endpoints, but it allows companies not have to change everything for adoption. So that's one. And I guess another use case is we have some companies that are in regulated industries and things like that where.

For certain applications, they need, you know, true virtualization for isolation as opposed to, you know, just cgroups and, you know, the isolation that comes with containers. Yeah. I feel like as an industry now too, everybody wants to containerize every service and makes sense in a lot of cases.

But I've also found that there are cases where putting it inside of Kubernetes or some sort of container orchestration layer doesn't always make sense. We're just doing it because that's the phase right now. It just seems like every trade show I go to, too, it's pushing this idea of containerization, which, again, microservice layers make sense.

There's certain applications, though, where I don't think putting bridges and stuff inside of a container makes sense, especially when you're dealing with services that can't be disrupted. It's an issue that I've come across a lot in the last year or two. Yeah, I mean, I would say that I'm probably on the opposite side of that. I containerize everything that I possibly can. But I mean, I also put the platform for it.

So, you know, kind of aligns with my belief there. But yeah, I containerize everything to the point that on Cycle, every virtual machine is also actually a container. And so that way we have container layer, we have a containerized hypervisor, and then you have your VM sitting inside of that.

And so the nice thing about that is that as we roll out new versions of the hypervisor, you know, given that it's containerized from that perspective, we're able to kind of roll out different versions of the hypervisor as users opt into it. So, you know, kind of little benefits from that. So that's interesting. So let's talk, I guess, a little bit more about the containerization aspect.

In Kubernetes, I'm dealing with PDBs, pod disruption budgets. I'm dealing with maybe Karpenter if I'm on AWS. So I'm having this scale up and down nodes as needed. And then if I'm dealing with spot instances, I could have disruptions to...

Workloads how does cycle handle that orchestration of services so there's two ways that cycle approaches that number one is that in general i believe that the idea of failover is a terrible idea and i've seen so many times in my career where you know you set up all these processes for failover and then things fail to fail over when things go wrong right and so um with that you know the idea that failover is you know kind of high risk.

It's also along the same kind of philosophy that, you know, less moving parts is better. So inside of cycle, when we talk about like stateful workloads and things like that, in general, the approach that we have is to run everything in active active 100 % of the time. So that way, if one side goes down, you're perfectly fine. Like, you know, when that side comes back up, you know, you'll recover.

Now, if you need to evacuate, you know, to another host, you can do that too. But there's a reason why, Mongo and so many of these modern database technologies have their own replication built into it.

And I guess maybe that's a little bit of a tangent is it bothers me so much when platforms try to automate storage replication, as opposed to letting the application that knows how to replicate it properly, replicate it. And so in Cycle's world, we try to, everything is active, active. We try to move things as little as possible.

And then if you are deploying a container and you mark it as stateful, the platform will treat that container separately, almost like a VM. It'll almost treat it like a VM. So that way its data will always move with it. Like we will like, instead of like creating like a volume claim in Kubernetes, we will create an attached LVM, like a raw LVM, and then migrate that as needed.

And if you need to scale up, we'll scale up, but we're going to rely on the underlying application to know how to replicate that data. Because again, if we're talking about like Mongo as an example, being able to have collection-level locking as you do migrations and things like that, as you have elections happening.

Great why should a platform try to do that in general what would be like a best bare metal use case right now given the trajectory of the industry i mean i think that For companies that are trying to reclaim ownership of their stack, bare metal is a great way of doing that. At that point, you're just consuming raw compute. The different services that you're kind of locked into are significantly less.

I mean, I guess it depends on what provider you're going with and things like that. But for the most part, your pricing is going to be significantly more stable. For example, we just signed a partnership today to be announced soon that includes 100 terabytes of bandwidth per service. Server on a 10 gig link.

And, you know, I mean, you know, when you start talking about 100 terabytes of bandwidth of bandwidth, per, you know, VM or, or whatever AWS, you're talking about a huge bandwidth bill.

Uh, so, uh, the fact that some of these, these bare metal providers are including a 100 terabytes, um, per server just out of the box with normal pricing is wild um and maybe i should take my earlier response back and instead talk about performance um and performance density right because like we as a company we've been buying some more bare metal um recently for we're getting ready to launch a european control plane so we're buying more infrastructure for that control plane and the fact that we were able to buy physical machines with 24 physical cores for $280 a month with dual 10-gig links and 100 terabytes of bandwidth included and 192 gigabytes of memory each and RAID 1 with terabyte NVMe drives.

Like, I mean, those same specs would be, I mean, I don't know the exact number, but I'm guessing probably three to four times over at AWS. And then you'd still have the virtualization overhead on top of that. What do you think is the reason for that? Do you think that these colo or server providers are trying to compete with that market? Or what's the reason for the discrepancy in pricing, you think?

Is it just because we're paying so much for the control plane at AWS? That's my theory. I don't actually know. It's wild that you can buy that level of performance for that price. And yeah, like, yes, there's... One of the cons that you typically get from most of these, these kind of bare metal providers today, one of the cons is that from a network perspective, you have way fewer PoPs, right?

So your latency from a network standpoint can be meaningfully higher where if you're using AWS and GCP, I mean, you'll have great latency to almost anywhere in the world where some of these bare metal providers, you might get one or two pops. Now, granted, there are some like Equinix and Megaport that that's what they do and they can still give you, you know, a really solid network. So there's...

You know, uh, pros and cons, but I don't know if having 10-plus PoPs, you know, at GCP is also worth a three X price increase on compute. I don't know if that's the reason why, but I think the network is like the network resiliency is probably the one con to bare metal today. Yeah. I guess it matters what your workloads are if you, if you're latency sensitive, but yeah.

Like if you're doing high traffic bidding applications where you need to bid for, for.

Creative or something i could see you needing to have that low latency for that but i don't think that most people if especially if you're serving websites or serving a web app i don't think for the most part it would matter but also i mean Level 3 is in northern virginia in the same area as you know you could probably buy colo space again i'm sure northern virginia area is is expensive just because dc metro area but You could probably buy Colo servers in that area competitively against AWS's rates as well.

Well, it's kind of funny that you mentioned that. Because of AWS's presence there, we have so many companies that when they're switching to bare metal and using Cycle as part of that process, we have companies that specifically need bare metal in that region. And it's like, hey, you can deploy bare metal anywhere in the world. And they're saying, no, no, no, no. I need to connect to Supabase or...

Some other service running on top of AWS. So even though they're trying to get off of AWS as a cloud provider, they still need to be in proximity to it because so many of the services they're communicating with are still sitting right on top of that infrastructure.

So we have companies that did initial test deployments and they were deployed to somewhere around New York City or somewhere to Atlanta and they would find out that extra. 12, 15 millisecond latency was too much for them. And so they had, so like, it's weird how there seems to be such a, and it's only on US East 1. We don't have any other companies that are like, oh, US East 2 or US West 1 is where I need to be.

But for whatever reason, US East 1, we just have an extreme number of companies that are like, I need to be close to that. And it's kind of coincidental because that's the one that always has an outage or at least it seems. And it's the control plane for the rest of the services.

So what I have found is when US East 1 goes down, if you have services that are in US West 2, maybe like an auto-scaling group that needs to scale up, you know, new instances, it's not able to because it's not able to reach out to the control plane, which lives in US East 1. Yeah. There's still marriage there as much as they maybe try to say that they're isolated. They're not.

Yeah, that is something I've absolutely noticed as well. When US East 1 goes down, you just have to assume that everything else is also impacted. Okay, so speaking about Cycle versus Kubernetes, what do you think that Kubernetes gets right? And where does it become self-inflicted pain?

So with Kubernetes versus Cycle and when is kind of a better fit for one versus the other, it really kind of comes down to the needs of individual companies from a customization perspective. For companies that are really, as I mentioned earlier, kind of engineering heavy, that are like, hey, we're mainly developers. We're very microservice heavy.

We might need object storage or like the requirements that they have are pretty, let's say, you know, commoditized, right? They need some disk. They need some, you know, object storage. They need kind of the primitives. That's where Cycle really shines. But at the same time, if you have companies that are like, hey, we need people to run on.

Very specific hardware with very specific kernel drivers and, you know, things like that. That's where Kubernetes is, you know, a better fit from that perspective because with Cycle, we ship a standardized OS to everyone. We don't provide SSH access. That OS is made to be as dumb and as tiny as possible. You know, it's 40 megabytes in size.

So that means that naturally it's going to be kind of limited in terms of what it can do. Like our goal is to target 80%, right? But if you have like really specific infrastructure you need to run or, you know, Supercomputing applications or some of these big AI models that people are buying a whole bunch of really sophisticated infrastructure for these days. Cycle today is not built for bring anything.

We are built for more of bring your typical x86 server you know with a few drives in it we'll get it up and running we'll get you what you need so it all comes back to whether do companies need just basic primitives and they don't want to be devops or do they have a very specific list of requirements um that you know extend from network to hardware to um maybe the oa the host os that those nodes need to run people who are coming to cycle are looking for an opinionated answer it's why most of the companies that are on Cycle today are companies that left Kubernetes for Cycle like you know they went that route they tested it um many of them were on Kubernetes for years before they decided like i think that it's natural for for all of us techies to want to play with new technologies but at some point people are like you know what i don't i don't want to play with all the bells and whistles anymore i just wanted to work yeah and it's something that you know i kind of talked about often is you know most people when they get their first smartphone or at least you know for me and most of my friends as we were growing up your first smartphone is an android right um and i know i'm probably about to piss off a whole bunch of people but you're for you know typically your first phone is an android phone because you want to you want to customize it you want to you know play with it you want to make it yours yeah yeah exactly um but eventually people are like i don't really care about that anymore i just want a phone that works i don't want to you know like let me change my background and i don't care and then they switch to an iphone um and so that's kind of what i've kind of always said about cycle like Companies, you know, they're going to go play with Kubernetes.

They're going to go play with Rancher. They're going to go kind of test out, you know, the latest and greatest. They want something where they can change every variable. But at some point, it no longer becomes about changing variables and playing with the latest and greatest of everything. They just want to get back to what they want to build.

And so that's where we built Cycle for companies that are like, yeah, just give me a standard opinionated platform and I'll just work with that. And so, yeah. Yeah, it makes sense. So, okay, wrapping up, what's one thing you'd do first if you had to modernize on-prem or hybrid without blowing up the team? This is going to go back to a conversation we had, you know, 10 minutes ago, but containerize, right?

Like, you know, I'm a big fan of containerizing everything. It makes applications typically way more portable. And if your goal is to chase portability, so that way you can kind of move to whether it's, you know, VMs or bare metal or on-prem, I mean, I guess that's bare metal too. Or mixture. Being able to standardize containers is, you know, I think why I'm in such favor of them.

Yeah, I think that would be my first step. I think that would absolutely be my first step from that standpoint. Now, if the question is, what if I'm already containerized and now I'm trying to like go towards, you know, bare metal? I think the next step there is, you know, look at your dependencies. If you're in a hyperscaler. And you're using things like Lambda and S3 and things like that.

Your next step is kind of to decide what services I'm going to try to replicate on top of bare metal. What am I going to be okay having still in the cloud? And do the primitives that I need allow me to be on bare metal? Because again, if you are tightly integrated into Lambda and some of these other things, bare metal might not be the best fit for you.

But if you're just running a whole bunch of containers, then you're kind of in a good spot. So assuming you're containerized, I think then the next part of that is just... Evaluating what third-party services you need and whether you can bring those in-house or whether you're okay with those staying in the cloud.

Have you noticed an increase in AI workloads and people building containers around AI-specific workloads using Cycle? So it's kind of interesting for how much, you know, you log into LinkedIn and Reddit and things and everything's about AI. So many of the companies that are on Cycle today, like, yes, you know, a number of them do have some AI component. In what they're doing.

But I think we only have one or two clients that are running like true models with GPUs. Yeah. I didn't mean like API calls to OpenAI, but yeah, like actual modeling locally. Yeah. Yeah, so Cycle out of the box supports all NVIDIA data center class GPUs. And so that's one of the drivers that we keep up to date. So we support that.

But we were hoping that we'd have more AI, you know, focused applications on top of the platform today. But I think that it's one of these kind of interesting things where it's kind of like with IoT and then crypto and now AI. Like, yes, you know, there are hype waves. And I'm not saying that AI is, you know, you know.

Equivalent to IoT or crypto or things like that but as we see these hype waves happen we always see people who kind of love bleeding edge technology chase them right like so many of my friends that were like really gung-ho on on crypto are now the same people that are really deep into you know ai and then the next thing that comes out they're going to chase that as well like it's what they do and those people typically are the same people that like changing all the, you know, the dials and they like playing and tweaking.

And so that's where they kind of, I think those people, they don't want an opinionated platform like Cycle. They want something where they can customize everything. It's kind of like the book of, you know, Crossing the Chasm, right? Cycle is there for the majority in the middle. They're like, hey, no, I just want to run stuff. I want to build stuff.

But then, you know, those super early adopters that are always chasing new technology, they want to be, they want more customization than Cycle will give them. And I guess to an earlier point, that's where Kubernetes probably makes more sense for them. But when applications kind of become more standardized and they just want to run them, then that's where Cycle starts to win. Makes sense. What is Cycle?

Why would someone choose Cycle? What would be a reason for choosing Cycle? Yeah, so for an organization that is really engineering heavy, a company that wants to spend more time focused on building versus maintaining. And when I say building versus maintaining, I'm talking about building the actual applications, the platforms, the services, etc.

Less on maintaining the host OS, the host kernel, the underlying infrastructure. That's where Cycle really shines for these organizations, especially teams that where they have. Really talented developers, but these developers really don't have interest in becoming DevOps engineers as part of that process is kind of where Cycle shines. Makes sense.

Okay, so wrapping up, where can people read more about you, find out about you, read more about Cycle? Where should they go? Yeah, so to learn more about Cycle, you can visit our website, which is Cycle.io. We also have a Slack community that we have a lot of developers and DevOps engineers that hang out in. That's slack.cycle.io.

And then for people who want to maybe learn more about me, I guess linkedin.com/in/jakewarner. Like it's kind of weird to hand out my or to use my LinkedIn as that as the primary source. But I think I keep it more up to date than anything else I do these days. That makes sense. I'll also put links for all of that in the show notes as well. So make it easier. Sounds good. Awesome.

Well, thank you, Jake, so much for coming on and telling me more about Cycle and containerization. Really appreciate it. Yeah, really, really appreciate you having me on, Brian. Really enjoyed the conversation. Always fun to be able to, you know, as you mentioned earlier about racking and stacking, meet people of like mind in terms of how we got to where we are today. Awesome. Thanks. All right.

That was my conversation with Jake Warner from Cycle .io. My biggest takeaway from this one. Is that the cloud versus bare metal debate is kind of the least interesting version of the conversation. The better question is, what are you actually trying to optimize for? Because sometimes the answer is cloud. Sometimes it is Kubernetes.

Sometimes it is managed services all the way down because your team does not have the time, people, or business reason to own more of the stack. But sometimes the answer is different. Sometimes the problem is cost. Sometimes it is performance density. Sometimes it is data sovereignty. Sometimes it is compliance. Sometimes it is latency.

Sometimes it is the fact that your developers just want to ship software and your platform team is slowly drowning in a pile of abstractions that were supposed to make everything easier. That is the part I think is worth paying attention to. A lot of teams do not necessarily want to go backwards. They want to go back to owning the parts that matter without reintroducing all of the old pain.

And that is a much more useful framing than pretending there is one correct infrastructure model for everybody. I also like Jake's point about opinionated platforms. Because as engineers, we love flexibility. We love knobs. We love knowing that technically, if we really wanted to, we could customize every piece of the stack. But there is a cost to that. Every knob becomes a decision.

Every decision becomes something to document. Every exception becomes something to support. And eventually, the platform that was supposed to help teams move faster becomes another system that needs its own platform team just to keep it sane. That does not always mean opinionated platforms are always better.

If you need very specific kernel drivers, specialized hardware, deep customization, or you are doing weird edge case infrastructure work, then Kubernetes or a more flexible platform may absolutely be the better fit.

But for a lot of teams, especially teams that mostly need containers, VMs, networking, storage, load balancers, and a sane way to deploy applications, there is a real argument for fewer choices and better defaults. And honestly, that is probably where a lot of infrastructure conversations are heading. Not everything should be cloud. Not everything should be Kubernetes. Not everyone should move back to bare metal.

More like what complexity is actually helping us? And what complexity are we just carrying? Because the industry told us this is what modern infrastructure is supposed to look like. I'll have links to Jake, Cycle.io, and their Slack community in the show notes. If you enjoyed this conversation, follow or subscribe to Ship It Weekly wherever you listen to podcasts.

It helps the show, and it makes sure you get both these conversation episodes and the weekly DevOps, SRE, platform, cloud, and security news recaps. You can also find the show notes and links over on shipitweekly.fm. Thanks for listening, and I'll see you later this week.

A repo named Private-CISA was reportedly public on GitHub, which is a pretty painful reminder that naming something private is not access control. This week, we've got leaked AWS keys, internal deployment docs, Terraform, Kubernetes manifests, Argo CD files, AI root cause analysis, agents clicking through legacy apps. Claude Code showing up in CI/CD.

And another Kubernetes security reminder hiding in a boring default. The theme is pretty simple. Automation is getting more powerful. But we are still leaking secrets, missing defaults, and writing postmortem action items that quietly go to die. I'm Brian Teller from Teller's Tech, and this is Ship It Weekly.

Welcome back to Ship It Weekly, the show where we look at the DevOps, SRE, cloud, platform, and security stories that actually matter when you are the person who eventually has to keep the thing running. This week, we're starting with the CISA contractor GitHub leak. Then we'll get into AWS DevOps agent and automated root cause analysis.

Microsoft Copilot Studio computer using agents, Atlassian adding cloud code to Bitbucket Pipelines, and CVE-2026-46333 with the Kubernetes seccomp angle. Then in the lightning round, we'll hit GitHub expanding OIDC support for Dependabot and code scanning. Java pods getting OOMKilled even when heap looks fine. And why LLM-generated SQL can be wrong in ways that still run.

And for the human closer, we'll talk about why postmortem action items die. So let's get into it. First up, the big one. A contractor for CISA reportedly had a public GitHub repository named Private-CISA. Again, Private-CISA. That is like naming an S3 bucket definitely not customer data and hoping AWS respects the vibe.

GitGuardian says it found the public repo on May 14th containing around 844 megabytes of exposed data. Reporting included plain text passwords, AWS tokens, Entra ID SAML certificates, and internal material tied to CISA systems. And this is the part that matters for us. This was not just one forgotten .env file.

The reporting describes CI/CD logs, Kubernetes manifests, Argo CD files, Terraform code, GitHub Actions workflows, deployment details, and internal docs. This is not only a secret leak. This is context leakage. Credentials are bad. AWS tokens are bad. Plain text passwords are bad.

But when you also leak deployment docs, infrastructure code, manifests, CI/CD logs, and internal workflows, you are giving someone more than a key. You are giving them the floor plan. Now, we should be careful. From the outside, we do not know the full operational impact. Public reporting says CISA was notified and the repo was taken down. But the lesson is already clear.

Secrets scanning matters, but it is not enough. The repo should not have been public. The archive should not have been there. The credentials should not have been valid. And once operational context leaks, the response cannot just be rotate the key and move on. You need to ask what the leaked context teaches an attacker. Most teams hear a story like this and think, wow, how does this happen?

And then somewhere in their own org, there is a repo called old prod migration backup final last touched by a contractor in 2022. Containing a zip file nobody wants to open because it feels cursed. Most of us have some version of this risk. Old repos, personal forks, contractor projects, build logs, Terraform state, kube configs, migration scripts, weird backup folders.

So the takeaway is not just don't commit secrets. That is true, but it is table stakes. The better takeaway is to inventory the weird stuff. Look for old repos, archived repos, contractor -owned repos, personal forks, internal examples, demo projects, and places where somebody may have dumped operational context while trying to move fast.

And when a leak happens, treat the context as compromise too, because sometimes the key is the headline, but the deployment map is the real prize. Next up, AWS published a post on using AWS DevOps Agent to automate root cause analysis across Datadog and Elasticsearch with CloudTrail and EKS involved too. This one sits right at the intersection of useful and slightly unsettling.

The AWS example starts with a Datadog alert. AWS DevOps Agent gets access to EKS. So it can describe Kubernetes objects, pull pod logs, and look at cluster events. It also correlates Elasticsearch logs, Datadog metrics, and CloudTrail deployment events to figure out what changed. And honestly, that sounds a lot like incident response. You start with one symptom. Then you try to rebuild the timeline. What changed?

What deployed? Which pod restarted? What metric moved first? Did someone push a config change? Did a dependency decide today was a great day to build character? Half of incident response is not fix the thing. Half of it is reconstructing the story fast enough that you fix the right thing. So if an agent can gather context and build a decent first-pass timeline, that is useful. Logs over here. Metrics over there.

CloudTrail in another tab. Kubernetes events in a terminal. Datadog on one monitor and Slack on another. And someone asking an update every 90 seconds, which is understandable and still emotionally damaging. But automated RCA can become very convincing very quickly.

A tool that says probable root cause with a clean summary can easily become the thing everybody believes, especially when the incident channel is noisy and everybody is tired. So I like the direction, but I would treat AI RCA like a fast incident scribe or junior investigator. Great at pulling threads. Useful for assembling context. Helpful for reducing time to understanding. But not the final authority on causality.

Humans still need to ask, is this correlation or cause? Did this happen before or after customer impact? What else changed? What evidence would prove this wrong? Because if the incident review action item is that AI said it was Elasticsearch, so we restarted Elasticsearch, that is not RCA. That is vibes with a dashboard. Before we get to the next story, a quick note from this week's sponsor, Guardsquare.

If you are building mobile apps, good enough security is usually a problem waiting to happen. Guardsquare focuses on actually protecting your code in addition to scanning it. That means code hardening, runtime protection, testing, and visibility into what's happening once your app is out in the wild.

So if you are responsible for shipping and securing mobile apps, Android or iOS, Definitely worth taking a look at guardsquare.com. Alright, back to the show. Third story. Microsoft says computer using agents in Copilot Studio are now generally available. This one is interesting because it changes the automation boundary. A lot of automation assumes there is an API. You call an endpoint. You get a response.

You wire it into a workflow. Everyone pretends the internal CRM is not held together by three workflows, a CSV export, and one person named Linda. Computer using agents are different. Microsoft describes these agents as interacting with graphical user interfaces, websites, desktop apps, screens, buttons, forms.

So instead of saying there is no API, so we cannot automate this, the pitch becomes the agent can use the UI like a human. And look. I get why this is appealing. Every company has legacy systems. Every company has some vendor portal that looks like it was designed during the emotional low point of enterprise software.

Every company has workflows where someone copies data from one screen into another and calls it a process. If an agent can take some of that away, great. Nobody needs a fulfilling career in manually clicking invoice screens. But this is also where the risk gets weird. API automation usually gives you structure. Scopes, endpoints, logs, schemas. UI automation is messier.

The agent is looking at screens, reading labels, clicking buttons, entering text, and deciding what to do next. Which means your automation path may now include a model looking at a webpage and deciding which button seems right. That sounds funny until the button says submit payment, delete record, approve request, or yes, I understand this is permanent. So the governance questions matter.

What apps can the agent access? What account does it use? Can it reach production admin screens? Are actions auditable? Can it pause before destructive steps? What happens if the UI changes slightly? And who owns the outcome when it clicks the wrong thing? Computer using agents may become useful because plenty of enterprise systems will never get good APIs.

But when the agent operates through a UI, treat that UI like an automation interface. Use restricted accounts. Use test environments. Use approval gates. Log actions. Limit destructive workflows. And be very suspicious of anything involving bulk update. A computer -using agent is not just a better macro. It is automation with eyeballs.

And like most things with eyeballs in enterprise software, it probably needs supervision. Fourth story. Atlassian says agentic pipelines now support Claude Code as a provider in bitbucket pipelines. This sounds like a feature announcement until you think about where it lives. It lives in CI/CD.

Atlassian's examples include README updates, security report triage, feature flag cleanup, PR descriptions, and other repetitive engineering chores. And this keeps coming back to a point I've made before. Developer tooling is production now. CI/CD is not just the stuff around production. It is the path code takes to become production.

So when you put agents inside pipeline workflows, you are not just making developers faster. You are changing the delivery path. And yes, some of these tasks are genuinely annoying. Security report triage, feature flag cleanup, PR descriptions, documentation updates. Please take them. Nobody is sitting around hoping for more stale flags. And slightly wrong README files.

But once an AI agent is part of a pipeline, the boring questions matter. What repository context does it get? What logs does it see? What secrets are visible to that step? Can it modify files? Can it open PRs? Can it change tests? Can it generate security triage notes that humans treat as authoritative? Can it make a flaky test look fixed by weakening the assertion? That last one is not me being dramatic.

If the task is fix the failing pipeline, a bad agentic workflow might make the pipeline pass without making the system better. And yes, humans do this too. We just call it temporary and let it survive three reorgs. Atlassian also points users to guidance around third-party agent providers and data handling.

That matters because if you bring Claude Code into a pipeline, you need to understand what code, prompts, logs, and generated context may leave your environment. That does not mean do not use it. It means do not discover your data flow by reading the invoice. Treat agentic pipeline steps like any other privileged CI step. Start with low-risk tasks. Avoid secrets. Avoid production deploy authority.

Make outputs reviewable. Require human approval when the agent changes code. And document what data goes to the provider. The agent does not need to be terrifying, but it also should not be a surprise guest in your delivery path. Fifth story. Let's talk about CVE-2026-46333 and Kubernetes seccomp defaults. This one is more technical, but it's a good grounding story after all of the AI agent stuff.

Because sometimes the most important security decision is not a new tool or a new model. Sometimes it is whether your pods are running with the syscall profile that you thought they were running with. NVD tracks CVE-2026-46333 as a Linux kernel issue in the ptrace path. Qualys describes it as a local privilege escalation and credential disclosure issue.

The Kubernetes angle is that unset or unconfined seccomp profiles can leave pods exposed to the tested path, while runtime default block the tested pidfd_getfd path. PSS restricted added more protection as well. The operator version is simple. Your Kubernetes security defaults matter. And they matter most when nobody is thinking about them.

Seccomp is easy to mentally file under container security stuff we should revisit someday. Which is engineering speak for future incident seasoning. Kubernetes docs say that if seccompDefault is enabled, pods use the RuntimeDefault seccomp profile when no other profile is specified. Otherwise, the default is unconfined. That difference matters. Because unset does not always mean safe.

Sometimes unset means congratulations. You are raw -dogging syscalls in production. Probably do not put that in your architecture diagram. So the takeaway is straightforward. Check whether RuntimeDefaults is actually being applied in your clusters. Review your pod security standards posture. Know where unconfined seccomp is allowed. Know where privileged pods exist. Know which namespaces have exceptions.

And if you use managed Kubernetes, do not assume the provider magically made your pod security posture sane because the control plane has a nice logo. Containers share the host kernel. That is the deal. So when there is a local kernel bug and your workload security posture allows the relevant path, the cluster configuration suddenly matters a lot. The boring defaults are not boring. They are latent decisions.

And every once in a while, a CVE shows up and asks what you decided. Now let's do a quick lightning round. First, GitHub expanded OIDC support for Dependabot and code scanning. GitHub says that Dependabot and code scanning now support OpenID Connect authentication for organization-level private registries for Cloudsmith and Google Artifact Registry.

The short version is fewer long-lived registry credentials sitting around as secrets. And that is good. OIDC -based access is not magic. But short-lived identity-based auth is usually healthier than, here's a token, Please don't leak it. Best of luck to everyone involved. Second, Java pods getting OOMKilled in Kubernetes even when the heap looks fine. Classic operator trap.

Someone sets -Xmx, looks at heap, and thinks we're fine. Then Kubernetes kills the pod anyway. And everyone stares at the graphs like the cluster betrayed them personally. But JVM heap is not the whole memory footprint. Metaspace, direct buffers, thread stacks, native memory, JIT. GC overhead, and off -heap usage still count towards the container memory limit.

Kubernetes does not care that your heap looked reasonable. It cares that the process crossed the limit. So if your Java pods are getting OOMKilled, do not only look at -Xmx. Look at total container memory and leave headroom. Because production loves punishing tight memory math. Third, LLM-generated SQL can be wrong in ways that still run. A broken query that fails loudly is annoying, but at least you know it failed.

A query that returns plausible nonsense is worse. The dashboard loads. The numbers look reasonable. Someone puts it in a slide deck. And now your business metric is powered by a hallucinated join and a missing filter. So text to SQL needs guardrails. Read-only roles. Query limits. Schema-aware validation. Known templates where possible. Human review for anything important.

The danger is not always that AI writes broken SQL. Sometimes it writes SQL that is wrong quietly. And quiet wrongness is how bad decisions get confidence. The Human Closer this week is about postmortem action items. Because if there is one place that engineering organizations consistently lie to themselves, it is the bottom of a postmortem document. Not maliciously, just optimistically. The incident happens.

People jump in. The team writes a timeline. There is a good discussion. Nobody blames anyone. Everybody agrees on what went wrong. Then the action items show up. Improve monitoring. Review runbooks. Add better alerting. Investigate retry behavior. Clean up ownership. These are not action items. These are wishes wearing a Jira costume. Incident .io had a good piece on why postmortem action items die.

The reasons are painfully familiar. No named owner, wrong tracking place, vague wording, and no follow -up cadence. That's basically the whole game. A postmortem action item without an owner is not an action item. It's a group hallucination. An action item that lives in a doc that nobody opens again is documentation tax. And an action item that says improve monitoring is not an action item. It is a mood.

A real action item sounds more like Maria adds a replication lag alert for the payment database by Friday. Or Kevin removes production deploy access from the old CI token before the next release. Named owner. Specific verb. Clear outcome. Real tracking location, due date. That does not make the work easy, but it makes it real. This connects back to every story this week.

The CISA leak is not fixed by saying review GitHub practices. AI RCA is not useful if the follow-up is improve incident response. Computer using agents are not safe because someone wrote ensure governance. Kubernetes seccomp is not handled because someone says harden workloads. This is the staff and principal engineer part of the job that does not always look exciting on a roadmap.

Turning vague risk into specific work. Turning incidents into system changes. That is where reliability actually improves. Because production does not care how good the postmortem sounded. Production cares whether anything changed. Okay, that's it for this week of Ship It Weekly.

We covered the CISA contractor GitHub leak, AWS DevOps agent and automated root cause analysis, Microsoft Copilot Studio computer using agents, Atlassian agentic pipelines with Claude code, CVE-2026-46333, and Kubernetes seccomp defaults, plus a lightning round on GitHub OIDC support, Java pods getting OOMKilled, and LLM-generated SQL.

If this episode was useful, follow or subscribe wherever you are listening or watching. If you're on YouTube, hit subscribe. If you're in a podcast app, follow the show there. And if you know somebody on a DevOps, SRE, platform, security, or engineering leadership team who is dealing with secrets, agents, Kubernetes defaults, or postmortem follow-up, send this one to them. It helps the show grow.

And honestly, it helps me keep making this kind of content for people who actually live with these systems. You can find the weekly brief at OnCallBrief.com and more episodes and show notes on ShipItWeekly .fm. I'm Brian from Teller's Tech, and thanks for listening.

And remember, if your repo is named Private-CISA, your agent can click buttons, your pipeline can call Claude, and your postmortem action item says improve monitoring, maybe take a breath then go find the owner the token the default and the ticket because production does not run on good intentions it runs on the stuff that someone actually fixed

AI agents just got APIs. They got identity. And they're starting to plug into the automation tools teams already use to change real systems. So the question is moving past, can AI write code? The better question is, what happens when AI can open pull requests, call tools, authenticate to services, and trigger operations workflows? Because at that point, you did not build a chatbot.

You built a coworker with API access. I'm Brian Teller from Teller's Tech, and this is Ship It Weekly. Welcome back to Ship It Weekly, the show where we look at DevOps, SRE, cloud, platform, and security stories that actually matter when you're the person who eventually has to keep the thing running.

This week we're looking at GitHub making Copilot cloud agent tasks available through a REST API, Auth0 bringing authentication to MCP servers, Red Hat positioning Ansible as an execution layer for agentic IT operations, and OpenAI Daybreak pushing AI deeper into security research and remediation.

Then we'll step away from the AI cycle for a really good Discord engineering story on automating ScyllaDB operations at scale. And in the lightning round, we'll hit AWS GuardDuty and crypto mining detection, queues and backpressure, and why an index scan can still ruin your day. The theme this week is authority, not intelligence, not productivity, authority. What can these agents reach? What can they change?

Who approved the action? And when something breaks, who owns it? That's the thread for this episode. So let's get into it. First up, GitHub Copilot Cloud Agent Tasks can now be started through the REST API. This is the right place to start because it sounds like a small product update, but it changes the shape of the thing.

GitHub says Copilot Business and Enterprise users can now programmatically start Copilot cloud agent tasks through a new Agent Tasks REST API, currently in public preview. The Copilot cloud agent works in the background in its own development environment. It can make code changes, validate those changes, and open a pull request. That part alone is already interesting. But the API is the bigger shift.

Because now this is not just a developer manually asking Copilot to work on something from inside GitHub. Now another system can kick it off. That means you could wire this into custom workflows. A support escalation, a bug triage process, a security finding, a dependency update workflow, a backlog grooming process. Or whatever else somebody decides to connect. And that's where this gets operationally interesting.

Because once an agent can be started by automation, it becomes part of your automation surface. It becomes something you need to reason about like any other system that can create change. What repos can it touch? What permissions does the token need? Who approved the task? What branch protection applies? Can it create a pull request but not merge one? Can it trigger CI? Can that CI deploy?

And if the workflow is kicked off by another tool, do you still have a clear human owner? That last one matters because it is very easy to imagine a chain like this. A vulnerability scanner opens a ticket. A workflow kicks off an AI agent. The AI agent makes a patch. CI passes. A PR gets opened. Somebody rubber-stamps it because the diff looks boring and the scanner says the vulnerability is resolved.

And maybe that is great. Maybe you just saved an engineer three hours. Or maybe you just created a subtle production issue from a change nobody really understood. The practical takeaway here is not don't use it. The practical takeaway is that agent workflows need the same boring controls we already expect from normal engineering workflows. Branch protection. Required reviews. Code owners. Scoped credentials.

Audit trails. Clear ownership. And a very bright line between agent can propose and agent can ship. The interesting part of AI agents is not that they can do work. The interesting part is that we have to decide how much authority that work gets. That leads nicely into the second story. Auth0 announced that Auth for MCP is generally available.

MCP or Model Context Protocol has become one of those terms that shows up everywhere now. It is basically a way for agents and AI tools to connect to external systems, tools, APIs, and data sources in a more standardized way. And that matters because agents are only as useful as the tools they can reach. A model sitting in a chat box can give advice. A model connected to tools can take action.

And once it can take action, authentication and authorization stop being side concerns. They become the whole game. Auth0's announcement is focused on putting an identity layer around MCP servers. They call out authentication, CIMD registration, and on behalf of token exchange. The plain-English version is this. If agents are going to call tools, those tools need to know who or what is calling them. On whose behalf?

And what that caller is actually allowed to do. That sounds obvious, but a lot on-behalf-of token exchange. The plain-English version that feels like local developer convenience first, production safety second. You spin up a server, you connect it to your agent, you give it access to some tools, and suddenly your agent can read things, write things, query things, maybe even change things. That's fine in a sandbox.

It is not fine... When the tools are attached to customer data, production infrastructure, internal admin APIs, CI/CD, billing systems, or cloud accounts. And this is where identity gets weird. Because with a normal user, we mostly know how to think about it. Brian logged in. Brian clicked a thing. Brian had these permissions. With an agent, the story is messier. Was the action taken by the agent?

By the user who asked the agent? By the application hosting the agent? By a service account? By a delegated token? And when something goes wrong, where does accountability land? That's why I think this Auth0 story is more important than it looks. MCP is not just a cute connector system for demos. It is becoming connective tissue for AI tooling.

And connective tissue needs identity, authorization, logging, and revocation. Otherwise, we're just building a faster way for something to call the wrong API with too much permission. For DevOps and platform teams, this is probably where the real work starts. Not how do we let every team use agents, but how do we let teams use agents without turning every MCP server into an ungoverned production backdoor?

Before we get to the next story, a quick note from this week's sponsor, Guardsquare. If you are building mobile apps, good enough security is usually a problem waiting to happen. Guardsquare focuses on actually protecting your code in addition to scanning it. That means code hardening, runtime protection, testing, and visibility into what's happening once your app is out in the wild.

So if you are responsible for shipping and securing mobile apps, Android or iOS, definitely worth taking a look at guardsquare.com. All right. Back to the show. Third story. Red Hat is pushing Ansible Automation Platform as a trusted execution layer for IT operations in the agentic era. That is a very enterprise sentence. But underneath the marketing language, this is actually a big deal.

Because Ansible is not theoretical. Ansible is already used to patch systems, restart services, configure servers, manage network gear, run operational tasks, and handle a bunch of work that is very close to production reality. So when you connect AI agents to Ansible, you are not just giving an agent a little toy function. You are connecting it to the machinery that already changes real systems.

Red Hat's angle is basically this. Agents may be good at reasoning, planning, or interpreting intent, but enterprises still need a governed, trusted, auditable execution layer. When it is time to actually do something. That is the right framing. Because the dangerous version of agentic operations is not an agent saying, here's the runbook. The dangerous version is the agent saying, I ran the runbook.

And then everyone hoping it did the right thing. Now, to be fair, this is also where something like Ansible can help. Because mature automation gives you structure. You have inventories. You have playbooks. You have idempotency, at least when things are written well. You have logs. You have a known execution path. You have a place to put approval gates.

That is much better than an agent freehanding shell commands on a production box because it read three Confluence pages and felt confident. But the same rules apply here. The agent should not get more authority than the automation deserves. If your existing playbooks are messy, overly broad, poorly scoped, or rely on tribal knowledge, an agent does not magically make them safe. It may just make them easier to invoke.

And that is the part I'd be nervous about. A bad script that an agent can discover and execute through a tool interface is a different class of problem. So the takeaway is not Ansible plus AI is bad. It is actually the opposite. If agentic ops is coming, I'd much rather see agents routed through controlled automation than improvised commands. But teams should treat this as a forcing function.

Clean up your automation. Narrow the blast radius. Split read-only diagnostics from mutating actions. Make destructive playbooks require approval. Add dry-run modes where possible. Make sure the logs clearly say who asked for the action, what agent or system executed it, and what changed.

Because if Ansible becomes the execution layer for agents, the quality of your automation becomes the quality of your agent safety model. Fourth story. OpenAI announced Daybreak, its cybersecurity initiative built around GPT-5.5 and Codex Security. I'm treating this as a follow-up to the Mythos and Project Glasswing episode, not a totally separate story. Because the broader trend is the same.

AI systems are getting better at vulnerability discovery, exploit reasoning, patch generation, and remediation validation. OpenAI describes Daybreak as a way to use AI for cyber defense. The pitch is that it can help identify threats, generate patches, and verify remediation across code and systems. And on one hand, this is exactly what we want. Most organizations are drowning in vulnerability backlog.

They have more findings than time. Some findings are noisy, some are real, some are technically real but not actually reachable. Some are buried in legacy code that nobody wants to touch. And even when the fix is obvious, there is still work. Open the issue. Find the owner. Understand the code path. Patch it. Test it. Get it reviewed. Deploy it. Verify the scanner is happy and hope nothing broke.

So an AI system that can help triage, validate, patch, and verify is genuinely useful. But here's the uncomfortable part. If defenders get this, attackers get some version of it too. Maybe not the same controlled access. Maybe not the same polished product. But the underlying capability trend is not one-sided. That means the bottleneck for security teams shifts. It is no longer just can we find vulnerabilities.

It becomes can we process, prioritize, patch, and safely ship fixes fast enough. And that lands right in the lap of DevOps, SRE, platform, and application teams. Because finding the bug is only step one. The real work is changing the system. And changing the system safely requires all the boring stuff.

Ownership, tests, CI/CD, feature flags, rollback plans, dependency strategy, runtime visibility, asset inventory, patch windows, and enough architectural knowledge to know when the easy fix is actually a trap. This is why I keep coming back to the same point. AI security tooling will probably find more issues. That is good. It will probably also create more pressure. That is complicated.

If your organization already struggles to patch known vulnerabilities, adding AI that finds more of them does not automatically make you safer. It may just make the backlog more honest. So the real question is not can Daybreak find things? The question is, can your engineering system absorb the findings? Can you validate them? Can you prioritize them? Can you patch them? Can you ship them?

Can you prove the fix worked? And can you do all of that without creating a second incident while fixing the first one? That is where this becomes a operations story, not just a security story. Now let's step away from AI for a minute. Because Discord published a really good write-up on how they automate ScyllaDB clusters at scale. And honestly, this is the kind of engineering story that I love.

Discord's persistence infrastructure team runs a lot of ScyllaDB. Over time, they had accumulated Python and shell scripts to help with operations. But those scripts had the usual problems. They were useful. They were also fragile. They were easy to misuse. They relied on humans understanding the right order of operations. And for complex cluster-wide workflows, that becomes a lot of operational risk.

So they built what they call the Scylla control plane. The goal was to safely automate and orchestrate cluster-wide workflows. Things like rolling restarts, replacing nodes, bootstrapping, and doing work that previously required a lot more manual supervision. One of the details that I liked from the write-up is that webhook notifications mattered more than they expected. That sounds small, but it is very real.

There is also a huge difference between babysitting a terminal for two hours and trusting the system to notify you when it needs attention. That's the difference between automation that technically works and automation that actually reduces human load. And that distinction matters. A lot of teams say they have automation, but what they really have is a pile of scripts.

A script can be automation, but it might not be safe automation. Safe automation needs state. It needs preconditions. It needs retries. It needs idempotency. It needs clear failure modes. It needs visibility. It needs a way to resume without making things worse. And it needs to know when to stop. That last one is underrated. Good automation is not automation that blindly completes the task no matter what.

Good automation is automation that knows when the world no longer matches its assumptions. If a node is unhealthy, stop. If the cluster is already degraded, stop. If replication is not where it should be, stop. If the previous step did not converge, stop. That is how you move from script that usually works to operational control plane. And this connects back to the AI stories in a weird way.

Because before we let agents run operational tasks, we need more automation that looks like this. Explicit. Recoverable. Observable. Constrained. Designed around failure. If the future is agents calling tools, then the tools need to be boring, safe, and well-structured. Discord's story is a reminder that the best automation is not magic.

It is just a lot of careful engineering around the parts where humans usually get tired, distracted, or inconsistent. Now let's do a quick lightning round. First, AWS GuardDuty and crypto mining. AWS published a guide on detecting and preventing crypto mining in AWS environments using GuardDuty. This is one of those classic cloud security problems where security, reliability, and cost all run into each other.

A compromised credential does not always turn into a dramatic data breach. Sometimes it turns into a compute bill. Someone gets access. They spin up resources. They run mining workloads. They try to persist. And by the time anyone notices, the incident is both a security problem and a finance problem. The practical question for teams is simple.

If somebody compromised a credential today and started mining in your AWS account, how fast would you know? Would it be GuardDuty? Would it be Cost Anomaly detection? Would it be Datadog? Would it be a budget alert? Would it be a developer asking why their workload is slow? Or would it be Finance two weeks from now forwarding a bill and asking what happened?

That is the difference between having a detection strategy and having a surprise. Next, queues and backpressure. There was a good piece making the point that queues do not absorb load forever. They delay failure. And that is exactly right. Queues are great for smoothing bursts. They are terrible when teams use them to hide sustained overload.

If messages are arriving faster than consumers can process them, the backlog will grow. A bigger queue does not fix that. It just gives you a bigger place to store the problem. Eventually, you hit freshness issues, storage limits, memory pressure, retry storms, customer-facing delay, or some downstream dependency that finally gives up. So the practical takeaway is simple. Monitor queue depth. Monitor message age.

Monitor consumer lag. Have backpressure. Have limits. Know when to shed load. And please, do not call a system resilient just because it has a queue in front of the fire. Last lightning item. Datadog had a nice PostgreSQL performance write-up about inefficient index scans. The short version is that using an index does not automatically mean a query is cheap.

Datadog walked through a production query where the plan used an index scan, but it was still expensive. They changed the indexing strategy and cut average latency from 300 milliseconds to 38 microseconds. That is a ridiculous improvement, and it is a good reminder. You cannot stop at the query uses an index.

You need to understand whether it is using the right index, how many rows it is touching, how selective the predicate is, what the access pattern looks like, and whether the index actually matches the way the query behaves in production. Sometimes the database is not slow. Sometimes your mental model is. The human closer this week is about authority because that is really what all these agent stories come down to.

Not intelligence, not productivity, not whether the model is impressive. Authority. What is this thing allowed to do? What can it read? What can it change? Can it trigger work? Can it authenticate? Can it call tools? Can it run automation? Can it open pull requests? Can it touch production? And maybe the hardest question, who owns what happens next? Because in real operations, ownership is not optional.

If I write a Terraform change and it breaks something, I own that. If I approve a bad pull request, I own that. If I run the playbook against the wrong environment, I own that. AI does not remove that responsibility. It just makes the path to action shorter. And shorter paths to action are great when the guardrails are good. They are terrifying when the guardrails are vibes.

That is where I think a lot of teams are going to struggle. They're going to treat agent adoption like a tooling rollout. Enable the feature, give access, write a quick policy, maybe do a lunch and learn. And then six months later, they will realize that they created a new automation layer that nobody fully owns. That is not a reason to panic. It is a reason to be deliberate. Start small.

Keep agents in proposal mode before execution mode. Treat MCP servers like production APIs. Treat agent tokens like service accounts. Treat agent created pull requests like code written by a junior engineer who is fast, confident, and occasionally very wrong. And before an agent can run a workflow, make sure the workflow itself is worth trusting. Because the future probably is not humans versus agents.

It is humans deciding which agents get authority, where the boundaries are, and what systems are safe enough to let them touch. That is engineering work. And honestly, it is probably some of the most important engineering work we are going to do over the next few years. That's it for this week's Ship It Weekly. We covered GitHub Copilot Cloud Agent Tasks through the REST API. Auth0 bringing identity to MCP servers.

Red Hat connecting Ansible to agentic IT operations. OpenAI Daybreak and the next phase of AI-assisted security. Discord ScyllaDB automation work. And a lightning round on GuardDuty crypto mining detection, queues, and database indexes. If you found this useful, follow the show. Share it with someone who is either excited or mildly terrified by agentic operations. And check out the weekly brief at OnCallBrief .com.

I'm Brian Teller from Teller's Tech. Thanks for listening. And remember, if your AI agent can open a pull request, call an MCP server, authenticate through your identity provider, and trigger Ansible, congratulations. You did not build a chatbot. You built a coworker with API access. Maybe give it a badge. But maybe don't give it production admin on day one.

A lot of the risk in modern infrastructure is no longer hiding in the parts we used to fear most. Sometimes it is a bad kernel bug. Sometimes it is a broken DNS signature at the TLD layer. Sometimes it is a GitOps upgrade that changes behavior under your feet. And sometimes it is an AI agent with way too much authority and not nearly enough guardrails. This week has a little bit of all of that.

And the common thread is pretty simple. Control planes are brittle. Automation is powerful, and identity still decides whether a mistake stays small or becomes a crater. Hey, I'm Brian Teller. I work in DevOps and SRE, and I run Teller's Tech. This is Ship It Weekly, where I filter the noise and focus on what actually changes how we run infrastructure and own reliability. Show notes and links are on shipitweekly .fm.

If the show's been useful, follow it wherever you listen. Ratings help way more than they should. And if you're watching on YouTube, subscribe there too. We've got six main stories today, then the lightning round, and we'll wrap with the human closer. We're starting with the Pocket PocketOS and Cursor incident because it is probably the most instantly gripping story in the set and also one of the most revealing.

Not because AI made a mistake, because an agent got access to something it should not have had in the first place. Then we're going to the .de DNSSEC outage and Cloudflare's response, which is one of those reminders that the internet's deepest layers are still capable of ruining everybody's afternoon, all at once.

After that, Bluesky's outage post- mortem, which is a really good incident story because it is weird, specific, and painfully real. Then Argo CD, with version 3.1.16 being the last 3 .1 release and version 3 .4 .1 bringing a behavior change that people really do need to notice. Then the Linux kernel Copy Fail bug, CVE-2026-31431.

Because active exploitation plus broad Linux exposure is not something ops teams get to casually defer. And finally, Google Cloud Agent Identity and AWS MCP Server GA. Because the cloud platforms are starting to treat AI agents as first -class actors instead of weird sidecars bolted onto existing IAM assumptions. Story one. PocketOS and Cursor is really an identity story. Let's start there.

This week's on -call brief summarized the incident this way. On April 25th, a Cursor AI agent reportedly deleted PocketOS's production database in under 10 seconds after a credential mismatch led it to access an API token it should not have had. The result, according to the reporting cited in the brief, was a full production data loss, including backups.

This is obviously a wild headline, but I think the more important part is not the speed or even the AI branding. It is the access path. Because if an agent can see a destructive credential, then the real problem started before the agent ever took action. That is why I think that this story is more useful than just look at what AI did.

It is really about credential sprawl, environment separation, and what happens when people start granting machine -assisted workflows access that was probably too broad even for a human. An AI agent did not invent bad blast radius design. It just exercised it fast. And that is the practical takeaway I'd want teams to hear.

If you are adding AI coding agents or operational agents into real environments, you do not get to treat identity as a cleanup task for later. The difference between helpful automation and production incident is often just one token, one role, one environment boundary, one missing approval step. So yeah, this story is dramatic. But the lesson is boring. And boring is good. Scope the credential.

Separate staging from prod. Assume the agent will eventually try something dumb, overconfident, or surprising. Because sooner or later, it will. Story 2. The The .de DNSSEC outage is a reminder that internet plumbing still has global blast radius. Next up, the .de outage.

Cloudflare says that on May 5th, at roughly 19:30 UTC, DENIC, the operator for the .de TLD, started publishing incorrect DNSSEC signatures for the .de zone. Any validating resolver that received those records was required by DNSSEC rules to reject them and return SERV- FAIL. Cloudflare says 1.1.1.1 was no exception.

That means a bad cryptographic state at the TLD layer was enough to make huge numbers of .DE domains effectively unreachable. That is a great outage story for ops people because it is one of those failures where everything is technically working exactly as designed and that is the problem. DNSSEC is there to make sure responses are authentic. The signatures were wrong.

Resolvers did the correct thing and refused to trust the data. And suddenly, correct behavior becomes mass breakage. That is such a brutal but important reliability lesson. Security mechanisms do not eliminate failure. They sometimes concentrate it. Cloudflare's write -up is also good because it walks through mitigation trade -offs instead of pretending there's always a clean answer.

When the trust anchor above you is wrong, your choices get ugly fast. Do you keep strict validation and break reachability? Or do you apply a temporary exception and accept the risk of bypassing a protection that exists for a reason? That is not really a DNS -only lesson either. It is a control plane lesson. When a higher order authority goes bad, the downstream systems that trust it do not fail independently.

They fail together. So if I'm taking one operator lesson from this, it is this. Any system built on centralized trust, centralized signing, or centralized metadata needs a very honest failure mode conversation. Because when the thing at the top is wrong, your elegant security chain can turn into synchronized outage machinery very quickly.

Before we get to the next story, a quick note from this week's sponsor, Guardsquare. If you are building mobile apps, good enough security is usually a problem waiting to happen. GuardSquare focuses on actually protecting your code, in addition to scanning it. That means code hardening, runtime protection, testing, and visibility into what's happening once your app is out in the wild.

So if you are responsible for shipping and securing mobile apps, Android or iOS, definitely worth taking a look at guardsquare.com. Alright, back to the show. Story 3. Bluesky's outage post -mortem is the kind of weird incident story worth studying. Now for my favorite kind of reliability story. Blue Sky published a post -mortem for its April outage.

And the post says that the service was intermittently down for about half its users for around eight hours. Jim Calabro's write -up says they saw pretty quickly that they were exhausting ports. But the harder part was figuring out exactly where and why. The root issue involved memcached traffic, ephemeral port exhaustion, and a cascade where the debugging path became part of the failure path.

What makes this one so good is that it is not just we ran out of ports. The postmortem explains that logging used blocking write syscalls, and under the load of trying to log huge volumes of errors while still serving traffic, the Go runtime spawned far more OS threads than normal. BlueSky says that extra thread pressure then hit the garbage collector, contributing to the broader failure cascade.

The write-up also shows a custom dialer workaround that randomizes loopback IPs to avoid ephemeral port exhaustion on a single address after restarts. That is such a real incident pattern. The original fault hurts. The observability path amplifies it. The recovery path gets noisy. And then the system that is supposed to help you reason about the incident starts participating in the incident. That is not hypothetical.

That is production. And I also like the honesty in the write -up. BlueSky basically says the signal Bluesky basically says the signal and that you need the discipline and the metrics to cut through it. That lands because it is true. A lot of outages are not invisible. They are just obscured by too many symptoms arriving at once. So my takeaway here is not just watch ephemeral ports. It is broader than that.

Watch where your debugging strategy becomes a scaling liability. Watch where synchronous behavior hides inside paths you think are harmless. And watch for failure modes where saturation causes your tooling to become part of the blast radius instead of part of the recovery. Story 4. Argo CD is giving people one quiet end-of-life and one not-so-quiet behavior change. Next up, Argo CD.

Argo's version 3.1.16 release is the final release in the 3 .1 series. The release notes are very explicit. As of May 6, 2026, 3 .1 has reached end of life and will no longer receive bug fixes or security updates. The same release notes tell operators to move to a supported version, meaning 3 .2, 3 .3, or 3 .4. That alone is worth a mention because GitOps tools have a way of becoming background furniture.

Teams stop thinking about the controller version because the controller is just there doing its thing. Right up until the day it matters a lot. But then there is the second part. Argo CD version 3 .4 .1 is the first release in the 3 .4 series. And the release notes call out an important change. Following Helm 3 .19 .0, Argo CD now aligns its interpretation of Kubernetes cluster version with Helm's behavior.

OnCallBrief points out that this affects application sets that filter clusters by Kubernetes version. That is exactly the kind of change that looks tiny in a release note and then quietly breaks assumptions in real environments. So for me, this is a classic operator story. Not sexy, not viral, very real. One branch is dead. One branch changes parsing behavior.

And if your GitOps setup depends on version -based selection logic, you need to actually test that logic instead of assuming a minor release means minor consequences. The practical lesson is simple. Treat controller upgrades like control plane changes, not package refreshes. Because that is what they are.

If the thing deciding what gets deployed, where, and when changes how it interprets your environment, That is production behavior, not housekeeping. Story 5. Copy fail Story 5. Copy Fail is the kind of kernel bug Now to CVE-2026-31431, also known as Copy Fail. NVD shows that this Linux kernel vulnerability is in CISA's Known Exploited Vulnerabilities catalog.

And CISA's entry gives federal agencies a remediation due date of May 15th. The flaw is an incorrect resource transfer between spheres issue in the Linux kernel. And the reporting around it says it enables local privilege escalation to root on a wide range of Linux distributions. AWS's security bulletin says that with the exception of specifically listed services, most AWS customers are not affected.

But it also lists update timelines for affected services such as Bottlerocket, ECS on EC2, EKS optimized AMIs, and EMR. So this is one of those stories where not everybody is affected should not turn into nobody on our side is affected. You still have to inventory what you run. That is the operational lesson here.

When a kernel LPE lands in KEV and there is an active exploitation, this is not the time for vague patch queues. This is the time for concrete exposure review. Which hosts? Which images? Which managed services? Which self -managed nodes? Which maintenance windows? Which compensating controls until patched?

And honestly, the thing I always come back to with stories like this is that kernel issues collapse abstraction layers really fast. You can have immaculate Kubernetes policy, good IAM, strong workload boundaries, and still get wrecked if the shared kernel underneath is vulnerable and unpatched. That is why these bugs matter.

They turn every higher -level control into a best -effort suggestion until the underlying system is fixed. Story 6. Google and AWS are both telling us agents need first -class infrastructure identity now. Last main story. I wanted to put Google Cloud Agent Identity and AWS MCP Server GA together. Because they are different products, but they point in the same direction.

Google Cloud's new IAM post says that the AI era needs a different security and governance model for autonomous agents. And it introduces agent identity as a new first -class principal type, distinct from human identities and generic service accounts.

Google says these identities are built on SPIFFE, are cryptographically protected, and strongly attested, and can be used to authenticate to MCP servers, cloud resources, endpoints, and other agents. It also ties this into agent gateway policy enforcement, least privilege controls, and runtime defense. AWS's side of the story is different in implementation, but similar in intent.

AWS says that the AWS MCP server is now generally available and gives AI agents and coding assistants secure, authenticated access to AWS services through a small fixed set of tools using existing IAM credentials. The official announcement frames it directly around the problem people keep asking. How do you give an agent real AWS access without just handing it keys to the kingdom?

That is why I think these stories belong together. Both clouds are basically admitting the same thing. Agents are not side features anymore. They are becoming infrastructure actors. And once that is true, the identity model has to change too. Not just more service accounts, not just more static tokens, not just vibes and trust. Real principals, real boundaries, real policy, real auditability.

So my read is that this is where cloud identity is headed next. Human identity was phase one. Workload identity was phase two. Agent identity is shaping up to be phase three. And if teams do not start treating that as real infrastructure design now, they're going to rediscover all of the old machine identity mistakes with an LLM in the loop and a much faster failure path. A few quick ones before we wrap.

Cilium published lessons from securing CI/CD for an open source project. And the reason that I like it is because it is practical. On -call brief highlights controls like tighter CI/CD security practices and lessons learned from operating a real open source pipeline. That is worth a read for anyone treating GitHub actions hardening like a someday project.

Velero version 1.18 .1-rc.2 includes a security fix for CVE-2026 -27141 by bumping golang.org/x/net to version 0.51.0. That is the kind of small release note that matters a lot when you realize it closes a known vuln in a tool people trust for recovery. Google's release notes also carry a couple of quieter but real operational changes.

Google Ads and related measurement APIs moved to a 37 -month retention policy for granular performance stats starting June 1st. And Google's distributed cloud's Kubernetes 1.35 platform update now requires cgroups v2 with cgroups v1 no longer supported for creation or upgrades. Those are not flashy changes, but both can absolutely break assumptions if nobody notices them.

AWS also published a cross -region disaster recovery walkthrough for Amazon EKS using AWS Backup. The post walks through creating backup vaults in source and DR regions, running on -demand backups, and initiating cross -region copies. It is not a glamorous story, but it is the kind of thing that people say they care about right up until they realize they have never actually rehearsed it.

I think the human thread underneath this week's episode is that modern reliability work keeps getting squeezed between two kinds The second kind feels newer: AI agents with too much authority, new cloud identity models that quietly become control planes, build and operational systems that act like sidecars until the day they absolutely are not. And what is tricky is that these two categories are not separate anymore.

The old failures now happen inside environments shaped by the newer automation. The newer automation inherits the blast radius of the old infrastructure. And ops teams end up responsible for both at the same time. So I think the real takeaway this week is pretty simple. Reliability is not just about uptime anymore. It is about authority. Who and what can act? What it can touch? How fast it can fail?

And whether your system design assumes mistakes will stay local when they almost never do. That sounds abstract until you line up the stories. A TLD signs bad data and millions of lookups fail. A social platform runs into port exhaustion and logging makes it worse. A GitOps controller hits end of life while version parsing changes in the next branch. A kernel bug drops into active exploitation.

An AI agent sees the wrong token and production disappears. Different systems, same lesson. Small authority problems turn into large reliability problems very quickly. All right, that's it for this week of Ship It Weekly. Quick recap. The Pocket OS and Cursor database wipe and why it is really an identity story. The .de DNSSEC outage and what happens when the trust chain itself goes wrong.

Bluesky's outage post-mortem and how observability can become part of the incident path. Argo CD 3 .1 going end of life while 3 .4 changes behavior. Copy fail and why active kernel exploitation still cuts through all the higher level abstractions. And Google Cloud agent identity plus AWS MCP server GA. Because the cloud providers are starting to formalize agents as real infrastructure actors.

Then in the lightning round, Cilium's CICD security lessons. Velero's CVE fix. Google's retention and cgroups v2 changes. And AWS's cross -region EKS disaster recovery. Links and show notes are on shipitweekly .fm. You can also find the video versions on YouTube. And if you want the source stack before the episode lands, check out this week's on -call brief.

If this episode was useful, follow or subscribe wherever you listen. And send it to the person on your team who still has to explain that reliability problems are not just outages anymore. Sometimes they are identity problems, tooling problems, and authority problems wearing outage clothes. I'm Brian and I'll see you next week.

AI is making it easier than ever to write code. That sounds great. Until you are the team responsible for everything that has to exist after the code gets written. The infrastructure. The deploy path. The secrets. The permissions. The cost. The rollback plan. The weird Terraform change someone generated at 4:30 on a Friday and swears is probably fine.

Because when app teams move faster, platform and infrastructure teams do not magically get less work. They get more change, more pressure, more risk, and a lot more questions about whether the systems around delivery can actually keep up. That is really what this conversation is about.

Not just AI, not just Terraform, not just infrastructure as code, but what happens when software delivery speeds up and the people running the infrastructure have to figure out how to keep things safe, reliable, governed, and still usable. Hey, I'm Brian Teller. I work in DevOps and SRE, and I run Teller's Tech.

Ship It Weekly is where I filter the noise and focus on what actually matters when you are the one running infrastructure and owning reliability. Most weeks, it's a quick news recap. In between those, I do conversation episodes with people who are building platforms, running infrastructure, organizing events, and thinking through where this whole industry is actually headed. Today is one of those conversations.

I'm joined by Gareth Kersey to preview IACConf 2026. IACConf is a free virtual conference focused on infrastructure as code, platform engineering, DevOps, and infrastructure operations. And the theme this year is basically how infrastructure teams keep pace when AI is changing the speed of software delivery. And I like that framing because it gets past the lazy version of the AI conversation.

This is not just can AI write Terraform. It probably can. Sometimes badly. Sometimes usefully. Sometimes like a tourist who learned just enough of a language to confidently order the wrong thing. The better question is, what happens after that? What happens when developers can generate more code, ship more changes, and push product roadmaps faster?

What does that mean for DevOps, SRE, platform, and infrastructure teams who still have to deal with the blast radius.

In this conversation, Gareth walks through the IACConf agenda, including Corey Quinn's keynote, sessions on AI and infrastructure operations, platform engineering panels, Kubernetes, Argo CD, AI agents managing infrastructure as code, governance, policy, and the risk of 10x code velocity turning into 10x operational risk.

We also talk about how IACConf has grown, why people seem hungry for an event that goes deeper on infrastructure as code, and how they are trying to keep it community focused instead of turning into just another vendor marketing conference. That part stood out to me because infrastructure as code is one of those topics that sits at the intersection of almost everything now.

Terraform, OpenTofu, Pulumi, Crossplane, Kubernetes, GitOps, Policy, Security, Cost. Developer experience, internal platforms, and now AI-generated changes being pushed into systems that were already complicated enough before the robots showed up.

So if you work around infrastructure, platform engineering, DevOps, SRE, or you are just trying to figure out how to keep your delivery system sane while everything around it speeds up, this one should be worth your time. All right, let's jump in. Today, I'm joined by Gareth Kersey from IACConf.

We're doing a fast preview of IACConf 2026, what the theme is, what talks are worth bookmarking, and what infra teams should be doing now that AI is changing the pace of shipping. Gareth, thank you for joining me. Thanks, Brian. Excited to be here. So I'm interested to hear, what does keeping pace mean?

Yeah, so it's not a surprise probably to anyone with whether you're in engineering, You're in any sort of field operations function. What AI is doing to kind of everyone's day -to -day function, right? Everyone's feeling the pressure to move faster, do more with less. And I think one of the biggest places that has been upended and really drastically changed is software engineering, right?

The rate at which you can spin up an application, you can code with cloud code.

What does that actually mean on the other side for operations teams well now you've got developers are shipping way faster that product roadmap is getting through a lot faster but that application code isn't always going to be the same maybe level of quality or the thoroughness that are security checks what does that mean for infrastructure all the kind of the downstream operations that are needed to actually sustain that application deployment is that actually keeping pace so that's what we mean by keeping pace and obviously with it being infrastructure as code conference, IACConf.

We're really focused on the infrastructure operations components of that. But we do have a lot of sessions that touch on more than just infrastructure as code, because realistically, whether you're a DevOps, SRE platform, IAC is one piece of the puzzle that you're working with. And so we've got several talks that talk a little more broadly about keeping pace in terms of like the app operations standpoint.

So that's a good precursor. What are the talks? Let's kind of dive into that. Yeah. I saw some of them. You had mentioned some of the speakers on the IACConf page. Can you give me just a high -level overview? Who is speaking? What are they speaking about? Yeah, for sure. Yeah, we've got a really good agenda this year.

It's a mix of, you know, a couple panels, a couple talks that are more, you know, framework, point of view on the market, and then a few talks which are down in the weeds, more demos. You know, so we've tried to create a nice array of topics. So this is, let's see, we did IACConf, the first one in May of 2025. That was 13 sessions, really great response, a lot of good feedback.

Then we did a security focused virtual spotlight in August. And then we did an AI focused one. And every time we do it, we collect feedback on what people want. And people like to have the in -talk or the in -depth demos, but they also, you know, like to hear the panel discussions and people just talking about what they're building. So we've tried to create that variety.

So I can kind of give you some of like the high level agenda overview and and kind of what these these different talks are shaking out, shaking out as. So first, we've got Corey Quinn. He's running the keynote. If you're involved with Amazon Web Services, you know, AWS, you probably know Corey Quinn. I think I came across him first, maybe like 2020. And then I saw him do the AWS reinvent like vendor crawl.

And it was just like he was like dressed up as a zookeeper, like going across these different vendors. And it was just hilarious. So his talk is what he's labeling AI speaks Terraform like a tourist. Really, really excited for him to kind of kick it off. When we were talking about his keynote, you know, there's no right answer in any of this is what it comes down to, right?

Whether AI writing Terraform is going to help in some areas, whether going completely away from IC. In general for some of these things and having AI just directly provisioned infrastructure and everywhere in between, the keynote is meant to kind of set the stage of, hey, there's a lot of places that you can go to. And obviously it has a little bit of Corey's humor built into it. Awesome.

So from there, we've got Matt Gowie, who's a founder and CEO of a consulting company called MasterPoint. They're a partner of Spacelift. He's got a great perspective on the industry. He has a lot of clients. So he's talking about. Kind of how AI is upending infrastructure operations. We're running two tracks at different times of the day. We're starting off with two tracks after Corey's keynote.

And we've got two folks, Emin Elmedar and Flavius Danu, who are practitioners. They've worked at Spacelift, but they're going to talk about how to get started pretty easily with infrastructure as code with an...

Ai flavor and they're actually going to run into an open source project called intent about how you can actually stand up very quickly some ai provisioning both with iac and then you know an ai driven workflow path from there we're going to jump into anta babenko who if you all know tf weekly he's uh you know a big voice in the terraform infrastructure's code space and he does a lot with terraform modules that's what he's known for producing these modules that everyone loves and goes to repeatedly so he's got a session specifically around replacing Terraform module forks with policy transformation rules.

We then jump into one of the first panel of the day. We've got two panels, and this one is really interesting. So it's going to be moderated by Luca, who is over at Platform Engineering, or if you know PlatformCon, Luca Galante is all things platform engineering. And we've got three folks on that panel. The first is Chris Haas, who is the CTO of Mondelez. So Mondelez is the massive food.

A manufacturer i think they own cad berries and all different sorts of uh confectionaries so he started as a director of platform engineering and he's made his way up and now he's the cto of the organization so great in -house expertise we've got an individual fasal who is the principal of platform engineering for a services firm called ahead so he's got a lot of experience going talking to all the services partners and his clients and then we've got Eric Maxwell, who's actually a co -author of DORA and works at Google.

So that panel is going to be really interesting perspectives from three different individuals all about platform engineering and what does it actually mean with now AI agents being consumers of your platform. So then we get into some more technical sessions. We run this on two tracks. We call the main stage and the builder stage.

On the main stage, we've got Amin talking about when 10x code velocity could mean 10x operational risks. So that idea of application. Code development is rapidly increasing, but what does it actually mean for your downstream operations? And then Davlet from Cloud Genie is going to be talking about how to safely deploy AI agents that write and manage your IAC.

On the build track, we've got a couple of interesting talks there. So one is this gentleman by the name of Joseph who... The title of this session is really interesting. I deleted 4 ,000 lines of AWS CDK and shipped faster. So he's going to get down into the details of a specific scenario he ran into. We actually simplified his AWS deployment, was able to go faster. So again, keep in pace.

And then Atesh Main from Philo is going to talk about infrastructure at scale using Argo CD. Kubernetes deployments, we're talking about what might be some future topics for IACConf. A lot of Kubernetes interest. Maybe not surprisingly, in a lot of session submissions and ones we've done in the past. So I think that's going to be a popular session. Around 2 .30 Eastern, we go into our second panel of the day.

And so this panel is actually a combination of different tech founders from a couple of different vendors in the DevOps platform engineering space. So we've got Marcin Wyszynski, who's the co -founder and head of R &D at Spacelift. We have Chris Evans, who's the field CTO and co -founder of Incident .io. And then we've got Ganesh Dutta, who is the CTO and co -founder of Cortex.

And what they're going to be talking about is... Yeah, we're seeing the, this is working for our customers. These are the people who are actually talking to people every day, building products that DevOps platform teams are using to try to keep that demand and how they're thinking about building those products.

I think those would be really interesting for maybe those platform engineering teams who are thinking about almost... Internal product development for their teams. What does that actually look like inside one of these vendors? This is also going to be moderated by a woman named Serena, who used to be the head of JP Morgan and Chase product developer platform from a UI UX standpoint.

So she's got some really interesting experience. She used to be at Red Hat on the product UI UX teams. That's going to be a very, I would say, product -centric panel thinking about how do you build this platform. For this 10x velocity and all the things that, again, they're hearing from customers, which I'm sure there'll be a lot of customers of any one or maybe a combination of those products in the conference.

Very cool. At three o 'clock, we're going to move over to, again, two tracks. So Dimitri Vlachos, who's the... CMO of Spacelift. This is our second year running the infrastructure automation report. And we surveyed 400 individuals across platform and DevOps engineering functions and asked them about their adoption of AI. How's it going?

What's the adoption rate of AI among their development teams versus their operations, specifically infrastructure operations? And what does that gap look like? There's some really interesting findings there. Drop a little bit of a hint.

Some of the biggest gaps are around the governance around those AI deployments and what does that actually mean in terms of productivity and time kept back of having to go back and, you know, check what's been done because of lack of guardrails that are in place. Alexander is going to run a session. This is very much a build session of something he's built around AI enforced architecture.

And then we're going to close out the day with John, who is a principal engineer at Sanofi, the large pharmaceutical company. And he's going to kind of round out the day about talking about. Platform engineering in general and how to do that without slowing developers down. Again, rounding out the theme around keeping pace and what does that mean in 2026? Interesting.

So I've seen like over the last year, IACConf has grown, especially with this next upcoming conference. And there's a lot of that just from feedback. Like you mentioned, we're doing two quote unquote stages now. There are panel discussions and what has caused like the changes and where do you see it going long term?

On the first one we did, which was in May of 2025, we were optimistic, but the response rate was more than we anticipated in terms of the registrations, the attendance. But then also after the event, we surveyed everybody. Hey, what did you like? What do you want to see more of? What suggestions do you have? And the response was overwhelmingly positive.

From the logistics standpoint, even people saying this is the type of event I would actually travel to because there was a... I mean, there still is a gap in the market in terms of something that's specific to infrastructure as code as a conference. You have your KubeCons, you've got, you know, the AWS Summits and the Reinvents, Google Next. Those are all great conferences, but they're quite broad, right?

And so this filled a gap for, you know, what is the niche of infrastructure as code? And I think with the content that we curated, it hit the mark. And so the feedback that we heard was... Well, one, keep it up. This is exactly what we want more of. People want stories. They want demos. They want real world scenarios.

I think there's always the balance of being able to have that kind of framework discussion of here's what good looks like versus like the actual real world scenario. Yes. And here's how I applied that in my actual environments. And here's how you can go on Monday.

Turn around and start doing it yourself so that's the biggest thing that we've tried to do every one of these events we run we go and survey the audience and say what did you like what did you not like what do you want to see more of and try to do that and we've also done that with trying to run these in person we've done two so far we did the first at cubecon when it was in atlanta in november and then we did a meetup in amsterdam back in march because The other half of it is people just want to connect, network, meet other people who are working with infrastructure as code and build their own network.

So I think that's the extension of what we want to make this more of is, yes, the primary is an events platform, allowing people to have a platform to share their story, share their knowledge, a place for people to come learn, but then also a platform for people to go connect with others who are working with infrastructure as code and whether that takes the shape of. The next place we've done was in -person events.

You know, from here, we want to build a content board, a jobs listing board. There was a lot of comments around, hey, we could have a Slack community or some sort of community where people can go and have real -life chat.

And so that's the objective is to keep building it, allow people to connect, still have the events platforms where people can share their stories and learn from each other, but create more of those forums, whether they're in -person or virtual, for people to actually connect with each other.

Do you see the cadence keeping up with that pace as far as like what one or two virtuals per year and then one or two in -persons or? Yeah. So we did Amsterdam. So for in -persons, we did Amsterdam. We're going to do Salt Lake City, which is, I think it's November. A lot of demand for doing something at reInvent.

Vegas is a bit crazy at reInvent, but it would be great to try to put something together there, even if it's a networking meetup style or a happy hour style. We have discussed some ideas of doing some more regional city -based events throughout the year, perhaps partnering with a local DevOps meetup or AWS has a lot of user groups. So I'd love to explore that. I'm in Boston.

Talking about trying to do that shortly after IACConf 2026. In terms of the virtual cadence, so yeah, we'll definitely keep with what we call Spotlight Series. So we have this annual event, 13 sessions. It runs from 11 to 4:30 Eastern. But then we did two of these spotlights last year. So one was security. So this intersection of infrastructure as code and security, that was three sessions, and that was August.

And then we did one on AI in January. Again, three sessions. And those went really well. So I definitely think there's more of those we can do. I mentioned Kubernetes being a really hot topic. Definitely think we can do a session based off of the submissions we've got and the interest we have on Kubernetes and that intersection of infrastructure and Kubernetes. Open source is another topic that's come up a lot.

And then there's, again, there's a lot of like those intersections we could have of infrastructure and going deep in one specific area. And then we've also talked about, we get a lot of submissions. I think we got 50 submissions for IAC Conf 2026, and we can only take so many. I'd love to be able to do one -off events. Hey, this is a submission we didn't get to.

We're going to run it on a Thursday in June and just give somebody that platform to share their story. You know, it helps them build their credibility. It's something they can add to their resume. They can add to their LinkedIn profile. This is a presentation I've done, but it also gives the community another.

Story they can go and listen to and think about how they can take that to their environment yeah for sure and so it is a free conference to be clear too so it's free yeah is it recorded so like let's say i couldn't attend can i see those talks after the fact yeah yep so we have a youtube channel so we post everything there typically within a week we'll get things posted up so yeah if you If you miss it, you can go catch it on YouTube again a week, 10 days later.

If you're also not sure you can attend, register because as soon as we do send an email out a few hours after the event, and then you can go access the events platform we use and you can go browse any of the sessions before they get to YouTube. So yeah, even if you're not sure you can attend, register and you can go access the platform immediately afterwards.

So given all the submissions that you have gotten, that maybe you weren't able to get to this time around. Have you seen any trend? Like, is there any industry trends around AI or around Kubernetes specifically that you've seen with these submissions? I have to go back and look a little bit.

I mean, so it was on this IACConf 2026, there was a pretty widespread of submissions around, again, this idea of keeping pace and what does that mean? We had some that were more keeping pace from the application side. And ways that they're thinking about the application development.

And I think there's ways to actually create some, there could be some interesting discussions around bringing people who are more focused on the application side and then someone else on the operations side. And maybe it's a team who's worked together to figure that out really well. On the Kubernetes, well, one, Kubernetes is such a broad topic.

I think people were, last year, Crossplane was one of the most, I would say, sought -after sessions we had at IACConf. Now, if you actually looked at the chatter and the questions that were coming up in that, and actually even the presentation itself, no one was super confident on Crossplane. Now, it doesn't mean, but people are still interested in how can I make Kubernetes work for...

My infrastructure and what does that intersection look like? So the demand is certainly there. Obviously, Kubernetes is not going away. We've got conferences dedicated to it. So there's plenty of people out there looking to learn and explore and see what's possible. I think my overarching, I guess, overarching takeaway in looking at the submissions is there's no right answer. People are still figuring this out.

There's a lot of experimenting. I think what's going to be really interesting from this is where the conversation kind of...

Centers around because the chat becomes really interesting during these sessions where people a speaker says something and then it seems to spark a conversation in the chat and you can like almost you know see through the screen people's like light bulbs going off like yes I get that that makes sense and there were certain sessions last year that triggered that and given that this year's topic is a bit more focused on this keeping pace and well just the environment May 2025 to May 2026.

The world of software engineering and what that means for operations has changed dramatically, right? So I think it'll be really interesting to see kind of what those spark moments are in some of these talks, because that will be a good indication for us. And we'll definitely publish it once we do that, because we did this last year.

We can see all the data on the back end, like what those topics were and what sparks those, what are the questions asked? So we get to actually kind of get some sort of an idea of what the themes are. We'll do a write -up and we'll definitely publish that once we have it, because I think. It'll be interesting for a lot of folks to read that.

So like a report around like the analytics, I guess, of what you're able to glean from the conference itself, like from the participants? Yeah, like what was the most attended session? Like what was the highest interest rates of the content? Yeah. Yeah. That sounds interesting. I'd be curious to read it for sure.

I mean, reading like the DORA report every year, it's very interesting to see just the state of the industry, where it's headed, especially over the last few years. As you've said already, AI has kind of changed everything for everybody. Yeah. There's been a real shift. Did you have Crossplane talks previously? Or you just had submissions around Crossplane? We had a Crossplane talk in IACConf 2025. Okay.

And how does that jive with Spacelift, your IC platform? Yeah. It's kind of like promoting a separate product in a way? I don't know. Yeah, yeah. So I'm on the Spacelift team. I'm employed by Spacelift. I'm on the marketing team. But we have been very deliberate in IACConf. Spacelift funds it, right? But we do honestly try to keep it separate. So that means we open up the CFPs.

We want to get people in talking about things. What are the popular topics? Crossplane was a popular topic. It was a great submission. And so... For the better of the community, that's what people want to hear about. It doesn't really matter if that, you know, is a competitor of Spacelift. In the end, it's not really a competitor of Spacelift.

You know, I think it's more of a, maybe a cultural difference of the team adopting it, right? If I want to go all in on Kubernetes versus I want to go all in on traditional, you know, terraforming GitOps approach, well, that's a separation once you make that decision. That's a bit of a competing, maybe philosophical, is that the right word? I don't know. Methodology of how you actually want to deploy infrastructure.

But even if it wasn't that, you know, GitOps Terraform space, again, if people, the community wants to hear about certain topics, that's what we want to create IOC Conf for. I've even thought about, it would be really interesting to have like a tacos panel, like get some of the founders from all these different platforms and have them talk about what they think the future is of, you know.

Terraform orchestration platforms. The IACConf is just the platform to make those conversations happen. It's great to hear. I mean, so I've been to a lot of like vendor sponsored conferences and sometimes they lean too heavily in marketing. It's nice to hear that IACConf is more interested in relaying the technologies that matter to the practitioners that are attending over trying to. Sponsor Spacelift as a product.

I mean, that's not the only motivating factor. Like I said, I don't want to name names, but I've been to other vendor -led conferences that were very heavy into that vendor ecosystem at all costs, which can be detrimental to the overall conference itself. So it's good to hear. Yeah, we have our own set of, you know, Spacelift events and sessions where we can talk Spacelift, but that's not what IACConf is for.

It's a platform for the community and, you know, Spacelift just helps fund it. So if there was, I had time to watch one talk, is there one or top three talks that you would say ahead of time that I should maybe check out? Let's say I was very busy. I just didn't have time. I don't want to put you on the spot. If you don't, if you don't want to name one, that's fine too.

But yeah, well, I haven't, all the, all like the first round drafts of presentations are due in like later this week. So I actually haven't seen anything yet, but I mean, there's a few talks that I think are going to be interesting. I don't know if I can like give you an order, but well, like Corey.

Again, if you've seen any of his videos, his sense of humor and his approach just talking about AWS and all things infrastructure is always entertaining. So I'm pretty excited about that. I'm really excited about this gentleman, Amin, whose session is when 10x code velocity could mean 10x operational risk. If you were to put a session, not so much focused on infrastructure, but on this keeping pace topic.

That's exactly what his session is about. And what does that actually mean for all the downstream operations teams? So I think that one will be really interesting. The panel in the agenda is the Platform Engineering Experts panel with Luca, Eric Maxwell from DORA, Frisal from AHEAD, and Chris from Mondelēz. I think that's going to be really interesting because you've got...

Three very different but aligned perspectives. You've got someone who's in -house, started as a director of platform engineering, now the CTO. You've got someone at a major services partner who's working with a lot of clients. And you've got someone at Google who co -authored Dora. So I think there's just going to be a lot of really interesting perspectives on that.

And then the one that I think is, you know, his background looks really interesting. The submission also looks really interesting is Alexandra from Anovo, AI -enforced architecture fitness functions at scale. On the builder's track, a bit more technical, a bit more down in the weeds. That one's at just around three o 'clock Eastern. I think that'll be a really interesting session that'll get some good engagement.

Very cool. So for anyone looking to register, where should they, how do they check you out? Yeah. So IACconf .com is the main website. We've got, there's an event section and you can see within there, it's the featured event. You can also find us on LinkedIn. We've got an IACconf profile page. We're releasing every few days, featuring a new speaker.

And so those are probably the two best places to go get some more information. Awesome. Great to hear it. Anything else you'd like to leave our listeners with, Gareth? Yeah, I'd say the reason that... IAC Conf has grown to what it has is by listening to feedback from the community.

So if infrastructure as code is of interest to you, you're either trying to learn it, use it daily, or you lead IEC initiatives at your organization, register, get on the list, attend, and let us know what you think. Again, we're building this community. For this audience. And so the best way to help us build is to let us know what you like, what you don't like, when you want to hear more of.

So this is a really exciting project. We want to do more of these events. And so, yeah, please join us on May 14th. I'm looking forward to it personally. Everyone else, please check it out as well. Gareth, thank you so much for coming on. Really appreciate your time. Thanks, Brian. All right. That was my conversation with Gareth Kersey previewing IACConf 2026.

My biggest takeaway from this one is that keeping pace sounds simple until you actually unpack it. Because for infrastructure and platform teams, keeping pace does not just mean moving faster. It means absorbing faster change without turning your platform into a junk drawer. It means giving developers better paths to production without giving up governance.

It means figuring out where AI helps, where it creates risk, and where human judgment still needs to sit in the loop. And honestly, that feels like the real infrastructure conversation right now. Not “AI is amazing.” Not “AI is useless.” More like, okay, the code is coming faster now. The pull requests are coming faster. The experiments are coming faster. The platform requests are coming faster.

So what has to be true for infrastructure teams to handle that without becoming the bottleneck or the cleanup crew? That is where infrastructure as code still matters. That is where policy matters. That is where reusable patterns matter. That is where platform engineering matters.

And that is where events like IACConf are interesting because they are bringing together people who are actually dealing with this stuff from different angles. Practitioners, platform teams, vendors, consultants, open source folks, people deep in Terraform, Kubernetes, GitOps, governance, and AI -driven workflows. I also appreciated Gareth being pretty clear that IACConf is trying to be a community event.

Not just a vendor event with a nicer logo. Spacelift helps fund it, but the topics are broader than Spacelift. They have had Crossplane talks. They are talking Kubernetes. They are talking Argo CD. They are talking platform engineering, DORA, incident management, governance, and the bigger operational impact of AI speeding up software delivery.

That matters because practitioners can smell a thinly disguised product pitch. From a mile away. And this seems more like a useful place for infrastructure people to compare notes on where the work is actually going. IACConf 2026 is free, virtual, and happening on May 14th.

Gareth mentioned that even if you cannot attend live, it is still worth registering because the sessions will be available afterward through the event platform and later on YouTube. I'll put the link in the show notes. If you enjoyed this episode, follow Ship It Weekly wherever you listen to podcasts. If you want the show notes and links from this conversation, head over to shipitweekly.fm.

Thanks for listening, and I'll see you later this week.