Back to all episodes
Ship It Conversations: The AI Agent That Ran Terraform Apply, With Davlet Dzhakishev of Cloudgeni on Read-Only Cloud Access, Sandboxes Beyond Docker, Agentic PR Reviews, the Real Cost of Running Agents, FinOps for AI, and Permissions Over Prompts
devopsSREplatform engineeringterraformDevOps learningagentic AIAI agentsInfrastructure as CodeFinOpsCloudgeni
Explore topics
In this episode, Davlet Dzhakishev discusses the challenges of integrating AI agents into cloud infrastructure, emphasizing the importance of guardrails in architecture.
Ship It Conversations: The AI Agent That Ran Terraform Apply, With Davlet Dzhakishev of Cloudgeni on Read-Only Cloud Access, Sandboxes Beyond Docker, Agentic PR Reviews, the Real Cost of Running Agents, FinOps for AI, and Permissions Over Prompts
Ship It Weekly
0:00
Current chapter: Hook
37:24
Chapters
Jump to a section in this episode.
Speed & share
Transcript
When you valet your car, you don't hand over
your key ring. You hand over the valet key. It
starts the car, but it won't open the trunk.
That's how an AI agent should get into your cloud.
My guest today builds AI agents for cloud infrastructure.
Early on, one of his agents hit a snag on a simple
Terraform change and decided the fix was to run
apply on staging. Resources got recreated. He
calls that his wake-up call. Now his agents
start with read-only access. Every change goes
through a PR and trust grows from there. I'm
Brian Teller from Teller's Tech and this is Ship It Weekly.
Welcome back to Ship It Weekly, where
I filter the noise and focus on what matters.
when you are the one running infrastructure and
owning reliability. Most weeks, it's a quick
DevOps, SRE, platform, cloud, and security news
recap. In between those, I do conversation episodes
with people building and operating these systems.
Ship It Weekly turns one this month. To celebrate
through mid-December, you are getting two conversations
a week, Mondays and Wednesdays, with the news every Friday.
Then we're back to our usual rhythm. Today, I'm
joined by Davlet, CEO and co-founder of Cloudgeni,
which builds AI agents for cloud infrastructure.
Davlet got into this after grinding through SOC
2 and ISO compliance prep, mostly by hand, and
asking what it would take for agents to fix those
problems reliably without making things worse.
We talk about the Terraform mistake that made
guardrails the first problem to solve. Using
agents to review infrastructure PRs against your
own standards. Why running your own agents can
cost a lot more than your AI subscription suggests.
And where a platform team should start without
building a monster. All right, let's jump in.
Davlet, thank you for joining me. Yeah, thanks,
Brian, for inviting us to your studio. Great
to be here. I appreciate it. So I guess let's
start with compliance, or at least like the risky
parts. So everyone wants to do AI agents, right?
That's the big thing right now with cloud and
infrastructure work. What needs to be solved
before that's a good idea? Yeah, that's a great
question, Brian. And this is basically the question
I had in my mind two years ago out of another
project I was working before that where I had
to go through, you know, still pretty much manual
process of going through compliance requirements
such as SOC 2 and ISO 27001 to be able to prepare
our company to service a Fortune 500 customer.
And even though some... the AI models were already
available there, GPT 3.5 and similar, they still
haven't been sufficient enough to help you in
a meaningful way. So that's where my question
popped in my mind. What do we need to build?
What kind of harness or agentic workflows, guardrails,
what do we need to help the agents, first of
all, to be able to identify and solve compliance
problems and solve them reliably. And in the
process of using agents, that we actually don't
make things worse as well. And this is the second
part that was interesting as well, because as
we started using agents more and more, they became
generally more useful, but they also became another
layer of potential vulnerabilities and breaches.
So this is, it kind of brought both solution,
but also... new set of problems to have to think
about and solve it. So concretely, what was the
first thing that we had in mind? Okay, let's
use all the data that we already have. There
is a lot of tools, open source commercial tools
that already scan all kinds of your assets from
infrastructure to your application code to maybe
organizational policies for security and compliance
findings. Everyone has them, thousands of alerts
in their hyperscalers and dashboards everywhere.
What do we do with those, right? Takes a lot
of time to plan for, takes a lot of time to go
through the backlog. And this was our thought.
Okay, how about we build an agentic workflow that
will use this deterministic data and understanding
of your... code, of your infrastructure as a
code, of your policies, and find the correct
solution to those findings. And we'll bring it
to you in the best possible way. And this goal
of finding the solution brought us finding out,
okay, how can we build those agents in a way
that you can use them in production? Because
we also encountered a lot of problems while doing
that. Yeah, some real stories. Luckily, our agents
didn't cause any production problems because
we were careful and we were thinking about in
advance before, you know, you had those headlines
where people were deleting their database and
things like that. But we had a similar incident
where an agent in the agentic mode, and that
is important to note, right? Because there is
different ways you can use AI agents. You can
use them as just helpers or just... You know,
you ask a question, you get the code back, or
you actually allow them to do the whole work
in iterative loop, right, until it actually completes
the goal that you set to it. And this is what
we call fully, you know, or not fully, but at
least autonomous mode or agentic mode. And in
this sort of mode, there is a higher risk that
things can go sideways. And so one example of
what happened is that agent was solving, you
know, simple... Simple Terraform problem, right?
Updating Terraform to update some environments.
Things didn't work exactly like it was supposed
to. And they decided that, okay, maybe applying
the changes would be a good idea, right? And
that was, you know, two years ago when we were
still early on on the guardrails. So it went
on and applied some changes on staging environment
that led to some resources being recreated, because naturally
apply can be destructive, right? So this was
the sort of wake-up call for me, where I understood,
okay, this is the more important problem that
we need to solve first before we can actually
allow agents to touch our infrastructure. And
so that's where we changed our platform and started
building all these guardrails on top of it as
the first class citizens. So what kind of guardrails?
Are we talking like OPA, Kyverno? Or are we talking
like pre prompts? Are we talking validating output?
What kind of guardrails do you have in place?
All of it. And we have a good open source guide,
infrastructure agents guide on GitHub that goes
through all the levels. But basically there is
around 13 or let's say 10 realistic levels that
you need to think through when using or designing.
Your system for agentic use, right? And it all
starts with architecture, right? How do you architect
actually the whole way of the system, the agentic
system, right? And in those levels, you decide,
okay, what kind of agents you're going to use
and how do you want to orchestrate them? Where
are you going to source them? Because that also
brings certain kinds of questions about... you
know, data privacy, you know, depending on where
you source them. Then where do you get your tools,
CLIs, skills? How do you actually enable agent
to do those things that we want them to be able
to do? How these agents run, right? Sandbox environments
or running versus running on your desktop. Of
course, you know, by definition, if you run it
in sandbox, it's going to be safer. There is
less likely that it will find some SSH file
on your desktop to gain keys to your systems
that you didn't intend to give, right? So these
kind of problems at the, you know, that wouldn't
be possible if you just run on sandbox environments.
And then credential management, right? How do
you actually provide it access to the things
you wanted to have access, right? You know, if
you want to solve with cloud infrastructure problems,
how do you give it permissions, right? What kind
of... Is it like a service principal? Are you
creating a special user for it? And what kind
of permissions you give it? And if you limit
it, by definition, it might not be able to apply
at all. Again, at every decision, you need to
kind of think through what is intended here and
can it do unintended. If you give it access to
workflows on GitHub, it will be able to apply
still. It will find some way to do things that
you didn't maybe intend it to do. So at architectural
level it shouldn't be possible even to do things
you don't want it to do. And then of course, you
know, what things you mentioned, policies and like
OPA, and those are those are also important part
of it. So it sounds like the foundational stuff
that we've been talking about for years, even
pre-AI, just making sure that's in place,
time-bound access, least privilege. Yeah, yeah. Okay.
And that's a fun part of it, right? I think this
is where good engineering is becoming even more
important. I can't remember who said it, but
it was recently a notable quote that, you know,
if you have a well-engineered system, right,
with good access, you already have a process
of onboarding junior engineers, right, where
they start with little permissions and then you
can grow from there as their autonomy increases,
you know. Same principles can be applied to introducing
a new teammate, right? In this case, it's AI.
And of course, if you have not mature processes,
then AI can make those things even worse. And
I think, you know, the platform teams will probably
suffer even more if they don't have a tight grip
and control already on those kinds of systems.
Do you think that agents should get direct cloud
credentials or like in practice? If they are
time-bound and least privileged? Yes. In practice you create
a special service principal or whatever way
that is designed for your cloud of choice, but
you limit the access to read-only if you want
to make sure that it doesn't change your infrastructure
directly. It should be able to see it. And then
your changes, the way how you suggest your changes,
should go through your typical change management
process, right? Through your Git repo, CI/CD pipelines,
tests, build tests. But it is useful when agent
understands and sees the data, right? What kind
of context should that agent have? Yeah. So typically
what we see works best is it needs a context
of your, how your infrastructure is structured,
right? The architecture, all the resources, all
the dependencies between them. That is important
information. Your infrastructure is a code, so
then it can start mapping between what what is
in the cloud and what is infrastructure as a
code. There is also very, you know, you can build
a lot of useful workflows around it, you know,
drift detection, migrating from ClickOps to IaC
is a very useful one. Observability data is very
useful so that it can help you with finding problems
or potentially alerting you. Security findings,
right? Whether it comes from AWS Security Hub
or Azure Defender or some third-party tools,
it's good when we have some more deterministic
or some tools that already scan you for security.
So then agents have it in the context. And cost
data is also useful if you are interested to
understand how infrastructure looks like from
the cost perspective and FinOps. Do you have
a human in the loop? Is it only in that approval
right before merge or do you have a human early
on in the loop? What's your thoughts on that? Yeah.
So the way we think about it is that humans kind
of get into this kind of agentic systems in two
places. In the beginning, when you need to set
it up, right? This is where you kind of need to
put some upfront effort to make sure, again, you
know, correct credentials, current access, and setting
up workflows. What, what do you want agents to
do? What kind of things, right? Do you want it to
do daily security checks and then creating patches
for you? Or do you want to rather come in and
ask it to do things as you wish on the need-by-need
basis, right? So it's like sort of inception
point. And then at the very end, after the agent's
done all the useful work, that would be the reviewer
role where you kind of need to verify the quality
of the work and if it matches the intent you
had. So are you using agents at all or are you
finding customers using agents to help review
Terraform or OpenTofu plans? And how does that
work in practice? Absolutely. And this is an
interesting thing because agents are also good
at reviewing or sometimes even better than humans
in some cases. And agents can review other agents'
work. So in a way, we can use agents to help
us to review work, right? Because it will be
very tedious if we have to go through each line
that produces pages. So as part of agentic work,
we also offer automatic PR reviews, agentic PR
reviews that review both agentic and human work
the same way. And the benefit, you know, these
agentic reviewers, they can have the same context
that you already provided. So they understand
how your infrastructure is structured, what are
the, you know, what are your compliance issues
so that it can, for example, notify you if you
make a change. Maybe it's a good change from
a security point of view, but it will block access
of your applications to it. For example, if you
put a private network, for example, around your
key vault, you might risk losing access yourself.
So then PR reviewer should not only find the
obvious errors, but also this kind of operational
impact. Am I going to have downtime or lose access
or will this recreate? So the PR reviewers can
have this knowledge if you give it the right
context and set it up correctly. And you can
also enrich it with your custom policies or your
organizational policies. Even if you have some
rules about styling or how your modules need
to be structured, all this knowledge you can
include. And then PR reviewers will help you
to keep your infrastructure standardized. This
is also one of the key findings and key features
that were important for one of our enterprise
customers, for example. What about drift? So
I know we deal with drift a lot with IaC. Maybe
not a lot. I guess it depends on the organization.
But if you're dealing with drift, you maybe don't
have the same context that you would for a PR
review. So how does the agent deal with that?
And how do you reconcile that agent doing that
work on the drift? Or do you not have agents
do drift work? Yeah, yeah. No, we do. So this
was one of the problems that we saw in the beginning,
right? We early on decided, okay, we don't want
agents, for example, touch infra directly, right?
Like this was for us a no-no. So how do we avoid
it? How do we circumvent it? We want to create
PRs and do changes to infra through CI/CD pipelines.
But to be able to do that, you need to have no
drift or minimal drift, right? You need to be
able to trust your infrastructure as a code.
So we built the whole workflow to be basically
to detect drift and use, again, agents to reconcile
everything that you have in your cloud back to
your infrastructure as a code. Because, of course,
the best way, you just have your infrastructure
code and from it you create your cloud infrastructure
and that's it. In reality, I have not... I haven't
worked with an organization yet that would be
able to do that, right? Unfortunately. So, and,
you know, we work with great organizations and
it's not that it's completely messed, but, you
know, there's still some important changes in
the cloud that you would not be able to just
overwrite with apply as purists might want to
do. So to account for real-world scenario, we
build this scanner that would find these drifts
and then use agents to help clean it up. And
bring everything into infrastructure as a code
to following your standards. So having that,
if you have that kind of information, then dealing
with drift is pretty simple. And you can build
it yourself, this kind of logic as well. We've
talked about drift. We've talked about the policy.
What we haven't really touched on is the financial
aspect of running agents in cloud. How do companies
deal with the financial implications of running
models? It's saving them time. It's maybe helping
with velocity. But how do they manage that cost
when it comes to running agentic workflows?
Yeah, I think right now the conversations are
pretty early, right? But they are starting to
show up. Until now, I think the mandate was very
simple. Just use tokens as much as possible.
Let's play with this. You know, people don't
know how to use it still. Let's not put any limitations.
Let's just, you know, enable it at organization
level. Unlimited usage oftentimes. And let's
see what happens. So what happens is that, of
course, you will end up in a situation where
people use a lot, right? I think there was a
new case where someone just built a non-work-related
game and used maybe a few hundred thousand, or not a few, tens of thousands
of dollars worth of tokens. Or other cases where
the budget was used up within three months, the
budget that was planned to be used for a year.
And I think also the disconnect also comes from
that we may be used to using AI from, you know,
ChatGPT, from OpenAI, from Claude Code, through
subscriptions, and they subsidize them a lot.
So if you use, let's say, if you build your own
agentic systems and you use tokens through API,
you might see that you will have to pay 10, 20
times more in real cost, you know, for the same
capabilities. So this makes it, of course, challenging
to invest into this kind of your own, building
your own workflows. And this is also something
that we, you know, we also offer subsidized in
a way service right now, right? But for companies
to be able to build, let's say, same capabilities
internally, they want similar to Cloudgeni,
they would have to pay 10 to 20 times more to
enable the same things. So I think understanding,
I think first question is understanding where
the cost comes from, right? To be able to track
it. Right now, there is no control, basically,
right? There is no... And this also goes back
to the security and policies part. There is no
audits on the agentic usage, right? No one knows
where is it used, what is it used for, what kind
of prompts people are running. Are there maybe
agentic... loops that just got stuck and using
money for no reason, right? It's, it can easily
just, you know, pull a non-existent API server
every day and use tokens and no one will know.
It is a completely new area, I think. And I think
the FinOps for AI is going to be big, a big problem
and big area to explore, just like we have in
cloud infrastructure, right? Orphan resources,
you know, orphaned agents that don't have purpose
anymore. We will see a lot of that. So I think
first step is actually having, if you're going
to build systems, agentic systems internally,
you need to design it with a way where you can
audit and see both costs, but also, you know,
the types of prompts and what they do and how
they do and measure some impact as well. So
this is what we also started realizing also with
Cloudgeni. And we started, we pull out the
foundation that, because in Cloudgeni, you
already have audits and control. So we pull out
this part and we made it open source, basically.
The foundation, the agentic builder, so to speak,
as an open source that anyone can use to build
their own workflows. Could be infrastructure
related, could be completely other area of business.
How do you correlate the spend in the API calls
for the agent versus the production output that
you're getting from that? How do you correlate
those two? Yeah, right now it is more vibes,
I would say, than hard data, definitely. But
we can see it already in our own case, for example,
right? We are able to achieve quite a lot with
extremely small engineering teams. We don't basically
have a dedicated DevOps engineer. We run everything
using our own product. And our engineering is
also quite thin. So what we focus on rather is
investing into those agentic workflows and automate
more and more engineering that way. And from
that point of view, it is still... cheaper or
there's less cost than hiring a person. And it
also less overhead in terms of knowledge sharing,
in terms of alignment. We would want to scale
though, definitely with more people as well,
but it also puts a certain requirement on engineers
to, instead of focusing on one-to-one work,
rather than focusing on, again, building agentic
capabilities and orchestrating agents to do their
work, right? So this is what we are focusing
on. And from that perspective, the economic impact
is visible, right? But we are yet to see it on
a larger scale, I would say, like in larger companies
where it can be more difficult to do so, these
kind of transformations. Okay, so given the cost
aspect, given the guardrails, given the models,
is there anything that you wouldn't trust an
AI agent with yet that you think it's maybe
not great at solving, maybe a person whether it
doesn't have the context or just a person is
better suited for solving it? Yeah, I still wouldn't
trust it to run, to make changes directly in infrastructure,
right? Like, unless I'm okay that this might, this
might, this is a staging environment or test environment
that is meant to, to be, right? And this is what
we are trying to solve as well in some limited
way. Can we let agent run, for example, our own
infrastructure without our oversight? And we have
internal experiments where we start looking into
it. But when it comes to, let's say, customer
data, privacy, losing data, losing access, where
there is big consequences. I would still want
to have some sort of guardrails before it, even
if it's semi-autonomous, but I would want to
have some deterministic blocks. For example,
it could be as simple as deletion block on the
resource group, for example, for production resources,
right? So it wouldn't be possible to delete it
without someone approving it, like a person,
a human engineer. Yes, but putting termination
protection on an instance as an example. At
the API level. So even if the agent did try to
run it, it couldn't without removing that protection.
Yeah. So I think at some point, if you have enough
of those guardrails and if you have sufficient
enough checks where you prove, you know, error-proof
your systems, basically, I do think there
will come a moment and maybe not very far in
the future where you would see fully autonomous
agentic systems, right? Where no human actually
looks at it. It will, you know, human will look
at dashboard to see like, okay, everything is
healthy. And this is what we're also building
for, right? Starting from smaller scale. Yeah.
Are you building like a risk scoring system where
like, if I'm adding tags to a resource, like
very low risk, right? I can probably let an agent
do that. But if I'm modifying VPC route table,
might not want an agent touching that without
a human. Like, are you? scoring that and how
do you score that like where we'd have a confidence
level for an agent to be able to make those changes
or not make those changes? Yeah, I think the way
we do it, let's say in Cloudgeni, we built, we
separated by workflows, right? Certain workflows
are more okay to do automatic, like for example
import. This is very safe action, there is no impact,
right? So we just let agents do completely. Security
and compliance patches, they're a little bit more
you know, where you need more control, right?
And this is where we put a little bit more guardrails
and checks. So I think for now we do it sort
of semi-manually. So for now it's up to a person,
I think, to decide because even tagging could
be quite important, right? Let's say someone...
If it's FinOps related or something, yeah. ...doesn't
want to mess it up as well. So for one team,
it could be like, ah, that's no problem. For
another team, it's like, no, no, we need to make
sure it's right. So I think you kind of need
this kind of flexibility. In that, for the future,
for that future product, basically where we think
agent will be able to run everything, we're sort
of starting from the foundation of it. But of
course, it will only be applied only to either
non-production environments or let's say small
startup environments, right? Where the impact
is lower. And from there, we will just, yeah.
If we know it's ephemeral, it doesn't really
matter as much, right? Yeah. And there we just
let agent go crazy and see where it breaks. And
then we can build, okay, this is where it breaks.
This is where it can cause problems. Let's build
a wall around this so that it will not do it.
Yeah, no, that makes sense. Okay, so if a cloud
or platform team wanted to start experimenting
with agents, where should they start without
creating like a monster? If they're very new,
they've used agents, maybe like they've used
Cursor, they've used Claude locally, but they
actually want to start using it in their IaC.
They want to start using it in their PR workflows.
Where should they start? Yeah. I can share how
we see, for example, some of our customers start,
right? There's different levels of adoption of
AI in general, and that can also correlate to
their willingness to adoption of it. All right.
If, let's say, they come from a background where
they just use it as an autocomplete or, you know,
some ChatGPT interface, so they haven't done
a lot of agentic, maybe safer way to adopt something
more agentic would be PR reviews, right? You
just enable connected to a couple of your environments,
a couple of your repositories, and start getting
reviews into your infrastructure's code, which
you can still write yourself or use agents to
help you. And that kind of builds trust. You
can start to see, okay, where agents can help
you, where there may be the limitations, where
they need a little bit more context. And next
step would be, well, now I can maybe let agents
to write infrastructure as a code for me, right?
Larger pieces of it, maybe larger refactorings.
Maybe, okay, I need a new set of resources. Here
is the template already. Agents are great at
following templates and examples. So if you need
a new setup, it will do it perfectly. And after
that is comfortable, I would say is automating
more and more work, right? Automating compliance
part, automating FinOps, right? Using agent
to finding orphan resources or so. This is where
it becomes more. But I think, you know, I would
say it's useful to tinker with it yourself, right?
Try to kind of build some, some automations yourself.
And this is why we also built OpenGeni because
we, I thought, you know, you can skip a few steps
and start building automations yourself. But I
think very quickly, if one gets very serious
about building agents, they will realize that
there is quite a lot of moving parts, just like
in anything else, right? In engineering. So if
you want to get quick results faster, there are
a few open source solutions, for example, for
SRE, for different things, right? And we see
Cloudgeni, for example, is one type of solution,
like Cloud Ops Management where you can start
managing your cloud using agents already without
having to invest into understanding how agents
should work and all the guardrails and all the
complexities. But it is still useful to understand
how agents work because then you will also be
able to interact with systems like Cloudgeni
in a much more efficient way, even if it doesn't
require you to be an expert in it. Okay, so wrapping
up, what about guardrails? Where do I start with
that? First guardrail, our first three guardrails.
I mean, 101, you know, run it in sandbox, right?
Maybe inside Docker, which I still don't consider
actually a proper sandbox because it still can
get out of your... So ideally, maybe somewhere
in the cloud, you know, run your agent in the
cloud in sandbox, give it a limited set of permissions,
right? To two things you wanted to have, let's
say, read access to, you know... subscription
on Azure or account on AWS, read-only. And then,
you know, connect maybe one infrastructure as
a code repo, only previous permission of PR,
creating PRs, right? So this is enough to kind
of say, hey, I can't do anything now. I can ask
my agent, delete this resource and it will not
interpret it. Even if it will interpret it in
the wrong way, it can't do anything. But most
likely it will just, it will understand, okay,
I'm in a sandbox, I have only read access, and
I can only create PRs. It will create PR where
it will remove it from Terraform instead of deleting
resource outright, or it will create a script
for this. And then you will be able to review
it. So very simple. No matter how you prompt
it, you make a mistake in the prompt, there is
no chance the agent will do a mistake there.
Makes sense. Anything to leave our audience with?
Any last thoughts? Yeah, I think the last thing,
you know, just to touch up on the human moment
as well. I think people are also, you know, there
is a different sentiment we see when it comes
to AI in engineering. And it also changed a lot
as we saw them from skepticism to, oh, wow, this
actually works. And now there's like, okay, there's
some concern, I think, naturally. What I see
instead is that I think especially platform and
DevOps people and in general, good engineers,
I think we are in a very important position right
now. We can decide how AI is going to be adopted
at a larger scale in organizations and our teams.
And instead of, let's say, ignoring it or instead
of avoiding it, what we can try to think of,
okay, how can we adopt it in a way that is actually
useful, that is safe, that is cost efficient,
that benefits us, our organization, our time.
How can we actually make it useful? And I think
if we kind of approach it with that set of mindset,
we can find new kind of fun with it as well.
Because it's actually, you know, it is still
building systems, but it's maybe a little bit
different kind of systems, right? Maybe not what
we're used to, but it is still, you know, a lot
of thinking that requires. It's not just a mindless
vibe coding that many people maybe think of when
they hear AI. If you go deep enough, there is
a lot of technical challenges, and they need
to be solved today before we see more outages,
problems, and costs and impacts on people. So
I think, yeah, that's what I would leave people.
Be curious. Try to figure out, okay, what can
I do with this, and how can I use it, and what's
required from me to enable it at its full capacity.
I think that will be fun. Yeah, well said. Where
can people find out more about you and Cloudgeni?
So I'm on LinkedIn, Davlet Dzhakishev. It's
a hard name, but maybe people can read it. It's
a cloud, Cloudgeni.ai, our commercial product. OpenGeni.ai
is the open source, MIT-licensed foundation
of it. And yeah, I think on LinkedIn is the best.
I have X and other stuff, but it's not very...
big yet. Give me a follow if you want to help.
It's Davlet Cloudgeni. So yeah. Awesome. I'll put links
for all those in the show notes as well. Davlet,
thank you so much for coming on. Really appreciate
your time. Awesome. It was great chatting with
you, Brian. Great conversation. Thank you. That
was my conversation with Davlet from Cloudgeni.
The biggest takeaway for me is that a safe agent
is mostly a permissions problem, not a prompting
problem. After that Terraform apply, his fix
wasn't a better prompt. It was guardrails at
the architecture level. So the agent can't do
whatever you don't want, however you word the
request. His starter kit is simple. Run the agent
in a real sandbox, ideally in the cloud. In his
book, Docker doesn't count. Give it read-only
cloud access, one IaC repo, and... permission
to open PRs. Ask it to delete a resource, and
the worst it can do is open a PR. That's the
valet key. His on-ramp is agentic PR reviews
first, then agents writing IaC from your templates,
then automation. And a good reviewer catches
more than typos. His example was a private network
around a key vault. Great for security, until
your own apps lose access. On cost, Davlet points
out that AI subscriptions are heavily subsidized
and running your own agents on API tokens can
cost 10 to 20 times more. He thinks FinOps for
AI is going to be big, orphaned agents and all.
You can't manage what you can't audit. You can
find Davlet on LinkedIn and X, plus Cloudgeni,
OpenGeni, and his Infrastructure Agents Guide
on GitHub through the links in the show notes.
Follow or subscribe to Ship It Weekly wherever
you listen and find previous episodes at shipitweekly.fm.
Thanks for listening and I'll see you Friday
for the news.
Ship It Conversations: The AI Agent That Ran…
0:00 / 37:24
Listen & Subscribe
Catch This Episode
Listen on your favorite podcast platform
Commentary
Host Commentary
Notes
Show Notes
This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps.
In this Ship It Conversations episode, I talk with Davlet Dzhakishev about what it takes to let AI agents work on your cloud infrastructure, why guardrails belong in the architecture and not the prompt, and what running your own agents really costs.
Davlet is the CEO and co-founder of Cloudgeni, which builds AI agents for cloud infrastructure. He got into this after working through SOC 2 and ISO 27001 compliance prep mostly by hand, and asking what agents would need to fix those problems reliably without making things worse.
We start with the incident that changed how Cloudgeni builds. An agent hit a snag on a simple Terraform change and decided the fix was to run apply. On staging. Resources got recreated, and Davlet calls it his wake-up call. Since then, guardrails come first: sandboxed runtimes, scoped credentials, read-only cloud access, and changes that go through Git, CI/CD and tests. At the architecture level, the agent shouldn't be able to do what you don't want it to do.
Then we get into agentic PR reviews, which check human and agent work the same way, and why a good reviewer catches more than the obvious errors. Davlet's example is a private network around a key vault: great for security, until your own apps lose access. We also cover drift, and why you need to reconcile it before you can trust agent PRs.
On cost, Davlet's view is that AI subscriptions are heavily subsidized, and running your own agents on API tokens can cost ten to twenty times more. Most agent usage also has no audit trail, so a stuck loop can poll a dead API server every day and no one will know. He thinks FinOps for AI is going to be big, orphaned agents and all. That need for audits is part of why Cloudgeni open-sourced its foundation as OpenGeni.
We finish with what he still wouldn't hand to an agent, and where a platform team should start without building a monster: PR reviews first, then agents writing IaC from your templates, then automation. His starter kit is a real sandbox, ideally in the cloud, because Docker doesn't count. Give it read-only access to an AWS account or Azure subscription, one IaC repo, and permission to open PRs. Ask it to delete a resource, and the worst it can do is open a PR.
The big takeaway: a safe agent is mostly a permissions problem, not a prompting problem. Give it read-only access and PR-only changes, and it can't make that mistake however you word the request.
Highlights
- The Terraform apply on staging that became his wake-up call
- Guardrails at the architecture level, not in the prompt
- Read-only access, with every change through Git, CI/CD and tests
- Agentic PR reviews that catch operational impact
- Why drift has to be reconciled before you can trust agent PRs
- Why API tokens can cost 10 to 20 times more
- FinOps for AI, stuck loops and orphaned agents
- A starter kit: a real sandbox, read-only cloud access, one IaC repo, PRs only
Links
Davlet on LinkedIn · Davlet on X · Cloudgeni · OpenGeni · Agents guide · More episodes · On Call Brief · LMGT Awards
Guest
Cloudgeni
Davlet Dzhakishev
CEO and co-founder
Davlet Dzhakishev is the CEO and co-founder of Cloudgeni, which builds AI agents for cloud infrastructure. He started working on agents after preparing a company for SOC 2 and ISO 27001 compliance largely by hand.
Made in Greencastle, Pennsylvania
More from Ship It Weekly
All episodes →
Ship It Conversations: Ned Bellavance of Ned in the Cloud on DevOps Beyond the Buzzwords, Terraform, AI, the Future of Infrastructure as Code, and Why Fundamentals Still Matter
In this episode, Ned Bellavance discusses the essence of DevOps, emphasizing principles over mere ceremonies. He explores the future of infrastructure as code, Terraform's challenges, and the impact of AI on code generation.
Ship It Conversations: AI Coding Outruns Software Delivery, With Darko Fabijan of SuperPlane on What Breaks After the Pull Request, Change Risk Scores, Onboarding AI Agents Like Engineers, Read-Only Ops Agents, and Why the Answer Is More Automation
In this episode, Darko Fabijan discusses the challenges of AI-generated code in software delivery, highlighting increased defect rates and the need for automation. He proposes a risk scoring system for changes and advocates for onboarding AI agents similarly to new engineers.
Cloudflare Saves 100TB of RAM, AI Drives Server Prices Up, AWS Adds a Fourth London AZ, Route 53 DNS Self-Service, AKS eBPF Routing, Go 1.27, and the Danger of Hidden Infrastructure Assumptions
In this episode, Ship It Weekly discusses Cloudflare's 100TB RAM savings through low-level cache optimizations and the implications of AI-driven price hikes in memory. AWS adds a fourth London Availability Zone, revealing hidden infrastructure assumptions.
GitHub Outage, PleaseFix Agentic Browser Vulnerability, AWS Certificate Manager Drops Email Validation, Cloudflare TypeScript CI Workflows, AI Observability Consolidation, and the Hidden Cost of “Simple” Platform Changes
This episode of Ship It Weekly discusses the recent GitHub outage affecting critical developer workflows and highlights the PleaseFix vulnerability in agentic browsers.
The thing that stuck with me most from this conversation with Davlet Dzhakishev is that a safe agent is mostly a permissions problem, not a prompting problem.
It's tempting to treat agent safety as something you solve in the prompt. Tell the agent to be careful. Tell it never to touch production. Add a line that says always ask first.
Davlet learned the hard way what happens when the guardrails come second.
Back when Cloudgeni was still early on its guardrails, one of their agents was working through a simple Terraform change. Things didn't work the way they were supposed to. So the agent decided that applying the changes might be a good idea.
On staging.
Some resources got recreated, because apply can be destructive. Davlet called it a "wake up call," and it changed how they built the platform. From then on, guardrails were built in as "first class citizens."
This happened in agentic mode, where the agent works in a loop until it reaches the goal you gave it. In that mode, he said, there's a higher risk things go sideways. My read: it's the mode everyone wants, and it's the one where nobody is reading every step.
If you've been on call long enough, the agent's reasoning probably sounds familiar. The plan doesn't look right, the pipeline is stuck, and somebody decides to just apply and see what happens. The difference is that most humans hesitate first.
So what was the fix?
When I asked what kind of guardrails he meant, policy engines like OPA and Kyverno, pre-prompts, validating output, his answer was all of it. But it all starts with architecture.
His line on this is the one I'd put on a slide: "At architectural level it shouldn't be possible even to do things you don't want it to do."
He walked through it in layers, and his team's infrastructure agents guide on GitHub covers them in detail: where the agents come from, which tools and skills they get, where they run, and what their credentials allow. Run an agent in a sandbox, and it can't stumble onto an SSH key sitting on your machine.
The part I keep thinking about is the indirect path.
You can take away the agent's permission to apply. But in his words, "If you give it access to workflows on GitHub, it will be able to apply still." It will find some way.
That's the same lesson we've been learning with IAM for years. Least privilege is about everything an identity can reach, including a CI system with far more power than the agent. If the agent can trigger the thing that can apply, the agent can apply.
That's why I like the valet key as an image. It starts the car, but it won't open the trunk. You're not asking the valet to promise not to open the trunk. The key just doesn't do it.
Map that onto what Davlet described, and the valet key is read-only cloud access. It's useful for the agent to see your infrastructure, but every change goes through your normal change management process: your Git repo, CI/CD pipelines, tests.
When I said that sounded like the foundational stuff we've talked about for years, time-bound access and least privilege, he agreed, and said good engineering is becoming even more important. He also named the flip side: "if you have not mature processes, then AI can make those things even worse."
Agents don't fix a messy platform. They amplify it.
Drift is a good example. If every agent change has to go through a PR, your infrastructure as code has to be something you can trust. Davlet said he hasn't worked with an organization yet that builds its cloud purely from IaC, even the great ones. Some changes in the cloud can't just be overwritten with apply, the way a purist might want.
The purist answer to drift is the same move his agent made on staging. Instead, Cloudgeni built a scanner that finds drift and uses agents to bring it back into code, following your standards, so the PR path reflects what's actually running.
Then there's review.
Cloudgeni runs agentic PR reviews that check human work and agent work the same way. And a good reviewer catches more than the obvious errors. His example was putting a private network around a key vault. That's a good change from a security point of view, right up until your own applications lose access to it.
That's the kind of change that passes every check and still pages someone. The plan tells you what will change. It doesn't tell you what will break. A reviewer with context on how your infrastructure fits together can ask whether it means downtime or lost access.
For anything with real consequences, like customer data or production resources, he still wants deterministic blocks in front of the agent. A deletion lock on a production resource group. Or, as I put it, termination protection at the API level, so even if the agent tries, it can't.
Trust grows by workflow, not all at once. Imports are safe, so agents can do them on their own. Security and compliance patches get more guardrails. Even tagging might be no problem for one team and a big deal for another. In non-production environments, where the impact is lower, he's happy to let an agent go crazy, see where it breaks, and then "build a wall around this."
That's how we've always hardened systems. Find the failure somewhere cheap, then make it impossible somewhere expensive.
The cost part of the conversation is the one I think a lot of teams are about to run into.
Davlet said the mandate so far has mostly been simple: use tokens as much as possible and see what happens. He mentioned cases where a budget planned for a year was used up in three months. And he pointed out that AI subscriptions are heavily subsidized. If you build your own agents and pay for tokens through the API, the same capabilities can cost ten to twenty times more.
The bigger problem is visibility. Right now there are no audits on most agent usage. Nobody knows where agents are running, what they're used for, or what prompts are going in. An agent loop can get stuck polling an API server that doesn't exist anymore, every day, burning tokens, and no one will know.
He thinks "FinOps for AI is gonna be big," and the parallel to cloud is right there. We already have orphaned resources. We're going to have "orphaned agents that don't have purpose anymore."
When I asked how you correlate agent spend with what you get back, he was honest: right now it's more vibes than hard data.
You can't manage what you can't audit. So if you're building agents internally, design for auditability from the start: cost, prompts, what each agent does, and some measure of impact. That need is part of why Cloudgeni open-sourced its foundation as OpenGeni.
So where should a platform team start?
Davlet's on-ramp is PR reviews first. They build trust, and they show you where agents help, where their limits are, and where they need more context. Then let agents write IaC from your templates, because agents are good at following templates and examples. Then automation: compliance, FinOps, finding orphaned resources.
His starter kit is small. Run the agent in a real sandbox, ideally in the cloud. Docker, he said, is something he still doesn't "consider actually a proper sandbox." Give it read-only access to an AWS account or an Azure subscription, one IaC repo, and permission to create PRs. That's it.
Then ask it to delete a resource. The worst it can do is open a PR that removes the resource from Terraform, and you review it. However badly you word the prompt, it can't make that mistake.
Davlet closed on something I agree with. Platform and DevOps engineers are in an important position right now: "We can decide how AI is gonna be adopted at a larger scale in organizations, in our teams." His advice was to be curious.
So my takeaway from this one is simple.
Stop trying to write the perfect prompt. Hand over the valet key.
Read-only access, PR-only changes, and an audit trail you can actually read. Get those right, and the prompt matters a lot less.