AWS Roles Trusting Qubole Abused to Steal Secrets, Claude Code's Don't Ask Again File Leaks Credentials, KubeCon Agent Sandboxes for Kubernetes & WorkOS Breach Traced to an AI Agent
Try reloading the player or open this episode directly on YouTube.
This episode of Ship It Weekly discusses a security breach where an attacker exploited Qubole's trusted access to AWS accounts, stealing secrets and S3 data.
Now Playing
AWS Roles Trusting Qubole Abused to Steal Secrets, Claude Code's Don't Ask Again File Leaks Credentials, KubeCon Agent Sandboxes for Kubernetes & WorkOS Breach Traced to an AI Agent
Ship It Weekly
0:00Current chapter: Cold open15:41
Chapters
Jump to a section in this episode.
Speed & share
Transcript
An attacker used Qubole's trusted access into
customer AWS accounts to read secrets and download
whole S3 buckets. Claude Code's Don't Ask Again
button has been saving passwords and API keys
into a file that keeps turning up on GitHub and
Docker Hub. And at KubeCon, sandboxing AI agents
became its own Kubernetes layer. I'm Brian Teller
from Teller's Tech, and this is Ship It Weekly.
Welcome back to Ship It Weekly, the show about
the DevOps, SRE, cloud, platform, and security
stories that matter when you are the person keeping
the thing running at three in the morning. For
the weekly story list and source links, check
out oncallbrief.com. For past episodes and show
notes, head over to shipitweekly.fm. This week,
AWS roles that trusted Qubole, the Claude Code
file that leaks secrets. KubeCon's new agent
sandboxes, and a WorkOS breach traced to a researcher's
AI agent. Then we have a quick lightning round
and a human closer about why incident resolution
time is the wrong thing to target. Let's get into it.
First up, AWS accounts breached through
a vendor's trusted access. Wiz reported Monday
that between October 14th and October 26th, an
attacker used IAM credentials from an AWS account
tied to Qubole, a data lake platform, to assume
the cross-account roles customers had created
to trust it. Once inside, the attacker went straight
for credentials, Lambda environment variables,
Secrets Manager values, and SSM parameters. It
also downloaded S3 data, sometimes entire buckets
in a single session. How it got Qubole's credentials
hasn't been disclosed. The interesting part is
the role trust itself. Every assume role call
Wiz tied to the attacker included the external
ID and every one succeeded. The external ID was
never meant to stop this. It protects against
the confused deputy problem, where another one
of the vendor's customers gets the vendor to
act on your account. It isn't a secret either.
It shows up in plain text in CloudTrail. If the
vendor's own credentials are stolen, the external
ID comes along for free. In most of these environments,
the attacker's calls were the only assume role
activity from that account in October. The integration
was idle. The trust policy was still live. So
find every IAM role whose trust policy names
an outside account and ask whether you still
use that vendor. If you don't, delete the role.
If you do, narrow the trust to the vendor-specific
role where they support it. Add a source IP or
VPC endpoint condition. Wiz says a source IP
condition alone would have blocked every role
assumption they saw. If you ever connected Qubole,
check CloudTrail from October 1st for its assume
role calls. Then rotate everything those roles
could have read and revoke active sessions. A
trust policy keeps working long after you've
stopped thinking about it.
Next, Claude Code has
been writing secrets into a file that ends up
in public repos and container images. When Claude Code
asks to run a shell command, and you pick yes
and don't ask again, it saves an allow rule to
.claude/settings.local.json in your project
folder. That rule is the full command, so any
password or token in it now sits there in plain
text. On November 6th, Apiiro reported that out
of about 104,000 public copies of that file
on GitHub, 1,122 held at least one secret. Those
included 379 database passwords and 23 AWS keys.
On Monday, GitGuardian counted more than 31,000
secrets pushed in that file since January 2025
and more than 5,300 worked when it found them.
The part I'd look at closely is Docker. GitGuardian
found 613 secrets in public Docker Hub images,
99 of them valid. If your Dockerfile does COPY
dot dot, it copies your working tree, not your
Git index. So a file that Git ignores still ships
in the image, unless .dockerignore excludes
it. Claude Code does add the file to your global
Git excludes, but that only covers your own laptop,
not a teammate's machine, a CI runner, or an
image build. And Apiiro says none of the related
bug reports have gotten a reply from Anthropic.
So rotate first, because git history, forks,
and image layers all keep copies. Then git rm
dash dash cached the file. Add .claude to your
.gitignore and .dockerignore and delete any
allow role with a literal secret in it. After
that, use environment variables for credentials
and approve command patterns instead of exact commands.
Third, KubeCon turned sandboxing AI
agents into a standard Kubernetes layer. At KubeCon
this week, the CNCF took in two new sandbox projects
for agents. Agent Substrate, contributed by Google,
is a zero-trust runtime for AI agents on Kubernetes.
OpenShell, led by NVIDIA, enforces policy on
which files, networks, credentials, and model
endpoints an agent can reach. And gVisor, Google's
application kernel, was contributed too. AWS
launched Nitro Sandboxes for EKS the same week.
Set a pod's runtime class name to isolated sandbox,
and that pod runs in its own lightweight virtual
machine on Nitro with its own kernel. It needs
Kubernetes 1.36 or later, and it's built into
EKS Auto Mode. AWS calls it available today without
saying GA or preview. Here's why this matters.
A normal container shares the host kernel. And
that's been fine for code your team wrote and
reviewed. An agent runs code nobody reviewed.
And a container escape puts it on the host. Next
to everyone else's pods. gVisor puts a user space
kernel in between. And Nitro and microVMs use
a separate virtual machine. Either way, the boundary
moves off the shared kernel. Two cautions though.
Sandbox stage at the CNCF means early. So treat
Agent Substrate and OpenShell as things to evaluate,
not deploy. And isolation limits what the agent
can do to the host, not what it can reach with
the credentials you hand it. If you run coding
agents or untrusted CI on Kubernetes, the default
runtime plus a broad service account plus open
egress is the combination to fix first.
Fourth, a breach that WorkOS traced to an AI agent. On
November 2nd, WorkOS, the developer platform,
disclosed unauthorized access to an internal
sales tool. The first access was October 25th,
and nobody noticed until records were pulled
in bulk on October 30th. The data was sales and
support material, including excerpts of sales
calls and customer Slack conversations. Then
on November 6th, a security researcher came forward.
They said they were running benchmark evaluations
with ARTEX, an AI offensive security framework,
when their agent escaped its isolated test environment
and ended up at WorkOS. WorkOS and an independent
forensics firm matched the researcher's evidence
to its logs, and they're confident it was this
person. The escape itself is the researcher's
account. Nobody has said how it happened, and
the claim that the data was deleted hasn't been
independently verified. The more useful part
is WorkOS's own list of what let it in, and none
of it needed a clever agent. The tool's API was
reachable from the internet. It accepted a predictable
credential meant for local development. There
was no rate limiting, and five days went by before
anyone noticed. It was a legacy internal tool
and most of us have one of those. WorkOS has
since cut all of its internal tools off from
the internet and now runs ARTEX against its own
systems. The sandboxes in the last story are
about containing the agents you run. This is
the other side. Agents someone else runs, on
purpose or not, that never get tired of trying
the dev credential you forgot about.
Quick lightning round. First, Microsoft's November Patch Tuesday
came with one bug already under attack. It's
an Outlook spoofing flaw. Open a crafted Office
file and classic Outlook leaks your NTLMv2 hash,
which can be used to impersonate you. CISA's
federal deadline is November 24th. And the fix
ships with Office updates, not Windows Update.
Azure DevOps Server also has a critical 9.9
remote code execution bug. Second, N-able. On
Sunday, a researcher published 33 N-central exploits
he calls Able2Shell. N-able took hosted N-central
offline on Monday and shipped 2026.4 Hotfix 2
on Tuesday with fixes for seven CVEs. It reports
scanning, but no successful exploitation. If
you run it on-prem, keep it offline until it's
upgraded. Then update agents and probes. Third,
KubeVirt graduated at the CNCF. It runs virtual
machines alongside containers on one Kubernetes
control plane. And recent releases added confidential
VMs and GPU claims through dynamic resource
allocation. If you are planning a move off VMware
onto Kubernetes, that's a maturity signal. And
fourth, Red Hat made OpenShift 5 generally available
on Kubernetes 1.36 as an in-place update from
4.22. Its AI troubleshooting is GA, with the
more autonomous agent features still in tech
preview. Standard support goes from 18 to 24 months.
The human closer this week comes from
Brent Chapman in a piece featured in SRE Weekly
called Why Incident Resolution Time Isn't Something
You Should Target. His argument is that resolution
time mostly depends on the incident, not on your
process. A config rollback and a cascading failure
at peak traffic are different problems. So one
target is either too easy or impossible. Once
it's a target, Goodhart's law takes over. The
number gets managed instead of the system. His
example is fire departments. They set targets
for dispatch, turnout, and travel time. But none
for time on scene. Because a kitchen grease fire
and a warehouse fire aren't the same job. So
target the steps that look the same every time
and that you control. How fast a page is acknowledged.
How fast the incident channel is staffed. The
first customer update. Track diagnosis and mitigation
as trends, and dig into the outliers. His test:
if you can write down two or three specific things
you change to improve a step, it's controllable
enough to target. That test fits this week. You
don't control whether a vendor you connected
years ago gets breached. You do control whether
that role still exists and which addresses it
accepts. You don't control what a coding tool
writes to disk. You do control your .dockerignore
and what your scanners check. And WorkOS couldn't
control an agent showing up, but five days to notice
is something a rate limit and a volume alert
can change. The parts that are yours to change
are usually smaller and more boring than a number
on a dashboard. That's it for this week of Ship
It Weekly. Follow or subscribe wherever you are
watching or listening. You can find the weekly
story list and source links at oncallbrief.com
and past episodes and show notes at shipitweekly.fm.
One quick note, Ship It Weekly is a finalist
in this year's DevOps Dozen for Best DevOps Community
Program or Event. And I'm a finalist for Enterprise
DevOps Champion of the Year. The public vote
counts, and it's open through December 31st.
The link's in the show notes. Thank you. In this
episode, we covered AWS roles that trusted Qubole,
Claude Code's leaky settings file, KubeCon's agent
sandboxes, and the WorkOS breach. I'm Brian Teller
from Teller's Tech. Thanks for listening. And
remember, target what you can change and track
the rest.
AWS Roles Trusting Qubole Abused to Steal Secrets,…
Every main story this week came down to one question: what can it touch?
Not what it's supposed to do. What it can actually reach, today, with the access it already has.
A vendor role nobody used anymore but nobody removed. An approval button that quietly saved the secret along with the command. An AI agent running code nobody reviewed, on a kernel it shares with everything else. And a legacy internal tool that was reachable from the internet with a credential meant for someone's laptop.
None of those were decisions anyone made. They were access that outlived the reason for it.
Start with Qubole.
Wiz reported that between October 14th and October 26th, an attacker used IAM credentials from an AWS account tied to Qubole, a data lake platform, to assume the cross-account roles customers had created to trust it. Once inside, it went straight for credentials: Lambda environment variables, Secrets Manager values and SSM parameters. It also downloaded S3 data, sometimes entire buckets in a single session. How it got Qubole's credentials hasn't been disclosed.
The part I keep thinking about is the External ID. Every assume role call Wiz tied to the attacker included it, and every one succeeded. That's not a bug. The External ID was never meant to stop this. It protects against the confused deputy problem, where another of the vendor's customers tricks the vendor into acting on your account. It isn't a secret either: it sits in plain text in CloudTrail. If the vendor's own credentials are stolen, the External ID comes along for free.
And here's the detail that should bother all of us. In most of these environments, the attacker's calls were the only assume role activity from that account in October. The integration was idle. The trust policy was still live.
My take: a cross-account role is a standing grant, and standing grants don't get re-reviewed unless something forces it. Nobody files a ticket to remove a vendor integration they stopped thinking about.
So go find every IAM role whose trust policy names an outside account, and ask whether you still use that vendor. If you don't, delete the role. If you do, narrow the trust to the vendor's specific role where they support it, and add a source IP or VPC endpoint condition. Wiz says a source IP condition alone would have blocked every role assumption they saw. And if you ever connected Qubole, check CloudTrail from October 1st, rotate everything those roles could read, and revoke active sessions.
Then Claude Code.
When Claude Code asks to run a shell command and you pick "Yes, and don't ask again," it saves an allow rule to .claude/settings.local.json in your project folder. That rule is the full command. So if the command had a password or a token in it, that secret now sits in a file in plain text.
Apiiro found about 104,000 public copies of that file on GitHub, and 1,122 of them held at least one secret, including 379 database passwords and 23 AWS keys. GitGuardian counted more than 31,000 secrets pushed in that file since January 2025, and more than 5,300 still worked when it found them.
The Docker part is the one I'd look at first. GitGuardian found 613 secrets in public Docker Hub images, 99 of them valid. If your Dockerfile does COPY . ., it copies your working tree, not your git index. A file git ignores still ships in the image unless .dockerignore excludes it. Claude Code does add the file to your global git excludes, but that only covers your own laptop, not a teammate's machine, a CI runner or an image build.
My take: any feature that remembers an approval is storing something. When the thing it remembers is a command, it can be storing a credential you never meant to save.
So rotate first, because git history, forks and image layers all keep copies. Then untrack the file, add .claude to both .gitignore and .dockerignore, and delete any allow rule with a literal secret in it. Going forward, keep credentials in environment variables and approve command patterns instead of exact commands.
KubeCon was the hopeful story.
The CNCF took in two new Sandbox projects built for agents. Agent Substrate, contributed by Google, is a zero-trust runtime for AI agents on Kubernetes. OpenShell, led by NVIDIA, enforces which files, networks, credentials and model endpoints an agent can reach. gVisor was contributed too. The same week, AWS launched Nitro Sandboxes for EKS: set a pod's runtime class to isolated-sandbox, and that pod gets its own lightweight virtual machine on Nitro, with its own kernel. It needs Kubernetes 1.36 or later. AWS calls it available today, without saying GA or preview.
Here's why it matters. A normal container shares the host's kernel, and that's been fine for code your team wrote and reviewed. An agent runs code nobody reviewed. A container escape puts it on the host, next to everyone else's pods. A sandbox moves the boundary off that shared kernel.
Two cautions. Sandbox stage at the CNCF means early, so evaluate Agent Substrate and OpenShell rather than deploy them. And isolation limits what an agent can do to the host. It doesn't limit what the agent can reach with the credentials you hand it.
If you run coding agents or untrusted CI on Kubernetes, my guess at the combination to fix first is the default runtime, plus a broad service account, plus open egress.
WorkOS is the same question from the other side.
On November 2nd, WorkOS disclosed unauthorized access to an internal sales tool. The first access was October 25th. Nobody noticed until records were pulled in bulk on October 30th. On November 6th, a security researcher came forward: they'd been running benchmark evaluations with ARTEX, an AI offensive security framework, when their agent escaped its test environment and ended up at WorkOS. WorkOS and an independent forensics firm matched the researcher's evidence to its logs. The escape itself is the researcher's account, and nobody has said how it happened.
What I find most useful is WorkOS's own list of what let it in. The tool should not have been reachable from the internet. Its API accepted a weak credential meant for local development. There was no rate limiting. And it took five days to notice. None of that needed a clever agent.
WorkOS has since cut all its internal tools off from the internet, and it now runs ARTEX against its own systems. That's the right instinct. Agents, whoever runs them and whether or not anyone meant to, never get tired of trying the dev credential you forgot about.
The human closer is where this lands for me.
Brent Chapman's piece argues that incident resolution time isn't something you should target. It mostly depends on the incident, not on your process, and once it's a target, the number gets managed instead of the system. His fire department example stuck with me: targets for dispatch, turnout and travel time, but none for time on scene, because a grease fire and a warehouse fire aren't the same job. His test for what to target: if you can name two or three specific things you'd change to improve a step, it's controllable enough to target.
Run this week through that test.
You don't control whether a vendor you connected years ago gets breached. You do control whether that role still exists, and which addresses it accepts. You don't control what a coding tool writes to disk. You do control your .dockerignore, and what your scanners check. And WorkOS couldn't control an agent showing up, but five days to notice is something a rate limit and a volume alert can change.
The parts that are yours to change are usually smaller, and more boring, than a number on a dashboard.
This week on Ship It Weekly, an attacker used Qubole's trusted access into customer AWS accounts to read secrets and pull whole S3 buckets. Claude Code's "don't ask again" button has been saving passwords and API keys into .claude/settings.local.json, a file that keeps turning up on GitHub and Docker Hub. At KubeCon, the CNCF took in Agent Substrate and OpenShell as Sandbox projects, and AWS launched Nitro Sandboxes for EKS. And WorkOS traced a breach of an internal sales tool to a security researcher's AI agent that escaped its test environment.
The thread: what can it touch? A vendor role nobody removed, an approval that saved the secret with the command, agents running unreviewed code, and an internet-facing tool with a guessable dev credential.
In the lightning round: an exploited Outlook bug (CVE-2026-124194) and a 9.9 Azure DevOps Server RCE in Patch Tuesday, N-able patching the Able2Shell exploits in N-central, KubeVirt's CNCF graduation, and OpenShift 5 going GA.
The human closer comes from Brent Chapman: incident resolution time isn't something you should target. Target what you can change, and track the rest.
Ship It Weekly is a 2026 DevOps Dozen finalist for Best DevOps Community Program or Event, and I'm a finalist for Enterprise DevOps Champion of the Year. The public vote is open through December 31.
Highlights
Why the External ID didn't stop a stolen vendor credential
Find every role that trusts an outside account, and delete the idle ones
settings.local.json: rotate, untrack, and add it to .dockerignore
Moving an agent's boundary off the shared kernel
WorkOS's own list: internet-facing API, dev credential, no rate limit
25 years in production: DevOps, SRE, platform, and cloud. Host of Kube Signals on KubeFM. Grafana Champion; Confluent Community Catalyst; IBM Influencer 2026; DevOps Institute & ITIL Ambassador.
Every main story this week came down to one question: what can it touch?
Not what it's supposed to do. What it can actually reach, today, with the access it already has.
A vendor role nobody used anymore but nobody removed. An approval button that quietly saved the secret along with the command. An AI agent running code nobody reviewed, on a kernel it shares with everything else. And a legacy internal tool that was reachable from the internet with a credential meant for someone's laptop.
None of those were decisions anyone made. They were access that outlived the reason for it.
Start with Qubole.
Wiz reported that between October 14th and October 26th, an attacker used IAM credentials from an AWS account tied to Qubole, a data lake platform, to assume the cross-account roles customers had created to trust it. Once inside, it went straight for credentials: Lambda environment variables, Secrets Manager values and SSM parameters. It also downloaded S3 data, sometimes entire buckets in a single session. How it got Qubole's credentials hasn't been disclosed.
The part I keep thinking about is the External ID. Every assume role call Wiz tied to the attacker included it, and every one succeeded. That's not a bug. The External ID was never meant to stop this. It protects against the confused deputy problem, where another of the vendor's customers tricks the vendor into acting on your account. It isn't a secret either: it sits in plain text in CloudTrail. If the vendor's own credentials are stolen, the External ID comes along for free.
And here's the detail that should bother all of us. In most of these environments, the attacker's calls were the only assume role activity from that account in October. The integration was idle. The trust policy was still live.
My take: a cross-account role is a standing grant, and standing grants don't get re-reviewed unless something forces it. Nobody files a ticket to remove a vendor integration they stopped thinking about.
So go find every IAM role whose trust policy names an outside account, and ask whether you still use that vendor. If you don't, delete the role. If you do, narrow the trust to the vendor's specific role where they support it, and add a source IP or VPC endpoint condition. Wiz says a source IP condition alone would have blocked every role assumption they saw. And if you ever connected Qubole, check CloudTrail from October 1st, rotate everything those roles could read, and revoke active sessions.
Then Claude Code.
When Claude Code asks to run a shell command and you pick "Yes, and don't ask again," it saves an allow rule to
.claude/settings.local.jsonin your project folder. That rule is the full command. So if the command had a password or a token in it, that secret now sits in a file in plain text.Apiiro found about 104,000 public copies of that file on GitHub, and 1,122 of them held at least one secret, including 379 database passwords and 23 AWS keys. GitGuardian counted more than 31,000 secrets pushed in that file since January 2025, and more than 5,300 still worked when it found them.
The Docker part is the one I'd look at first. GitGuardian found 613 secrets in public Docker Hub images, 99 of them valid. If your Dockerfile does
COPY . ., it copies your working tree, not your git index. A file git ignores still ships in the image unless.dockerignoreexcludes it. Claude Code does add the file to your global git excludes, but that only covers your own laptop, not a teammate's machine, a CI runner or an image build.My take: any feature that remembers an approval is storing something. When the thing it remembers is a command, it can be storing a credential you never meant to save.
So rotate first, because git history, forks and image layers all keep copies. Then untrack the file, add
.claudeto both.gitignoreand.dockerignore, and delete any allow rule with a literal secret in it. Going forward, keep credentials in environment variables and approve command patterns instead of exact commands.KubeCon was the hopeful story.
The CNCF took in two new Sandbox projects built for agents. Agent Substrate, contributed by Google, is a zero-trust runtime for AI agents on Kubernetes. OpenShell, led by NVIDIA, enforces which files, networks, credentials and model endpoints an agent can reach. gVisor was contributed too. The same week, AWS launched Nitro Sandboxes for EKS: set a pod's runtime class to
isolated-sandbox, and that pod gets its own lightweight virtual machine on Nitro, with its own kernel. It needs Kubernetes 1.36 or later. AWS calls it available today, without saying GA or preview.Here's why it matters. A normal container shares the host's kernel, and that's been fine for code your team wrote and reviewed. An agent runs code nobody reviewed. A container escape puts it on the host, next to everyone else's pods. A sandbox moves the boundary off that shared kernel.
Two cautions. Sandbox stage at the CNCF means early, so evaluate Agent Substrate and OpenShell rather than deploy them. And isolation limits what an agent can do to the host. It doesn't limit what the agent can reach with the credentials you hand it.
If you run coding agents or untrusted CI on Kubernetes, my guess at the combination to fix first is the default runtime, plus a broad service account, plus open egress.
WorkOS is the same question from the other side.
On November 2nd, WorkOS disclosed unauthorized access to an internal sales tool. The first access was October 25th. Nobody noticed until records were pulled in bulk on October 30th. On November 6th, a security researcher came forward: they'd been running benchmark evaluations with ARTEX, an AI offensive security framework, when their agent escaped its test environment and ended up at WorkOS. WorkOS and an independent forensics firm matched the researcher's evidence to its logs. The escape itself is the researcher's account, and nobody has said how it happened.
What I find most useful is WorkOS's own list of what let it in. The tool should not have been reachable from the internet. Its API accepted a weak credential meant for local development. There was no rate limiting. And it took five days to notice. None of that needed a clever agent.
WorkOS has since cut all its internal tools off from the internet, and it now runs ARTEX against its own systems. That's the right instinct. Agents, whoever runs them and whether or not anyone meant to, never get tired of trying the dev credential you forgot about.
The human closer is where this lands for me.
Brent Chapman's piece argues that incident resolution time isn't something you should target. It mostly depends on the incident, not on your process, and once it's a target, the number gets managed instead of the system. His fire department example stuck with me: targets for dispatch, turnout and travel time, but none for time on scene, because a grease fire and a warehouse fire aren't the same job. His test for what to target: if you can name two or three specific things you'd change to improve a step, it's controllable enough to target.
Run this week through that test.
You don't control whether a vendor you connected years ago gets breached. You do control whether that role still exists, and which addresses it accepts. You don't control what a coding tool writes to disk. You do control your
.dockerignore, and what your scanners check. And WorkOS couldn't control an agent showing up, but five days to notice is something a rate limit and a volume alert can change.The parts that are yours to change are usually smaller, and more boring, than a number on a dashboard.
Target what you can change, and track the rest.
Additional Links
SRE Weekly #542
N-able restores N-central cloud services (Cyber Press)