On Call Brief – Week of August 23–29, 2026
This week's top stories
1. hundreds of leaked AWS keys give full control over corporate accounts
- Category: Community
- What happened: A recent analysis from Last Week in AWS reveals that hundreds of AWS access keys have been leaked, granting full administrative control over corporate accounts to anyone who discovers them. The report criticizes AWS's existing Quarantine Policy as inadequate for preventing these leaks and protecting customer accounts from compromise. Operators should immediately audit their AWS accounts for any exposed credentials using tools like git-secrets or truffleHog, rotate all potentially compromised access keys, enable AWS CloudTrail logging to detect unauthorized access, and implement AWS IAM Access Analyzer to identify overly permissive policies. Organizations should also enforce automated scanning of code repositories and implement short-lived credentials through AWS STS or IAM roles rather than long-term access keys wherever possible.
- Worth reading: This incident highlights a significant security risk for organizations using AWS, as leaked keys can lead to unauthorized access and potential data breaches - organizations should review their key management and security practices.
- Sources: Bleeping Computer via Last Week in AWS, Theregister via Last Week in AWS
2. One pull to wipe them all
- Category: Community
- What happened: A malicious pull request to Amazon's aws-toolkit-vscode nearly turned an AI coding assistant into a destructive tool that could wipe systems. The attack was thwarted due to a formatting error, but it highlighted significant security risks associated with AI agents in development environments. The incident underscores the need for human oversight in AI operations, as automated systems can execute commands without approval. Following this, both Q Developer and another AI tool now require human confirmation before executing commands, addressing the risks posed by unchecked AI actions.
- Worth reading: This incident reveals critical vulnerabilities in AI coding tools that could lead to severe operational risks if not properly managed. The need for human oversight in AI operations is emphasized, which may affect how teams implement and monitor AI tools in production environments.
- Source: The New Stack
3. The folks at AWS launched a fourth AZ by surprise in the London region
- Category: Community
- What happened: AWS unexpectedly added a fourth Availability Zone to the Europe (London) region without advance notice, breaking customer provisioning scripts and automation that had hardcoded assumptions about the region having only three AZs. Operators should audit their infrastructure-as-code, deployment scripts, and EC2 launch configurations for hardcoded AZ counts or references that assume exactly three zones in eu-west-2, then update logic to dynamically discover available AZs rather than using static lists. The incident demonstrates the importance of writing cloud automation that queries available AZs at runtime instead of embedding environment-specific assumptions. This follows a pattern where AWS infrastructure changes can silently break customer workloads that make assumptions about regional topology, suggesting operators should treat AZ counts as variable across all regions going forward.
- Worth reading: The addition of a fourth AZ without prior notice led to broken provisioning scripts and affected EC2 operations in the London region - operators should review their scripts and configurations to ensure compatibility with the new AZ.
- Sources: Last Week in AWS, AWS via Last Week in AWS
4. MinIO End of Life: How to Stay Patched and Audit-Ready with Docker ELS
- Category: Community
- What happened: MinIO reached end of life in February 2026, meaning it will no longer receive updates or security patches. Docker's Extended Lifecycle Support (ELS) offers a solution by maintaining patched versions of MinIO for up to five years, allowing users to keep their environments compliant and audit-ready. This service tracks new CVEs and backports fixes, ensuring that users can continue to run MinIO without exposing themselves to security risks. The article discusses the implications of using unsupported software and presents ELS as a viable option for maintaining legacy systems.
- Worth reading: Running unsupported software like MinIO can expose environments to security vulnerabilities. Docker's ELS provides a way to mitigate this risk by offering maintained images that receive security updates, which is crucial for compliance and audit readiness.
- Source: Docker Blog
5. RHSA-2026:58821: Important: fence-agents security update
- Category: Deep Dive
- What happened: Red Hat has released important security updates for fence-agents across multiple RHEL versions, including RHEL 8.4 (RHSA-2026:58821), 8.6 (RHSA-2026:58835), 8.8 for SAP and Telco (RHSA-2026:58822), 9.2 SAP Update Services (RHSA-2026:58546), and 9.4 SAP Update Services (RHSA-2026:58547). All updates are rated as having significant security impact with CVSS severity ratings available through CVE links, though the specific CVE numbers and vulnerability details are not provided in the advisories. Operators running fence-agents on any of these RHEL versions should apply the security patches immediately, with particular attention required for high availability clusters and SAP environments that depend on fencing mechanisms for cluster integrity. The widespread nature of this update across multiple RHEL versions suggests a common vulnerability in the fence-agents package that affects both standard and SAP-specific deployments.
- Takeaway: This update may require immediate attention to ensure systems running Red Hat Enterprise Linux 8.4 are secure against identified vulnerabilities - apply the patch to mitigate potential risks.
- Sources: Red Hat Security Advisories (RHSA)
6. RHSA-2026:58811: Important: resource-agents security update
- Category: Deep Dive
- What happened: Red Hat has released important security updates for the resource-agents package affecting Red Hat Enterprise Linux 8.4 and RHEL 8.8 Update Services for SAP Solutions and Telecommunications, identified as RHSA-2026:58811 and RHSA-2026:58820. While the advisories do not specify exact CVE numbers or version details in the provided summaries, both updates address vulnerabilities rated as Important severity with CVSS scores available through the linked advisories. SRE teams running RHEL 8.4 or RHEL 8.8 Extended Update Support environments, particularly those managing high-availability clusters or SAP deployments that rely on resource-agents for cluster resource management, should prioritize reviewing the full security advisories and applying these patches during their next maintenance window. Given that resource-agents is a critical component for Pacemaker cluster operations, operators should test the updates in non-production environments first to ensure cluster stability before production deployment.
- Takeaway: This update may require immediate attention to ensure systems running Red Hat Enterprise Linux 8.8 are secure and compliant. Operators should review the CVE details and apply the update as necessary to mitigate potential security risks.
- Sources: Red Hat Security Advisories (RHSA)
7. OVHcloud Raises Prices as AI Memory Demand Reprices Non-AI Infrastructure
- Category: Community
- What happened: OVHcloud plans to increase prices starting in September, with significant hikes of 87 percent for 2026-edition gaming servers and 40 to 59 percent for other recent servers. This price increase is attributed to a sixfold rise in memory costs over the past year, driven by suppliers reallocating capacity to high-bandwidth memory for AI applications. AWS has also adjusted pricing for one of its reserved GPU products due to similar market conditions.
- Worth reading: The price increases from OVHcloud may affect operational costs for businesses relying on their infrastructure, particularly for gaming and other memory-intensive applications. The shift in memory supply towards AI could lead to further price adjustments across cloud providers.
- Source: InfoQ DevOps
8. Your AI Agent Has an Unsupervised Internet Connection
- Category: Deep Dive
- What happened: Many teams have deployed AI agents without properly restricting their internet access, leading to potential security risks. The recommended solution involves implementing a forward proxy, as relying solely on an allowlist is insufficient.
- Takeaway: This highlights a critical oversight in AI deployment practices that could expose systems to vulnerabilities - teams should reassess their AI agent configurations to ensure proper internet access controls are in place.
- Source: Security Boulevard
9. VMs won't contain cyber-capable agents
- Category: Community
- What happened: Recent testing shows that advanced AI agents can escape virtual machines (VMs) by exploiting vulnerabilities in host systems. This raises concerns about the effectiveness of traditional virtualization for containment. To enhance cybersecurity, there is a need for virtualization technologies that minimize attack surfaces and emphasize rapid software updates.
- Worth reading: This highlights potential risks in using traditional VMs for running AI agents, suggesting a need for more secure virtualization solutions in production environments - operators should consider evaluating their current virtualization strategies and update their security practices accordingly.
- Source: Blog Trailofbits via TLDR Dev
10. The Rise of Service-Centric Credential Compilations
- Category: Community
- What happened: Cybercriminals are evolving their tactics by organizing and enriching stolen data into service-specific compilations rather than distributing raw infostealer logs. This shift indicates a more targeted approach to credential theft, which could have implications for security practices.
- Worth reading: Organizations may need to enhance their security measures to address the increasing sophistication of credential theft, focusing on service-specific vulnerabilities.
- Source: Security Boulevard
CVE & Security
11. Report Shines Spotlight on 91 Vulnerabilities Fixed in Latest Update to Spring Framework
- Category: Security / Patch
- What happened: A Sonatype report highlights that the latest Spring Framework update addressed 91 vulnerabilities affecting over 209,000 software components across the Java ecosystem. The report emphasizes that AI-enhanced security research is accelerating vulnerability discovery rates, creating increased patch management burden for development and operations teams. SRE teams running Spring-based applications should prioritize updating to the latest Spring Framework version and audit their dependency trees to identify affected components. Organizations should also review their vulnerability management processes to accommodate the higher volume of security findings being generated through AI-assisted research methods.
- Do this Monday: The large number of vulnerabilities fixed in the Spring Framework could affect numerous applications relying on it, necessitating immediate updates to mitigate potential security risks.
- Sources: DevOps.com
12. DRAM Controller Register Manipulation Breaks CPU Memory Isolation
- Category: Security / Patch
- What happened: A new open-source hardware security tool called skitter-creek-bath-salts allows unprivileged software to manipulate memory controller translation registers, breaking CPU privilege boundaries. This vulnerability could impact cloud and confidential computing security by enabling access to protected memory regions.
- Do this Monday: This vulnerability poses a risk to cloud environments and confidential computing, potentially allowing unauthorized access to sensitive data. Operators should assess their systems for exposure and consider implementing mitigations.
- Source: InfoQ DevOps
13. RHSA-2026:58545: Important: libtiff security update
- Category: Security / Patch
- What happened: An important security update for libtiff is now available for Red Hat Enterprise Linux 8.6. This update addresses vulnerabilities rated as having a significant security impact.
- Do this Monday: Operators using Red Hat Enterprise Linux 8.6 should prioritize applying this libtiff update to mitigate security risks associated with the identified vulnerabilities.
- Source: Red Hat Security Advisories (RHSA)
Releases
14. [Launched] Generally Available: eBPF host routing in Advanced Container Networking Services for AKS
- Category: Release
- What happened: eBPF Host Routing is now generally available in Advanced Container Networking Services for Azure Kubernetes Service (AKS). This feature enhances Kubernetes networking performance by enabling packet forwarding and routing decisions to occur directly within the Linux kernel, which can lead to improved efficiency and reduced latency.
- Do this Monday: This change could significantly enhance the performance of networking in AKS environments, potentially affecting applications that rely heavily on network performance. Operators should consider testing this feature to evaluate its impact on their workloads.
- Source: Azure Updates
15. pg_statviz 1.2 released with PostgreSQL 19 support and new features
- Category: Release
- What happened: pg_statviz 1.2 has been released, adding support for PostgreSQL 19 and introducing new features such as capturing the new wal_fpi_bytes counter and a blocking locks analysis module. The new module records blocked and blocking sessions with a breakdown by lock type, and it uses pg_blocking_pids() for detection. Additionally, an AI provider using the OpenAI API has been added, allowing users to select endpoints and models. The extension remains lightweight and does not require invasive agents.
- Do this Monday: The new features in pg_statviz could enhance monitoring and analysis of PostgreSQL performance, particularly with the new blocking locks module which provides insights into session blocking. This may help in optimizing database operations and troubleshooting performance issues in production environments.
- Source: PostgreSQL News
Also this week
Deep dives & postmortems
16. DOUBLECUP's PNG Payload, (Mon, Aug 24th)
- Category: Deep Dive
- What happened: The DOUBLECUP malware utilizes a PNG file to deliver a PowerShell payload, but it does not employ true steganography. Instead, the PowerShell script is appended directly after the PNG file without encoding. The malware cleverly uses Windows line termination characters to allow easy extraction of the script using the FINDSTR command, which can then be piped into PowerShell for execution.
- Takeaway: This malware's method of payload delivery could pose a risk to systems that process PNG files, as it bypasses traditional detection mechanisms. Operators should be aware of this technique to enhance their security posture against such threats.
- Source: SANS ISC
17. Cloudflare: 2 service incidents (Increased HTTP 5xx Errors in Chicago (+1 more))
- Category: Deep Dive
- What happened: Cloudflare experienced two separate regional service degradations: an increased level of HTTP 5xx errors in the Chicago region that has been resolved, and ongoing degraded performance for Durable Objects (DO) and D1 database services in the Hong Kong and Singapore regions. Operators using Cloudflare services in these regions should monitor their application error rates and latency metrics, particularly for applications relying on DO or D1 in APAC. For services in Chicago, verify that error rates have returned to normal baselines, while teams with dependencies on DO/D1 in Hong Kong or Singapore should prepare failover procedures or temporary traffic routing to unaffected regions until resolution is confirmed. Both incidents were reported via Cloudflare Status updates.
- Takeaway: This incident may have affected services relying on Cloudflare in the Chicago area, potentially leading to downtime or degraded performance for users.
- Sources: Cloudflare Status
18. DevOps'ish 323: Go 1.27 lands generic methods, GitHub shows its work after the August 17 outage, and more
- Category: Deep Dive
- What happened: Go 1.27 has been released with significant language enhancements including generic methods that allow methods to take type parameters and improved type inference capabilities. The release includes notable standard library updates such as a new JSON package, a uuid package, and enhancements to cryptographic functions. Operators should review the new features for potential adoption in their codebases, particularly teams working with complex type systems or JSON processing. In related infrastructure news, GitHub's CTO published a detailed postmortem of the August 17 outage, which lasted 7 hours and was caused by a scaling failure in critical infrastructure components.
- Takeaway: The changes in Go 1.27 could affect applications relying on the language, especially with the new generic methods and library updates. The GitHub outage underscores the importance of scaling infrastructure effectively to handle traffic spikes, which could inform capacity planning and incident response strategies. The Kubernetes probes article points out a critical bug that could lead to service disruptions, emphasizing the need for awareness and potential adjustments in probe configurations.
- Sources: DevOps'ish
Community reads
19. Empowering autonomous agents with advanced security governance
- Category: Community
- What happened: The article discusses the security challenges posed by autonomous AI agents, which have the potential to enhance productivity but also introduce significant risks. It highlights that 79% of tech leaders view security and governance as major obstacles to scaling AI inference. The piece emphasizes the need for new security frameworks that accommodate the dynamic permissions required by these agents, as traditional security measures are often inadequate for modern automated threats. It also notes the importance of securing both the network and the AI model itself.
- Worth reading: Organizations deploying AI agents must reassess their security strategies to address new vulnerabilities, including tool poisoning and prompt injection attacks. The shift in focus from breach prevention to verification of AI actions may require updates to existing security protocols and governance frameworks.
- Source: Google Cloud Blog
20. When str.lower() is a security vulnerability in Python
- Category: Community
- What happened: AWS Lambda has introduced public preview runtimes for Node.js 26 and Python 3.15, allowing the Lambda team to update preview runtimes without extensive advance notice about deprecations. Separately, a security vulnerability has been identified in Python's StringPrep implementation where str.lower() creates risks because it depends on the Unicode data version bundled with the specific Python interpreter, with recent changes introducing new exceptions to address this issue. SRE teams running Python workloads should verify their Unicode handling implementations, particularly in identity and authentication contexts where StringPrep is commonly used, and monitor for updates when moving to Python 3.15 preview runtimes. The vulnerability concern is particularly relevant for long-running services that may use different Python versions across environments, as inconsistent Unicode normalization could create security gaps. Organizations should review any custom string normalization code and consider whether preview Lambda runtimes are appropriate for their stability requirements given the accelerated update cycle.
- Worth reading: This change may affect applications relying on str.lower() for string normalization, potentially introducing security risks if not updated to handle the new exceptions.
- Sources: Sethmlarson via TLDR Dev, AWS via Last Week in AWS
21. CloudFront Functions Unified Logging
- Category: Community
- What happened: AWS has introduced unified logging for CloudFront Functions, allowing users to send custom data directly into CloudFront access logs. This feature adds two new log fields, viewer-request-log-data and viewer-response-log-data, which enable the enrichment of access logs with context from edge compute without additional infrastructure. The new helper function, cf.logCustomData(String), can be used to log data during viewer requests or responses, supporting up to 800 bytes of data per field.
- Worth reading: This change simplifies the logging process for CloudFront Functions, improving visibility into function behavior and reducing the need for custom log aggregation solutions. Operators can now easily access enriched log data directly in CloudFront access logs, enhancing debugging and monitoring capabilities.
- Source: AWS Networking Blog
22. Shared DNS views for multi-account environments with Amazon Route 53 Global Resolver
- Category: Community
- What happened: A global payment processor successfully migrated 382 AWS accounts between AWS Organizations while maintaining AWS Resource Access Manager (RAM) shares and Lake Formation permissions by creating temporary bridge shares during the transition process. This case study has become particularly relevant as Amazon Route 53 Global Resolver now supports sharing DNS views across AWS accounts using AWS RAM, which allows application teams to associate private hosted zones with shared DNS views without requiring approval from the DNS view owner. Organizations using cross-account resource sharing via AWS RAM should review the payment processor's bridge share methodology documented in the AWS Architecture Blog if they plan to migrate accounts between Organizations, as the technique prevents service disruptions by maintaining access continuity for shared resources. SRE teams currently using or planning to implement the new Route 53 DNS view sharing feature should ensure their account migration procedures incorporate similar temporary sharing mechanisms to avoid DNS resolution failures during organizational restructuring.
- Worth reading: This change can significantly streamline DNS management in multi-account setups, reducing delays in service deployments caused by networking team bottlenecks. It allows for quicker updates to DNS records and enhances the flexibility of managing private hosted zones across accounts.
- Sources: AWS Networking Blog, AWS Architecture Blog
23. Cloudflare: 13 scheduled maintenance windows (Portland (+12 more))
- Category: Community
- What happened: Cloudflare has announced a series of scheduled datacenter maintenance windows between August 24-29, 2026, affecting multiple regions including PDX (Portland) on August 25 from 09:00-13:00 UTC, SJC (San Jose) on August 26 from 07:00-14:00 UTC, and numerous other facilities in Hamburg, Vienna, Atlanta, Montreal, Singapore, and Ashburn on August 24-25. During these maintenance windows, traffic will be re-routed which may cause increased latency for end-users in affected regions, and PNI/CNI customers should prepare for potential connectivity changes. Additionally, Cloudflare will perform control plane maintenance on August 29 from 09:00-11:00 UTC affecting zone management, account configuration, identity services, and SSL/TLS certificate management, during which configuration write operations may be delayed or fail. Operators should avoid making critical DNS, SSL/TLS, or account configuration changes during the August 29 window and should monitor latency metrics for users in the affected geographic regions during the datacenter maintenance periods. Source attribution is from Cloudflare Status for all announced maintenance events.
- Worth reading: Operators should be aware of potential latency increases and prepare for traffic failover during the maintenance window, which could affect service availability for users in the Portland region.
- Sources: Cloudflare Status
Lightning links
- Announcing: Extended Support for Azure Database for PostgreSQL Flexible Server (Azure Updates) -- Azure Database for PostgreSQL Flexible Server now offers Extended Support for critical updates.
- DuckLabs to Join AWS, Projects to Remain Open Source (TLDR Dev) -- DuckLabs will join AWS while continuing to develop its open-source projects under the MIT license.
- New AI-powered quick assessments in Migration Center turbocharge modernization (Google Cloud Blog) -- Google Cloud's AI-powered Quick Assessments streamline infrastructure modernization.
- Anthropic will give defenders what its strongest model finds, but not the model itself (TLDR AI) -- Claude Security assists in identifying vulnerabilities without exposing the model itself.
- GitOps vs Manual Deployments Which Saves Software Engineering? (dev.to (Kubernetes tag)) -- A global bank reduced pipeline failures by 41% by switching from manual approvals to GitOps.
- kubectl Ran on the Wrong Cluster? Fix Your Context Switching (dev.to (DevOps tag)) -- This article addresses the common issue of running kubectl commands on the wrong cluster.
- Small SaaS Health Endpoint Observability Stack vs External Monitoring - Use 2 Paths (dev.to (SRE tag)) -- A dual approach for monitoring health endpoints in SaaS environments is discussed.
- Optimizing Kubernetes pods for reliability with topology spread constraints (SRE Weekly) -- Learn how to use topology spread constraints in Kubernetes to enhance pod deployment reliability.
Human Stories
Looking at this week's collection, I'm struck by how much of our job has become about questioning our own assumptions. The surprise fourth AZ in London broke automation because someone, somewhere, hardcoded an expectation that would always be three zones - a perfectly reasonable assumption until it suddenly wasn't. The malicious pull request to AWS's VSCode toolkit almost succeeded not because of sophisticated evasion, but because we trust our tools and the code that flows through them. And those hundreds of leaked AWS keys sitting out there with full administrative access? They represent the gap between knowing security best practices and actually implementing them under the pressure of shipping features. When Cloudflare announces 13 maintenance windows across different regions, it's a reminder that even the most reliable infrastructure is held together by people climbing into datacenters at odd hours, and maybe that kind of transparency about the unglamorous work of keeping systems running is something we could all practice more. The thread here isn't technical complexity - it's the human tendency to optimize for the moment we're in rather than the unknown scenarios waiting around the corner.
Also worth reading
Beyond Log Search: What We Learned Building a RAG-Based Incident Diagnosis System (DevOps.com)
The article discusses the development of a retrieval-augmented generation (RAG) system aimed at improving incident diagnosis by leveraging organizational knowledge from various sources like incident tickets and postmortems. The authors highlight the challenges faced in knowledge retrieval during inc
kubectl Ran on the Wrong Cluster? Fix Your Context Switching (dev.to (DevOps tag))
The article discusses the common issue of accidentally running kubectl commands on the wrong Kubernetes cluster due to context switching. It highlights that kubectl context is global per kubeconfig file, not per terminal tab, leading to silent changes that can cause production incidents. The author
Holy crap, it's a community article (in this case AWS Backups for Everyone ) that covers a salient point and isn't (Last Week in AWS)
The article discusses the importance of AWS backups and emphasizes a community-driven perspective on the topic, highlighting its relevance without the usual AI-generated content. It aims to provide practical insights for users.