On Call Brief – Week of November 15–21, 2026

2026-11-15 — 2026-11-21 Briefing: 2026-11-15 Last updated 17 hours ago (Nov 15, 2026 3:50 am EST) 11 min read
Share
Category:
Tags:

This week's top stories

1. Flights halted at airports across US due to air traffic control communications problems, FAA says

  • Category: Deep Dive
  • What happened: The FAA grounded all flights across the United States on November 15, 2026, due to a telecommunications outage that affected regional air traffic control centers, resulting in over 800 flight diversions before the ground stop was lifted. The incident exposed single points of failure not only in aviation systems but also in dependent infrastructure like payment processors, which faced disruptions to ACH transfer timing windows and vendor availability during the outage. Operators managing systems with strict timing dependencies or third-party integrations should review failover mechanisms and vendor SLAs to ensure critical operations can continue during telecommunications failures. The FAA is still investigating the root cause of the telecommunications outage that triggered the nationwide disruption.
  • Takeaway: Informational: significant disruption in air traffic control communications.
  • Sources: Apnews via DevOps'ish, dev.to (SRE tag)
  • Tags:

2. GitHub Actions: 8 related updates

  • Category: Community
  • What happened: A series of GitHub Actions troubleshooting patterns and optimization techniques emerged in November 2026, covering common workflow performance and configuration issues. Slow checkouts typically result from specific inputs overriding the default fetch-depth of 1, not from inadequate shallow clone settings, while the fromJSON matrix expansion failures trace to producer jobs not writing output, invalid JSON formats, empty lists, or mangled JSON syntax. The "reserveCache failed" error in version 1 cache service stems from distinct root causes including DNS failures, size limit violations, and job race conditions that require separate diagnostic approaches. The "unrecognized named-value: inputs" error occurs when expressions reference inputs in unsupported contexts such as action input defaults or workflow-level keys where the inputs context is not available. For runner infrastructure decisions, GitHub Actions Runner Controller (ARC) provides greater control but demands Kubernetes expertise, while managed runners like Blacksmith offer simpler setup at $0.004 per minute for 2 vCPU Linux instances compared to GitHub's $0.006 rate for private repositories as of November 2026.
  • Worth reading: Evaluate your team's expertise in Kubernetes before choosing ARC over managed runners for GitHub Actions.
  • Sources: dev.to (DevOps tag), dev.to (DevOps tag) (2), dev.to (DevOps tag) (3) (+5 more)
  • Tags:

3. See what your Argo CD applications cost

  • Category: Community
  • What happened: Cost Intelligence for Argo CD is an extension that adds a cost tab to applications, querying OpenCost for monthly spend projections, flagging idle resources, and exporting cost metrics to Prometheus. It has limitations, including only accessing the cluster where both Argo CD and OpenCost run and relying on list prices without cloud billing data.
  • Worth reading: Consider integrating the Cost Intelligence extension for Argo CD to monitor application costs and optimize resource usage.
  • Source: Octopus via DevOps'ish
  • Tags:

4. gcloud resource_exhausted quota ci: rate quota or allocation

  • Category: Community
  • What happened: The article explains the different error messages produced by Google Cloud when exceeding quotas, specifically focusing on the 'RESOURCE_EXHAUSTED' error. It details the causes of these errors, such as exceeding rate quotas due to parallel jobs or accumulating resources against allocation quotas. The author emphasizes the importance of understanding which type of quota has been exceeded to determine the appropriate response, such as waiting for a reset or adjusting resource usage.
  • Worth reading: Monitor your gcloud usage patterns to identify if you're exceeding rate or allocation quotas and adjust your deployment strategy accordingly.
  • Source: dev.to (DevOps tag)
  • Tags:

5. What we write down after every database restore drill

  • Category: Deep Dive
  • What happened: The article discusses the importance of documenting database restore drills to capture valuable insights beyond just successful restores. It emphasizes the need for detailed logs that include restore times, the specific backup used, and any issues encountered during the drill. A tool called backupdrill is introduced, which helps automate the logging of these drills for Supabase projects. The article also highlights the significance of measuring restore times accurately and understanding the factors that affect them, such as the environment in which the restore is performed.
  • Takeaway: Implement a detailed logging process for database restore drills using the provided template to capture insights and improve future drills.
  • Source: dev.to (SRE tag)
  • Tags:

6. Cutover Needs a No-Go: Microsoft, Workday, the FAA, and Starship Flight 15

  • Category: Deep Dive
  • What happened: The article discusses four significant events in mid-November 2026, focusing on lessons learned about change discipline and verification during cutovers. It highlights a Microsoft 365 outage caused by failed server health checks during a network capacity upgrade, emphasizing the importance of reliable health checks as gates in production environments. Additionally, it mentions a Workday incident where a disaster-recovery test inadvertently affected production tenants, underscoring the risks associated with testing in live environments. The author advocates for a disciplined approach to software cutovers, using their tool MigrateForge as an example of maintaining control during transitions.
  • Takeaway: Informational: highlights the importance of reliable health checks and disciplined cutover practices.
  • Source: dev.to (SRE tag)
  • Tags:

7. Financial Research Dashboard Retrieval: Scheduling PDF Chunks Without Full Reindexing

  • Category: Community
  • What happened: The article discusses strategies for improving the retrieval of financial research data from a dashboard, focusing on incremental indexing and the use of staged retrieval with explicit collections. It emphasizes the importance of defining retrieval contracts and maintaining separate observable stages for ingestion, querying, and answer citation. The author suggests using Infrai for managing vector collections and highlights the need for alerting on discrepancies between accepted and retrievable versions to avoid stale data.
  • Worth reading: Implement incremental indexing and alerting on retrieval gaps to ensure data freshness in financial dashboards.
  • Source: dev.to (SRE tag)
  • Tags:

8. Docker manifest unknown when pulling in CI

  • Category: Community
  • What happened: The article explains the 'Docker manifest unknown' error that occurs when pulling images in CI, detailing common causes such as incorrect tags, missing platforms in the manifest list, format disagreements between client and registry, and deleted or moved tags. It emphasizes that this error is not related to authentication issues and provides insights on how to diagnose and fix the problem.
  • Worth reading: Ensure that all tags are correctly pushed and verify that the correct platform is included in the manifest list to avoid 'Docker manifest unknown' errors.
  • Source: dev.to (DevOps tag)
  • Tags:

9. Docker net/http request canceled on a registry call

  • Category: Community
  • What happened: The article discusses the 'net/http request canceled' error encountered during Docker registry operations, explaining that it typically occurs due to transient network issues, slow name resolution, or proxy/firewall configurations. It emphasizes that this error is often intermittent and suggests implementing a retry mechanism with backoff to address it effectively.
  • Worth reading: Implement a retry mechanism with backoff for Docker registry operations to handle transient network issues.
  • Source: dev.to (DevOps tag)
  • Tags:

10. KubeVirt: 2 related updates

  • Category: Community
  • What happened: KubeVirt has graduated from the Cloud Native Computing Foundation in November 2026 after ten years of development since its inception by Red Hat. The project, which enables running virtual machines on Kubernetes, now has over 2,000 contributors and is deployed in production by major organizations including ByteDance and Cloudflare. Operators running KubeVirt in their Kubernetes clusters should review the graduated project's stability commitments and consider it for production workloads requiring VM orchestration alongside containers. The graduation status indicates the project has met CNCF's criteria for production readiness, sustainability, and community governance.
  • Worth reading: Informational: KubeVirt's graduation may influence VM deployment strategies in Kubernetes.
  • Sources: CNCF via DevOps'ish, DevOps'ish
  • Tags:

Releases

11. Cloudflare Introduces Private Origins for Application Services

  • Category: Release
  • What happened: Cloudflare has launched Application Services for Private Origins, enabling public traffic to access applications on private networks while keeping their IP addresses hidden and not requiring open inbound firewall ports. However, private-to-private connectivity is not supported yet.
  • Do this Monday: Evaluate the use of Cloudflare's Application Services for Private Origins to enhance security for applications hosted on private networks.
  • Source: InfoQ DevOps
  • Tags:

12. Visual Studio Code 1.147 (Insiders)

  • Category: Release
  • What happened: Visual Studio Code Insiders 1.147 introduces features such as agent session management, chat resources, and enhancements to Markdown editing.
  • Takeaway: Informational: a new version of Visual Studio Code Insiders has been released.
  • Source: VS Code Blog
  • Tags:

Also this week

Community reads

13. Adyen Releases kubectl-rexec Tool for Auditable Kubernetes Pod Access

  • Category: Community
  • What happened: Adyen has released a tool called kubectl-rexec, which simplifies the process of executing commands in Kubernetes pods while maintaining an audit trail. This tool is designed to enhance security and compliance by ensuring that all kubectl exec actions are logged.
  • Worth reading: Informational: a new tool for Kubernetes auditing.
  • Source: GitHub via DevOps'ish
  • Tags:

14. could not get lock /var/lib/dpkg/lock, and who is holding it

  • Category: Community
  • What happened: The article explains the 'could not get lock /var/lib/dpkg/lock' error in CI environments, detailing its causes and implications. It describes three scenarios that lead to this error: a process holding the lock, a missing process name due to timing, and permission issues. The author emphasizes that understanding the specific error message is crucial for diagnosing the problem, particularly in automated environments where package management tasks may overlap.
  • Worth reading: Informational: understanding apt lock errors can help troubleshoot CI failures.
  • Source: dev.to (DevOps tag)
  • Tags:

Status pages & maintenance

15. HashiCorp: Emergency Maintenance Notification – HCP Radar

  • Category: Status Page
  • What happened: Emergency maintenance is ongoing for the HCP Radar service, causing brief unavailability of billing, metering, and onboarding agent flows. The maintenance is taking longer than expected, with a new completion time set for 09:30 UTC.
  • Heads up: Monitor the status of HCP Radar and prepare for potential service disruptions until maintenance is complete.
  • Source: HashiCorp Status
  • Tags:

16. Cloudflare: 23 scheduled maintenance windows (Austin, Tokyo, Singapore, +11 more)

  • Category: Status Page
  • What happened: Cloudflare has scheduled datacenter maintenance across 17 locations between November 16 and November 20, 2026, including Singapore (SIN) on November 16-20, Hong Kong (HKG) on November 17, Newark (EWR) on November 17-19, Boston (BOS) on November 17, Austin (AUS), Tokyo (NRT), Athens (ATH), Guadalajara (GDL), and Guatemala City (GUA) on November 18, as well as Anchorage (ANC), Sacramento (SMF), Albuquerque (ABQ), San Antonio (SAT), and Oklahoma City (OKC) on November 17. During the specified maintenance windows (ranging from 01:00-05:00 UTC to 20:00-00:00 UTC depending on location), operators should expect traffic rerouting and increased latency for regional users. PNI/CNI customers should prepare for potential traffic failover and temporary unavailability of network interfaces during these windows. No action is required beyond monitoring regional traffic patterns and ensuring failover configurations are operational if you serve traffic through these specific Cloudflare datacenters.
  • Heads up: Prepare for potential traffic failover and increased latency for users in the AUS region during the maintenance window.
  • Sources: Cloudflare Status, Cloudflare Status (2), Cloudflare Status (3) (+20 more)
  • Tags:

(No additional items this week.)

Human Stories

The FAA telecommunications outage that grounded flights nationwide and the HCP Radar emergency maintenance running longer than expected both remind us that our systems exist in a world that doesn't pause when things break. What strikes me about this week's stories is how much of our maturity as an industry shows up in what we do before and after the crisis, not just during it. The team writing down lessons from every database restore drill understands something fundamental: the muscle memory we build in calm moments is what saves us when we're running on adrenaline and caffeine at 3am. Meanwhile, tools like cost intelligence for Argo CD and the detailed breakdowns of Google Cloud quota errors represent the same philosophy applied differently - making the invisible visible before it becomes a problem. KubeVirt's ten-year journey to graduation is a quiet reminder that the most important infrastructure work often happens in the unglamorous middle years, when nobody's watching and the headlines have moved on.