AWS CloudFormation Express Mode, Spark 4.2 Vector Search, AI Speeds Coding but Not Delivery, Platform Governance Developers Won’t Hate, and Why Could Is Not the Same as Should
Try reloading the player or open this episode directly on YouTube.
This episode of Ship It Weekly explores AWS CloudFormation Express mode, which accelerates deployment feedback but may mislead on resource readiness.
Now Playing
AWS CloudFormation Express Mode, Spark 4.2 Vector Search, AI Speeds Coding but Not Delivery, Platform Governance Developers Won’t Hate, and Why Could Is Not the Same as Should
Ship It Weekly
0:0018:34
Chapters
Jump to a section in this episode.
Speed & share
Transcript
AWS CloudFormation Express mode can report a
deployment complete up to four times faster,
but the resources may still be stabilizing in
the background. So the green check arrives faster,
but it means something different. Apache Spark
4.2 adds native vector operations, allowing
some AI data workloads to stay inside the platform
teams already operate. GitLab says that AI is
making developers faster at producing code, while
review, testing, governance, and deployment struggle
to keep up. And a platform team at Sevdesk has
a question for anyone rolling out new controls.
Will your internal users hate you by the time
you are finished? Put those together, and the
theme this week is local optimization. Making
one part of the system faster does not guarantee
the whole system gets better. Sometimes it just
moves the waiting, risk, or ownership problem
somewhere else. I'm Brian Teller from Teller's
Tech, and this is Ship It Weekly. Welcome back
to Ship It Weekly, the show about the DevOps,
SRE, cloud, platform, and security stories that
matter when you are the person keeping the thing
running at 3 a.m. For the weekly story list
and source links, check out OnCallBrief.com.
For full show notes and past episodes, head over
to ShipItWeekly.fm. This week: CloudFormation
Express mode, vector functionality in Spark 4.2,
GitLab's warning that AI is accelerating
code faster than software delivery, and minimum
viable governance for platform teams. In the
lightning round, OpenShift security updates,
a GitHub Actions outage, Cloudflare dropping
some POST requests, and GitHub Code Quality becoming
generally available with billing attached. And
we close with Reid Savage on the difference between
what a capable team could do and what it should
do. Let's get into it. First up, AWS introduced
CloudFormation Express mode. AWS says it can
reduce deployment completion time by up to four
times in some workflows, but it does not necessarily
make the underlying resources provision four
times faster. It changes when CloudFormation
declares the operation complete. In standard
mode, CloudFormation applies the configuration
and waits for resources to pass stabilization
checks. Express mode reports completion once
the configuration has been applied, while the
resource may continue becoming operational in
the background. AWS showed an SQS deployment
dropping from 64 seconds to around 10 seconds.
That is meaningful when you are repeatedly changing
and testing infrastructure, especially for AI
agents that are generating, deploying, and correcting
configurations. But deployment status is a contract.
Pipelines, dashboards, agents, and engineers
all make decisions based on what "complete" means.
What happens immediately afterwards? Do you run
integration tests? Deploy an application? Shift
traffic? Start another stack? With Express mode,
those steps may need their own readiness checks.
CloudFormation can be finished while the resource
is not ready. Express mode also disables rollback
by default, although teams can turn it back on.
That makes sense for development where you may
not want every failed experiment followed by
a long rollback. In production though, faster
confirmation without readiness validation, monitoring,
or rollback could let the pipeline move ahead
too early. The takeaway is not that Express mode
is unsafe. It looks useful. The takeaway is that
teams need to separate three states they often
treat as one. The configuration was accepted.
The resource finished provisioning. And the application
is ready for users. For development, testing,
and agent-driven workflows, the faster loop
could be a major improvement. For production,
the pipeline needs to understand exactly what
that green check now means. Faster confirmation
is useful. Confirmation is not readiness. Second
story: Apache Spark 4.2 adds native support
for more vector-oriented operations. Spark SQL
now includes functions for similarity, distance
calculations, normalization, and vector aggregation.
It also adds NEAREST BY, a top-K ranking join
for matching rows based on similarity or distance.
That supports workloads like recommendations,
entity matching, candidate generation, and offline
retrieval. This does not mean Spark replaces
every vector database. A specialized system serving
low-latency application requests is solving a
different problem. But not every vector workload
is an interactive search request. Sometimes the
data is already in Spark. Sometimes you are evaluating
embeddings, enriching large datasets, or generating
CloudFormation reports success faster. Spark absorbs more AI-oriented workloads. Coding agents generate more code. Platform teams automate more controls. Each of those can be a real improvement, but improving one stage does not automatically improve the system around it.
Sometimes the bottleneck disappears. Sometimes it just moves.
CloudFormation Express mode is a good example. AWS can now return deployment completion before every resource has finished stabilizing. That could make development and agent-driven infrastructure loops much faster, especially when you are repeatedly changing a template and waiting several minutes to discover one bad property.
But the speed comes from changing what “complete” means.
CloudFormation may be done while the resource is still becoming operational. That is not necessarily a problem, but pipelines have to understand the distinction. A green deployment should not automatically trigger integration tests, traffic shifts, or dependent work unless something else confirms that the resource is actually ready.
The uncertainty did not disappear. CloudFormation just stopped waiting for it.
Spark 4.2 raises a different version of the same question. Native vector operations and nearest-neighbor joins mean some teams can keep recommendation, matching, retrieval, and embedding workloads inside Spark rather than introducing another specialized platform.
That does not make every vector database unnecessary. Low-latency application serving is different from large-scale batch analysis. But it should push teams to start with the workload instead of the product category.
Do we need a dedicated serving layer? Or is the data already in Spark, with the work happening offline?
Every new platform comes with credentials, networking, scaling, monitoring, patching, backups, cost, and another operational boundary. Sometimes the specialized system is absolutely worth it. Other times, the best platform decision is not adding one.
The GitLab research gets closer to the organizational side of this. Developers say AI is helping them produce code faster, but review and validation are not accelerating at the same pace.
That feels predictable once you stop treating software delivery as typing.
Software delivery is a chain of queues. Implementation feeds review. Review feeds testing. Testing feeds deployment. Deployment feeds operations. If one stage suddenly produces twice as much work, the next stage does not magically gain twice the capacity.
The pull request gets created faster, but someone still has to understand it. Terraform gets generated in minutes, but someone still has to evaluate the blast radius. More changes reach production, but the on-call team still owns what happens afterward.
A local productivity gain can become a system-level slowdown when it increases work in progress faster than the organization can validate it.
The traceability issue may be even more important. When something fails, can the team determine where the change came from, what it was intended to do, what context the agent had, what validation ran, and who accepted responsibility for it?
Those are not just AI governance questions. They are basic incident-response questions.
Generating code is becoming cheaper. Understanding and operating it is not getting cheaper at the same rate.
The Sevdesk platform-governance story shows what happens when teams forget that internal platforms are still products.
The technically simple approach is to publish a long list of requirements, set a deadline, and start blocking deployments. The controls may be correct, but the rollout can still fail because developers receive noise instead of useful guidance.
Sevdesk took a more gradual approach: build visibility, identify ownership, group findings, provide feedback, soft-enforce, and then enforce. They called it minimum viable governance.
I like that framing because it does not mean minimum standards. It means starting with the smallest set of controls that meaningfully reduces the company’s risk and introducing them in a way the organization can absorb.
If every finding is urgent, none of them are.
If every policy failure becomes a ticket, the platform becomes another queue.
And if the only thing developers experience is a failed deployment, the platform team may win the policy argument while losing adoption entirely.
The platform team’s product is not the policy. The product is changed behavior.
The lightning-round stories reinforced the same theme in smaller ways.
OpenShift being supported does not mean it is current. GitHub’s autoscaling configuration was valid, but it was based on thresholds that no longer matched the live system. Cloudflare showed why a successful GET request does not prove that POST requests, logins, purchases, or webhooks are working. GitHub Code Quality showed how a preview feature can quietly become both a production dependency and a production bill.
In each case, the surface signal was incomplete.
Supported is not patched.
Valid is not correct.
Reachable is not functional.
Enabled is not free.
That leads naturally into Reid Savage’s article about their first year managing an SRE team.
“Could versus should” sounds like a question about ownership, and it is. A capable SRE team could take the ticket, operate the service, debug the deployment, or absorb another responsibility that does not have a clear home.
But Reid’s larger point is about judgment.
Sometimes a manager should make the call. Sometimes they should recommend a direction. Sometimes they should add context. Sometimes they should stay quiet and let the team own the decision.
The challenge is not knowing what you are capable of doing. It is understanding which kind of participation improves the outcome.
That applies to technology decisions too.
CloudFormation can return faster. Should this production pipeline use that mode?
Spark can perform the vector operation. Should the workload live there?
AI can generate more code. Should all of it immediately enter the review queue?
The platform can block every violation. Should blocking be the first way developers learn that a policy exists?
Capability expands the decision space. It does not make the decision for us.
Good engineering increases what is possible.
Good judgment decides what is useful.
Faster is not the same as finished.
And could is not the same as should.
📝 Notes
Show Notes
This week on Ship It Weekly: AWS CloudFormation Express mode promises faster infrastructure feedback by reporting deployments complete before extended resource stabilization finishes. Apache Spark 4.2 adds native vector operations and nearest-neighbor joins, giving some teams a way to keep AI data workloads closer to the platforms they already run.
GitLab’s latest research says AI is helping developers generate and commit code faster, but review, testing, governance, and deployment are not accelerating at the same pace. Then, a platform engineering case study from Sevdesk shows how minimum viable governance, useful feedback, and progressive enforcement can improve compliance without turning the platform team into another approval queue.
The theme this week: making one part of the system faster does not automatically improve the whole system.
In the lightning round: OpenShift 4.22.5 receives an important security update, outdated autoscaling thresholds disrupt GitHub Actions, Cloudflare experiences an incident where some POST requests fail to reach customer origins, and GitHub Code Quality becomes generally available—with automatic billing attached.
The episode closes with Reid Savage reflecting on their first year managing Honeycomb’s SRE team and the difference between what a capable team could do and what it should do.
This episode is about local optimization.
CloudFormation reports success faster. Spark absorbs more AI-oriented workloads. Coding agents generate more code. Platform teams automate more controls. Each of those can be a real improvement, but improving one stage does not automatically improve the system around it.
Sometimes the bottleneck disappears. Sometimes it just moves.
CloudFormation Express mode is a good example. AWS can now return deployment completion before every resource has finished stabilizing. That could make development and agent-driven infrastructure loops much faster, especially when you are repeatedly changing a template and waiting several minutes to discover one bad property.
But the speed comes from changing what “complete” means.
CloudFormation may be done while the resource is still becoming operational. That is not necessarily a problem, but pipelines have to understand the distinction. A green deployment should not automatically trigger integration tests, traffic shifts, or dependent work unless something else confirms that the resource is actually ready.
The uncertainty did not disappear. CloudFormation just stopped waiting for it.
Spark 4.2 raises a different version of the same question. Native vector operations and nearest-neighbor joins mean some teams can keep recommendation, matching, retrieval, and embedding workloads inside Spark rather than introducing another specialized platform.
That does not make every vector database unnecessary. Low-latency application serving is different from large-scale batch analysis. But it should push teams to start with the workload instead of the product category.
Do we need a dedicated serving layer? Or is the data already in Spark, with the work happening offline?
Every new platform comes with credentials, networking, scaling, monitoring, patching, backups, cost, and another operational boundary. Sometimes the specialized system is absolutely worth it. Other times, the best platform decision is not adding one.
The GitLab research gets closer to the organizational side of this. Developers say AI is helping them produce code faster, but review and validation are not accelerating at the same pace.
That feels predictable once you stop treating software delivery as typing.
Software delivery is a chain of queues. Implementation feeds review. Review feeds testing. Testing feeds deployment. Deployment feeds operations. If one stage suddenly produces twice as much work, the next stage does not magically gain twice the capacity.
The pull request gets created faster, but someone still has to understand it. Terraform gets generated in minutes, but someone still has to evaluate the blast radius. More changes reach production, but the on-call team still owns what happens afterward.
A local productivity gain can become a system-level slowdown when it increases work in progress faster than the organization can validate it.
The traceability issue may be even more important. When something fails, can the team determine where the change came from, what it was intended to do, what context the agent had, what validation ran, and who accepted responsibility for it?
Those are not just AI governance questions. They are basic incident-response questions.
Generating code is becoming cheaper. Understanding and operating it is not getting cheaper at the same rate.
The Sevdesk platform-governance story shows what happens when teams forget that internal platforms are still products.
The technically simple approach is to publish a long list of requirements, set a deadline, and start blocking deployments. The controls may be correct, but the rollout can still fail because developers receive noise instead of useful guidance.
Sevdesk took a more gradual approach: build visibility, identify ownership, group findings, provide feedback, soft-enforce, and then enforce. They called it minimum viable governance.
I like that framing because it does not mean minimum standards. It means starting with the smallest set of controls that meaningfully reduces the company’s risk and introducing them in a way the organization can absorb.
If every finding is urgent, none of them are.
If every policy failure becomes a ticket, the platform becomes another queue.
And if the only thing developers experience is a failed deployment, the platform team may win the policy argument while losing adoption entirely.
The platform team’s product is not the policy. The product is changed behavior.
The lightning-round stories reinforced the same theme in smaller ways.
OpenShift being supported does not mean it is current. GitHub’s autoscaling configuration was valid, but it was based on thresholds that no longer matched the live system. Cloudflare showed why a successful GET request does not prove that POST requests, logins, purchases, or webhooks are working. GitHub Code Quality showed how a preview feature can quietly become both a production dependency and a production bill.
In each case, the surface signal was incomplete.
Supported is not patched.
Valid is not correct.
Reachable is not functional.
Enabled is not free.
That leads naturally into Reid Savage’s article about their first year managing an SRE team.
“Could versus should” sounds like a question about ownership, and it is. A capable SRE team could take the ticket, operate the service, debug the deployment, or absorb another responsibility that does not have a clear home.
But Reid’s larger point is about judgment.
Sometimes a manager should make the call. Sometimes they should recommend a direction. Sometimes they should add context. Sometimes they should stay quiet and let the team own the decision.
The challenge is not knowing what you are capable of doing. It is understanding which kind of participation improves the outcome.
That applies to technology decisions too.
CloudFormation can return faster. Should this production pipeline use that mode?
Spark can perform the vector operation. Should the workload live there?
AI can generate more code. Should all of it immediately enter the review queue?
The platform can block every violation. Should blocking be the first way developers learn that a policy exists?
Capability expands the decision space. It does not make the decision for us.
Good engineering increases what is possible.
Good judgment decides what is useful.
Faster is not the same as finished.
And could is not the same as should.