Apache Airflow · Monthly Virtual Town Hall · July 10, 2026

New Sandbox provider for
ephemeral task isolation

Fan out work in milliseconds. Run code you don’t trust in a fresh computer.

One fresh microVM per task try. Destroyed when the task ends.
apache/airflow #68847 · issue #68845 · dev@airflow discussion

Yossi Eliaz — Incredibuild · more writing ·

01 · Why now

Code nobody wrote is entering our DAGs

“Airflow lacks native sandboxing mechanisms… DAG authors can execute arbitrary code on workers.” — Airflow security model docs. Mitigation: code review.

That model assumed a human wrote and reviewed every task. With common.ai (Airflow’s provider for AI-agent tasks), agents now write task code at runtime — and Airflow runs it on the worker, with the worker’s credentials, reviewed by no one.

Agents also read the outside world, and an LLM can’t tell instructions from data — anything it reads can steer the code it writes. The fix: a sandbox — a microVM, a tiny full computer that boots in milliseconds, created per task try and deleted when it ends.

airflow.apache.org/docs/…/security/security_model.html

02 · Where Airflow stands, mid-2026

AIP-72 built a credential boundary — not a sandbox

AIP = Airflow Improvement Proposal — how the community designs and votes on big changes.

Mechanism	Actual boundary	Isolation
Task SDK / AIP-72	Workers hold zero DB credentials; per-task JWT via Execution API. But task code = ordinary OS process.	credential-only
Local / Celery	Shared long-lived worker: one filesystem, network identity, pip env for every task.	none
Multi-team / AIP-67	“Logical isolation… not complete isolation” — docs’ words, not mine.	logical
KubernetesExecutor	Pod per task — a real boundary, paid for in image-pull + schedule + bootstrap.	pod · slow
ECS / Fargate executor	Container per task, Fargate by default. ~50–60s startup latency per task (amazon-provider docs).	container · slow

Sources: Airflow security-model, multi-team & executor docs; amazon-provider ECS executor docs

03 · The trap

Today you choose:
fast-and-shared or isolated-and-slow

fast & shared

Celery-style workers start tasks instantly — into a process sharing everything on the node: one filesystem, one network identity, one pip env (your torch pin fights their pandas pin).

isolated & slow

Pod-per-task and Fargate give a real wall — and charge seconds-to-minutes of cold start on every try. Billed twice: warm fleets idle between bursts; cold starts tax retries.

No Airflow executor targets ephemeral cloud sandboxes.
That’s the gap issue #68845 names.

04 · The primitive

Two Airflow use cases, one disposable computer

fast fan-out / fan-in

Milliseconds to N workers

Start a hot snapshot, fork it across mapped tasks, collect artifacts or XComs, then delete every machine.

Eval shards, test shards, data probes, notebook cells, package-heavy Python.
Copy-on-write means N sandboxes cost one warm parent plus deltas.
Retries get clean computers instead of polluted workers.

controlled untrusted execution

A sandbox, plus a gate

Agent-written code, tenant code, partner scripts, and eval contestants run off the worker with scoped egress.

No worker credentials in sandbox memory by default.
Gateway injects allowed secrets per request and audits the call.
Network allow/deny rules define what the code can touch.

Airflow already has the shape: map out, fan in, retry, teardown. The missing part is the per-try computer.

The whole shift in one line: “the efficiency of multitenancy with the security of single tenancy.” — Modal

05 · What the agent platforms figured out

Per-task cold start, on a log scale

≈ 3–4 orders of magnitude faster — at a stronger isolation boundary.

Firecracker: AWS / NSDI ’20 (~125 ms boot, <5 MiB overhead, 150 VMs/s/host) · E2B docs · Firecracker snapshot docs · Airflow amazon-provider docs

06 · The mechanism

Snapshots replaced warm pools.
Forks replaced setup.

Pay for warm state once, reuse it N times. A “golden” parent runs pip install, imports, cache warmup — every task try forks from it, already hot.
Modal: import torch — ~5 s cold → ~1.05 s from a memory snapshot.
Cognition (Devin): 20 GB machine snapshot — ~30 min → ~200 ms with blockdiff.
Morph: fork a running VM — processes, memory, in-flight state — in < 250 ms.
AWS Lambda MicroVMs (Jun 2026): Firecracker sandboxes built to “safely run code that the application developer did not write” — near-instant launch/resume. One task, one VM is now an AWS primitive.
Copy-on-write economics: N forks ≈ 1 parent’s memory + deltas. One-sandbox-per-task becomes affordable.

modal.com/blog/mem-snapshots · cognition.ai blockdiff · morph.so/blog/infinibranch · aws.amazon.com/lambda/lambda-microvms

07 · The proposal — PR #68847

One ephemeral sandbox per task try

Scheduler

Queues task tries exactly as today. No core changes — rides AIP-51’s public executor interface.

→

SandboxExecutor

Creates a sandbox per try, uploads the workload, then only reconciles exit state via a polling watcher.

→

microVM sandbox

The task heartbeats its state to Airflow (EdgeExecutor-style) and ships logs to remote storage — no extra plumbing on the worker.

→

Destroy

Task ends, sandbox dies. Nothing to clean, nothing to leak into the next try.

Plus SandboxOperator and @task.sandbox for per-task opt-in — mix executors per deployment (AIP-61) and route only untrusted or heavy tasks to sandboxes.

github.com/apache/airflow/pull/68847 · at talk time (Jul 2026): +3,216 lines · 54 files · docs, tests, example DAG

08 · Pluggable backends

One contract, many vendor promises

SandboxProvider          # abstract base — backends/base.py
  authenticate() → create_sandbox(spec) → upload_workload()
  → run() → poll_status() → fetch_logs() → destroy()

SandboxSpec(image, cpu, memory_mb, disk_gb, timeout, env, …)
SandboxState: PENDING · RUNNING · SUCCEEDED · FAILED · GONE · UNKNOWN
# UNKNOWN ≠ GONE — transient API blips never kill a healthy task

localsubprocess · no creds · dev

Daytonasub-90 ms create · snapshots · network controls

E2BFirecracker · ~150 ms create · AI code runtime

ModalgVisor (sandboxed runtime) · memory snapshots

isloours — microVM agent computer · gatewayed secrets

Same shape across the ecosystem — instant sandboxes and agent computers are converging

Tensorlake“sandbox-native cloud for agents” · fork/suspend microVMs

AWS Lambda MicroVMsFirecracker · new Jun 2026

exe.devKVM VMs · SSH/root/systemd · gatewayed integrations

crabbox.shOSS contract · lease → sync → run → release · dozens of backends

Config: [sandbox] provider = local|daytona|e2b|modal|islo — or any module:Class path · exe.dev/Tensorlake/AWS/crabbox are market signals, not current PR backends

09 · What it looks like

Same DAG, different blast radius

from airflow.sdk import dag, task

@dag
def nightly_train():

    @task.sandbox(
        image="python:3.12-slim",
        cpu=2, memory_mb=2048,
    )
    def train():
        # returns the command the sandbox runs
        return "python train.py --epochs 3"

    train()

# airflow.cfg — all-in; or opt in
# per task via AIP-61 multi-executor
[core]
executor = SandboxExecutor

[sandbox]
provider = e2b
poll_interval = 5
creation_batch_size = 8

The callable returns the shell command that runs inside the sandbox; logs ship out before it dies, so every try leaves a complete record.

Status (Jul 2026): open PR. Update: merged 2026-08-13 as SandboxToolset in common.ai · standalone executor: github.com/zozo123/airflow-provider-sandbox

10 · What the community said — and the pivot

Feedback received. Direction changed.

“My first main concern… is how would this integrate with Common AI Provider and why should it be separate provider instead of part of common.ai.”

Kaxil Naik · converted PR to draft pending dev-list discussion

“It likely requires an AIP… your current PR and implementation will be treated as one potential solution — and mostly a starting point.”

Jarek Potiuk · plus: threat model required

“+1 to Kaxil’s comments… we discussed this offline earlier in the context of agentic workflows… interaction with the common.ai provider would be the logical first step.”

Vikram Koka · on the dev@ thread

So: the work pivots into common.ai as a SandboxToolset (proposed) — a first-class toolset like SQLToolset / MCPToolset. Agent-written code runs off the worker — so it no longer executes with the worker’s credentials.

The pivot · standalone executor lives on as a third-party package

11 · Honest limits — and what you can do today

What a sandbox doesn’t fix

still open

DAG processor & Triggerer still hold DB credentials
Tasks can still read every secret their deployment can — per-task scoping or an egress gateway is future work
Snapshot sharp edges: TCP connections die, cloned entropy needs VMGenID, GPU state largely unsnapshottable

today, zero airflow changes

gVisor / Kata (sandboxed container runtimes) via runtimeClassName in a KubernetesExecutor pod_override
gVisor (Google’s sandboxed container runtime) runs on any K8s cluster today — no special hardware, modest overhead; GKE Sandbox is the managed flavor
Route only sensitive tasks there with multiple executors (AIP-61)

This is proposal-stage engineering — the caveats are part of the design discussion, not footnotes.

12 · Where this goes — with your help

The community owns the outcome

Dev-list discussion — thread open on dev@airflow since June 24
AIP — scope, use cases, threat model & security properties
Focused common.ai PR — SandboxToolset, reviewed on its own
Dev-call demo — live @task.sandbox run, warts and all

Read #68845 & #68847.
Tell us your use case:
fan-out/fan-in? untrusted code? gatewayed agents?

Long-term maintainers committed (Yossi Eliaz, Yuval Raz — Incredibuild). Thanks to Elad Kalif (eladkal) for offering to sponsor, and to Kaxil, Jarek & Vikram for the early steering.

Make Airflow safe for code nobody wrote. · THANK YOU — QUESTIONS IN #TOWNHALL-SPEAKERS

New Sandbox provider forephemeral task isolation