SIGN IN SIGN UP
apache / airflow UNCLAIMED

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

44796 0 0 Python

Add gunicorn support for API server with rolling worker restarts (#60940)

Add optional gunicorn server type for the API server that provides:
- Memory sharing via preload + fork copy-on-write
- Rolling worker restarts through GunicornMonitor
- Correct FIFO signal handling (SIGTTOU kills oldest worker)

New configuration options in [api] section:
- server_type: uvicorn (default) or gunicorn
- worker_refresh_interval: seconds between refresh cycles (0=disabled)
- worker_refresh_batch_size: workers to refresh per cycle
- master_timeout: gunicorn master timeout
- reload_on_plugin_change: reload on plugin file changes

Requires apache-airflow-core[gunicorn] extra for gunicorn mode.

* Run GunicornMonitor in main thread instead of daemon thread

Matches Airflow 2's webserver pattern: monitor runs in main thread,
so if it crashes, the whole process exits (fail-fast). No silent
degradation where gunicorn keeps running without worker recycling.

Also triggers monitor when reload_on_plugin_change is enabled,
even if worker_refresh_interval is 0.

* Refactor gunicorn support to use custom Arbiter instead of external monitor

This refactor changes the gunicorn worker monitoring architecture from an
external thread-based approach to using a custom Arbiter subclass, which
is gunicorn's recommended extension pattern.

Changes:
- New gunicorn_app.py with AirflowArbiter and AirflowGunicornApp
- AirflowArbiter integrates worker refresh into manage_workers() loop
- Removed gunicorn_monitor.py (no longer needed)
- Simplified api_server_command.py (no subprocess, direct gunicorn API)
- Updated tests for new architecture

Benefits:
- Simpler architecture (no separate thread or subprocess)
- Direct access to worker state via self.WORKERS
- Uses gunicorn's internal spawn_worker/kill_worker methods
- Follows gunicorn's documented extension pattern
K
Kaxil Naik committed
b906080fe82cd72c9024dd62cf1b64c15f403ed8
Parent: d12f444
Committed by GitHub <noreply@github.com> on 2/4/2026, 9:45:09 AM