Installing Airflow

Airflow can be installed several ways depending on your goal: a quick pip install for local experimentation, a Docker Compose stack for a more production-like local environment, or a managed service like Astronomer, MWAA, or Cloud Composer for production. Because Airflow has many optional dependencies (providers for AWS, GCP, Postgres, and so on), the Apache project publishes constraint files pinned to specific Python versions to keep installs reproducible — installing without them is one of the most common sources of dependency conflicts.

For local development, the simplest path is a virtual environment plus a pip install using the official constraints URL, which pins every dependency to a version known to work with your chosen Airflow and Python versions. For a more realistic setup that includes the scheduler, webserver, and a Postgres metadata database running as separate containers, the official docker-compose.yaml from the Airflow documentation is the standard starting point — it spins up all the core services with one command and is what most tutorials and this course assume you're using.

Airflow stores all of its state — DAG definitions metadata, task instance history, connections, variables, and user accounts — in a metadata database, usually Postgres or MySQL in production (SQLite only for the quickest local trials, since it can't handle the LocalExecutor's parallelism). Before Airflow can run anything, you must initialize this database with airflow db init or, on newer versions, airflow db migrate, which creates the schema, and typically create an admin user with airflow users create so you can log into the web UI.

Once services are up, visit localhost:8080 and log in with the admin credentials you created; you should see the DAGs list view with a handful of example DAGs (unless you disabled load_examples in airflow.cfg). A healthy install shows the scheduler and webserver both reporting green/healthy status in the UI's admin health panel, and the example DAG example_bash_operator should be triggerable and complete successfully within a minute or two.