|
16 | 16 | under the License. |
17 | 17 |
|
18 | 18 |
|
19 | | -
|
20 | 19 | Quick Start |
21 | 20 | ----------- |
22 | 21 |
|
23 | | -This quick start guide will help you bootstrap an Airflow standalone instance on your local machine. |
24 | | - |
25 | | -.. note:: |
| 22 | +Install Airflow on your machine and start it with ``airflow standalone``, which runs the scheduler, Dag |
| 23 | +processor, triggerer and API server together on a SQLite database. Use it to try Airflow and to develop |
| 24 | +Dags locally, not in production. For other ways to install and run Airflow, see :doc:`/installation/index`. |
26 | 25 |
|
27 | | - Successful installation requires a Python 3 environment. Airflow supports Python 3.11, 3.12, 3.13, 3.14. |
| 26 | +Before you start |
| 27 | +'''''''''''''''' |
28 | 28 |
|
29 | | - Officially supported installation methods are ``pip`` or ``uv``. |
| 29 | +You need a Python version that this Airflow release supports; see :doc:`/installation/prerequisites`. |
30 | 30 |
|
31 | | - Run ``pip install apache-airflow[EXTRAS]==AIRFLOW_VERSION --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-AIRFLOW_VERSION/constraints-PYTHON_VERSION.txt"``, for example :subst-code:`pip install "apache-airflow[celery]==|version|" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.11.txt"` to install Airflow in a reproducible way. You can also use - much faster - ``uv`` - by adding ``uv`` before the command. |
| 31 | +On Windows, set up WSL2 first: run ``wsl --install`` in PowerShell as administrator and restart, as described |
| 32 | +in `Install WSL <https://learn.microsoft.com/en-us/windows/wsl/install>`__. Then run every command below in |
| 33 | +the Ubuntu terminal that WSL2 installs. |
32 | 34 |
|
| 35 | +Install Airflow |
| 36 | +''''''''''''''' |
33 | 37 |
|
| 38 | +Create a virtual environment and install Airflow into it with the |
| 39 | +:ref:`constraints file <installation:constraints>`, which pins every dependency to the version this release |
| 40 | +was tested with. A later release of a dependency then does not change what you install. |
34 | 41 |
|
35 | | - While there have been successes with using other tools like `poetry <https://python-poetry.org/>`_ or |
36 | | - `pip-tools <https://pypi.org/project/pip-tools/>`_, they do not share the same workflow as |
37 | | - ``pip`` or ``uv`` - especially when it comes to constraint vs. requirements management. |
38 | | - Installing via ``Poetry`` or ``pip-tools`` is not currently supported. |
| 42 | +.. tab-set:: |
39 | 43 |
|
40 | | - If you wish to install Airflow using those tools you should use the constraint files and convert |
41 | | - them to appropriate format and workflow that your tool requires. |
| 44 | + .. tab-item:: uv |
| 45 | + :sync: uv |
42 | 46 |
|
43 | | - This guide will help you quickly set up Apache Airflow using ``uv``, a fast and modern tool for managing Python environments and dependencies. ``uv`` makes the installation process easy and provides a |
44 | | - smooth setup experience. |
| 47 | + If you do not have uv yet, install it from the |
| 48 | + `uv installation guide <https://docs.astral.sh/uv/getting-started/installation/>`__. Then create the |
| 49 | + virtual environment and install Airflow: |
45 | 50 |
|
46 | | - If you are on Windows, you have to use WSL2 (Linux environment for Windows). |
| 51 | + .. code-block:: bash |
| 52 | + :substitutions: |
47 | 53 |
|
48 | | - .. code-block:: bash |
| 54 | + uv venv |
| 55 | + source .venv/bin/activate |
49 | 56 |
|
50 | | - wsl --install |
| 57 | + AIRFLOW_VERSION=|version| |
| 58 | + PYTHON_VERSION="$(python -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")')" |
| 59 | + CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt" |
| 60 | + uv pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}" |
51 | 61 |
|
52 | | -1. **Set Airflow Home (optional)**: |
| 62 | + ``uv venv`` uses a Python it finds on your machine. To choose a supported version instead, pass it |
| 63 | + with ``--python``, and uv downloads it if it is missing. |
53 | 64 |
|
54 | | - Airflow requires a home directory, and uses ``~/airflow`` by default, but you can set a different location if you prefer. The ``AIRFLOW_HOME`` environment variable is used to inform Airflow of the desired location. This step of setting the environment variable should be done before installing Airflow so that the installation process knows where to store the necessary files. |
| 65 | + .. tab-item:: pip |
| 66 | + :sync: pip |
55 | 67 |
|
56 | | - .. code-block:: bash |
| 68 | + On Debian and Ubuntu, including WSL2, install the ``venv`` module first with |
| 69 | + ``sudo apt update && sudo apt install python3-venv``. Without it, ``python3 -m venv`` fails with |
| 70 | + ``The virtual environment was not created successfully because ensurepip is not available``. |
57 | 71 |
|
58 | | - export AIRFLOW_HOME=~/airflow |
| 72 | + Then create the virtual environment and install Airflow: |
59 | 73 |
|
60 | | -2. Install Airflow in a virtual environment using ``uv`` since it is a faster alternative that creates the ``venv`` automatically for you. It is an efficient alternative to using ``pip`` and ``venv``. |
| 74 | + .. code-block:: bash |
| 75 | + :substitutions: |
61 | 76 |
|
62 | | - .. rst-class:: centered |
| 77 | + python3 -m venv .venv |
| 78 | + source .venv/bin/activate |
| 79 | + pip install --upgrade pip |
63 | 80 |
|
64 | | - Install uv: `uv Installation Guide <https://docs.astral.sh/uv/getting-started/installation/>`_ |
| 81 | + AIRFLOW_VERSION=|version| |
| 82 | + PYTHON_VERSION="$(python -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")')" |
| 83 | + CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt" |
| 84 | + pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}" |
65 | 85 |
|
| 86 | +If the virtual environment's Python is a version this release does not support, no constraints file exists for |
| 87 | +it and the install stops at the constraints URL. pip reports ``ERROR: 404 Client Error: Not Found for url:`` |
| 88 | +and uv reports ``error: Error while accessing remote requirements file:``, each followed by the URL. Recreate |
| 89 | +the virtual environment with a supported Python. |
66 | 90 |
|
67 | | - For creating virtual environment with ``uv``, refer to the documentation here: |
68 | | - `Creating and Maintaining Local virtual environment with uv <https://github.com/apache/airflow/blob/main/contributing-docs/07_local_virtualenv.rst#creating-and-maintaining-local-virtualenv-with-uv-recommended>`_ |
| 91 | +Start Airflow |
| 92 | +''''''''''''' |
69 | 93 |
|
70 | | -For installation using ``pip`` and ``venv``, carry out the following steps. |
71 | | -On Debian/Ubuntu systems, Python may enforce |
72 | | -externally managed environments (PEP 668), so use a virtual environment |
73 | | -before running ``pip install`` commands: |
| 94 | +With the virtual environment activated, start every component with one command: |
74 | 95 |
|
75 | 96 | .. code-block:: bash |
76 | | - :substitutions: |
77 | | -
|
78 | | - # For Windows after WSL2 install, restart computer, then in WSL Ubuntu terminal |
79 | | - sudo apt update |
80 | | - sudo apt install python3-pip python3-venv |
81 | | -
|
82 | | - # Go to Linux home directory (not Windows mount) |
83 | | - cd ~ |
84 | | -
|
85 | | - # Create airflow directory |
86 | | - mkdir -p ~/airflow |
87 | | - cd ~/airflow |
88 | | -
|
89 | | - # Create virtual environment |
90 | | - python3 -m venv airflow_venv |
91 | | -
|
92 | | - # Activate |
93 | | - source airflow_venv/bin/activate |
94 | | -
|
95 | | - # Upgrade pip |
96 | | - pip install --upgrade pip |
97 | | -
|
98 | | - # Install Airflow with correct Python version constraints |
99 | | - pip install apache-airflow[celery]==|version| --constraint https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.12.txt |
100 | | -
|
101 | | - # Verify installation |
102 | | - airflow version |
103 | | -
|
104 | | -3. Install Airflow using the constraints file, which is determined based on the URL we pass: |
105 | | - |
106 | | - .. code-block:: bash |
107 | | - :substitutions: |
108 | | -
|
109 | | -
|
110 | | - AIRFLOW_VERSION=|version| |
111 | | -
|
112 | | - # Extract the version of Python you have installed. If you're currently using a Python version that is not supported by Airflow, you may want to set this manually. |
113 | | - # See above for supported versions. |
114 | | - PYTHON_VERSION="$(python -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")')" |
115 | 97 |
|
116 | | - CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt" |
117 | | - # For example this would install |version| with python 3.11: https://raw.githubusercontent.com/apache/airflow/constraints-|version|/constraints-3.11.txt |
| 98 | + airflow standalone |
118 | 99 |
|
119 | | - uv pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}" |
| 100 | +On the first run, Airflow creates ``~/airflow`` with its configuration file ``airflow.cfg`` and a SQLite |
| 101 | +database, and generates a password for the ``admin`` user. The password appears near the start of the |
| 102 | +output. Once the API server, scheduler, Dag processor and triggerer are all running, Airflow prints a ready |
| 103 | +banner. The components' logs fill the lines in between: |
120 | 104 |
|
121 | | -4. Run Airflow Standalone: |
| 105 | +.. code-block:: text |
122 | 106 |
|
123 | | - The ``airflow standalone`` command initializes the database, creates a user, and starts all components. |
| 107 | + standalone | Starting Airflow Standalone |
| 108 | + [warning ] SimpleAuthManager is active but the deployment shape looks like production (non-sqlite backend, non-local API host, or a distributed executor). ... |
| 109 | + Simple auth manager | Password for user 'admin': <generated-password> |
| 110 | + ... |
| 111 | + standalone | Airflow is ready |
| 112 | + standalone | Airflow Standalone is for development purposes only. Do not use this in production! |
124 | 113 |
|
125 | | - .. code-block:: bash |
| 114 | +The ``SimpleAuthManager is active but the deployment shape looks like production`` warning is expected for a |
| 115 | +local standalone run: the API server listens on ``0.0.0.0`` by default, which the warning treats as a |
| 116 | +non-local host. |
126 | 117 |
|
127 | | - airflow standalone |
| 118 | +Press ``Ctrl+C`` to stop every component. |
128 | 119 |
|
129 | | - .. note:: |
| 120 | +Airflow keeps its files in ``~/airflow`` unless you set ``AIRFLOW_HOME``. If you set it, export the same value |
| 121 | +in every terminal before you run ``airflow``: a terminal without it uses ``~/airflow``, with a separate |
| 122 | +database and password. To start Airflow again later, open a terminal in the directory where you created the |
| 123 | +virtual environment, run ``source .venv/bin/activate``, then ``airflow standalone``. |
130 | 124 |
|
131 | | - In Airflow 3.x, the admin password may not always be displayed in the terminal output when running ``airflow standalone``. |
| 125 | +Log in and run a Dag |
| 126 | +'''''''''''''''''''' |
132 | 127 |
|
133 | | - The password is automatically generated and stored in: |
134 | | - |
135 | | - ``$AIRFLOW_HOME/simple_auth_manager_passwords.json.generated`` |
136 | | - |
137 | | - To retrieve it, run: |
138 | | - |
139 | | - .. code-block:: bash |
140 | | -
|
141 | | - cat ~/airflow/simple_auth_manager_passwords.json.generated |
142 | | -
|
143 | | - Use this password to log in to the web interface instead of default credentials. |
144 | | - |
145 | | -5. Access the Airflow UI: |
146 | | - |
147 | | - Visit ``localhost:8080`` in your browser and log in with the admin account details shown in the terminal. Enable the ``example_bash_operator`` DAG on the home page. |
148 | | - |
149 | | -Upon running these commands, Airflow will create the ``$AIRFLOW_HOME`` folder |
150 | | -and create the "airflow.cfg" file with defaults that will get you going fast. |
151 | | -You can override defaults using environment variables, see :doc:`/configurations-ref`. |
152 | | -You can inspect the file either in ``$AIRFLOW_HOME/airflow.cfg``, or through the UI in |
153 | | -the ``Admin->Configuration`` menu. The PID file for the webserver will be stored |
154 | | -in ``$AIRFLOW_HOME/airflow-api-server.pid`` or in ``/run/airflow/airflow-webserver.pid`` |
155 | | -if started by systemd. |
156 | | - |
157 | | -As you grow and deploy Airflow to production, you will also want to move away |
158 | | -from the ``standalone`` command we use here to running the components |
159 | | -separately. You can read more in :doc:`/administration-and-deployment/production-deployment`. |
160 | | - |
161 | | -Here are a few commands that will trigger a few task instances. You should |
162 | | -be able to see the status of the jobs change in the ``example_bash_operator`` Dag as you |
163 | | -run the commands below. |
| 128 | +Open ``http://localhost:8080`` and log in as ``admin`` with the password from the |
| 129 | +``Password for user 'admin'`` line. Airflow prints that line only on the first run; later runs print a line |
| 130 | +saying the password was previously generated, with the path of the file that holds it. To read the file: |
164 | 131 |
|
165 | 132 | .. code-block:: bash |
166 | 133 |
|
167 | | - # run your first task instance |
168 | | - airflow tasks test example_bash_operator runme_0 2015-01-01 |
169 | | - # run a backfill over 2 days |
170 | | - airflow backfill create --dag-id example_bash_operator \ |
171 | | - --from-date 2015-01-01 \ |
172 | | - --to-date 2015-01-02 |
173 | | -
|
174 | | -If you want to run the individual parts of Airflow manually rather than using |
175 | | -the all-in-one ``standalone`` command, you can instead run: |
176 | | - |
177 | | -.. code-block:: bash |
178 | | -
|
179 | | - airflow db migrate |
180 | | -
|
181 | | - airflow users create \ |
182 | | - --username admin \ |
183 | | - --firstname Peter \ |
184 | | - --lastname Parker \ |
185 | | - --role Admin \ |
186 | | - --email spiderman@superhero.org |
| 134 | + cat "${AIRFLOW_HOME:-$HOME/airflow}/simple_auth_manager_passwords.json.generated" |
187 | 135 |
|
188 | | - airflow api-server --port 8080 |
| 136 | +It maps each user to their password: |
189 | 137 |
|
190 | | - airflow scheduler |
| 138 | +.. code-block:: text |
191 | 139 |
|
192 | | - airflow dag-processor |
| 140 | + {"admin": "<generated-password>"} |
193 | 141 |
|
194 | | - airflow triggerer |
| 142 | +Airflow loads a set of example Dags, all paused. Open **Dags**, find ``example_bash_operator``, and switch on |
| 143 | +its toggle to unpause it. The scheduler starts a run within a few seconds; click the Dag name to follow its |
| 144 | +tasks. |
195 | 145 |
|
196 | | -.. note:: |
197 | | - ``airflow users`` command is only available when :doc:`apache-airflow-providers-fab:auth-manager/index` is enabled. |
| 146 | +What's next |
| 147 | +''''''''''' |
198 | 148 |
|
199 | | -What's Next? |
200 | | -'''''''''''' |
201 | | -From this point, you can head to the :doc:`/tutorial/index` section for further examples or the :doc:`/howto/index` section if you're ready to get your hands dirty. |
| 149 | +* :doc:`/tutorial/index` to write your first Dag |
| 150 | +* :doc:`/installation/installing-from-pypi` for extras, providers, and how constraints files work |
| 151 | +* :doc:`/configurations-ref` for the settings in ``airflow.cfg`` |
| 152 | +* :ref:`starting-components-separately` to run the API server, scheduler, Dag processor, |
| 153 | + and triggerer as separate processes |
| 154 | +* :doc:`/howto/docker-compose/index` or :doc:`helm-chart:index` for container and Kubernetes setups |
| 155 | +* :doc:`/administration-and-deployment/production-deployment` when you move beyond standalone |
| 156 | +* :doc:`/howto/index` for common configuration tasks |
0 commit comments