Skip to content

Commit 68e4c4d

Browse files
authored
Remove Backport Providers (#14886)
We are removing support for Backport Providers now. The last release was sent yesterday- as planned, on 17 March 2021 - the last release of the Backport Providers. As agreed before, and documented here: https://github.com/apache/airflow/blob/master/dev/PROJECT_GUIDELINES.md#support-for-backport-providers > Backport providers within 1.10.x, will be supported for critical fixes for three months (March 17, 2021) from Airflow 2.0.0 release date (Dec 17, 2020). For the future reference, if anyone would like to build backport providers with cherry-picking any fixes, the branch to start from is `legacy-backport-cutoff-point`. The documentation and tools to build the backports are there, but there will be no more community releases for backports. Good Bye Backport Providers.
1 parent 2f32df7 commit 68e4c4d

459 files changed

Lines changed: 332 additions & 23403 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/build-images-workflow-run.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ jobs:
120120
sourceRunId: ${{ github.event.workflow_run.id }}
121121
notifyPRCancel: true
122122
jobNameRegexps: >
123-
["^Pylint$", "^Static checks", "^Build docs$", "^Spell check docs$", "^Backport packages$",
123+
["^Pylint$", "^Static checks", "^Build docs$", "^Spell check docs$",
124124
"^Provider packages", "^Checks: Helm tests$", "^Test OpenAPI*"]
125125
- name: "Extract canceled failed runs"
126126
# We use this step to build regexp that will be used to match the Source Run id in

‎.github/workflows/ci.yml‎

Lines changed: 0 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -486,62 +486,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
486486
github.event_name == 'push'
487487
run: aws s3 sync --delete ./files/documentation s3://apache-airflow-docs
488488

489-
prepare-backport-provider-packages:
490-
timeout-minutes: 40
491-
name: "Backport packages: ${{ matrix.package-format }}"
492-
runs-on: ${{ fromJson(needs.build-info.outputs.runsOn) }}
493-
needs: [build-info, ci-images]
494-
strategy:
495-
matrix:
496-
package-format: ['wheel', 'sdist']
497-
env:
498-
# In this case we want to install airflow from the latest released 1.10 version
499-
# all provider packages are installed from wheels or .tar.gz files
500-
INSTALL_AIRFLOW_VERSION: "1.10.14"
501-
AIRFLOW_EXTRAS: "all"
502-
PYTHON_MAJOR_MINOR_VERSION: ${{needs.build-info.outputs.defaultPythonVersion}}
503-
BACKPORT_PACKAGES: "true"
504-
VERSION_SUFFIX_FOR_PYPI: "dev"
505-
VERSION_SUFFIX_FOR_SVN: "dev"
506-
PACKAGE_FORMAT: ${{ matrix.package-format }}
507-
GITHUB_REGISTRY: ${{ needs.ci-images.outputs.githubRegistry }}
508-
if: needs.build-info.outputs.image-build == 'true'
509-
steps:
510-
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
511-
uses: actions/checkout@v2
512-
with:
513-
persist-credentials: false
514-
- name: "Setup python"
515-
uses: actions/setup-python@v2
516-
with:
517-
python-version: ${{ env.PYTHON_MAJOR_MINOR_VERSION }}
518-
- name: "Free space"
519-
run: ./scripts/ci/tools/ci_free_space_on_ci.sh
520-
- name: "Prepare CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
521-
run: ./scripts/ci/images/ci_prepare_ci_image_on_ci.sh
522-
- name: "Prepare provider documentation"
523-
run: ./scripts/ci/provider_packages/ci_prepare_provider_documentation.sh
524-
- name: "Prepare provider packages: ${{ matrix.package-format }}"
525-
run: ./scripts/ci/provider_packages/ci_prepare_provider_packages.sh
526-
- name: "Install and test provider packages and airflow via ${{ matrix.package-format }} files"
527-
run: ./scripts/ci/provider_packages/ci_install_and_test_provider_packages.sh
528-
env:
529-
INSTALL_PROVIDERS_FROM_SOURCES: "false"
530-
- name: "Upload package artifacts"
531-
uses: actions/upload-artifact@v2
532-
if: always()
533-
with:
534-
name: airflow-backport-packages
535-
path: "./dist/apache*"
536-
retention-days: 7
537-
- name: "Upload documentation artifacts"
538-
uses: actions/upload-artifact@v2
539-
if: always() && matrix.package-format == 'wheel'
540-
with:
541-
name: airflow-backport-readmes
542-
path: "./files/airflow-readme-*"
543-
retention-days: 7
544-
545489
prepare-provider-packages:
546490
timeout-minutes: 40
547491
name: "Provider packages ${{ matrix.package-format }}"
@@ -1112,7 +1056,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
11121056
- tests-postgres
11131057
- tests-mysql
11141058
- tests-kubernetes
1115-
- prepare-backport-provider-packages
11161059
- prepare-provider-packages
11171060
- test-provider-packages-released-airflow
11181061
- prod-images
@@ -1162,7 +1105,6 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
11621105
- tests-postgres
11631106
- tests-mysql
11641107
- tests-kubernetes
1165-
- prepare-backport-provider-packages
11661108
- ci-images
11671109
- verify-ci-images
11681110
- docs

‎.rat-excludes‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,6 @@ rat-results.txt
7979
apache-airflow-.*\+source.tar.gz.*
8080
apache-airflow-.*\+bin.tar.gz.*
8181
PULL_REQUEST_TEMPLATE.md
82-
BACKPORT_PROVIDER_CHANGES*.md
8382
PROVIDER_CHANGES*.md
8483
manifests/*
8584
redirects.txt

‎BREEZE.rst‎

Lines changed: 9 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -562,9 +562,6 @@ There are several commands that you can run in Breeze to manage and build packag
562562
Preparing provider readme files is part of the release procedure by the release managers
563563
and it is described in detail in `dev <dev/README.md>`_ .
564564

565-
You can prepare provider packages - by default regular provider packages are prepared, but with
566-
``--backport`` flag you can prepare backport packages.
567-
568565
The packages are prepared in ``dist`` folder. Note, that this command cleans up the ``dist`` folder
569566
before running, so you should run it before generating airflow package below as it will be removed.
570567

@@ -582,13 +579,6 @@ providers that you would like to build. By default ``both`` types of packages ar
582579
583580
./breeze prepare-provider-packages google amazon
584581
585-
You can also prepare backport provider packages, if you specify ``--backport`` flag. You can read more
586-
about backport packages in `dev <dev/README.md>`_
587-
588-
.. code-block:: bash
589-
590-
./breeze prepare-provider-packages --backports google amazon
591-
592582
You can see all providers available by running this command:
593583

594584
.. code-block:: bash
@@ -1789,10 +1779,6 @@ This is the current syntax for `./breeze <./breeze>`_:
17891779
17901780
Default: both
17911781
1792-
--backports
1793-
1794-
Prepares backport providers rather than regular ones.
1795-
17961782
-v, --verbose
17971783
Show verbose information about executed docker, kind, kubectl, helm commands. Useful for
17981784
debugging - when you run breeze with --verbose flags you will be able to see the commands
@@ -2149,7 +2135,7 @@ This is the current syntax for `./breeze <./breeze>`_:
21492135
Detailed usage for command: prepare-provider-documentation
21502136
21512137
2152-
breeze prepare-provider-documentation [FLAGS] [YYYY.MM.DD] [PACKAGE_ID ...]
2138+
breeze prepare-provider-documentation [FLAGS] [PACKAGE_ID ...]
21532139
21542140
Prepares documentation files for provider packages.
21552141
@@ -2158,23 +2144,14 @@ This is the current syntax for `./breeze <./breeze>`_:
21582144
If no packages are specified, readme for all packages are generated.
21592145
If no date is specified, current date + 3 days is used (allowing for PMC votes to pass).
21602146
2161-
You can also specify --backport flag to prepare backport providers documentation and in this
2162-
case you can also optionally specify CALVER version as first parameter.
2163-
21642147
Examples:
21652148
21662149
'breeze prepare-provider-documentation' or
2167-
'breeze prepare-provider-documentation --version-suffix-for-pypi rc1' or
2168-
'breeze prepare-provider-documentation --backports 2020.05.10' or
2169-
'breeze prepare-provider-documentation --backports 2020.05.10 https google amazon'
2150+
'breeze prepare-provider-documentation --version-suffix-for-pypi rc1'
21702151
21712152
General form:
21722153
2173-
'breeze prepare-provider-documentation YYYY.MM.DD <PACKAGE_ID> ...'
2174-
2175-
* YYYY.MM.DD - is the CALVER version of the package to prepare. Note that this date
2176-
cannot be earlier than the already released version (the script will fail if it
2177-
will be). It can be set in the future anticipating the future release date.
2154+
'breeze prepare-provider-documentation <PACKAGE_ID> ...'
21782155
21792156
* <PACKAGE_ID> is usually directory in the airflow/providers folder (for example
21802157
'google' but in several cases, it might be one level deeper separated with
@@ -2183,7 +2160,7 @@ This is the current syntax for `./breeze <./breeze>`_:
21832160
Flags:
21842161
21852162
-S, --version-suffix-for-pypi SUFFIX
2186-
Adds optional suffix to the version in the generated backport package. It can be used
2163+
Adds optional suffix to the version in the generated provider package. It can be used
21872164
to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
21882165
21892166
-N, --version-suffix-for-svn SUFFIX
@@ -2200,10 +2177,6 @@ This is the current syntax for `./breeze <./breeze>`_:
22002177
22012178
Default: both
22022179
2203-
--backports
2204-
2205-
Prepares backport providers rather than regular ones.
2206-
22072180
-v, --verbose
22082181
Show verbose information about executed docker, kind, kubectl, helm commands. Useful for
22092182
debugging - when you run breeze with --verbose flags you will be able to see the commands
@@ -2225,7 +2198,7 @@ This is the current syntax for `./breeze <./breeze>`_:
22252198
22262199
breeze prepare-provider-packages [FLAGS] [PACKAGE_ID ...]
22272200
2228-
Prepares backport packages. You can provide (after --) optional list of packages to prepare.
2201+
Prepares provider packages. You can provide (after --) optional list of packages to prepare.
22292202
If no packages are specified, readme for all packages are generated. You can specify optional
22302203
--version-suffix-for-svn flag to generate rc candidate packages to upload to SVN or
22312204
--version-suffix-for-pypi flag to generate rc candidates for PyPI packages. You can also
@@ -2235,9 +2208,6 @@ This is the current syntax for `./breeze <./breeze>`_:
22352208
prepared there so make sure you run prepare-provider-packages first,
22362209
and prepare-airflow-packages second.
22372210
2238-
You can also specify --backport flag to prepare backport providers or --package-format to
2239-
prepare one or both types of supported formats.
2240-
22412211
Examples:
22422212
22432213
'breeze prepare-provider-packages' or
@@ -2250,7 +2220,7 @@ This is the current syntax for `./breeze <./breeze>`_:
22502220
22512221
General form:
22522222
2253-
'breeze prepare-provider-packages [--backports] [--package-format PACKAGE_FORMAT] \
2223+
'breeze prepare-provider-packages [--package-format PACKAGE_FORMAT] \
22542224
[--version-suffix-for-svn|--version-suffix-for-pypi] <PACKAGE_ID> ...'
22552225
22562226
* <PACKAGE_ID> is usually directory in the airflow/providers folder (for example
@@ -2269,12 +2239,8 @@ This is the current syntax for `./breeze <./breeze>`_:
22692239
22702240
Default: both
22712241
2272-
--backports
2273-
2274-
Prepares backport providers rather than regular ones.
2275-
22762242
-S, --version-suffix-for-pypi SUFFIX
2277-
Adds optional suffix to the version in the generated backport package. It can be used
2243+
Adds optional suffix to the version in the generated provider package. It can be used
22782244
to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
22792245
22802246
-N, --version-suffix-for-svn SUFFIX
@@ -2790,10 +2756,10 @@ This is the current syntax for `./breeze <./breeze>`_:
27902756
Default: All
27912757
27922758
****************************************************************************************************
2793-
Flags for generation of the backport packages
2759+
Flags for generation of the provider packages
27942760
27952761
-S, --version-suffix-for-pypi SUFFIX
2796-
Adds optional suffix to the version in the generated backport package. It can be used
2762+
Adds optional suffix to the version in the generated provider package. It can be used
27972763
to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
27982764
27992765
-N, --version-suffix-for-svn SUFFIX

‎CI.rst‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -265,10 +265,10 @@ You can use those variables when you try to reproduce the build locally.
265265
+-----------------------------------------+-------------+-------------+------------+-------------------------------------------------+
266266
| Version suffix variables |
267267
+-----------------------------------------+-------------+-------------+------------+-------------------------------------------------+
268-
| ``VERSION_SUFFIX_FOR_PYPI`` | | | | Version suffix used during backport |
268+
| ``VERSION_SUFFIX_FOR_PYPI`` | | | | Version suffix used during provider |
269269
| | | | | package preparation for PyPI builds. |
270270
+-----------------------------------------+-------------+-------------+------------+-------------------------------------------------+
271-
| ``VERSION_SUFFIX_FOR_SVN`` | | | | Version suffix used during backport |
271+
| ``VERSION_SUFFIX_FOR_SVN`` | | | | Version suffix used during provider |
272272
| | | | | package preparation for SVN builds. |
273273
+-----------------------------------------+-------------+-------------+------------+-------------------------------------------------+
274274
| Git variables |
@@ -706,8 +706,6 @@ This workflow is a regular workflow that performs all checks of Airflow code.
706706
+---------------------------+----------------------------------------------+-------+-------+------+
707707
| Spell check docs | Spell check for documentation | Yes | Yes | Yes |
708708
+---------------------------+----------------------------------------------+-------+-------+------+
709-
| Backport packages | Prepares Backport Packages for 1.10 Airflow | Yes | Yes | Yes |
710-
+---------------------------+----------------------------------------------+-------+-------+------+
711709
| Trigger tests | Checks if tests should be triggered | Yes | Yes | Yes |
712710
+---------------------------+----------------------------------------------+-------+-------+------+
713711
| Tests [Pg/Msql/Sqlite] | Run all the Pytest tests for Python code | Yes(2)| Yes | Yes |

‎CONTRIBUTING.rst‎

Lines changed: 2 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -629,8 +629,8 @@ The dependency list is automatically used during PyPI packages generation.
629629

630630
Cross-dependencies between provider packages are converted into extras - if you need functionality from
631631
the other provider package you can install it adding [extra] after the
632-
``apache-airflow-backport-providers-PROVIDER`` for example:
633-
``pip install apache-airflow-backport-providers-google[amazon]`` in case you want to use GCP
632+
``apache-airflow-providers-PROVIDER`` for example:
633+
``pip install apache-airflow-providers-google[amazon]`` in case you want to use GCP
634634
transfer operators from Amazon ECS.
635635

636636
If you add a new dependency between different providers packages, it will be detected automatically during
@@ -764,19 +764,6 @@ providers.
764764
You can read more about out approach for tests in `TESTING.rst <TESTING.rst>`_ but here
765765
are some highlights.
766766

767-
768-
Backport providers
769-
------------------
770-
771-
You can also build backport provider packages for Airflow 1.10. They aim to provide a bridge when users
772-
of Airflow 1.10 want to migrate to Airflow 2.0. The backport packages are named similarly to the
773-
provider packages, but with "backport" added:
774-
775-
* ``apache-airflow-backport-provider-*``
776-
777-
Those backport providers are automatically refactored to work with Airflow 1.10.* and have a few
778-
limitations described in those packages.
779-
780767
Dependency management
781768
=====================
782769

‎IMAGES.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -493,7 +493,7 @@ based on example in `this comment <https://github.com/apache/airflow/issues/8605
493493
--build-arg AIRFLOW_SOURCES_FROM="empty" \
494494
--build-arg AIRFLOW_SOURCES_TO="/empty" \
495495
--build-arg ADDITIONAL_AIRFLOW_EXTRAS="slack" \
496-
--build-arg ADDITIONAL_PYTHON_DEPS="apache-airflow-backport-providers-odbc \
496+
--build-arg ADDITIONAL_PYTHON_DEPS="apache-airflow-providers-odbc \
497497
azure-storage-blob \
498498
sshtunnel \
499499
google-api-python-client \

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,8 @@ pip install apache-airflow[postgres,google]==2.0.1 \
179179
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.0.1/constraints-3.7.txt"
180180
```
181181

182-
For information on installing backport providers check [backport-providers.rst](docs/apache-airflow/backport-providers.rst).
182+
For information on installing provider packages check
183+
[providers](http://airflow.apache.org/docs/apache-airflow-providers/index.html).
183184

184185
## Official source code
185186

0 commit comments

Comments
 (0)