|
| 1 | +# Contributing to `mkl_fft` |
| 2 | + |
| 3 | +This document covers the development |
| 4 | +workflow: how to get a working build, how to run the checks, and how the code is |
| 5 | +laid out so a change lands in the right layer. |
| 6 | + |
| 7 | +For end-user installation and API usage, see [README.md](README.md). Security |
| 8 | +vulnerabilities go through the process in [SECURITY.md](SECURITY.md). |
| 9 | + |
| 10 | +--- |
| 11 | + |
| 12 | +## Development setup |
| 13 | + |
| 14 | +Building requires a C compiler, oneMKL headers and libraries (`mkl-devel`), and |
| 15 | +NumPy. A conda environment is the least surprising way to get them: |
| 16 | + |
| 17 | +```sh |
| 18 | +conda create -n mkl_fft-dev -c conda-forge python=3.12 pip mkl-devel numpy \ |
| 19 | + meson-python ninja cmake cython pytest scipy mkl-service |
| 20 | +conda activate mkl_fft-dev |
| 21 | +``` |
| 22 | + |
| 23 | +Then build in place, which reuses the environment's MKL and NumPy: |
| 24 | + |
| 25 | +```sh |
| 26 | +pip install -e ".[test]" --no-build-isolation --verbose |
| 27 | +``` |
| 28 | + |
| 29 | +The `[test]` extra pulls in `pytest`, `scipy`, and `mkl-service`. Other extras |
| 30 | +are declared in `pyproject.toml`: `scipy_interface` for the SciPy adapter at |
| 31 | +runtime and `benchmark` for the ASV suite. |
| 32 | + |
| 33 | +`README.md` documents the non-editable install paths, including the isolated |
| 34 | +build that resolves its own `mkl` and `numpy`. |
| 35 | + |
| 36 | +### Rebuilding |
| 37 | + |
| 38 | +`meson-python` rebuilds the extension on import for editable installs, so |
| 39 | +editing `.pyx`, `.c.src`, or `meson.build` and rerunning `pytest` is usually |
| 40 | +enough. Generated sources and the compiled extension live under `build/<tag>/` |
| 41 | +rather than in the source tree. If a build gets into a bad state, `rm -rf build` |
| 42 | +and reinstall. |
| 43 | + |
| 44 | +## Running the checks |
| 45 | + |
| 46 | +```sh |
| 47 | +pytest mkl_fft/tests # test suite |
| 48 | +pre-commit run --all-files # lint and format hooks |
| 49 | +``` |
| 50 | + |
| 51 | +Install the hooks once with `pre-commit install` and they run on each commit. |
| 52 | +`.pre-commit-config.yaml` is the source of truth for the tooling; today it |
| 53 | +covers `black`, `isort`, `flake8`, `pylint` (errors only), `cython-lint`, |
| 54 | +`clang-format`, `codespell`, `shellcheck`, `gitleaks`, and `actionlint`. Line |
| 55 | +length is 80 for Python, Cython, and TOML. |
| 56 | + |
| 57 | +### What CI runs |
| 58 | + |
| 59 | +`.github/workflows/*.yml` is canonical for platform and Python matrices. In |
| 60 | +outline: |
| 61 | + |
| 62 | +| Workflow | Purpose | |
| 63 | +| --- | --- | |
| 64 | +| `conda-package.yml` | conda build and test against the Intel channel | |
| 65 | +| `conda-package-cf.yml` | conda build and test against conda-forge only | |
| 66 | +| `build_pip.yml` | editable pip build, including pre-release NumPy | |
| 67 | +| `build-with-clang.yml` | build with the IntelLLVM `icx` compiler | |
| 68 | +| `build-with-standard-clang.yml` | build with upstream clang | |
| 69 | +| `pre-commit.yml` | lint and format | |
| 70 | +| `coverity.yml` | static analysis (see `coverity/README.md`) | |
| 71 | +| `openssf-scorecard.yml`, `zizmor.yml` | supply-chain and workflow security | |
| 72 | + |
| 73 | +To reproduce a conda packaging failure locally, build the recipe the same way CI |
| 74 | +does — `conda build --python <ver> --numpy <ver> -c <channels> --override-channels conda-recipe` |
| 75 | +(or `conda-recipe-cf` for the conda-forge variant). The recipe directories are |
| 76 | +canonical for packaging intent and dependency pins. |
| 77 | + |
| 78 | +## How the code fits together |
| 79 | + |
| 80 | +Build configuration lives in `pyproject.toml` (with `meson-python` as the build |
| 81 | +backend) and `meson.build`. The version is read from `mkl_fft/_version.py` by |
| 82 | +`meson.build`, so that file is the single place a version is set. |
| 83 | + |
| 84 | +A transform call flows down through these layers: |
| 85 | + |
| 86 | +``` |
| 87 | +mkl_fft.interfaces.numpy_fft / scipy_fft drop-in NumPy/SciPy adapters |
| 88 | +mkl_fft (__init__.py) public FFT API |
| 89 | +mkl_fft/_mkl_fft.py, _fft_utils.py argument handling, normalization, dispatch |
| 90 | +mkl_fft/_pydfti.pyx Cython bindings |
| 91 | +mkl_fft/src/mklfft.c.src -> mklfft.c C backend, generated at build time |
| 92 | +oneMKL DFTI |
| 93 | +``` |
| 94 | + |
| 95 | +Directories: |
| 96 | + |
| 97 | +- **`mkl_fft/`** — the package. `__init__.py` is the public API surface; |
| 98 | + `_mkl_fft.py` and `_fft_utils.py` hold the Python-level FFT logic; |
| 99 | + `_pydfti.pyx` is the Cython binding layer. |
| 100 | +- **`mkl_fft/src/`** — the C backend, written as `*.c.src` templates. At build |
| 101 | + time `_vendored/process_src_template.py` expands `mklfft.c.src` into |
| 102 | + `mklfft.c`, which is compiled into the `_pydfti` extension. The generated |
| 103 | + `.c` is regenerated on every build, so only template edits survive. |
| 104 | +- **`mkl_fft/interfaces/`** — adapters presenting `numpy.fft`- and |
| 105 | + `scipy.fft`-shaped APIs. `numpy_fft.py` and `scipy_fft.py` are the public |
| 106 | + modules; the `_`-prefixed siblings are implementation. Upstream signatures and |
| 107 | + semantics are the contract here. |
| 108 | +- **`mkl_fft/`** patching layer — `patch.py`, `with_patch.py`, `_patch_numpy.py`, |
| 109 | + `_patch_startup.py`, and the `__main__.py` CLI implement the monkey-patching |
| 110 | + entry points documented in the README. The contract is that patching stays |
| 111 | + reversible and observable: anything installed can be uninstalled, and |
| 112 | + `is_patched()` reports the truth. |
| 113 | +- **`mkl_fft/tests/`** — the suite. `helper.py` holds shared utilities and |
| 114 | + `third_party/` carries tests adapted from upstream projects. |
| 115 | +- **`_vendored/`** — build-time code-generation helpers vendored from NumPy. |
| 116 | + They are excluded from `black` and `isort` in `pyproject.toml`. |
| 117 | +- **`conda-recipe/`**, **`conda-recipe-cf/`** — Intel-channel and conda-forge |
| 118 | + packaging. |
| 119 | +- **`benchmarks/`** — ASV benchmarks, run with the `benchmark` extra. |
| 120 | + |
| 121 | +Each of these directories has an `AGENTS.md` stating the same boundaries for |
| 122 | +coding agents; [`AGENTS.md`](AGENTS.md) at the root indexes them and is a useful |
| 123 | +orientation map for humans too. |
| 124 | + |
| 125 | +## Dos and don'ts |
| 126 | + |
| 127 | +**Do** |
| 128 | + |
| 129 | +- Keep changes atomic and single-purpose. |
| 130 | +- Preserve NumPy/SciPy FFT compatibility. This package is used as a drop-in |
| 131 | + replacement, so a behavioral difference is a bug even when the new behavior is |
| 132 | + arguably better. Call out an intentional break in the PR. |
| 133 | +- Add tests in `mkl_fft/tests/` alongside behavior changes, and a regression |
| 134 | + test with every bug fix. |
| 135 | +- Keep tests deterministic. |
| 136 | +- Edit the `*.c.src` templates for C backend changes. |
| 137 | +- Keep patching reversible and observable. |
| 138 | +- Cite the source-of-truth file for mutable details: `pyproject.toml`, |
| 139 | + `meson.build`, `conda-recipe*/meta.yaml`, `.github/workflows/`. |
| 140 | +- Give benchmark numbers reproducible context — hardware, versions, and the |
| 141 | + command you ran. |
| 142 | + |
| 143 | +**Don't** |
| 144 | + |
| 145 | +- Commit generated artifacts, or hand-edit a generated `.c`. |
| 146 | +- Hardcode versions, build flags, CI matrices, or channel URLs in documentation. |
| 147 | +- Assert on timing or throughput in the test suite. |
| 148 | +- Refactor `_vendored/` opportunistically. Keep local diffs minimal and send |
| 149 | + fixes upstream where you can. |
| 150 | +- Introduce ISA-specific assumptions outside explicit build configuration. |
| 151 | + |
| 152 | +## Submitting a change |
| 153 | + |
| 154 | +Work on a branch: the `no-commit-to-branch` hook blocks direct commits to |
| 155 | +`master` and `maintenance/*`. |
| 156 | + |
| 157 | +Add a `CHANGELOG.md` entry under `## [dev]` in the matching section, with a |
| 158 | +`[gh-NNN](https://github.com/IntelPython/mkl_fft/pull/NNN)` link. The format |
| 159 | +follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project |
| 160 | +follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). |
| 161 | + |
| 162 | +Then open the PR and fill in the template, including what you verified locally |
| 163 | +and what you left to CI. |
| 164 | + |
| 165 | +By contributing you agree that your contributions are licensed under the |
| 166 | +BSD-3-Clause terms in [LICENSE.txt](LICENSE.txt). |
0 commit comments