Skip to content

Reduce CI time 30% via marker-based multi-rank test filtering #356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Copilot wants to merge 12 commits into from
Closed

Reduce CI time 30% via marker-based multi-rank test filtering #356

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
c23e64f
Initial plan
Copilot Feb 4, 2026
f6ba201
Add pytest markers and update test infrastructure
Copilot Feb 4, 2026
506eb27
Complete CI workflow updates for marker-based test filtering
Copilot Feb 4, 2026
90e9f9b
Fix marker assignment script and reapply markers correctly
Copilot Feb 4, 2026
91460a5
Merge remote changes and use corrected markers
Copilot Feb 4, 2026
f8ed164
Fix marker assignment - properly handle existing markers and file str…
Copilot Feb 4, 2026
45e5d12
Simplify implementation - use pytest marker filtering instead of work…
Copilot Feb 4, 2026
7c7a2b8
Simplify markers - assume multi-rank by default, only mark single_ran…
Copilot Feb 4, 2026
17c1c28
Remove excessive blank lines from test files
Copilot Feb 4, 2026
ff22726
Reset files with only whitespace changes to original state
Copilot Feb 4, 2026
3f2a993
Remove test optimization documentation file
Copilot Feb 4, 2026
15f10e9
Apply Ruff auto-fixes
github-actions[bot] Feb 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/scripts/run_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,16 @@ fi
$INSTALL_CMD

# Run tests in the specified directory
# Skip single_rank tests when running with multiple ranks
MARKER_ARG=\"\"
Copy link
Collaborator

@mawad-amd mawad-amd Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot do we need this?

Copy link
Contributor Author

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this line initializes MARKER_ARG to empty for NUM_RANKS=1 (single-rank runs). The logic on lines 97-100 is the core of the optimization - it adds -m 'not single_rank' to pytest when NUM_RANKS > 1, which skips single_rank tests on multi-rank configurations. Without this, single_rank tests would run on all 4 rank configs instead of just rank 1, and we wouldn't get the 30% CI time reduction.

If you'd prefer a different approach, I can simplify further - for example, we could skip the initialization and just use the conditional directly in the pytest command. Let me know!

if [ \"$NUM_RANKS\" -gt 1 ]; then
MARKER_ARG=\"-m 'not single_rank'\"
fi

for test_file in tests/$TEST_DIR/test_*.py; do
if [ -f \"\$test_file\" ]; then
echo \"Testing: \$test_file with $NUM_RANKS ranks (install: $INSTALL_METHOD)\"
python tests/run_tests_distributed.py --num_ranks $NUM_RANKS \"\$test_file\" -v --tb=short --durations=10
python tests/run_tests_distributed.py --num_ranks $NUM_RANKS \"\$test_file\" $MARKER_ARG -v --tb=short --durations=10
fi
done
"
6 changes: 6 additions & 0 deletions pytest.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# SPDX-License-Identifier: MIT
# Copyright (c) 2025 Advanced Micro Devices, Inc. All rights reserved.

[pytest]
markers =
single_rank: Tests validating tensor properties (shape, dtype, values) - run on 1 rank only. All other tests run on all rank configurations by default.
3 changes: 3 additions & 0 deletions tests/unittests/test_arange.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


def test_arange_basic_functionality():
"""Test basic arange functionality with various argument combinations."""
shmem = iris.iris(1 << 20)
Expand Down
3 changes: 3 additions & 0 deletions tests/unittests/test_empty.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


@pytest.mark.parametrize(
"dtype",
[
Expand Down
3 changes: 3 additions & 0 deletions tests/unittests/test_full.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


@pytest.mark.parametrize(
"fill_value",
[
Expand Down
3 changes: 3 additions & 0 deletions tests/unittests/test_linspace.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


@pytest.mark.parametrize(
"dtype",
[
Expand Down
3 changes: 3 additions & 0 deletions tests/unittests/test_ones.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


@pytest.mark.parametrize(
"dtype",
[
Expand Down
3 changes: 3 additions & 0 deletions tests/unittests/test_rand.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


@pytest.mark.parametrize(
"dtype",
[
Expand Down
3 changes: 3 additions & 0 deletions tests/unittests/test_randint.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


@pytest.mark.parametrize(
"dtype",
[
Expand Down
3 changes: 3 additions & 0 deletions tests/unittests/test_randn.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


@pytest.mark.parametrize(
"dtype",
[
Expand Down
3 changes: 3 additions & 0 deletions tests/unittests/test_zeros.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


@pytest.mark.parametrize(
"dtype",
[
Expand Down
3 changes: 3 additions & 0 deletions tests/unittests/test_zeros_like.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
import iris


pytestmark = pytest.mark.single_rank


@pytest.mark.parametrize(
"dtype",
[
Expand Down