Skip to content

Made easy_toolbox tests use multithreading by default#769

Open
malevitzch wants to merge 1 commit into
sio2project:masterfrom
malevitzch:Multithreaded-tests
Open

Made easy_toolbox tests use multithreading by default#769
malevitzch wants to merge 1 commit into
sio2project:masterfrom
malevitzch:Multithreaded-tests

Conversation

@malevitzch

Copy link
Copy Markdown
Member

Made the easy_toolbox test command utilize all cpu cores by default. On a good processor, this gave me around 5x speedup when testing locally, and I don't see why it shouldn't be the default.

Nightly tests already use pytest's "-n auto" option anyways, so this is not a relaibility issue.

Runtimes for singlethreaded vs multithreaded tests for reference:
image
image

@malevitzch malevitzch requested a review from MasloMaslane as a code owner June 15, 2026 09:48
@sonarqubecloud

Copy link
Copy Markdown

@otargowski

Copy link
Copy Markdown
Contributor

Unfortunately, some tests have depdendencies on state from other ones (??), which causes them to fail occasionally. On a 20-thread cpu I got one failure out of ten runs:

FAILED oioioi/teachers/tests.py::TestAddUserToContestForm::test_add_user_to_contest_as_pupil - AssertionError: "User is already added" does not match "["Użytkownik jest już dodany: 'test_user'"]"
FAILED oioioi/teachers/tests.py::TestAddUserToContestForm::test_add_user_to_contest_as_teacher - AssertionError:
"User is already added" does not match "["Użytkownik jest już dodany: 'test_user'"]"

I recall that there were more such tests. Running with --dist=loadscope would help (but not entirely), though it seems that in the last year or two running without that flag got much more reliable.

@malevitzch

malevitzch commented Jun 18, 2026

Copy link
Copy Markdown
Member Author

Then there is a problem in nightly tests as well and the changes for the CI pipeline should be reverted. In #467, the nightly tests have been modified to use -n auto, which is what this PR was inspired by. In that case, the part that parallelizes CI actions should probably be reverted as well, right? This will be a major slowdown for the pipelines.

@malevitzch

Copy link
Copy Markdown
Member Author

I can maybe try to locate badly isolated tests and fix them so they don't depend on each other

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants