Feature/baseline expression by Tobi1kenobi · Pull Request #64 · opentargets/pts

Tobi1kenobi · 2026-02-18T16:08:17Z

Barring some git rebase failures this should generate the baseline expression dataset for the March PPP release

Aligns the config with the transformer's updated signature that expects named keys (disease_path, phenotype_path, mondo_path) instead of a list.

…ession

javfg

To save some time, I just had a quick glance over to scaffolding of the entrypoints, and made some comments to sort them out a bit.

You can apply those in all of the pyspark programs, changing some things.

Co-authored-by: Javier Ferrer <jferrer@ebi.ac.uk>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…xpression.py Agent-Logs-Url: https://github.com/opentargets/pts/sessions/bfeb784f-b3c7-49e0-a549-a6d86f83a2e6 Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Agent-Logs-Url: https://github.com/opentargets/pts/sessions/1dc49aaa-a5e7-4373-9a52-8b90e650e473 Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

… before quartile calculation Agent-Logs-Url: https://github.com/opentargets/pts/sessions/5bb32345-ea72-43ee-a0cd-01ab2c938fb9 Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…f.local/self.json directly Agent-Logs-Url: https://github.com/opentargets/pts/sessions/5786f5d2-6904-4e69-afaf-3d0b98a412d8 Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Agent-Logs-Url: https://github.com/opentargets/pts/sessions/3656125d-3c20-42cc-bac7-96321416ea7f Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Tobi1kenobi · 2026-05-18T11:04:22Z

@copilot OK now I've fixed most of your issues from code review can you go back and focus on the stylistic choices used across the pts repo and check if the baseline expression syntax aligns?

Agent-Logs-Url: https://github.com/opentargets/pts/sessions/e28a56ee-7d7e-4c87-8b28-d255d4b8d60e Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Copilot · 2026-05-18T11:07:20Z

@copilot OK now I've fixed most of your issues from code review can you go back and focus on the stylistic choices used across the pts repo and check if the baseline expression syntax ...

Aligned the baseline expression files with the repo-wide ruff style in commit eef4c08. Ran ruff format + ruff check --fix over all PR-touched files, then manually fixed the remaining warnings (ambiguous ∪ in a comment in baseline_expression.py, missing docstring period in expression_utils/expression.py). The auto-format mostly added trailing commas (COM rule) and collapsed/expanded chained calls per the project's 120-char line-length. ruff check and ruff format --check now pass on all 12 PR-modified Python files.

…pts into feature/baseline_expression

…ession

Tobi1kenobi and others added 27 commits December 18, 2025 11:08

feat: add new baseline expression harmonisation

4cd4280

feat: Add baseline DICE processing

b730a76

Update config

692747e

GTEx working

6b4620e

Pseudobulking working

170274c

Fix aggregate step

d834074

fix: cellex paths and tissue_celltype handling

4b2ed31

fix: cellex paths and tissue_celltype handling 2

50b62f2

Restructure harmonise scriptt

f6974d6

treat sources and destinations in a standard way

8cc0e4f

update disease_phenotype config to use dict source format

f732e8d

Aligns the config with the transformer's updated signature that expects named keys (disease_path, phenotype_path, mondo_path) instead of a list.

Migrating association otf logic from ETL

0a75cdc

Adding tests for association otf

8d93465

adding novelty to atof dataset

e190ee9

fix: regex and raw strings

eb8a7d5

feat: adding VEP view logic

7cda279

fix: Minor tweaks to get into working state

12df69d

feat: Add PRIDE functionality

9a6795c

feat: Merge functional

1638d9f

fix: ruff line too long

184e6a8

Merge remote-tracking branch 'origin/main' into feature/baseline_expr…

3153125

…ession

feat: add baseline local files

7b26d0c

Rebase error, ta merging commits to association otf

f57dfc8

rebase failure, ta merging

4be7cbe

fix: minor anndata reading bug

d885bf1

fix: increase memory for aggregate task

cad170b

fix: adjust single-cell memory usage

83aee4d

javfg requested changes Feb 20, 2026

View reviewed changes

Comment thread src/pts/pyspark/baseline_expression_aggregate.py Outdated

Comment thread src/pts/pyspark/baseline_expression_aggregate.py Outdated

Comment thread src/pts/pyspark/baseline_expression_aggregate.py Outdated

Comment thread src/pts/pyspark/baseline_expression_aggregate.py Outdated

Tobi1kenobi and others added 2 commits February 23, 2026 12:08

Apply suggestions from code review

fffe863

Co-authored-by: Javier Ferrer <jferrer@ebi.ac.uk>

Apply suggestion from @javfg

ebaa759

Co-authored-by: Javier Ferrer <jferrer@ebi.ac.uk>

Tobi1kenobi and others added 2 commits May 18, 2026 11:30

Potential fix for pull request finding

4e0959a

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Replace bare col/coalesce/explode imports with f. style in baseline_e…

8378184

…xpression.py Agent-Logs-Url: https://github.com/opentargets/pts/sessions/bfeb784f-b3c7-49e0-a549-a6d86f83a2e6 Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Copilot finished work on behalf of Tobi1kenobi May 18, 2026 10:32

Copilot started work on behalf of Tobi1kenobi May 18, 2026 10:36 View session

Copilot AI and others added 2 commits May 18, 2026 10:37

Remove unused within_donor_mean method from AggregateExpression

fbf4724

Agent-Logs-Url: https://github.com/opentargets/pts/sessions/1dc49aaa-a5e7-4373-9a52-8b90e650e473 Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Potential fix for pull request finding

f01502b

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot finished work on behalf of Tobi1kenobi May 18, 2026 10:38

Copilot started work on behalf of Tobi1kenobi May 18, 2026 10:45 View session

Copilot AI and others added 2 commits May 18, 2026 10:46

Wire apply_qc_threshold into baseline_expression_aggregate entrypoint…

757ed11

… before quartile calculation Agent-Logs-Url: https://github.com/opentargets/pts/sessions/5bb32345-ea72-43ee-a0cd-01ab2c938fb9 Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Potential fix for pull request finding

b8ae482

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot finished work on behalf of Tobi1kenobi May 18, 2026 10:47

Copilot started work on behalf of Tobi1kenobi May 18, 2026 10:47 View session

Remove redundant local/json params from pack_data_for_output; use sel…

cc396fc

…f.local/self.json directly Agent-Logs-Url: https://github.com/opentargets/pts/sessions/5786f5d2-6904-4e69-afaf-3d0b98a412d8 Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Copilot finished work on behalf of Tobi1kenobi May 18, 2026 10:49

Potential fix for pull request finding

9a41da5

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of Tobi1kenobi May 18, 2026 11:01 View session

Give each baseline expression pipeline step a unique name in config.yaml

2b4f9b9

Agent-Logs-Url: https://github.com/opentargets/pts/sessions/3656125d-3c20-42cc-bac7-96321416ea7f Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

Copilot finished work on behalf of Tobi1kenobi May 18, 2026 11:04

Copilot started work on behalf of Tobi1kenobi May 18, 2026 11:04 View session

Apply ruff format and lint fixes to baseline expression files

eef4c08

Agent-Logs-Url: https://github.com/opentargets/pts/sessions/e28a56ee-7d7e-4c87-8b28-d255d4b8d60e Co-authored-by: Tobi1kenobi <25749859+Tobi1kenobi@users.noreply.github.com>

chore: tidy

655dcdf

Copilot finished work on behalf of Tobi1kenobi May 18, 2026 11:08

Tobi1kenobi added 3 commits May 18, 2026 11:09

Merge branch 'feature/baseline_expression' of github.com:opentargets/…

ccff161

…pts into feature/baseline_expression

Merge remote-tracking branch 'origin/main' into feature/baseline_expr…

f50f773

…ession

bump version

513ac5d

Tobi1kenobi requested a review from DSuveges May 18, 2026 13:26

DSuveges mentioned this pull request May 19, 2026

Feature/baseline expression clean #133

Merged

3 tasks

DSuveges closed this May 27, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/baseline expression#64

Feature/baseline expression#64
Tobi1kenobi wants to merge 60 commits into
mainfrom
feature/baseline_expression

Tobi1kenobi commented Feb 18, 2026

Uh oh!

javfg left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Tobi1kenobi commented May 18, 2026

Uh oh!

Copilot AI commented May 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

Tobi1kenobi commented Feb 18, 2026

Uh oh!

javfg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Tobi1kenobi commented May 18, 2026

Uh oh!

Copilot AI commented May 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants