`ValueError: ('Failed to normalize to within expected bounds', 0.0, 3.0, 9.0, 3.0)` in genmod compound

[fellen31]

Command executed:

genmod \
        compound \
        --temp_dir ./ \
        --processes 2 \
        --outfile test_genmod_compound_compound.vcf \
        test_genmod_score_score.vcf

Command error:

      File "/opt/conda/lib/python3.11/site-packages/genmod/score_variants/score_variant.py", line 148, in as_normalized_max_min
        raise ValueError(
    ValueError: ('Failed to normalize to within expected bounds', 0.0, 3.0, 9.0, 3.0)
    
    [INFO/CompoundScorer-3] process shutting down
    [INFO/CompoundScorer-3] process shutting down
    [INFO/CompoundScorer-3] process shutting down
    Process CompoundScorer-3:
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
        self.run()
      File "/opt/conda/lib/python3.11/site-packages/genmod/score_variants/compound_scorer.py", line 360, in run
        raise e
      File "/opt/conda/lib/python3.11/site-packages/genmod/score_variants/compound_scorer.py", line 357, in run
        return self._run(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/genmod/score_variants/compound_scorer.py", line 260, in _run
        if compound_rank_score > get_rank_score(
                                 ^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/genmod/score_variants/compound_scorer.py", line 54, in get_rank_score
        return as_normalized_max_min(
               ^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/genmod/score_variants/score_variant.py", line 148, in as_normalized_max_min
        raise ValueError(
    ValueError: ('Failed to normalize to within expected bounds', 0.0, 3.0, 9.0, 3.0)
    [INFO/CompoundScorer-3] process exiting with exitcode 1
    [INFO/CompoundScorer-3] process exiting with exitcode 1
    [INFO/CompoundScorer-3] process exiting with exitcode 1
    [ERROR/MainProcess] Worker <CompoundScorer name='CompoundScorer-3' pid=101 parent=41 stopped exitcode=1> failed
    [ERROR/MainProcess] Worker <CompoundScorer name='CompoundScorer-3' pid=101 parent=41 stopped exitcode=1> failed
    [ERROR/MainProcess] Worker <CompoundScorer name='CompoundScorer-3' pid=101 parent=41 stopped exitcode=1> failed
    [INFO/MainProcess] Removing temp file
    [INFO/MainProcess] Removing temp file
    [INFO/MainProcess] Removing temp file
    Aborted!
    [INFO/MainProcess] process shutting down
    [INFO/MainProcess] process shutting down
    [INFO/MainProcess] process shutting down
    [INFO/MainProcess] sending shutdown message to manager
    [INFO/MainProcess] sending shutdown message to manager
    [INFO/MainProcess] sending shutdown message to manager
    [INFO/SyncManager-1] process shutting down
    [INFO/SyncManager-1] process shutting down
    [INFO/SyncManager-1] process shutting down
    [INFO/SyncManager-1] process exiting with exitcode 0
    [INFO/SyncManager-1] process exiting with exitcode 0
    [INFO/SyncManager-1] process exiting with exitcode 0
    [INFO/MainProcess] calling join() for process VariantPrinter-4
    [INFO/MainProcess] calling join() for process VariantPrinter-4
    [INFO/MainProcess] calling join() for process VariantPrinter-4

Running genmod 3.10.2 with test_genmod_score_score.vcf.txt

[fellen31]

Maybe you can tell why this happens @torbjorgen?

[torbjorgen]

    score_normalized: float = (score - min_score_value) / (max_score_value - min_score_value)
    if not (MIN_SCORE_NORMALIZED <= score_normalized <= MAX_SCORE_NORMALIZED):
        raise ValueError(
            "Failed to normalize to within expected bounds",
            min_score_value,
            max_score_value,
            score,
            score_normalized,
        )

The purpose of this value error is to notify user that something is off with the scores in relationship to the config or VCF RankScore or RankScoreMinMax.
One should expect a value between (0, 1) for the normalized score, not 3.0.

I think the CompoundScorer.threshold=9to be the culprit.

The normalization min, max values are computed from the rank ini plugins configs. i.e. a score of 9.0 is not expected by the scoring config -> that is RankScoreMinMax fields in the VCF.

The code throws the error because CompoundScorer.threshold is never expected to exceed the maximum value given by the scoring config.

Suggest to either:

1. Change the threshold argument to fit within RankScoreMinMax bounds, in this case (0, 3) given by the test data.
2. Update the test data file to fit to the new threshold config -> RankScoreMinMax=justhusky:0.0:9.0(hack for your data)

I expect this to happen to the CompoundScorer.penalty as well.

Tested this and compound scoring succeeds using threshold=penalty=3 on your data.

@fellen31

[fellen31]

Thank you @torbjorgen!