do not use global variables in modules

[kelly-sovacool]

Numerous modules have global variables set which are then used by process definitions within the file. These should be passed as inputs to each process that needs them.

example of problem code

//References
GENOMEREF=file(params.genomes[params.genome].genome)
GENOMEFAI=file(params.genomes[params.genome].genomefai)

//Octopus
SOMATIC_FOREST=params.genomes[params.genome].octopus_sforest
GERMLINE_FOREST=params.genomes[params.genome].octopus_gforest



process octopus_tn {
    container "${params.containers.octopus}"
    label 'process_somaticcaller_high'
    errorStrategy { task.attempt <= 3 ? 'retry' : 'terminate' }

    input:
        tuple val(tumorname), path(tumor), path(tumorbai),
        val(normalname), path(normal), path(normalbai), path(bed)

    output:
        tuple val("${tumorname}_vs_${normalname}"),
        path("${tumorname}_vs_${normalname}_${bed.simpleName}.octopus.vcf.gz")

    script:
    """
    octopus \\
        -R ${GENOMEREF} -I ${normal} ${tumor} \\
        --normal-sample ${normalname} \\
        -C cancer \\
        --annotations AF AC AD DP SB \\
        -t ${bed} \\
        --threads ${task.cpus} \\
        ${GERMLINE_FOREST} \\
        ${SOMATIC_FOREST} \\
        -B ${task.memory.toGiga()}Gb \\
        -o ${tumorname}_vs_${normalname}_${bed.simpleName}.octopus.vcf.gz
    """

    stub:
    """
    touch "${tumorname}_vs_${normalname}_${bed.simpleName}.octopus.vcf.gz"
    """

}

list of known global variables that need to be removed

grep -r "^[A-Z]*=" modules

modules/local/purple.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/somalier.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/octopus.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/octopus.nf:GENOMEFAI=file(params.genomes[params.genome].genomefai)
modules/local/annotsv.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/annotsv.nf:ANNOTSVGENOME=params.genomes[params.genome].annotsvgenome
modules/local/varscan.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/varscan.nf:GENOMEDICT=file(params.genomes[params.genome].genomedict)
modules/local/sage.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/sage.nf:GENOMEFAI=file(params.genomes[params.genome].genomefai)
modules/local/sage.nf:SOMATICHOTSPOTS=params.genomes[params.genome].SOMATICHOTSPOTS
modules/local/sage.nf:PANELBED=params.genomes[params.genome].PANELBED
modules/local/sage.nf:HCBED=params.genomes[params.genome].HCBED
modules/local/sage.nf:ENSEMBLCACHE=params.genomes[params.genome].ENSEMBLCACHE
modules/local/sage.nf:GENOMEVER=params.genomes[params.genome].GENOMEVER
modules/local/muse.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/muse.nf:GENOMEFAI=file(params.genomes[params.genome].genomefai)
modules/local/muse.nf:GENOMEDICT=file(params.genomes[params.genome].genomedict)
modules/local/muse.nf:DBSNP=file(params.genomes[params.genome].dbsnp)
modules/local/annotvep.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/annotvep.nf:VEPCACHEDIR=file(params.genomes[params.genome].vepcache)
modules/local/annotvep.nf:VEPSPECIES=params.genomes[params.genome].vepspecies
modules/local/annotvep.nf:VEPBUILD=params.genomes[params.genome].vepbuild
modules/local/deepsomatic.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/kraken.nf:BACDB=file(params.genomes[params.genome].KRAKENBACDB)
modules/local/gridss.nf:BWAGENOME=file(params.genomes[params.genome].bwagenome)
modules/local/gridss.nf:BLACKLIST=file(params.genomes[params.genome].GRIDSSBLACKLIST)
modules/local/gridss.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/lofreq.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/lofreq.nf:DBSNP=file(params.genomes[params.genome].dbsnp)
modules/local/combinefilter.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/combinefilter.nf:GENOMEFAI=file(params.genomes[params.genome].genomefai)
modules/local/combinefilter.nf:GENOMEDICT=file(params.genomes[params.genome].genomedict)
modules/local/gatk_varianteval.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/gatk_varianteval.nf:DBSNP=file(params.genomes[params.genome].dbsnp) //dbsnp_138.hg38.vcf.gz"
modules/local/vardict.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/qc.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/qc.nf:DBSNP=file(params.genomes[params.genome].dbsnp) //dbsnp_138.hg38.vcf.gz"
modules/local/qc.nf:BACDB=file(params.genomes[params.genome].KRAKENBACDB)
modules/local/strelka.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/lancet2/lancet2.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/mutect2.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/mutect2.nf:GENOMEDICT=file(params.genomes[params.genome].genomedict)
modules/local/mutect2.nf:GNOMADGERMLINE=params.genomes[params.genome].gnomad
modules/local/mutect2.nf:PON=file(params.genomes[params.genome].PON)
modules/local/mutect2.nf:TONLYPON=file(params.genomes[params.genome].tonly_PON)
modules/local/svaba.nf:BWAGENOME=file(params.genomes[params.genome].bwagenome)
modules/local/svaba.nf:INDELREF=file(params.genomes[params.genome].INDELREF)
modules/local/manta.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/deepvariant.nf:GENOMEREF=file(params.genomes[params.genome].genome)
modules/local/deepvariant.nf:MODEL="/opt/models/wgs/"

[kelly-sovacool]

Current code

octopus module

LOGAN/modules/local/octopus.nf

Lines 1 to 41 in f78bc10

	//References
	GENOMEREF=file(params.genomes[params.genome].genome)
	GENOMEFAI=file(params.genomes[params.genome].genomefai)
	//Octopus
	SOMATIC_FOREST=params.genomes[params.genome].octopus_sforest
	GERMLINE_FOREST=params.genomes[params.genome].octopus_gforest
	process octopus_tn {
	container "${params.containers.octopus}"
	label 'process_somaticcaller_high'
	errorStrategy { task.attempt <= 3 ? 'retry' : 'terminate' }
	input:
	tuple val(tumorname), path(tumor), path(tumorbai),
	val(normalname), path(normal), path(normalbai), path(bed)
	output:
	tuple val("${tumorname}_vs_${normalname}"),
	path("${tumorname}_vs_${normalname}_${bed.simpleName}.octopus.vcf.gz")
	script:
	"""
	octopus -R $GENOMEREF -I ${normal} ${tumor} --normal-sample ${normalname} \
	-C cancer \
	--annotations AF AC AD DP SB -t ${bed} \
	--threads $task.cpus \
	$GERMLINE_FOREST \
	$SOMATIC_FOREST \
	-B 92Gb \
	-o ${tumorname}_vs_${normalname}_${bed.simpleName}.octopus.vcf.gz
	"""
	stub:
	"""
	touch "${tumorname}_vs_${normalname}_${bed.simpleName}.octopus.vcf.gz"
	"""
	}

subworkflow that calls octopus

LOGAN/subworkflows/local/workflows.nf

Lines 579 to 581 in f78bc10

	//Octopus TN
	if ("octopus" in call_list){
	octopus_in=octopus_tn(bambyinterval) | bcftools_index_octopus

genome config

LOGAN/conf/genomes.config

Lines 29 to 30 in f78bc10

	octopus_sforest= "--somatic-forest /data/CCBR_Pipeliner/Pipelines/LOGAN/resources/hg38/octopus/somatic.v0.7.4.forest"
	octopus_gforest= "--forest /data/CCBR_Pipeliner/Pipelines/LOGAN/resources/hg38/octopus/germline.v0.7.4.forest"

Example solution

octopus module

the parameters are now inputs to the process, and the global variables are deleted from the module. also, the process memory should not be hard-coded in the script.

process octopus_tn {
    container "${params.containers.octopus}"
    label 'process_somaticcaller_high'
    errorStrategy { task.attempt <= 3 ? 'retry' : 'terminate' }

    input:
        tuple val(tumorname), path(tumor), path(tumorbai),
        val(normalname), path(normal), path(normalbai), path(bed), path(genome_ref), path(genome_fai), path(germline_forest), path(somatic_forest)

    output:
        tuple val("${tumorname}_vs_${normalname}"),
        path("${tumorname}_vs_${normalname}_${bed.simpleName}.octopus.vcf.gz")

    script:
    """
    octopus -R ${genome_ref} -I ${normal} ${tumor} --normal-sample ${normalname} \
    -C cancer \
    --annotations AF AC AD DP SB -t ${bed} \
    --threads $task.cpus \
    --forest ${germline_forest} \
    --somatic-forest ${somatic_forest} \
    -B ${task.memory.toGiga()}Gb \
    -o ${tumorname}_vs_${normalname}_${bed.simpleName}.octopus.vcf.gz
    """

    stub:
    """
    touch "${tumorname}_vs_${normalname}_${bed.simpleName}.octopus.vcf.gz"
    """

}

subworkflow that calls octopus

the subworkflow passes in the parameters as inputs

    //Octopus TN
    if ("octopus" in call_list){
        octopus_in = bambyinterval
          | combine(Channel.fromPath(file(params.genomes[params.genome].genome)))
          | combine(Channel.fromPath(file(params.genomes[params.genome].genomefai)))
          | combine(Channel.fromPath(file(params.genomes[params.genome].octopus_sforest)))
          | combine(Channel.fromPath(params.genomes[params.genome].octopus_gforest))
          | octopus_tn
          | bcftools_index_octopus

genome config

remove the flag from the parameter, it should only be the file path itself

            octopus_sforest = "/data/CCBR_Pipeliner/Pipelines/LOGAN/resources/hg38/octopus/somatic.v0.7.4.forest"
            octopus_gforest = "/data/CCBR_Pipeliner/Pipelines/LOGAN/resources/hg38/octopus/germline.v0.7.4.forest"