Problem
filter-and-trim removes leading bases by fixed offset only
(src/filter_trim.rs:192, let start = p.trim_left;). There is no
sequence-aware primer matching anywhere in the module.
That is fine when the primer begins at a constant position. It breaks on the
now-common library designs that insert heterogeneity spacers — 0–N nt of
variable padding before the forward primer, used to boost base diversity on
patterned flow cells (Fadrosh-style).
With variable offsets there is no correct --trim-left:
- too small → spacer bases remain on some reads
- too large → real biological bases are removed from others
Both produce the same biological sequence as several distinct ASVs.
Measured impact
On a public NovaSeq 6000 16S run (BioProject PRJNA1504839), 341F was found at
5 distinct offsets (0–4 nt) in 99.96% of merged reads, with the forward
primer retained. Stripping spacer + primer post-hoc collapsed:
| arm |
post-chimera ASVs |
after stripping |
collapse |
loess |
26,097 |
15,849 |
39.3% |
binned-qual |
35,314 |
19,019 |
46.1% |
So roughly 4 in 10 reported ASVs were offset duplicates of another ASV. It also
inflated an errfun A/B: a −26.1% ASV difference was −16.7% once corrected.
Current workaround
Use cutadapt (or R's removePrimers()) upstream for primer removal, then
filter-and-trim for quality filtering only. This is what our R workflow does.
Worth documenting explicitly either way, since --trim-left looks like it is
the primer-removal knob and silently is not.
Possible directions
- Add primer-sequence-aware trimming:
--primer-fwd / --primer-rev accepting
IUPAC-degenerate sequences, matched within a small search window with a
mismatch allowance, trimming through the match end. This is what
removePrimers() does and what the data above requires.
- Optionally
--max-primer-mismatch and an orientation check.
- At minimum: document that
--trim-left is a positional trim and is not
safe for spacer-containing or variable-offset designs, and point at cutadapt.
Notes
- Degenerate primers (341F is
CCTACGGGNGGCWGCAG) mean a plain literal match is
insufficient; IUPAC expansion is required.
- Detection is cheap as a diagnostic even before trimming is implemented: search
a known primer across the first ~30 nt and report the offset distribution. A
multi-offset result is the signal that positional trimming is unsafe. This
would pair naturally with the read-overlap QC screen discussed in the binned
quality findings.
Context: docs/findings/binned-quality-illumina-novaseq.md.
Problem
filter-and-trimremoves leading bases by fixed offset only(
src/filter_trim.rs:192,let start = p.trim_left;). There is nosequence-aware primer matching anywhere in the module.
That is fine when the primer begins at a constant position. It breaks on the
now-common library designs that insert heterogeneity spacers — 0–N nt of
variable padding before the forward primer, used to boost base diversity on
patterned flow cells (Fadrosh-style).
With variable offsets there is no correct
--trim-left:Both produce the same biological sequence as several distinct ASVs.
Measured impact
On a public NovaSeq 6000 16S run (BioProject PRJNA1504839), 341F was found at
5 distinct offsets (0–4 nt) in 99.96% of merged reads, with the forward
primer retained. Stripping spacer + primer post-hoc collapsed:
loessbinned-qualSo roughly 4 in 10 reported ASVs were offset duplicates of another ASV. It also
inflated an errfun A/B: a −26.1% ASV difference was −16.7% once corrected.
Current workaround
Use
cutadapt(or R'sremovePrimers()) upstream for primer removal, thenfilter-and-trimfor quality filtering only. This is what our R workflow does.Worth documenting explicitly either way, since
--trim-leftlooks like it isthe primer-removal knob and silently is not.
Possible directions
--primer-fwd/--primer-revacceptingIUPAC-degenerate sequences, matched within a small search window with a
mismatch allowance, trimming through the match end. This is what
removePrimers()does and what the data above requires.--max-primer-mismatchand an orientation check.--trim-leftis a positional trim and is notsafe for spacer-containing or variable-offset designs, and point at cutadapt.
Notes
CCTACGGGNGGCWGCAG) mean a plain literal match isinsufficient; IUPAC expansion is required.
a known primer across the first ~30 nt and report the offset distribution. A
multi-offset result is the signal that positional trimming is unsafe. This
would pair naturally with the read-overlap QC screen discussed in the binned
quality findings.
Context:
docs/findings/binned-quality-illumina-novaseq.md.