Recent updates
by Austin G. Marshall, Kristen D. Curry, Eddy Huang, Michael G. Nute and Todd J. Treangen
The Emu development team recently came across the Savont tool and preliminary results, which reported Savont's performance on an ONT 16S mock community dataset and, importantly, highlighted a recurring theme in benchmarking: tools can look broadly consistent while still differing meaningfully in why they make their mistakes. In that write-up, Dr. Jim Shaw compared Savont and Emu on some well characterized, albeit simplistic, microbial mock communities. Both tools produced similar high-level profiles, but Emu appeared to generate a larger tail of low-abundance false positives. Good science benefits from constructive dialogue, so we welcomed the opportunity to learn more about both methods. Also, while Emu has undergone regular maintenance and some workflow tooling (shoutout to Kristen and Samuel Lampa), we have not made any major modifications to it since it was published in 2022.
For this post, it made sense to start by reproducing Dr. Shaw's comparison and dig deeper into the differences between the two tools and the particulars of the false positives that were observed. When Emu was first published, the most striking point of comparison between it and all other methods available was the dramatic reduction in false positives that it accomplished, so it is a reflection of the advancement in this area since then that this particular challenge has ebbed so heavily. This reflects both an improvement in the methods but also of course the reduction (but not elimination) of error rates in ONT reads through the R10 chemistry and more sophisticated basecalling, but more on that later…
First, a quick aside to discuss Savont, a promising new tool for full length 16S microbiome profiling. Savont is a tool for microbial amplicon profiling on long-reads with low-ish but still non-trivial error rates, and was developed by Dr. Jim Shaw, postdoctoral scientist in Dr. Heng Li's group at Harvard. The problem with long-reads (non-HiFi division) is familiar: the length gives additional information but the error rate (particularly when that includes lots of indels) make it so that naive per-read alignment is also high error. Savont’s approach to this is to create amplicon sequence variants (ASVs) from the sample (essentially clusters of reads that putatively originate from a single organism with a single true sequence).
This is a popular approach for amplicon sequencing using Illumina reads, for example using the popular DADA2 tool, although in that case the problem is mathematically and computationally quite distinct given Illumina and PacBio HiFi reads have far fewer insertions or deletions (indels). For ONT reads that is not the case, and as the indel rate gets away from zero, clustering becomes more difficult. So it is a noteworthy accomplishment to have developed a method that credibly estimates ASVs from ONT amplicon reads. Additionally, Savont runs in a few minutes where each of Emu’s iterations may take 15-20 minutes (or longer), depending on the size of the dataset etc.
Savont’s documentation suggests that it is best suited for long reads with at least 98% accuracy, which is achievable today with the latest chemistry and basecalling methods although even then it’s not guaranteed. Accuracy rates vary for SNPs vs. Indels and also depend on the coverage, plus a 98% accuracy is generally a median value and will come with plenty that are below that mark. Also, of course, there are plenty of cases where the latest and greatest chemistry is not an option (as well as simply old datasets that exist on the NCBI SRA). Mainly, this highlights the point though that Emu was developed, optimized, and tested on R9 sequencing data with median SNP+Indel rates in the 8% to 13% range. So as we think about improving Emu, taking advantage of better quality data is an obvious leverage point.
We ran Emu v3.5.5 on what we believe is the same dataset used in Savont's preliminary results. This dataset comes from ONT’s validation of their new Microbial Amplicon Barcoding Kit, where they provide four replicates of the Zymo Gut Microbiome Standard (Cat #D6331). We processed each replicate independently with Emu 3.5.5 and Savont v0.3.2, using the original Emu database.
Our results were consistent with Dr. Shaw's narrative: the main community members were recovered similarly, but Emu produced more low-abundance calls that looked like false positives. One point that we’d like to add is that Emu is able to correctly call S. enterica in 2 of the 4 samples while Savont does not. The true concentration of this microbe is expected to be in the 0.01% range, so it’s a challenging case for sure and it’s hard to draw a systematic conclusion from one false negative.
But the larger point about Emu with respect to false positives remains and is worth discussing. The first thing to notice is that one of these FP’s is not like the others… Can you spot it?
If you said the Laceyella sacchari, give yourself a pat on the back. We’ll cover that one in the next section because it comes with a fresh commit to get rid of that bug. But that one should jump out because it’s the only FP in any of the charts that isn’t extremely closely related to one of the True Positives in the sample (Shigella is a close relative of Escherichia and is taxonomically separated largely out of historical accident, while all other FPs share a genus with a TP). Not Laceyella though, that one is out of left field.
For the rest though, note that Emu calls each of these at an abundance below 0.1% while the concentration of the closely related TP is an order of magnitude higher, which is a reflection of how the EM algorithm likely drove their estimated concentration down with each iteration, as it should. But it’s still just a plain EM algorithm with no hard-cap between iterations, so it would naturally generate an abundance vector with a long tail of non-zero estimates. That decision was motivated largely by the 13% error rate that prevailed at the time, because with that much noise the distinction between a handful of reads and none at all was absorbed by the noise. To have what FPs did show up appear at such low concentration compared to the truth was a breakthrough.
But those were the old days, and that doesn’t solve the problem in these figures, but it illustrates how Emu is still operating as it did when the problem was far noisier than now and there is significant potential for improvements by calibrating it to sharpen up its estimates when the data quality is available. The results above are probably a reasonable harbinger of things to come: the false positives can likely be cleaned up either by additional polishing or with a sharper statistical model related to the SNPs and Indels. At the same time, as with Salmonella, true positives at low abundance may now be discernable and we may be able to reliably detect microbes at 0.01% concentration.
Ok, back to the Laceyella case. As noted, this was a bizarre thing to observe in such a simple community. As part of our testing we had also run both methods on the Zymo Standard Communiday (D6300), and wouldn’t you know it but the same Laceyella species was in that one too, and at the same concentration! So that’s either a wild coincidence or Emu has a garden-variety bug somewhere (the computer kind).
So the easiest way to start is to look at the reads that map and see the alignment with your own eyes. We used Emu’s --keep-files option to save the SAM alignments, then used grep on the results to find that genus:
(nute) mn56@gho:~$ cat $samfile | grep '37482' -n | grep 'NM:i:0' | head
177857:5acdf05d-40c7-48c3-8a0f-462245a20ebc 2064 37482:emu_db:37435 987 1 139H61M763H * 0 0 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
291125:dbbe829a-6459-4c3e-a6bd-1784792f9c98 2064 37482:emu_db:37435 987 2 676H63M1102H * 0 0 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGG
291126:dbbe829a-6459-4c3e-a6bd-1784792f9c98 2048 37482:emu_db:37435 987 1 1219H61M561H * 0 0 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
291127:dbbe829a-6459-4c3e-a6bd-1784792f9c98 2048 37482:emu_db:37435 987 1 1248H61M532H * 0 0 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
291128:dbbe829a-6459-4c3e-a6bd-1784792f9c98 2048 37482:emu_db:37435 987 2 1284H43M514H * 0 0 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Well that’s 5 reads and we think we’ve found the problem. All 5 alignments to Laceyella are between 50-60bp long, with all hard-clips or soft-clips elsewhere. And on those 50bp wouldn’t you know it but every single matching character is an ‘A’, so one hunch is that if we look at the Laceyella reference sequences, we are going to find one with a homopolymer run that was never caught, and in fact that was correct:
>37482:emu_db:37435 ["37482:rrn:16062 ['Laceyella sacchari|GCF_002893785.1|NZ_CP025943.1|Chromosome: ANONYMOUS|124359..125863 +']"]
(sequence truncated)
...CGGCCGCAAGGCTGAAACTCAAAGGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGAAGCAACGCGAAGAACCTTACCAGGGCTTGACATCCCGCTGACCGCTCCAGAGATGGAGCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGGGTTCAGTCCCGCAACGAGCGCAACCCTTATCGCTAGTTGCCAGCATTCAGTTGGGCACTCTAGCGAGACAGCCGGTGAAAGCCGGAGGAAGGTGGGGATGACGTCAAATCATCATGCCCCTTATGTCC…
(sequence truncated)
So this one was quick, and the fix will be to remove that particular sequence from our reference DB, which we are doing and committing along with this post. For Laceyella aficionados, don’t worry, we have 12 other reference sequences for that one.
You may be wondering: why Emu was relying on a 50bp alignment match for an amplicon that is supposed to be 1400bp long? Is there no filter after the read-mapping to ensure that the full gene is used? Given we had observed that allowing some flexibility in alignment length would rescue signal, we didn’t implement a strict filter. However, a filter on the length of the alignment would be a good QC option for users in general and especially those using other custom databases where this issue might also arise, and it is simple to implement. To address this, we added minimum and maximum alignment length filters (--min-aln-len and --max-aln-len) and reran the dataset (prior to the Laceyella removal). In our quick tests, the Laceyella sacchari false positive disappeared after filtering:
The Savont post notes that:
“V. rogosae is in the community rather than V. parvula. However, their 16s sequences are similar, and the ASV is best classified as V. parvula according to Emu's database. BLASTN indeed gives the ASV as V. parvula.”
Indeed, there are often cases where distinct species have 16S sequences that are nearly indistinguishable (or where intraspecies diversity overlaps interspecies diversity). In those cases, the correct species-level taxonomic call may be fundamentally underdetermined from 16S alone, leading to the loss of species-level taxonomic resolution [1].
To highlight one such example, we collected all of the 16S sequences from the Emu database for both species and created a multiple sequence alignment, shown below.
We see anywhere between 10 and 20 SNPs between the V. rogosae 16S sequence and the remainder V. parvula sequences. The key takeaway is clear: some sequences labeled as V. parvula are more similar to the lone V. rogosae sequence than to other V. parvula sequences. While we are currently looking at modifying Emu to push the classifications up to a genus level classification in these situations. The original V. rogosae paper [2] indicated the following regarding the ability of full length 16S sequences to distinguish between these two very closely related species:
“We suggest that as 16S rRNA gene sequence comparisons are not suitable for differentiating between all members of the genus Veillonella, comparisons based on rpoB gene sequences represent a valid method for species characterization”
The emergence of the promising new tool Savont served as a timely "check engine light" for Emu. We are very grateful to Dr. Shaw for taking the time to compare Savont to Emu, and inspiring a constructive dialogue that helped us improve Emu. It’s been a few years since our 2022 release, and although Emu has performed well in external validations over the past few years [3,4,5], this comparison highlighted exactly where we needed to sharpen the tool for the higher-accuracy R10 (10.4.1 and beyond) era.
Thanks to the aforementioned analysis, we’ve pushed two important updates to Emu:
-
Code Fixes: We’ve implemented
--min-aln-lenand--max-aln-lenparameters to prevent short, spurious alignments from impacting results. - Database Cleanup: We’ve purged the problematic Laceyella sacchari reference and are undergoing a broader audit to remove similarly problematic sequences that trigger false positives.
All of which, are included in the Emu 3.6.1 release.
What's next?
- Algorithm Re-calibration: We are working on a refined statistical model that accounts for lower error rates, ensuring our EM algorithm doesn't produce a long tail of noise when the data quality allows for cleaner species-level calls.
- Database updates: We are focusing on updating the Emu 16S database and plan to release an updated database by the end of the month.
-
Commichaux S, Luan T, Muralidharan HS, Pop M (2024) Database size positively correlates with the loss of species-level taxonomic resolution for the 16S rRNA and other prokaryotic marker genes. PLoS Comput Biol 20(8): e1012343. doi:10.1371/journal.pcbi.1012343
-
Arif N, Do T, Byun R, et al. Veillonella rogosae sp. nov., an anaerobic, Gram-negative coccus isolated from dental plaque. Int J Syst Evol Microbiol. 2008;58(Pt 3):581-584. doi:10.1099/ijs.0.65093-0
-
Esberg, A., Fries, N., Haworth, S. & Johansson, I. Saliva microbiome profiling by full-gene 16S rRNA Oxford Nanopore Technology versus Illumina MiSeq sequencing. npj Biofilms Microbiomes 10, 149 (2024). doi:10.1038/s41522-024-00634-1
-
Yergaliyev, T., Rios-Galicia, B. & Camarinha-Silva, A. NaMeco - Nanopore full-length 16S rRNA gene reads clustering and annotation. BMC Genom. 27, 62 (2025). doi:10.1186/s12864-025-12415-x
-
Gupta, A., Cooper, V. S. & Zemke, A. C. Evaluation of V3–V4 and FL-16S rRNA amplicon sequencing approach for microbiota community analysis of tracheostomy aspirates. mSphere 10, e00388-25 (2025). doi:10.1128/msphere.00388-25