Accept AGC archives as input - #494
Merged
Merged
Conversation
-i now accepts an AGC (.agc) archive. Sequence names, used for the PanSN check and for the haplotype and sequence counts, are enumerated with 'agc listctg' instead of a samtools .fai, and partition-before-pggb extracts each community with 'agc getctg' instead of 'samtools faidx'; wfmash and seqwish read the archive directly. Requires agc on PATH. An archive that repeats a contig name across samples is rejected: pggb needs unique sequence names, and 'agc getctg' writes the bare contig name as the FASTA header, so those records could not be extracted apart.
Adds AGC v3.2.1 (PLATFORM=sse2, to keep the image portable): the agc binary is installed for pggb's own listing and extraction, and wfmash is configured with -DAGC_ROOT so it links libagc and can read .agc directly. Only the files wfmash links against are kept; the AGC source tree is removed in the same layer. seqwish is now a Rust crate, so it moves below the Rust toolchain, is built with cargo, and is bumped to a revision that reads AGC input. Documents that .agc input needs agc on PATH and an AGC-enabled wfmash.
Builds an AGC archive from the DRB1-3123 test sequences and induces the graph from both the FASTA and the archive, reusing one alignment so the only difference is where the sequences come from; the seqwish graphs must be identical. Also covers agc in the no-AVX portability check, since the image now ships it.
seqwish carries .cargo/config.toml with target-cpu=native, so building it with cargo baked the build machine's instruction set into the image and the binary died with SIGILL on CPUs without AVX (the portability job caught it). Set RUSTFLAGS to the x86-64-v2 baseline used for the other tools, which takes precedence over the crate's config: the binary then contains no ymm/zmm instructions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
-i/--input-fastanow accepts an AGC (.agc) archive in addition to FASTA/FASTQ, in bothpggbandpartition-before-pggb.An
.agcarchive has no samtools.fai, so:agc listctginstead ofcut -f1 *.fai;partition-before-pggbextracts each community withagc getctginstead ofsamtools faidx(identical FASTA headers, verified);Requires the
agcbinary onPATH(checked up front alongside the other tools) and needs the AGC support in wfmash (waveygang/wfmash#400) and seqwish (pangenome/seqwish#144), both merged.An archive that holds the same contig name in more than one sample is rejected with a clear message: pggb needs unique sequence names, and
agc getctgwrites the bare contig name as the FASTA header, so those records could not be told apart after extraction. PanSN naming (sample#haplotype#contig) gives unique names by construction.Verified on the LPA test data and on 8 yeast genomes (96 Mbp): the
seqwish.gfathat pggb produces is byte-identical whether the input is the FASTA or the AGC archive built from it (the final.smooth.final.gfais not compared because smoothxg is not deterministic run to run, as two runs on the same FASTA also differ). Runtime and memory are at parity with FASTA input, and the archive is roughly 4x smaller on disk.