A place to keep one-line pieces of code that might come in handy
- Find files of a certain type and copy to another location:
find . -name \*.xls -exec cp {} newDir \;
- Find files of a certain type and copy to a new location retaining file structure:
find top_level_folder/ -name '*.tsv' | cpio -pdm new_folder_name/
- Find files of a certain name and copy them to a new folder using the name of the folder they are in as the name of the file (useful for examples such as the one below where a pipeline puts out a generic name, say for a fasta file)
find . -type f -name "contigs.fa" -printf "/%P\n" | while read FILE ; do DIR=$( dirname "$FILE" );\cp "./$FILE" "../fastas_for_emmtyper/$DIR.fa";done
Note also in this example, you can test it by adding echo immediately before cp.
From: https://askubuntu.com/questions/746860/rename-a-file-to-parent-directorys-name-in-terminal
- Generate a input tsv from a list of files of a given type (.fa used here) and their paths in multiple subdirectories:
find $PWD -name '*.fa' -type f -print0 | while IFS= read -r -d '' file; do echo -e "$(basename "$file" .fa)\t$file"; done > input.tab
Note, you can modify this as necessary to change the type of file or the amount of the file name removed to form the sample name in the first column
- Check a sample list (samples.txt) against a directory of paired end fastq files to find missing samples:
while read sample; do
ls data/"$sample"*.fastq.gz > /dev/null 2>&1 || echo "$sample is missing"
done < samples.txt
- Count total reads in a FASTQ file:
awk 'NR % 4 == 0' filename.fastq | wc -l- Count the number of sequences in a FASTA file:
grep -c "^>" filename.fasta- List unique sequence IDs from a multi-FASTA file (useful for identifying duplicates):
grep "^>" file.fasta | sed 's/^>//' | sort -u- Identify files containing a specific sequence ID:
grep -l "SequenceID" *.fastq- Check for non-standard bases (anything not A, C, G, T, or N) in a sequence:
grep -v "^>" filename.fasta | grep -i "[^ACGTN]"- Count "N" bases in a FASTA file:
grep -v "^>" filename.fasta | grep -o "[N]" | wc -l- Count occurrences of 'G' (or any other base) in a FASTA:
grep -v "^>" filename.fasta | grep -o "[G]" | wc -l- List IDs of sequences containing non-standard bases:
grep -v "^>" filename.fasta | grep "[^ACGTN]" | cut -d' ' -f1 | sort -u- Count total number of "Other" bases (anything not A, C, G, T, or N):
grep -v "^>" filename.fasta | grep -o "[^ACGTN]" | wc -l- Extract headers from a FASTA file (stripping the
>):
grep "^>" filename.fasta | sed 's/^>//'The below scripts may be of use:
| Script | Comment | Source |
|---|---|---|
| Fasta parser | Python script to parse a multi fasta file and report composition as well as produce file listing entries that contain non-ATGC bases | made by Co-pilot |