...
The info lines for each read in parsed_NS*_R1.fastq and parsed_NS*_R2.fastq have the locus, the forward mid, the reverse mid, and the sample name. These can be used with the demux key to separate reads into loci, projects, and samples, in the folder sample_fastq/. The reads are in separate files for each sequenced sample, including replicates. The unique combination of forward and reverse MIDs (for a locus) is part of the filename and allows replicates to be distinguished and subsequently merged.
run_splitFastq_fwd.sh
Below This Point is yet to be done
and run_splitFastq_rev.sh run splitFastq_manyInputfiles.pl, which steps through the many pairs of files to split reads by sample and project, and place them in /project/microbiome/data_queue/seq/NS5/rawdata/sample_fastq/.
splitFastq.pl and splitFastq_manyInputfiles.pl will need tweaking in the future, whenever sample names and the format of the key for demultiplexing and metadata changes. The number of columns has differed among some of early sequence lanes, which necessitated changes to this parsing script.
Below This Point is yet to be done
Calculate summary statistics on reads
...
In /project/microbiome/data_queue/seq/psomagen_6mar20/coligoISD, /project/microbiome/data/seq/psomagen_26may20/coligoISD, and /project/microbiome/data/seq/psomagen_29jan21novaseq1c/coligoISD, there are 16S and ITS directories for all projects. These contain a file named coligoISDtable.txt with counts of the coligos and the ISD found in the trimmed forward reads, per sample. The file run_slurm_mkcoligoISDtable.pl has the code that passes over all of the projects and uses vsearch for making the table.