Page Comparison

...

Demultiplexing and splitting

Demultiplexing

The raw sequence data are in /project/microbiome/data/seq/gtl_tests/iSeq100Pilot1_brew_30nov20/rawdata.

1. I uncompressed the forward and reverse read files with unpigz on an interactive node (could also used single-threaded gunzip).

2. I fixed the line-endings (from MS DOS line-endings) in the Brew_20_DEMUX.csv with dos2unix Brew_20_DEMUX.csv

3. I ran sbatch run_slurm_parse_count.sh after editing run_slurm_parse_count.sh to have the correct filenames and string for the sequencer id.

Splitting

The info lines for each read in parsed_*_R1.fastq and parsed_*_R2.fastq have the locus, the forward mid, the reverse mid, and the sample name. These can be used with the demux key to separate reads into loci, projects, and samples, in the folder sample_fastq/. The reads are in separate files for each sequenced sample, including replicates. The unique combination of forward and reverse MIDs (for a locus) is part of the filename and allows replicates to be distinguished and subsequently merged.

run_splitFastq_fwd.sh and run_splitFastq_rev.sh run splitFastq.pl, which split reads by sample and project, and place them in /project/microbiome/data/seq/gtl_tests/iSeq100Pilot1_brew_30nov20/rawdata/sample_fastq/.

splitFastq.pl will need tweaking in the future, until sample names and the format of the key for demultiplexing and metadata stabilizes. The number of columns has differed among some of our completed sequence lanes.