Module filter 

Methods for converting sequences into kmers, filtering observed kmers before De Bruijn graph construction, and summarizing ‘color’ annotations.

Functions

bucket

check which bucket a k-mer has to be sorted into according to first four bases

filter_kmers

Process DNA sequences into kmers and determine the set of valid kmers, their extensions, and summarize associated label/‘color’ data. The input sequences are converted to kmers of type K, and like kmers are grouped together. All instances of each kmer, along with their label data are then proccessed with SummaryData::summarize, which generates an implementation of SummaryData, which is specified with the generic SD, decides if the k-mer is ‘valid’ based on the parameters given in summary_config, and summarizes the the individual label into a single label data structure for the kmer. Care is taken to keep the memory consumption small.

filter_kmers_parallel

Process DNA sequences into kmers and determine the set of valid kmers, their extensions, and summarize associated label/‘color’ data. The input sequences are converted to kmers of type K, and like kmers are grouped together. All instances of each kmer, along with their label data are then proccessed with SummaryData::summarize, which generates an implementation of SummaryData, which is specified with the generic SD, decides if the k-mer is ‘valid’ based on the parameters given in summary_config, and summarizes the the individual label into a single label data structure for the kmer. Care is taken to keep the memory consumption small.

remove_censored_exts

Remove extensions in valid_kmers that point to censored kmers. Use this method in a non-partitioned context when valid_kmers includes all kmers that will ultimately be included in the graph.

remove_censored_exts_sharded

Remove extensions in valid_kmers that point to censored kmers. A censored kmer exists in all_kmers but not valid_kmers. Since the kmer exists in this partition, but was censored, we know that we can delete extensions to it. In sharded kmer processing, we will have extensions to kmers in other shards. We don’t know whether these are censored until later, so we retain these extension.