Hot answers tagged

7

Turns out, simply keeping track of the next candidate line (after sorting the sample line numbers) fixes the performance issue, and most of the remaining slowness seems to be due to the overhead of actually reading the file so there’s not very much to improve. Since I don’t know how how to do this in sed, and it’s not trivial in awk either, here’s a Perl ...


5

Hmm, it's hard to think of a super efficient way of doing this (assuming the files aren't ordered the same - if they are then this whole answer is basically redundant). And also assuming the read ids for both files aren't a perfect intersection. Off the top of my head you probably want to build a set of read ids for the fastq file and another for the bam. ...


5

Perl should be fairly fast with this when using a hash set to store the list of lines. A structure like this also works for subsetting based on a field value, where the comparison would be with the field rather than "$.": #!/usr/bin/perl use strict; use warnings; my $lines_file = $ARGV[0]; my %include_lines = (); open my $lines_fh, '<', $lines_file or ...


4

Some related questions appear in other sites, with potentially interesting solutions, which I report here: To sample approximately 1% of the non-empty lines: awk 'BEGIN {srand()} !/^$/ { if (rand() <= .01) print $0}' input_file (from https://stackoverflow.com/a/692321/1878788) To select 1000 random lines: shuf -n 1000 input_file (from https://...


3

It's safer to import everything. You'll want all the data for normalization and dispersion estimates.


2

I think your issue is simply not assigning the newly-subset object in to a new object. Assuming rownames(subset_DR1) is indeed a list of cell names seuratObj_subset_dr1 <- SubsetData(seuratObj, cells=rownames(subset_DR1)) will work.


2

Here's an approach that uses native R, using the vectorized (fast) family of apply() functions. > df2 <- df[df$A == 'a', which(names(df) %in% c("B","C"))] > apply(df2, 1, prod) Use apply/lapply/mapply/etc. where you can in place of alternatives, where you don't know their run characteristics. Here's a more concrete example: > df A B C D 1 ...


1

If you are going to use idents like that, make sure that you have told the software what your default ident category is. This works for me, with the metadata column being called "group", and "endo" being one possible group there. Idents(combined.all) <- "group" endo_subset <- subset(combined.all, idents = c("endo"...


1

I wrote a command-line (C++14) tool called subset which is up on Github: https://github.com/alexpreynolds/subset This should be reasonably memory efficient and fast. The subset tool does not store input lines in a table, but instead streams through the file once, storing a 4 or 8k buffer chunk of the input file (depending on OS). It stores line numbers in ...


Only top voted, non community-wiki answers of a minimum length are eligible