I have used the following script for calculating the median for one file (gene). However, I have 20,000 files for which I want to run this R script and get 200 separate output files
#!/usr/bin/R
setwd("/location folder")
d <- read.table("tgf.txt", header=FALSE)
x <- aggregate(d[, 2], list(d$V3), median)
write.table(x, "tgf.output.txt", quote=FALSE, row.names=TRUE)
q()
Another option is to use list.files and grep to select only the .txt in your directory. A for loop can read-aggregate and save the files one at the time.
all.files <- list.files("/location folder")
txt.files <- grep(".txt",all.files,value=T)
for(i in txt.files){
d <- read.table(paste("/location folder",i,sep=""),header=F)
x <- aggregate(d[, 2], list(d$V3), median)
write.table(x, paste0(i,".output.txt"), quote=FALSE, row.names=TRUE)
}
list.files(pattern = "*.txt")? The grep is not required.
$\endgroup$
paste(..., sep="") on line 5 and paste0() on line 7.
$\endgroup$
paste and paste0 achieve the same but in different ways (e.g. sep=""). I tend to split commands involving list.files and grep as it makes easier to read and to reuse. The content of all.files is hard to miss. Finally the usage of the applyfamily is not easy to grasp for new user, while loops are much intuitive. Furthermore, apply is not always better than a nice for, especially in an example like this.
$\endgroup$
apply family solves a lot of problems, especially when there is a learning opportunity such as this one, where the task is well defined and how apply works can be easily understood.
$\endgroup$
Please do not use setwd in a script, it changes working folder without explicit specifications from the user, so the results could be unpredictable for people that are not familiar with the content of the script.
Create a tab-separated file with input filename and output filename, or at least a file with all input files, unless you can pick them using a list.files(pattern = "<pattern_here>"). Read this file as a data.frame and use it with an apply, so a function processes the df row-by-row. Let's say the file (named metadata.txt) is:
in_file out_file
tgf.txt tgf.output.txt
abc.txt abc.filtered.txt
xyz.txt xyz.randomname.txt
Define a function called process_row() in the Rscript:
process_row <- function(r) {
d <- read.table(r[[1]], header = FALSE);
x <- aggregate(d[, 2], list(d$V3), median);
write.table(x, r[[2]], quote=FALSE, row.names=TRUE);
}
And do the actual work:
meta_data <- read.table('./metadata.txt', sep = ' ', header = TRUE, stringsAsFactors = FALSE);
apply(meta_data, 1, process_row(r))
# You don't need a q() in an Rscript because it exits after the last line of code anyway.
./metadata.txt, which is in the working directory where you're running R?
$\endgroup$