Using preprocessing/alignment functions on the server

Question

I am new to bash and the processes behind cluster computing in general and need some help with understanding some basics.

After looking all over the internet and this forum (+ askUbuntu) I found nothing that addresses this issue.

I have a series of raw RNA sequencing files: .bam and wish to utilize samtools (or others) to begin the data preprocessing steps defined in many workflows.

I have installed samtools and many other programs necessary for the workflow, however, once I am on the remote server in bash, and try and run a function over the files of interest, it gives me the error: -bash-samtools: command not found

When I try to install the package, say samtools, by running sudo apt install samtools, I get an error stating I do not have permission to do such.

What do I do? Must I bring up this issue to the server provider or is there a way around it?

Thank you for your time,

Answer 1

The short answer is to use conda. In the bioconda channel we have most of the tools used in bioinformatics, such as samtools. You do not need administrator permissions to install conda or packages with it, so your lack of sudo ability is not an issue.

I have installed samtools and many other programs necessary for the workflow, however, once I am on the remote server in bash, and try and run a function over the files of interest, it gives me the error

Installing packages on your local machine won't affect the remote server unless they share a file system, so this is expected.

As an aside, BAM files aren't really "raw", they're considered processed data (fastq/fast5 files are considered truly "raw" data in the sequencing world).

Answer 2

At a very wild guess

-bash-samtools

This looks shaky. For a start you need a space between the command and the application in all Linux/Unix/Mac OS X applications (its complicated to explain right now [see below] just accept it).

---

Wait you are using sudo and thats how you are installing on a local machine. sudo on a remote machine will never work, because you are not the systems admin. sudo on a local machine is not cool (see below)

---

Anyway I need to do some next-gen and have 6 MiSeq kits waiting for action so I installed it from here: http://www.htslib.org/download/

Okay so here's my tcsh history

 4  21:44   cd sam* # like I was in the Desktop
 7  21:44   more README # I needed to know how they wanted the installation
 8  21:45   ./configure # okay there are setting the paths for me - saves me a job
11  21:46   make # like fairytale stuff for a load of C scripts
16  21:49   ls | grep samtools # I wanna know if its compiled and I see samtools* ... yep
17  21:49   ln -s ./samtools* ~/bin # I'm gonna pop a link in my bin so after I type
18  21:49   rehash # 'cause I still insist on tcsh rather than bash (don't do it in bash)

However, the following is a bit creepy because I get loads of perl scripts in my /usr/local/bin ...

make install

Anyway,

samtools

It took 4 mins on an OS X machine, using tsch, after I just quaffed 2 pints at my pub (10pm London time). So probability is set against me ....

OUTPUT

Program: samtools (Tools for alignments in the SAM format)
Version: 0.1.18 (r982:295)

Usage:   samtools <command> [options]

Command: view        SAM<->BAM conversion
         sort        sort alignment file
         mpileup     multi-way pileup
         depth       compute the depth
         faidx       index/extract FASTA
         tview       text alignment viewer
         index       index alignment
         idxstats    BAM index stats (r595 or later)

etc ... BTW seriously, fun over, avoid "sudo" if you can, for lots of reasons (which I'm not gonna explain). Its a last resort and usually means your paths are incorrectly set.

sudo will be refused on a remote machine, without question. You COULD ask systems admin to do this for you. HOWEVER, I'd try and install yourself without sudo and your bioinformatics will benefit.

---

Program: samtools (Tools for alignments in the SAM format)
Version: 1.9 (using htslib 1.9)

Usage:   samtools <command> [options]

Commands:
  -- Indexing
     dict           create a sequence dictionary file
     faidx          index/extract FASTA
     fqidx          index/extract FASTQ
     index          index alignment

Output following make install