# What are the ways to keep track of branches in the analysis?

I'm going through an RNA-seq pipeline in R/Bioconductor and want to try multiple parameters at subsequent steps, for example, running clustering with different settings, running RegressOut or not on unwanted effects etc. That's a lot of "versions", even if I don't do combinations of these steps.

How can I keep track of this, and my conclusions? Not necessarily want to save the results.

• Save the different scripts with git (seems overkill)
• Make notes in the script itself

The main purpose of git is to version code, which usually means sequential improvement of the codebase. While it is possible to use branches for multiple variants of the software, permanent branches are traditionally used for gradual integration of new features (i.e. dev/testing/master branches). Supporting multiple independent branches requires some investment, i.e. distributing common changes among branches via merge or cherry-pick. This is hard to manage when you have more than two-three branches.

If you compare different methods of analysis, you probably want to compare the results between methods. Having them on separate branches makes it hard.

In my opinion you should integrate all methods of analysis into the master branch. To avoid copy & paste, it is better to put common code in a library or an independent script. You can also specify a method as a run-time parameter of your pipeline, and create a meta-script which will execute all methods of interests.

Once you performed benchmarking, you shouldn't remove unused methods from you master branch. Having them is important for reproducible research, and your scripts could be used in the future for new datasets.

• "You can also specify a method as a run-time parameter of your pipeline, and create a meta-script which will execute all methods of interests." This is the approach I use. Allows maximum flexibility for code reuse and keeps a straightforward record of everything that I tried. – heathobrien Jun 15 '17 at 12:36
• Don't forget to store the output of the code (with the sessionInfo) if you run the meta-script.R – llrs Jun 15 '17 at 12:48

Save the different scripts with git (seems overkill)

Whoa. I did an actual double take when reading this:1 it’s the opposite of overkill.

Version controlling your scripts (using Git or something similar) is the absolute minimum, and should become completely automatic. For every new project I begin, one of the very first steps is to issue the git init command, and to set up a remote repository (on Github).

To keep track of different analyses, I use a combination of the following approaches:

1. Write reusable functions/scripts and parametrise. The parameters are kept either inside the script itself (that then calls the relevant function repeatedly), or in a Makefile (I recommend Snakemake).
2. Document the alternative analysis approaches; once again, this could be a Makefile with different rules for alternative analyses, or a set of notebooks (via R Markdown).
3. Have different Git branches for mutually exclusive approaches. At the end of the analysis one of these branches gets merged into master, and published. If I want to publish several analysis approaches, I merge all these branches into master, and use approaches (1) or (2) enable them simultaneously.

In fact, I recommend creating a Makefile for every analysis; I have found that this is the most practical, self-documenting way to run an analysis. It most closely resembles a wet-lab lab notebook. The advantage over a single R Markdown document is that rerunning just parts of the analysis can be completely automated, and dependencies in the workflow are apparent from the dependencies of the Makefile rules. This is much harder in R Markdown.

Some time ago I create an example analysis workflow to show how this can be structured. Nowadays I would use Snakemake instead of GNU make.

Make notes in the script itself

“Notes” are a dangerous beasts: documentation is important, but experience shows that it’s sometimes very hard to keep documentation synchronised with the code. There is no mechanism that ensures that documentation and code actually agree. Differences between presumed analysis and actually executed analysis can become very problematic.

People therefore prefer using self-documenting code as much as possible; that is: writing code so that its meaning becomes immediately clear, without comments, even to somebody who hasn’t worked on the code before. Doing this well is hard and takes practice, but improves overall code quality. Once again, using a Makefile helps here because the dependencies between the rules are self-documenting the kind of analysis that was performed.

Jeff Atwood has written two seminal essays on this subject:

They are two of the best pieces of advice on programming that I could give.

1 To emphasise: take a look at the edit history of this answer.

• I appreciate the edit, but it does distort the meaning of my answer: The outrage implicit in the “What????!” is entirely intentional. I’ll change it to something else since people seem to disapprove. – Konrad Rudolph Jun 15 '17 at 15:43
• +1 for the emphasis on makefiles. Makefile and version control make an awesome combo for reproducibility! – Kamil S Jaron Jun 15 '17 at 15:59
• Maybe I should have stated that I meant branching. That seemed to be a lot of overhead for running an R function with a different parameter setting. I'd like to accept your answer too.. but can accept only one. – Peter Jul 4 '17 at 9:56