# Merging files gives memory allocation error

I have two files like

> head(a)
chr position
1   1   136962
2   1   562020
3   1   672948
> class(a)
[1] "data.frame"
>


And

> dim(b)
[1] 1855235      10
SNP chr position
1:  rs62635286          1    13116
2:  rs75454623          1    14930
3:    rs806731          1    30923
4: rs200943160          1    49298
5: rs116400033          1    51479
6: rs141149254          1    54490
> class(b)
[1] "data.table" "data.frame"
>


I want to find corresponding SNP in file b matched to position in file a so I did like this that gives memory allocation error.

> unique(merge(a,b,x.by=a$$position,y.by=b$$position))
Error: cannot allocate vector of size 137.2 Gb


I also tried rstudio cloud but uploading the bigger file never completes

Can you help me with task please?

The expected output would be something like

rs62635286  1   13116
rs376723915 1   91515
rs147061536 1   92858
rs62642131  1   135982
rs371474651 1   158006
rs201293782 1   665401
rs138476838 1   668374
rs1401137   1   2063094
rs3128293   1   2065339


EDITED

> setDT(a)
> setkey(a, chr, position)
> setkey(b, chr, position)
Error in setkeyv(x, cols, verbose = verbose, physical = physical) :
some columns are not in the data.table: chr,position
> setkey(b, Chromosome, Position)
> a[b, nomatch=0]
Error in bmerge(i, x, leftcols, rightcols, roll, rollends, nomatch, mult,  :
Incompatible join types: x.chr (factor) and i.Chromosome (integer). Factor columns must join to factor or character columns.
>


You could try something like this :

First you should work with the same class of object : data.table for fast merging operation

# Turning a to a data.table
setDT(a)
# b is already a data.table


Then you might have to set the keys of both table :

# I added chr in case you need to match the same position on a different chromosome.
setkey(a, chr, position)
setkey(b, chr, position)


Finally perform the joining operation :

a[b, nomatch=0]


Because you didn't provide the expected output, it might need some tuning !

• Thank you I added the output of your command in my main post. I obtained error please can you have a look? Dec 18 '19 at 16:42
• This error means the type of your chromosome columns don't match. You can check the type of all columns with str(a). I suggest you keep the the integer format of the b table. This should update a column from factor to integer : a[, chr := as.integer(chr)]. It may work then ! Dec 18 '19 at 18:00