2
$\begingroup$

I am having strange issues lifting this dataset to hg37 from hg38. I am using the HGDP WGS data from ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/. The data set split by chromosome, so I first converted them to BED then merged them to a single file using plink1.9. However, when trying to lift to hg37/19 using either liftOver or CrossMap v0.5.1 I get strange errors. I tried both the chain from ensemble.org and from UCSC, I tried them both compressed and uncompressed. I am starting to think I need to try the conversion on the VCF files but these are so large I would prefer not to duplicate them unnecessarily.

Data

The data looks fine. If we look at the BIM:

user@desktop:/media/luks8tb1/data/genomics/HGDP$ head hgdp_hg38_chrpos.bim
1   1:10153 0   10153   G   A
1   1:10163 0   10163   C   T
1   1:10180 0   10180   C   T
1   1:10250 0   10250   C   A
1   1:10257 0   10257   C   A
1   1:10291 0   10291   T   C
1   1:10297 0   10297   T   C
1   1:10327 0   10327   C   T
1   1:10330 0   10330   A   C
1   1:10333 0   10333   T   C
user@desktop:/media/luks8tb1/data/genomics/HGDP$ tail hgdp_hg38_chrpos.bim
Y   Y:56887228  0   56887228    C   T
Y   Y:56887285  0   56887285    C   A
Y   Y:56887316  0   56887316    A   G
Y   Y:56887463  0   56887463    A   C
Y   Y:56887491  0   56887491    A   C
Y   Y:56887503  0   56887503    T   G
Y   Y:56887583  0   56887583    G   A
Y   Y:56887631  0   56887631    T   C
Y   Y:56887837  0   56887837    A   G
Y   Y:56887844  0   56887844    C   T

The data are in chrpos variant names here but I had the same result using the original rsid.

Chains

Chains look fine, and I am able to read them into R without problems:

user@desktop:/media/luks8tb1/data/genomics/HGDP$ head GRCh38_to_GRCh37.chain
chain 1 1 248956422 + 10000 297968 1 249250621 + 10000 267719 2
167417 80249 50000
40302

chain 1 1 248956422 + 347968 501617 1 249250621 - 248779253 248932902 3
153649

chain 1 1 248956422 + 585988 12949384 1 249250621 + 521368 13009210 4
1044707 1 0
3709 3 0
user@desktop:/media/luks8tb1/data/genomics/HGDP$ tail GRCh38_to_GRCh37.chain

chain 1 chrY 57227415 + 56821509 57217415 Y 59373566 + 58967656 59363566 11183
85168 0 1
78580 0 1
63496 0 1
3158 1 0
30382 0 1
54147 0 1
80974

CrossMap

user@desktop:/media/luks8tb1/data/genomics/HGDP$ CrossMap.py bed GRCh38_to_GRCh37.chain hgdp_hg38_chrpos.bed hgdp_hg37_chrpos.bed
@ 2020-08-25 20:04:26: Read the chain file:  GRCh38_to_GRCh37.chain
Traceback (most recent call last):
  File "/home/user/.local/bin/CrossMap.py", line 166, in <module>
    crossmap_bed_file(mapTree, in_file, out_file)
  File "/home/user/.local/lib/python3.6/site-packages/cmmodule/mapbed.py", line 32, in crossmap_bed_file
    for line in ireader.reader(inbed):
  File "/home/user/.local/lib/python3.6/site-packages/cmmodule/ireader.py", line 27, in reader
    yield l.decode('utf8').strip().replace("\r", "")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdd in position 20: invalid continuation byte

liftOver

user@desktop:/media/luks8tb1/data/genomics/HGDP$ liftOver hgdp_hg38.bed hg38ToHg19.over.chain hgdp_hg37.bed hgdp_hg37_errors.bed
Reading liftover chains
Mapping coordinates
Data format error: expecting at least 3 fields in BED file (hgdp_hg38.bed)

Trying the PED file with liftOver

I converted BED file to PED with plink1.9:

plink --bfile hgdp_hg38 --recode --out hgdp_hg38

Then:

user@desktop:/media/luks8tb1/data/genomics/HGDP$ liftOver hgdp_hg38.ped hg38ToHg19.over.chain hgdp_hg37.ped hgdp_hg37_errors.ped
Reading liftover chains
Mapping coordinates
invalid unsigned integer: "HGDP00001"

There's a few question on Google about this, but nothing that looks useful.

CrossMap does not support PED.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

BED format can refer to the ".txt" file format that stores chromosomes and positions of genomic features as well as some other things like name. bed format can also refer to the plink format, which belongs to bed, bim, fam trio. The plink bed format stores all the genetic variants, but the program needs to refer to bim file for their positions.

Genome liftover obviously only needs the coordination of each SNPs, and it does not need individual genotypes at all. In fact, the bed format in PLINK does not provide any useful information. Knowing this means that providing the plink bed file is wrong. And this is pretty clear from the error message.

Moreover, since the three plink file always needs to be used together, most programs refer to them as plink file format. So whenever people say BED format, it is the txt file format.

Improve this answer
$\endgroup$
0
$\begingroup$

Not sure about crossMap but liftOver requires the BED format where column 1 is the chromosome, column 2 the 0-based start and column 3 the end coordinate. Additional columns are optional and can be used to store the other columns you have. Simply convert BIM to BED and run liftOver.

Improve this answer
$\endgroup$
2
  • $\begingroup$ BIM files are part of the plink trio of BED, BIM and FAM. The data above are already in BED format. $\endgroup$
    – CoderGuy123
    Commented Aug 25, 2020 at 22:27
  • $\begingroup$ Data format error: expecting at least 3 fields in BED file (hgdp_hg38.bed) According to this, they are not. $\endgroup$
    – user3051
    Commented Aug 26, 2020 at 7:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.