0079-【生信软件】-人类基因组hg19、hg38构建bwa索引

在临床数据分析时，使用的人类参考基因组为UCSC上面的hg19，hg39版本，且常常将除1-22，X,Y,M，以外的染色体去除，避免数据的干扰。

hg19索引构建

进入UCSC的官网，hg19的ftp网址 http://hgdownload.soe.ucsc.edu/goldenPath/hg19/bigZips/
参考基因组版本说明，与NCBI的对应版本

This directory contains the Feb. 2009 assembly of the human genome (hg19,
GRCh37 Genome Reference Consortium Human Reference 37 (GCA_000001405.1)),
as well as repeat annotations and GenBank sequences.

The Feb. 2009 human reference sequence (GRCh37) was produced by the
Genome Reference Consortium:
	http://www.ncbi.nlm.nih.gov/projects/genome/assembly/grc/

需要下载的文件

chromFa.tar.gz - The assembly sequence in one file per chromosome.
    Repeats from RepeatMasker and Tandem Repeats Finder (with period
    of 12 or less) are shown in lower case; non-repeating sequence is
    shown in upper case.

md5sum.txt - checksums of files in this directory

hg19.chrom.sizes - Two-column tab-separated text file containing assembly
    sequence names and sizes.

检测文件完整性

# 打开码文件
cat md5sum.txt
## 将chromFa.tar.gz的码写入一个新文件
 echo "ec3c974949f87e6c88795c17985141d3  chromFa.tar.gz" > check_md5sum.txt

## 检测
 md5sum -c check_md5sum.txt
 ##显示 chromFa.tar.gz: OK

查看染色体，看染色体命名规律

cat hg19.chrom.sizes |head -n 25
## 显示
chr1    249250621
chr2    243199373
chr3    198022430
chr4    191154276
chr5    180915260
chr6    171115067
chr7    159138663
chrX    155270560
chr8    146364022
chr9    141213431
chr10   135534747
chr11   135006516
chr12   133851895
chr13   115169878
chr14   107349540
chr15   102531392
chr16   90354753
chr17   81195210
chr18   78077248
chr20   63025520
chrY    59373566
chr19   59128983
chr22   51304566
chr21   48129895
chr6_ssto_hap7  4928567

解压参考基因组，显示所有染色体单个文件

tar -zxf chromFa.tar.gz

将不要的染色体，移动到暂存的文件夹

mv chrUn_gl0002* chr*_random.fa  chr*ctg* chr*apd* chr*cox* chr*hap* temp

#mkdir chrfa
#mv chromFa.tar.gz chrfa/

当面目录只剩下

chr1.fa chr2.fa chr3.fa chr4.fa chr5.fa chr6.fa chr7.fa chr8.fa chr9.fa chr10.fa chr11.fa chr12.fa chr13.fa chr14.fa chr15.fa chr16.fa chr17.fa chr18.fa chr19.fa chr20.fa chr21.fa chr22.fa chrX.fa chrY.fa chrM.fa

将需要的染色体进行合并，并移动到单个fa文件暂存文件夹

for i in chr1.fa chr2.fa chr3.fa chr4.fa chr5.fa chr6.fa chr7.fa chr8.fa chr9.fa chr10.fa chr11.fa chr12.fa chr13.fa chr14.fa chr15.fa chr16.fa chr17.fa chr18.fa chr19.fa chr20.fa chr21.fa chr22.fa chrX.fa chrY.fa chrM.fa
do
	echo $i;
	#cat $i >> hg19.fa
	mv $i chrfa
done

bwa进行索引构建

bwa index -a bwtsw -p hg19.fa hg19.fa

索引构建完后显示

[BWTIncConstructFromPacked] 670 iterations done. 6154206942 characters processed.
[BWTIncConstructFromPacked] 680 iterations done. 6177547390 characters processed.
[bwt_gen] Finished constructing BWT in 687 iterations.
[bwa_index] 2423.30 seconds elapse.
[bwa_index] Update BWT... 19.05 sec
[bwa_index] Pack forward-only FASTA... 16.77 sec
[bwa_index] Construct SA from BWT and Occ... 1192.17 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index -a bwtsw -p hg19.fa hg19.fa
[main] Real time: 3814.250 sec; CPU: 3676.297 sec

文件生成查看

toucan@DELL5577:/mnt/j/BIO001-bioinfos-refseq-doc/UCSC/hg19$ ls -lht
total 8.0G
-rwxrwxrwx 1 toucan toucan 3.0G Sep 27 09:53 hg19.fa

-rwxrwxrwx 1 toucan toucan 1.5G Sep 27 11:17 hg19.fa.sa
-rwxrwxrwx 1 toucan toucan 6.6K Sep 27 10:57 hg19.fa.amb
-rwxrwxrwx 1 toucan toucan  944 Sep 27 10:57 hg19.fa.ann
-rwxrwxrwx 1 toucan toucan 739M Sep 27 10:57 hg19.fa.pac
-rwxrwxrwx 1 toucan toucan 2.9G Sep 27 10:57 hg19.fa.bwt

-rwxrwxrwx 1 toucan toucan 3.5K Sep 27 13:16 work_index.log
-rwxrwxrwx 1 toucan toucan  685 Sep 27 10:06 work_index.sh
drwxrwxrwx 1 toucan toucan 4.0K Sep 27 09:58 chrfa
drwxrwxrwx 1 toucan toucan 4.0K Sep 27 09:17 temp
drwxrwxrwx 1 toucan toucan 4.0K Sep 27 09:00 download

索引构建完成

hg38构建索引

进入UCSC官网 http://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/
查看下载文件

hg38.fa.gz - "Soft-masked" assembly sequence in one file.
    Repeats from RepeatMasker and Tandem Repeats Finder (with period
    of 12 or less) are shown in lower case; non-repeating sequence is
    shown in upper case.

 hg38.fa.gz - "Soft-masked" assembly sequence in one file.
    Repeats from RepeatMasker and Tandem Repeats Finder (with period
    of 12 or less) are shown in lower case; non-repeating sequence is
    shown in upper case.

md5sum.txt - checksums of files in this directory

hg38.chrom.sizes - Two-column tab-separated text file containing assembly
    sequence names and sizes.

检查文件完整性

toucan@DELL5577:/mnt/j/BIO001-bioinfos-refseq-doc/UCSC/hg38/genome/downloads$ echo "1c9dcaddfa41027f17cd8f7a82c7293b  hg38.fa.gz" >> check_md5sum.txt
toucan@DELL5577:/mnt/j/BIO001-bioinfos-refseq-doc/UCSC/hg38/genome/downloads$ echo "a5aa5da14ccf3d259c4308f7b2c18cb0  hg38.chromFa.tar.gz" >> check_md5sum.txt
toucan@DELL5577:/mnt/j/BIO001-bioinfos-refseq-doc/UCSC/hg38/genome/downloads$ cat check_md5sum.txt
1c9dcaddfa41027f17cd8f7a82c7293b  hg38.fa.gz
a5aa5da14ccf3d259c4308f7b2c18cb0  hg38.chromFa.tar.gz

toucan@DELL5577:/mnt/j/BIO001-bioinfos-refseq-doc/UCSC/hg38/genome/downloads$ md5sum -c check_md5sum.txt
hg38.fa.gz: OK
hg38.chromFa.tar.gz: OK

查看染色体命名规则

toucan@DELL5577:/mnt/j/BIO001-bioinfos-refseq-doc/UCSC/hg38/genome/downloads$ cat hg38.chrom.sizes |head -n 25
chr1    248956422
chr2    242193529
chr3    198295559
chr4    190214555
chr5    181538259
chr6    170805979
chr7    159345973
chrX    156040895
chr8    145138636
chr9    138394717
chr11   135086622
chr10   133797422
chr12   133275309
chr13   114364328
chr14   107043718
chr15   101991189
chr16   90338345
chr17   83257441
chr18   80373285
chr20   64444167
chr19   58617616
chrY    57227415
chr22   50818468
chr21   46709983

解压染色体单个文件

tar -zxf hg38.chromFa.tar.gz
cd chroms/

合并需要的染色体文件

 mkdir chrfa

 cat work.sh
for i in chr1.fa chr2.fa chr3.fa chr4.fa chr5.fa chr6.fa chr7.fa chr8.fa chr9.fa chr10.fa chr11.fa chr12.fa chr13.fa chr14.fa chr15.fa chr16.fa chr17.fa chr18.fa chr19.fa chr20.fa chr21.fa chr22.fa chrX.fa chrY.fa chrM.fa
do
        echo $i;
        cat $i >> ../hg38.fa
        mv $i ../chrfa
done

toucan@DELL5577:/mnt/j/BIO001-bioinfos-refseq-doc/UCSC/hg38$ rm -rf chroms/
toucan@DELL5577:/mnt/j/BIO001-bioinfos-refseq-doc/UCSC/hg38$ mv hg38.chromFa.tar.gz chrfa/

查看结果

toucan@DELL5577:/mnt/j/BIO001-bioinfos-refseq-doc/UCSC/hg38$ cat hg38.fa |grep ">"|uniq -c
      1 >chr1
      1 >chr2
      1 >chr3
      1 >chr4
      1 >chr5
      1 >chr6
      1 >chr7
      1 >chr8
      1 >chr9
      1 >chr10
      1 >chr11
      1 >chr12
      1 >chr13
      1 >chr14
      1 >chr15
      1 >chr16
      1 >chr17
      1 >chr18
      1 >chr19
      1 >chr20
      1 >chr21
      1 >chr22
      1 >chrX
      1 >chrY
      1 >chrM

bwa构建索引

bwa index -a bwtsw -p hg38.fa hg38.fa

输出显示：

[BWTIncConstructFromPacked] 630 iterations done. 6018708242 characters processed.
[BWTIncConstructFromPacked] 640 iterations done. 6055482898 characters processed.
[BWTIncConstructFromPacked] 650 iterations done. 6088164258 characters processed.
[BWTIncConstructFromPacked] 660 iterations done. 6117207522 characters processed.
[BWTIncConstructFromPacked] 670 iterations done. 6143017202 characters processed.
[BWTIncConstructFromPacked] 680 iterations done. 6165952882 characters processed.
[bwt_gen] Finished constructing BWT in 686 iterations.
[bwa_index] 2564.67 seconds elapse.
[bwa_index] Update BWT... 19.27 sec
[bwa_index] Pack forward-only FASTA... 16.36 sec
[bwa_index] Construct SA from BWT and Occ... 1230.34 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index -a bwtsw -p hg38.fa hg38.fa
[main] Real time: 3969.074 sec; CPU: 3855.406 sec

输出文件

-rwxrwxrwx 1 toucan toucan 3150052305 Oct  5 15:44 hg38.fa*
-rwxrwxrwx 1 toucan toucan      16843 Oct  5 16:34 hg38.fa.amb*
-rwxrwxrwx 1 toucan toucan        954 Oct  5 16:34 hg38.fa.ann*
-rwxrwxrwx 1 toucan toucan 3088286508 Oct  5 16:33 hg38.fa.bwt*
-rwxrwxrwx 1 toucan toucan  772071602 Oct  5 16:34 hg38.fa.pac*
-rwxrwxrwx 1 toucan toucan 1544143256 Oct  5 16:54 hg38.fa.sa*

其他索引构建

fai结尾

samtools faidx hg19.fa

## 显示生成
hg19.fa.fai

samtools faidx hg38.fa

dict结尾

gatk CreateSequenceDictionary -R hg19.fa -O hg19.dict

0079-【生信软件】-人类基因组hg19、hg38构建bwa索引

hg19索引构建

hg38构建索引

fai结尾

dict结尾

猜你喜欢