GBrowse2.0中将GFF3数据转存到MySQL中

概述

在实际项目中GFF3的文件数据量非常大，当GFF3数据量大于1000时就应该考虑使用数据进行存储GFF3数据了，GBrowse支持大量的数据库进行数据存储比如MySQL，SQLite...

数据存储转换

我所用到的数据及账号信息（均需替换成自己的）：

Malus_Gene_Zh.gff3 苹果基因数据
MySQL用户名 root 密码 root(本机测试，没使用复杂的密码，建议使用安全性高的密码) 数据库名 Malus

1、建立相应的数据库

    
    
     
     mysql -uroot -proot #用户名 密码
     
     create database Malus_zh  #数据库名称

2、将GFF3数据转换到MySQL中

    
    
     
     /usr/bin/bp_seqfeature_load -a DBI::mysql -d Malus_zh -u root -p root --create Malus_Gene_Zh.gff3

出现如下的结果就表明数据正在转换并且成功了：

    
    
     
     loading Malus_Gene_Zh.gff3...
     
     Building object tree...70.36s.19s                                                                                                          
     
     load time: 766.59s
     
     Building summary statistics for coverage graphs...
     
     402000 features processed
     
     coverage graph build time: 295.58s
     
     total load time: 1062.17s

注： bp_seqfeature_load.pl脚本创建和存储GFF3数据到MySQL（该文件所在的位置为 /usr/ bin / bp_seqfeature_load . pl ），并且需要注意权限的问题，一般用root登录操作就没有问题

文档信息

    
    
     
     Usage: /usr/bin/bp_seqfeature_load.pl [options] gff_file1 gff_file2...
     
       Options:
     
               -d --dsn        The database name (dbi:mysql:test)
     
               -s --seqfeature The type of SeqFeature to create (Bio::DB::SeqFeature)
     
               -a --adaptor    The storage adaptor to use (DBI::mysql)
     
               -v --verbose    Turn on verbose progress reporting
     
                  --noverbose  Turn off verbose progress reporting
     
               -f --fast       Activate fast loading (only some adaptors)
     
               -T --temporary-directory  Specify temporary directory for fast loading (/tmp)
     
               -c --create     Create the database and reinitialize it (will erase contents)
     
               -u --user       User to connect to database as
     
               -p --password   Password to use to connect to database
     
               -S --subfeatures   Turn on indexing of subfeatures (default)
     
                  --nosubfeatures Turn off indexing of subfeatures
     
               -i --ignore-seqregion 忽视序列区域
     
                               If true, then ignore ##sequence-region directives in the
     
                               GFF3 file (default, create a feature for each region)
     
               -z --zip        If true, database tables will be compressed to save space

也可以使用man手册进行查看

    
    
     
      man /usr/bin/bp_seqfeature_load

使用实例

    
    
     
     重新创建并且刷新频率表
     
     bp_seqfeature_load.pl -a DBI::mysql -d <db> -u <user> -p <passwd> --create *.gff3 *.fasta

3、设置MySQL访问权限

    
    
     
     mysql -uroot -p password -e 'grant all privileges on genomegff3.* to me@localhost'
     
     mysql -uroot -p password -e 'grant select on genomegff3.* to apache@localhost'

或者（直接使用下面的方法比较简单）

    
    
     
      grant all privileges on `Malus_zh`.* to 'www-data'@localhost identified by 'root';

最终结果如下图：

配置GBrowse

1、编辑GBrowse全局配置文件（添加相应的配置项）注：我们需要添加的数据在最后

    
    
     
     # This is the global configuration for gbrowse
     
     # It contains setting common to all data sources as well 
     
     # as the various constants formerly scattered amongst scripts and libraries
     
     
     
     [GENERAL]
     
     config_base            = /etc/gbrowse   # overridden by environment variable GBROWSE_CONF
     
     htdocs_base            = /usr/share/gbrowse/htdocs
     
     url_base               = /gbrowse2
     
     tmp_base               = /var/cache/gbrowse
     
     persistent_base        = /var/lib/gbrowse/databases
     
     userdata_base          = /var/lib/gbrowse/databases/userdata
     
     db_base                = /var/lib/gbrowse/databases/databases
     
     
     
     # These paths are relative to the url base
     
     buttons       = images/buttons
     
     balloons      = images/balloons
     
     openid        = images/openid
     
     gbrowse_help  = .
     
     js            = js
     
     
     
     # These paths are relative to the config base
     
     plugin_path    = plugins
     
     language_path  = languages
     
     templates_path = templates
     
     moby_path      = MobyServices
     
     
     
     # session settings
     
     session lock type = default
     
     
     
     # If no session driver is set, then GBrowse will pick one for you.
     
     # It will use db_file for the driver and storable for the serializer
     
     # if available; otherwise falling back to the file driver and default serializer.
     
     # Override driver guessing by setting these options
     
     # The safest, but slowest session driver...
     
     #session driver    = driver:file;serializer:default
     
     #session args      = Directory /var/lib/gbrowse2/sessions
     
     
     
     # to use the berkeley DB driver comment out the previous
     
     # line and uncomment these two
     
     #session driver = driver:db_file;serializer:default
     
     #session args   = FileName /var/lib/gbrowse2/sessions.db
     
     
     
     # DBI backend to use for uploaded userdata.
     
     # The SQLite option is the easiest to use and the best tested.
     
     # if this option is commented out, then GBrowse will
     
     # try 'DBI::SQLite', 'berkeleydb', 'DBI::mysql' and finally the 'memory'
     
     # backend.
     
     
     
     # NOTICE the double semicolon! This is a DBI Perl module, NOT a DBI connection string.
     
     
     
     # For the DBI::mysql adaptor to work, you must give the web user
     
     # permission to create databases named userdata_% using the following
     
     # mysql command:
     
     #    mysql> grant all privileges on `userdata\_%`.* to 'www-data'@localhost identified by 'foobar';
     
     # Note the backquotes around the database name, and do be sure to replace "foobar" with
     
     # a more secure password!
     
     
     
     # for SQLite
     
     #upload_db_adaptor = DBI::SQLite
     
     
     
     # for Berkeleydb
     
     #upload_db_adaptor = berkeleydb
     
     
     
     # for mysql
     
     #upload_db_adaptor = DBI::mysql
     
     #upload_db_host    = localhost
     
     #upload_db_user    = www-data
     
     #upload_db_pass    = foobar
     
     
     
     # Debug settings
     
     debug                  = 0
     
     debug_external         = 0
     
     debug_plugins          = 0
     
     
     
     # Performance settings
     
     renderfarm             = 1
     
     slave_timeout          = 45
     
     global_timeout         = 60
     
     search_timeout         = 15
     
     max_render_processes   = 4   # try double number of CPU/cores
     
     
     
     # Clean up settings (used by the gbrowse_clean script)
     
     expire session  = 1M  # expire unused sessions after a month
     
     expire cache    = 2h  # expire cached data if unmodified for >2 hours
     
     expire uploads  = 6w  # expire uploaded data if unused for >6 weeks
     
     
     
     # Appearance settings
     
     truecolor     =  1   # better appearance at the expense of larger image files
     
     
     
     # The #include line following this one defines a transparent theme.
     
     # Replace "transparent_colors" with "solid_gray_colors" 
     
     # or "warm_colors" for different themes.
     
     
     
     #include "themes/warm_colors"
     
     # #include "themes/transparent_colors"
     
     # #include "themes/solid_gray_colors"
     
     
     
     balloon tips        = 1
     
     titles are balloons = 1
     
     plugins             = FastaDumper RestrictionAnnotator SequenceDumper TrackDumper
     
     overview grid       = 0
     
     region grid         = 0
     
     detail grid         = 1
     
     image widths        = 450 640 800 1024
     
     default width       = 800
     
     pad_left            = 60
     
     pad_right           = 30
     
     too many landmarks  = 100
     
     track listing style = categories   # either "categories" or "facets"
     
     
     
     # Loads more details image data than can fit on the screen.  This lets the user drag and drop the details
     
     # tracks, without loading more data from the server.  A value of 1 is default (no drag and drop). A value
     
     # of 3 loads one full width on each side.
     
     details multiplier = 3
     
     
     
     # where to link to when user clicks in detailed view
     
     link          = AUTO
     
     
     
     # HTML to insert inside the <head></head> section
     
     head = 
     
     
     
     # At the top of the HTML...
     
     header =
     
     
     
     # At the footer
     
     footer = <hr />
     
              <p style="font-size:small">The Generic Genome Browser. For questions about the data
     
              at this site, please contact its webmaster. For support of the
     
              browser software <i>only</i>, send email to
     
              <a href="mailto:[email protected]">gmod-gbrowse@lists.sourceforge.net</a> 
     
              or visit the <a href="http://www.gmod.org">GMOD Project</a> web pages.
     
              </p>
     
     
     
     # Various places where you can insert your own HTML -- see configuration docs
     
     html1 = 
     
     html2 = 
     
     html3 = 
     
     html4 = 
     
     html5 = 
     
     html6 = 
     
     
     
     # Limits on genomic regions (can be overridden in datasource config files)
     
     region segment         = 200000
     
     max segment            = 5000000
     
     default segment        = 5000
     
     zoom levels            = 100 200 1000 2000 5000 10000 20000 50000 100000 200000 500000 1000000
     
     region sizes           = 1000 5000 10000 20000
     
     default region         = 5000
     
     fine zoom              = 10%
     
     
     
     # keyword search maxima
     
     max keyword results    = 1000
     
     
     
     ###### Authorization ######
     
     # uncomment this to use the PAM authentication plugin
     
     # authentication plugin = PamAuthenticate
     
     
     
     
     
     ####### User Account Registration Database ######
     
     # If no authentication plugin is defined, and
     
     # "user_accounts" is true, then GBrowse
     
     # will attempt to use its internal user accounts database
     
     # to authenticate and/or register users.
     
     user_accounts               = 0
     
     user_accounts_registration  = 0
     
     user_accounts_openid        = 0
     
     
     
     # Path to the database -- you will need to create this database and grant all
     
     # privileges on it to the indicated user.
     
     
     
     user_account_db          = DBI:SQLite:/var/lib/gbrowse2/databases/users.sqlite
     
     
     
     # For SQLite
     
     # user_account_db        = DBI:SQLite:/var/lib/gbrowse2/databases/users.sqlite
     
     
     
     # For MySQL
     
     #user_account_db        = DBI:mysql:gbrowse_login;user=root;password=root
     
     
     
     # The number of public files to display
     
     public_files       = 10
     
     
     
     # What email gateway to use for outgoing registration confirmation messages.
     
     # The full format is
     
     #      <smtp.server.com>:<port>:<encryption>:<username>:<password>
     
     # Only the first field, the server name, is required.
     
     #    The port is assumed to be 25 unless ssl encryption is specified, in
     
     #               which case it defaults to 465.
     
     #    protocol is either "plain" or "ssl", "plain" assumed.
     
     #    username and password may be required by the gateway for authentication
     
     #
     
     # here are some common options
     
     # smtp_gateway         = localhost                                 # localhost has properly configured outgoing gateway
     
     # smtp_gateway         = smtp.oicr.on.ca                           # indicated machine will forward email for you
     
     # smtp_gateway         = smtp.gmail.com:465:ssl:joe.user:secret    # use gmail with account "joe.user" and password "secret"
     
     # smtp_gateway         = none                                      # disable outgoing email
     
     smtp_gateway           = none  # disable outgoing email
     
     
     
     # These values are used in the login confirmation message sent during
     
     # user registration. You may customize
     
     application_name       = GBrowse
     
     application_name_long  = The Generic Genome Browser
     
     email_address          = noreply@gmod.org
     
     
     
     # name of the "superuser" who can add public tracks
     
     admin_account          = admin
     
     admin_dbs              = /var/lib/gbrowse2/databases/admin_uploads
     
     
     
     
     
     ######## DEFAULT DATASOURCE #########
     
     default source = yeast
     
     
     
     ###############################################################################################
     
     # Global settings for plugins (used for the PamAuthenticate plugin only at this point)
     
     ###############################################################################################
     
     
     
     [PamAuthenticate:plugin]
     
     login hint = your UNIX account
     
     login help = <span style="font-size:9pt">Please see your system administrator for help<br>if you have lost your password.</span>
     
     pam service name = gbrowse
     
     
     
     ###############################################################################################
     
     #
     
     # DATASOURCE DEFINITIONS
     
     # One stanza for each configured data source
     
     #
     
     ###############################################################################################
     
     
     
     [yeast]
     
     description   = Yeast chromosomes 1+2 (basic)
     
     path          = yeast_simple.conf
     
     
     
     [yeast_advanced]
     
     description   = Yeast chromosomes 1+2 (advanced)
     
     path          = yeast_chr1+2.conf
     
     
     
     [yeast_renderfarm]
     
     description  = Renderfarm demo (gbrowse_slave must be running!)
     
     path         = yeast_renderfarm.conf
     
     
     
     [pop_demo]
     
     description   = Population Display Demo
     
     path          = pop_demo.conf
     
     
     
     [volvox]
     
     description  = Tutorial database
     
     path         = volvox.conf
     
     
     
     [Malus_zh] #添加Malux基因信息
     
     description  = Malus_zh database
     
     path         = Malus_zh.conf

2、配置数据源配置文件

    
    
     
     vim /etc/gbrowse/Malus_zh.conf

数据项配置

    
    
     
     [GENERAL]
     
     db_adaptor    = Bio::DB::SeqFeature::Store #数据库适配器
     
     db_args       = -adaptor DBI::mysql # MySQL适配器参数
     
     		-dsn	dbi:mysql:database=Malus;host=localhost #数据库连接信息
     
     		-user	root #MySQL用户名
     
     		-pass	root #MySQL密码
     
     
     
     # just the basic track dumper plugin
     
     plugins     = FastaDumper RestrictionAnnotator SequenceDumper TrackDumper Submitter S
     
     autocomplete = 1
     
     # list of tracks to turn on by default
     
     default tracks = Contigs Scaffolds #默认显示的track
     
     
     
     # size of the region
     
     #region segment         = 120000
     
     
     
     # examples to show in the introduction
     
     examples = chr1:1..120000
     
     	   chr2:1..120000
     
     	   chr3:1..120000
     
     
     
     # feature to show on startup
     
     initial landmark = chr1:1..120000 #用户直接进入网页浏览到的数据段
     
     
     
     ########################
     
     # Default glyph settings
     
     ########################
     
     
     
     [TRACK DEFAULTS] #默认track信息
     
     glyph         = generic
     
     height        = 10
     
     bgcolor       = lightgrey
     
     fgcolor       = black
     
     font2color    = blue
     
     label density = 25
     
     bump density  = 100
     
     # where to link to when user clicks in detailed view
     
     link          = AUTO
     
     
     
     ################## TRACK CONFIGURATION ####################
     
     # the remainder of the sections configure individual tracks
     
     ###########################################################
     
     
     
     [Contigs] #用户自定义的数据信息,在这里是contig
     
     feature      = contig
     
     glyph        = generic
     
     bgcolor      = green
     
     height       = 10
     
     key          = Contigs
     
     
     
     [Scaffolds] #在这里是ultracontig
     
     feature      = ultracontig
     
     glyph        = generic
     
     bgcolor      = blue
     
     height       = 10
     
     key          = Scaffolds

3、访问测试

    
    
     
     在浏览器中输入 http://localhost/cgi-bin/gbrowse/gbrowse/Malus_zh/

可以看到下面的效果就表明安装成功：

GBrowse2.0中将GFF3数据转存到MySQL中

猜你喜欢