rqalpha source - data source and data agency

The underlying data source

If you do not configure the data source will be provided in the event source configuration items in the default data source. The default data source is disposed at a position data / bundle /

    if not env.data_source:
        # print(f"配置的默认的数据源是：{config.base.data_bundle_path}") 
        env.set_data_source(BaseDataSource(config.base.data_bundle_path))

Update the underlying data source

After each run, I will try to update the data source:

     try:
        update_bundle()
    except Exception as e:
        print(f"fail, reasion: {e}")

Source Location: main.py

def update_bundle(data_bundle_path=None, locale="zh_Hans_CN", confirm=True):
    # 设置语言环境
    set_locale(locale)
    # 指定默认的数据下载路径 若没有显式指定路径 即使用该路径
    default_bundle_path = os.path.abspath(os.path.expanduser("~/.rqalpha/bundle/"))
    if data_bundle_path is None:
        data_bundle_path = default_bundle_path
    else:
        # 在执行文件夹下面创建 bundle 文件夹用以存放数据
        data_bundle_path = os.path.abspath(os.path.join(data_bundle_path, './bundle/'))
    # 如果指定的文件夹中已存在 不是默认文件夹 且不为空
    # 向用户确认是否进行删除
    if (confirm and os.path.exists(data_bundle_path) and data_bundle_path != default_bundle_path and
            os.listdir(data_bundle_path)):
        # click.confirm 的作用是展示一个  [y/N]: y 的选项 用户选 y 返回 True
        click.confirm(_(u"""
[WARNING]
Target bundle path {data_bundle_path} is not empty.
The content of this folder will be REMOVED before updating.
Are you sure to continue?""").format(data_bundle_path=data_bundle_path), abort=True)  # 否定的答案将终止程序

    day = datetime.date.today()
    # 创建临时文件
    tmp = os.path.join(tempfile.gettempdir(), 'rq.bundle')

    while True:
        url = 'http://7xjci3.com1.z0.glb.clouddn.com/bundles_v3/rqbundle_%04d%02d%02d.tar.bz2' % (
            day.year, day.month, day.day)
        six.print_(_(u"try {} ...").format(url))
        r = requests.get(url, stream=True)
        if r.status_code != 200:
            # 下载当天的最新数据 如果没有的话 就把时间向前推进一天
            day = day - datetime.timedelta(days=1)
            continue

        out = open(tmp, 'wb')
        total_length = int(r.headers.get('content-length'))

        # 在终端显示进度条
        with click.progressbar(length=total_length, label=_(u"downloading ...")) as bar:
            for data in r.iter_content(chunk_size=8192):
                bar.update(len(data))
                out.write(data)

        out.close()
        break
    # 先下载到临时文件夹里面 然后将用户指定的文件夹清空 将临时文件压缩放入
    shutil.rmtree(data_bundle_path, ignore_errors=True)
    os.makedirs(data_bundle_path)
    tar = tarfile.open(tmp, 'r:bz2')
    tar.extractall(data_bundle_path)
    tar.close()
    os.remove(tmp)
    six.print_(_(u"Data bundle download successfully in {bundle_path}").format(bundle_path=data_bundle_path))

Data source interface

Data source class implements the interface AbstractDataSource defined, can call env.set_data_sourceto replace the default data source. By: class: DataProxyfurther package provides an interface easier to use the upper layer.
For example:
get_all_instruments: get all the contracts the code
get_trading_calendar: acquisition transaction Calendar
get_yield_curve: acquiring Treasury rates
get_dividend: obtaining information dividend stocks and futures, etc. ...
Source Location: interface.py [AbstractDataSource]

Implemented here is to read our updated pull down the data, and then packaged into a data processing interface, providing strategies to use. Analysis here here temporarily read and encapsulation. We can replace any source data source, data files or databases.
It can be called directly by the data acquisition package type data:

from data.base_data_source import BaseDataSource

bb = BaseDataSource("/Users/furuiyang/codes/trade/data/bundle")

# 获取某只股票基金的分红信息
res1 = bb.get_dividend("000949.XSHE")
print(res1, type(res1))

Data Agent

Data provided by the agent to env env.set_data_proxy (DataProxy (env.data_source)).

Data Agent is a layer of encapsulating data source, through inheritance InstrumentMixin, TradingDatesMixin achieve access to some data and trading hours of the stock.

源码： data/data_proxy.py
data/trading_dates_mixin.py
data/instrument_mixin.py