Foreword
Although financial planning data applications like Flush good enough, but still have their own custom impulse, naturally not the data will not be much better than the former, but according to their own ideas to customize is good.
aims
Interface available through the free data, daily incremental update subject of historical transaction data, and then do the analysis and visualization of data through Kibana.
In fact, through visualization framework echarts like to do visualization is also a good choice, but the upfront costs too much. There is pandas + matplotlib been sufficient to meet most of the demand, but a sense of interaction is too weak, so the aid of a visualization application is necessary, where the choice is kibana, its competing products have Grafana.
This goal should have been to go on, you can get the code via the following link
https://github.com/youerning/stock_playground
Environment Configuration
Python3 (recommended Anaconda installation)
Install its dependencies:
pip install -r requirement.txtConfiguring eleasticsearch, kibana environment (recommended docker)
Elasticsearch, Logstash, Kibana 7.2.0
data source
Access to data There are many ways, fee or free of charge, as an amateur free of natural selection, here choose tushare.pro, but in fact tushare be a bit limited, such as data acquisition frequency of certain restrictions, and the interface is also limited, You need a lot of points. If you are interested in this register, register through my referral link chant, so that I can share more about data visualization content with you, and I will share the downloaded data out.
https://tushare.pro/register?reg=277890
It is noteworthy that, tushare there is also a free version of almost no restrictions, but more full pro version of the data, in order to avoid post-maintenance costs, so I chose pro version.
In fact, there are other free data acquisition mode, you can try it yourself
retrieve data
Configure your token
import tushare as ts
ts.set_token("<your_token>")
pro = ts.pro_api("<your_token>")On Access Token can refer to the links
Try to manually acquired data
通过日期取历史某一天的全部历史
df = pro.daily(trade_date='20190725')
df.head()
ts_code trade_date open high low close pre_close change pct_chg vol amount value
0 000032.SZ 20190725 9.49 9.60 9.47 9.56 9.49 0.07 0.7376 12658.35 12075.625 8906.981000
1 000060.SZ 20190725 4.39 4.40 4.35 4.36 4.39 -0.03 -0.6834 129331.65 56462.292 -38586.330353
2 000078.SZ 20190725 3.37 3.38 3.35 3.38 3.37 0.01 0.2967 76681.00 25795.633 7653.564311
3 000090.SZ 20190725 5.66 5.66 5.56 5.61 5.64 -0.03 -0.5319 105582.72 59215.389 -31496.665409
4 000166.SZ 20190725 4.97 4.98 4.93 4.96 4.97 -0.01 -0.2012 268122.48 132793.120 -26717.975744
获取某一只股票的日线行情数据
data = ts.pro_bar(ts_code="601668.SH", adj='qfq', start_date="20120101")
data.head()
ts_code trade_date open high low close pre_close change pct_chg vol amount
0 601668.SH 20190726 6.01 6.06 5.98 6.03 6.04 -0.01 -0.17 696833.16 419634.547
1 601668.SH 20190725 6.05 6.07 6.02 6.04 6.04 0.00 0.00 543074.55 327829.380
2 601668.SH 20190724 6.09 6.11 6.02 6.04 6.05 -0.01 -0.17 788228.12 477542.609
3 601668.SH 20190723 5.93 6.07 5.92 6.05 5.94 0.11 1.85 1077243.46 650250.021
4 601668.SH 20190722 6.02 6.03 5.92 5.94 6.00 -0.06 -1.00 811369.73 485732.343The acquired data is naturally to be automated, but due to the limitations of the interface, you need to consider the following questions.
- Stock list
- Determine whether the interface beyond the limit, and if so, then pause for some time
The key code section
def save_data(code, start_date, fp):
print("下载股票(%s)日线数据到 %s" % (code, fp))
try:
data = ts.pro_bar(ts_code=code, adj='qfq', start_date=start_date)
# 当超过调用次数限制返回None
if data is None:
time.sleep(10)
return
pass_set.add(code)
except Exception:
time.sleep(10)
print("股票: %s 下载失败" % code)
return
if len(data) == 0:
pass_set.add(code)
return
try:
data.trade_date = pd.to_datetime(data.trade_date)
data = data.sort_values("trade_date")
if path.exists(fp):
data.to_csv(fp, mode="a", header=False, index=False)
else:
data.to_csv(fp, index=False)
except Exception:
print("股票:%s 保存失败" % code)We can refer to my GitHub repository save_data.py, you can automatically download the data via the following command
python save_data.pyStart time code inside the configuration of 2012-01-01, there is a need to change the class on their own, it is worth noting that you need to configure a config.json in the same directory, as follows
{
"token": "<your_token>"
}Coupled with their token
Arrangement elasticsearch, kibana
As used herein, is docker configuration.
# 拉取镜像
docker pull sebp/elk:720
# 启动docker环境
docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -v /home/elasticsearch/:/var/lib/elasticsearch -itd sebp/elk:720Dump data
Upload data to elasticsearch inside for data analysis
Configuration settings.py
# 将ip:port改成自己elasticsearch地址，如192.168.56.102:9200
config["es_host"] = ["ip:port"]Run the code
# 上传股票数据
python cmd.py dump
# 上传上证指数数据
python cmd.py dump_indexVisualization
Configuration kibana will take some time, but fortunately kibana now most configurations support import and export, so we can directly import files by export.ndjson my warehouse
Show results
Now that the interface is limited, access to the stock factor is limited, so wait for my points more, I will add more dashboard, and visualization.
postscript
We hope to complete their transaction system from scratch to build a series of articles, then to a working time and place of the Freedom Trail.
Not for absolute financial freedom, time and place of hope free ^ _ ^