Python处理JSON格式数据（出租车轨迹数据）

1.主要是先通过观察发现JSON数据的规律，我们可以使用Notepad++工具查看JSON数据的结构，了解数据的层次。

即使文件不是真正的json格式，例如有可能是多个json格式放在一起，每一行是一个json文件。

例如

{json1}

{json2}

……

2.很大的文本我们可以先用python读取前500或前1000个字符来查看JSON格式都有什么

当我们了解了结构之后，可以先按行读取文件。

扫描二维码关注公众号，回复： 3162484 查看本文章

fname=open("m.txt","r")
print(fname.read(1000))

处理json格式，我们需要json和pandas库

例如我们的文件名为m.txt

内容大致为：(实际每一行不一样)

{ "_id" : "729c3318ad3b4d53b7a92a31010fd085" , "fid" : "BJJYJ_095994" , "y" : "2012" , "m" : "03" , "d" : "01" , "sp" : 728 , "sl" : 235650.0 , "st1" : 40534.0 , "st2" : 40534.0 , "bb" : "116.2164536,39.8669052,116.6208801,40.0528488" , "pts" : [ { "1" : 0 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.0 , "5" : 0.0 , "6" : "2012-03-01 06:28:28" , "7" : "2012-03-01 06:28:33" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 1 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.06486486486486487 , "5" : 0.0 , "6" : "2012-03-01 06:29:36" , "7" : "2012-03-01 06:29:41" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 2 , "2" : 39.9216499 , "3" : 116.6060028 , "4" : 1.894736842105263 , "5" : 0.0 , "6" : "2012-03-01 06:31:27" , "7" : "2012-03-01 06:31:34" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 3 , "2" : 39.9216805 , "3" : 116.6056519 , "4" : 16.494545454545456 , "5" : 23.0 , "6" : "2012-03-01 06:32:24" , "7" : "2012-03-01 06:32:30" , "8" : "0" , "9" : 354.0 , "10" : 1} ]}
{ "_id" : "729c3318ad3b4d53b7a92a31010fd085" , "fid" : "BJJYJ_095994" , "y" : "2012" , "m" : "03" , "d" : "01" , "sp" : 728 , "sl" : 235650.0 , "st1" : 40534.0 , "st2" : 40534.0 , "bb" : "116.2164536,39.8669052,116.6208801,40.0528488" , "pts" : [ { "1" : 0 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.0 , "5" : 0.0 , "6" : "2012-03-01 06:28:28" , "7" : "2012-03-01 06:28:33" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 1 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.06486486486486487 , "5" : 0.0 , "6" : "2012-03-01 06:29:36" , "7" : "2012-03-01 06:29:41" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 2 , "2" : 39.9216499 , "3" : 116.6060028 , "4" : 1.894736842105263 , "5" : 0.0 , "6" : "2012-03-01 06:31:27" , "7" : "2012-03-01 06:31:34" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 3 , "2" : 39.9216805 , "3" : 116.6056519 , "4" : 16.494545454545456 , "5" : 23.0 , "6" : "2012-03-01 06:32:24" , "7" : "2012-03-01 06:32:30" , "8" : "0" , "9" : 354.0 , "10" : 1} ]}

则处理代码如下

import json
import pandas as pd
j=0
with open(r'm.txt') as file:
    for line in file:
        m=json.loads(line)
        n = m["pts"]
        data = pd.DataFrame(n)
        filename = "%d.csv"%j
        f = open(filename , 'w')
        j=j+1
        data.to_csv(filename)
        f.close()

处理的结果如图

每个文件的内容

,1,10,2,3,4,5,6,7,8,9
0,0,1,39.9216461,116.6059799,0.0,0.0,2012-03-01 06:28:28,2012-03-01 06:28:33,0,0.0
1,1,1,39.9216461,116.6059799,0.06486486486486487,0.0,2012-03-01 06:29:36,2012-03-01 06:29:41,0,0.0
2,2,1,39.9216499,116.6060028,1.894736842105263,0.0,2012-03-01 06:31:27,2012-03-01 06:31:34,0,0.0
3,3,1,39.9216805,116.6056519,16.494545454545456,23.0,2012-03-01 06:32:24,2012-03-01 06:32:30,0,354.0
4,4,1,39.9234695,116.6038361,24.588679245283018,23.0,2012-03-01 06:33:19,2012-03-01 06:33:24,0,302.0
5,5,1,39.9262543,116.6060333,16.599999999999998,21.0,2012-03-01 06:34:12,2012-03-01 06:34:18,0,36.0

这样json文件就被处理成可以使用的csv文件了，可以用于ArcGIS后期分析处理了

Python处理JSON格式数据（出租车轨迹数据）

猜你喜欢