Foreword
Kettle addition to conventional data processing may also be an analog transmission HTTP client / post, REST client.
Experimental background
This Tuesday, the teacher arranged an experiment:
The establishment of a conversion, a cat's eye API json hit movies, and generated as xls file.
Cat's eye hit movie interface: http://m.maoyan.com/#movie , find the API interface from the inside: http://m.maoyan.com/ajax/movieOnInfoList . We need to get inside: the name of the movie, ratings, starring information.
Process flow
I am using a kettle of HTTP GET requests access to the opal film provides an interface
HTTP Client kettle assembly position is:
kettle processing flow as follows:
parameter settings
HTTP Client request for the setting, HTTP client call interface through url, there are two ways. The first one is directly given url, the second step by step in the past among the url passed over.
Returned after the call data will generally exist in the form of a field which the json. We can "result field name" on json named after acquisition by setting the following convenient operation.
When calling HTTP client request header can be added if access is denied.
kettle analysis json
INPUT kettle parsing component json json mainly through the inner input.
When using this component, it must parse the json format to ensure full compliance with resolution rules have written.
Read local files do not need to check the source parsing defined in a field.
After checking field contains json may acquire from among the steps, its resolution. This field is usually the result of http client output field name.
HTTP client when parsing json structure is unknown, parsing rules need to write our own.
Json in the following example, we need access to information corresponding json parsing rules nm, star, sc, showInfo, rt and other results are as follows
{
"coming":[]
,"movieIds":[503342,1230121,1249366,342146,1277939,345419,1250700,1227005,1219701,1215348,1256872,1225993,1251174,296020,1178432,359377,1211270,345397,1190383,1260354,1245196,1243361,1277982,1240752,334625,78519,668097,1197625,1291398,1180993,1242130]
,"stid":"576591972453269000"
,"stids":[{"movieId":503342,"stid":"576591972453269000_a503342_c0"}
,{"movieId":1230121,"stid":"576591972453269000_a1230121_c1"}
,{"movieId":1249366,"stid":"576591972453269000_a1249366_c2"}
,{"movieId":342146,"stid":"576591972453269000_a342146_c3"}
,{"movieId":1277939,"stid":"576591972453269000_a1277939_c4"}
,{"movieId":345419,"stid":"576591972453269000_a345419_c5"}
,{"movieId":1250700,"stid":"576591972453269000_a1250700_c6"}
,{"movieId":1227005,"stid":"576591972453269000_a1227005_c7"}
,{"movieId":1219701,"stid":"576591972453269000_a1219701_c8"}
,{"movieId":1215348,"stid":"576591972453269000_a1215348_c9"}
,{"movieId":1256872,"stid":"576591972453269000_a1256872_c10"}
,{"movieId":1225993,"stid":"576591972453269000_a1225993_c11"}]
,"total":31
,"movieList":[
{"id":503342,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/movie/c437c3e340eff266f5cc7dcc79bead6b2406517.jpg","version":"v3d imax","nm":"双子杀手","preShow":false,"sc":0,"globalReleased":false,"wish":102860,"star":"威尔·史密斯,玛丽·伊丽莎白·温斯特德,克里夫·欧文","rt":"2019-10-18","showInfo":"今天51家影院放映58场","showst":4,"wishst":0}
,{"id":1230121,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/movie/cddf92d0ac6a0db837a1bc488b241c42267927.jpg","version":"v2d imax","nm":"中国机长","preShow":false,"sc":9.4,"globalReleased":true,"wish":561040,"star":"张涵予,欧豪,杜江","rt":"2019-09-30","showInfo":"今天126家影院放映1637场","showst":3,"wishst":0}
,{"id":1249366,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/moviemachine/ff399baffe786445b96e4e3d24fe1eeb675719.jpg","version":"","nm":"航海王：狂热行动","preShow":false,"sc":0,"globalReleased":false,"wish":337092,"star":"田中真弓,冈村明美,中井和哉","rt":"2019-10-18","showInfo":"今天4家影院放映4场","showst":4,"wishst":0}
,{"id":342146,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/movie/6392cd350d00860921042cdac7f2ed797389667.jpg","version":"","nm":"犯罪现场","preShow":false,"sc":8.9,"globalReleased":true,"wish":110692,"star":"古天乐,张继聪,宣萱","rt":"2019-10-12","showInfo":"今天125家影院放映1369场","showst":3,"wishst":0}
,{"id":1277939,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/moviemachine/b2c5c74d33e45745fd3462e44b3698e18336620.jpg","version":"v2d imax","nm":"我和我的祖国","preShow":false,"sc":9.7,"globalReleased":true,"wish":578308,"star":"黄渤,张译,韩昊霖","rt":"2019-09-30","showInfo":"今天126家影院放映1201场","showst":3,"wishst":0}
,{"id":345419,"haspromotionTag":false,"img":"http://p1.meituan.net/w.h/moviemachine/c68e3fff79463952b33cbc1e1b45442b1772992.jpg","version":"v3d imax","nm":"沉睡魔咒2","preShow":false,"sc":0,"globalReleased":false,"wish":227645,"star":"安吉丽娜·朱莉,艾丽·范宁,切瓦特·埃加福特","rt":"2019-10-18","showInfo":"今天34家影院放映34场","showst":4,"wishst":0}
,{"id":1250700,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/movie/4c01895cfd53e82f7c3048c407974a6b4739229.jpg","version":"v2d imax","nm":"攀登者","preShow":false,"sc":9.4,"globalReleased":true,"wish":558579,"star":"吴京,章子怡,张译","rt":"2019-09-30","showInfo":"今天124家影院放映700场","showst":3,"wishst":0}
,{"id":1227005,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/movie/980cddbf2fe1296ae3657c968e536ea56541174.png","version":"","nm":"侠路相逢","preShow":false,"sc":0,"globalReleased":false,"wish":24896,"star":"姜武,邵兵,姚娆","rt":"2019-10-18","showInfo":"2019-10-18 本周五上映","showst":4,"wishst":0}
,{"id":1219701,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/movie/c8c17eb7c9c82d189771e3bfdd4d95061444262.jpg","version":"","nm":"急速逃脱","preShow":false,"sc":7.7,"globalReleased":true,"wish":24799,"star":"沃坦·维尔克·默林,汉娜·赫茨施普龙,克里斯蒂安娜·保罗","rt":"2019-10-12","showInfo":"今天41家影院放映106场","showst":3,"wishst":0}
,{"id":1215348,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/movie/6bce8da6f1daa7f69bed91cd00b049a42186478.jpg","version":"","nm":"天作谜案","preShow":true,"sc":0,"globalReleased":false,"wish":18799,"star":"西达尔特·马尔霍特拉,索娜什·辛哈,阿克夏耶·坎纳","rt":"2019-10-25","showInfo":"2019-10-25 下周五上映","showst":4,"wishst":0}
,{"id":1256872,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/movie/db82e48befc1c7e2e98425165b2500d76222872.jpg","version":"v3d","nm":"为国而歌","preShow":false,"sc":0,"globalReleased":false,"wish":8700,"star":"王雷,古力娜扎,海一天","rt":"2019-10-18","showInfo":"2019-10-18 本周五上映","showst":4,"wishst":0}
,{"id":1225993,"haspromotionTag":false,"img":"http://p0.meituan.net/w.h/movie/cbfd89ee598b5837c7d6b7bb6f6d9e0a1744668.jpg","version":"v3d","nm":"雪人奇缘","preShow":false,"sc":9.3,"globalReleased":true,"wish":59473,"star":"汪可盈,张子枫,丹增·诺盖·特雷纳","rt":"2019-10-01","showInfo":"今天38家影院放映57场","showst":3,"wishst":0}]
}