No need for python2 streamlined DataX under Windows

Streamlined DataX

table of Contents

Overview

Ali DataXLite, you can windowslower rely only Java8you can run. It recommends linuxthe use of original lower DataX. Sorry, I received a little point as a reward for the weekend work, thank you. Download link for the lite version: https://download.csdn.net/download/killingbow/12158389

1. Start quickly

Execution run.batcan convert inall the csv files in the outdirectory to the directory, and csvuse taba comma as a separator to use as a csvseparator. This conversion job/csv2csv.jsonis defined in.

Note: csv2csv.jsonThe windowsversion and linuxversion cannot be mixed. As inputstreamof csvthe encodingthough is actually utf8encoded in the windowsversion to be written gbk, it will have the correct output. This may be caused by a bug in DataX, but there is no problem if it is in Linux.

2. Advanced

jobThere are several *.json files under the path, which can be realized by modifying the configuration to realize the mutual conversion of data between sqlserver, text files, streamand ftp. The "csv2csv" run.batin the first line set JOB_NAME=csv2csvof the jobfile represents the name of the file csv2csv.json, which run.batwill run this csv2csv.jsonconfiguration and output a log with this name. You JOB_NAMEcan change the task to be run by modifying this .

Regarding jobthe jsonformat configuration file, its content can be composed of settingand contenttwo parts. settingThe partial channelsetting is "1". contentThe readerand writermay refer to pluginthe directory readerand writerplug folder directory corresponding to the plugin_job_template.jsoncontent. Detailed configuration documents can be found in dataxthe githubofficial source code directory. Each readerand writerhas very detailed attribute definitions including whether it is required, default values, enumerations, and value ranges.

Three, customized lite version

We want to add more modules to the Starter Edition, available from the official version of pluginthe copy readerand writerthe lower corresponds to the current directory. These readerand writerthe libdirectory to the public libin sharing and achieve the purpose of streamlining.

on

• Compose: tlw
• Log:
  • 20200215: Created