003_SSIS webservice data extraction

Objective: To extract electronic transaction data

FIG complete process is as follows:

Implementation steps:

1. Perform sql sql task as follows:

select (current date - 1 days)||' 00:00:00' as start_date,(current date - 1 days)||' 23:59:59' as end_date from sysibm.sysdummy1

The results of the query in a variable configuration:

2. Because the interface to extract data only once a day, so to historical data, using foreach container, using ADO enumerator, the incoming variable result

In the section for variables mapped to the start time with the deadline

3. New http connection manager as

4. New web service mission editor, as, first of all now WSDL file, download the WSDL document may not be available, probably due to downloaded

There are one hundred WSDL interfaces, which may be used is 70, and the rest is superfluous, can not be resolved. Solution: First make sure the interface is good to extract data

Name, then you can just keep this one interface.

WSDL file format is as follows, in accordance with the following format preserves the required interfaces, close attention to labels, when the input tab to see the interface name, description WSDL file that is configured successfully.

<definitions>

<types>
   definition of types........
</types>

<message>
   definition of a message....
</message>

<portType>
   definition of a port.......
</portType>

<binding>
   definition of a binding....
</binding>

</definitions>

Foreach the variable passed to the input tab, TokenRing provided upstream fixed value

And outputs the acquired content into a variable

The next step is generated by c # script xml file and step on the User :: HDTYPE_2 variables passed, the script reads as follows, because there are some problems that interface,

Webservice returns the data contained xsd content, so this can not generate xsd xml file, the data processing method is extracted with XPath content, generates a

只含有数据的xml文件，然后再由ssis工具生成xsd文件，如图：

xml文件位置固定，这样每次有新的xml生成就会覆盖原有文件，生成的xsd需要注意的是，系统生成的xsd是由当前xml中的数据生成的，

数据库中为varchar类型的，但此次xml中只含有数字的话，xsd就认为这里为int类型，这里要人工将xsd中的数据类型与数据库中的数据

类型一一对照，手动修改。

public void Main()
{
// TODO: Add your code here
//String content = Dts.Variables["HDTYPE_2"].Value.ToString();
//MessageBox.Show(content);

System.Xml.XmlDocument xdoc = new System.Xml.XmlDocument();
xdoc.InnerXml = Dts.Variables["HDTYPE_2"].Value.ToString();
String text = "<?xml version=\"1.0\" encoding=\"utf-16\"?><DataSet><NewDataSet><Status Result=\"ok\" />";
String text2 = "";
XmlNodeList listNodes = xdoc.SelectNodes("//c_product");
foreach (XmlNode node in listNodes)
{
text +="<c_product>" + node.InnerXml + "</c_product>";

}

XmlNodeList listNodes2 = xdoc.SelectNodes("//c_product_control");
foreach (XmlNode node2 in listNodes2)
{
text2 += "<c_product_control>" + node2.InnerXml + "</c_product_control>";

}
String eXml = text2 + "</NewDataSet></DataSet>";
String Xml = text + eXml;
/*StreamWriter sw = new StreamWriter(@"D:\app\Microsoft SQL Server\MSAS11.HBDCAPP\OLAP\Backup\cigaTransService\GetCigaBaseDATAforJAVA.xml");
sw.WriteLine(Xml);
sw.Close();*/
System.Xml.XmlDocument xdoc2 = new System.Xml.XmlDocument();
xdoc2.InnerXml = Xml;

xdoc2.Save(@"D:\app\Microsoft SQL Server\MSAS11.HBDCAPP\OLAP\Backup\cigaTransService\GetCigaBaseDATAforJAVA.xml");

Dts.TaskResult = (int)ScriptResults.Success;
}

xml源处理好后，进行数据类型的转换，转换也要跟数据库中进行一一对应，精确度，字符串的长度等

处理好后，即可插入到数据库中，结束后对重复的数据进行删除

做完该项抽取后，可以对该包做一个加密处理，双击控制流空白处，点击属性，在如下图中进行设置

003_SSIS webservice data extraction

Guess you like