htmlunit small note --------- web pages

   HtmlUnit Kaiyuan is a Java page analysis tool that can be used effectively analyze page content htmlunit Han, the project can be simulated browser to run, known as the Kaiyuan implementation of the Java browser, this is not the browser interface

 

Using the API

    Simulate a particular browser

        Example: WebClient webClient = new WebClient (BrowserVersion.FIREFOX_3); // analog Firefox

 

Find a specific element

 Obtained by the get method

        HtmlPage page = WebClient.getPage ( "URL"); // Get the page source

        HtmlDivision div = (HtmlDivision) page.getElementById ( "hed"); // Get hed id attribute value of the element.

 

By Xpath acquisition,

        HtmlDivision div=(HtmlDivision)page.getByXPath("//div").get(0);

System.out.println (div.asXml ()); // output code

 

Proxy Server Configuration

    Proxy Configuration is very simple, only you need to configure the address, port, user name and password

    Example: // Create Object

 = New new WebClient WebClient WebClient (BrowserVersion.CHROME, " http://127.0.0.1", 8087);      // simulation browser, the proxy IP address and port number

 

 DefaultCredentialsProvider credentialsProvider=webClient.getCredentialsProvider();  

 

// Set the username and password credentialsProvider.addCredentials ( "username", "password");