1. Download the corresponding Jar package:
<!-- httpClient component--> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.5.3</version> </dependency> <!-- Jsoup component--> <dependency> <!-- jsoup HTML parser library @ http://jsoup.org/ --> <groupId>org.jsoup</groupId> <artifactId>jsoup</artifactId> <version>1.10.2</version> </dependency>
2. Create an instance of httpclient:
// Create an instance of httpclient HttpClientBuilder httpClientBuilder = HttpClientBuilder.create(); CloseableHttpClient httpclient = httpClientBuilder.build();
3. Call the get, post method:
HttpGet httpGet = new HttpGet(String url); HttpPost httpPost=new HttpPost(String url);4. Create a return object:
CloseableHttpResponse response = null;5. Use this returned object to receive the request for execution:
response = httpclient.execute(httpGet);
need try-catch;
6. Content of the consuming entity:
HttpEntity entity = response.getEntity();
String html = EntityUtils.toString(entity, "utf-8");
7. The output can get the web content we want
8. Use a proxy:
HttpHost proxy = new HttpHost(String host, int port);
Builder globalConfig = RequestConfig.custom(); globalConfig.setConnectTimeout(10000);// Set the longest connection time globalConfig.setSocketTimeout(30000);// Set the read time httpGet.setConfig(globalConfig.setProxy(proxy).build());9. Proxy ip using account and password:
CredentialsProvider credsProvider = null; credsProvider = new BasicCredentialsProvider(); credsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("username", "password"));