curl is a commonly used command line tool to request a web server. Its name means the URL tool of the client.
Its function is very powerful, as many as dozens of command line parameters. If you are proficient, you can completely replace graphical interface tools like Postman.
This article introduces its main command line parameters as a daily reference for easy reference. The content is mainly translated from "curl cookbook" . In order to save space, the following example does not include the runtime output. For beginners, you can read the "curl tutorial for beginners" I wrote before .
Without any parameters, curl will issue a GET request.
$ curl https://www.example.com
The above command sends www.example.com a GET request to the server, and the content returned by the server will be output on the command line.
-A
-A The parameter specifies the user agent header of the client, ie User-Agent. The default user agent string for curl is curl/[version].
$ curl -A 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36' https://google.com
The above command will be User-Agent changed to Chrome browser.
$ curl -A '' https://google.com
The above command will remove the User-Agent header.
You can also -H directly specify the header through parameters and change it User-Agent.
$ curl -H 'User-Agent: php/1.0' https://google.com
-b
-b Parameters are used to send cookies to the server.
$ curl -b 'foo=bar' https://google.com
The above command will generate a header Cookie: foo=barand send a cookie with a name fooand a value to the server bar.
$ curl -b 'foo1=bar;foo2=bar2' https://google.com
The above command sends two cookies.
$ curl -b cookies.txt https://www.google.com
The above command reads the local file cookies.txt, which contains the cookie set by the server (see -cparameters), and sends it to the server.
-c
-c The parameter writes the cookie set by the server to a file.
$ curl -c cookies.txt https://www.google.com
The above command writes the cookies set by the server's HTTP response to a text file cookies.txt.
-d
-d The parameter is used to send the data body of the POST request.
$ curl -d'login=emma＆password=123'-X POST https://google.com/login # 或者 $ curl -d 'login=emma' -d 'password=123' -X POST https://google.com/login
After using -d parameters, HTTP requests will automatically add headers Content-Type : application/x-www-form-urlencoded. And it will automatically convert the request to the POST method, so it can be omitted -X POST.
-d The parameter can read the data of the local text file and send it to the server.
$ curl -d '@data.txt' https://google.com/login
The above command reads data.txt the content of the file and sends it to the server as a data body.
--data-urlencode
--data-urlencode The parameters are equivalent to -dthe data body of the POST request. The difference is that the sent data will be automatically URL-encoded.
$ curl --data-urlencode 'comment=hello world' https://google.com/login
In the above code, hello world there is a space between the sent data , which needs to be URL encoded.
-e
-e The parameter is used to set the HTTP header Referer, indicating the source of the request.
curl -e 'https://google.com?q=example' https://www.example.com
The above command sets the Referer header to https://google.com?q=example.
-H Parameters can be directly added to the header Refererto achieve the same effect.
curl -H 'Referer: https://google.com?q=example' https://www.example.com
-F
-F Parameters are used to upload binary files to the server.
$ curl -F '[email protected]' https://google.com/profile
The above command will add a header to the HTTP request Content-Type: multipart/form-data, and then upload the file photo.pngas a filefield.
-F The parameter can specify the MIME type.
$ curl -F '[email protected];type=image/png' https://google.com/profile
The above command specifies the MIME type image/png, otherwise curl will set the MIME type application/octet-stream.
-FThe parameter can also specify the file name.
$ curl -F '[email protected];filename=me.png' https://google.com/profile
In the above command, the original file name is photo.png, but the file name received by the server me.png.
-G
-G The parameters are used to construct the query string of the URL.
$ curl -G -d 'q=kitties' -d 'count=20' https://google.com/search
The above command will issue a GET request, and the actual requested URL is https://google.com/search?q=kitties&count=20. If omitted --G, a POST request will be issued.
If the data needs URL encoding, it can be combined with --data--urlencodeparameters.
$ curl -G --data-urlencode 'comment=hello world' https://www.example.com
-H
-H The parameter adds the header of the HTTP request.
$ curl -H 'Accept-Language: en-US' https://google.com
The above command adds HTTP headers Accept-Language: en-US.
$ curl -H 'Accept-Language: en-US' -H 'Secret-Message: xyzzy' https://google.com
The above command adds two HTTP headers.
$ curl -d '{"login": "emma", "pass": "123"}' -H 'Content-Type: application/json' https://google.com/login
The above command adds the header of the HTTP request Content-Type: application/json, and then -dsends the JSON data with parameters.
-i
-i The parameter prints out the HTTP header of the server response.
$ curl -i https://www.example.com
After the above command receives the server's response, it first outputs the header of the server's response, then a blank line, and then outputs the source code of the webpage.
-I
-I The parameter sends a HEAD request to the server, and then the HTTP header returned by the server is printed out.
$ curl -I https://www.example.com
The above command outputs the server's response to the HEAD request.
--headThe parameters are equivalent to -I.
$ curl --head https://www.example.com
-k
-k The parameter specifies to skip SSL detection.
$ curl -k https://www.example.com
The above command will not check whether the server's SSL certificate is correct.
-L
-L The parameter causes the HTTP request to follow the redirection of the server. curl does not follow redirects by default.
$ curl -L -d 'tweet=hi' https://api.twitter.com/tweet
--limit-rate
--limit-rate Used to limit the bandwidth of HTTP requests and responses to simulate a slow Internet speed environment.
$ curl --limit-rate 200k https://google.com
The above command limits the bandwidth to 200K bytes per second.
-O
-o The parameter saves the server's response as a file, which is equivalent to a wgetcommand.
$ curl -o example.html https://www.example.com
The above command will be www.example.com saved as example.html.
-O
-O The parameter saves the server response as a file, and uses the last part of the URL as the file name.
$ curl -O https://www.example.com/foo/bar.html
The above command saves the server response as a file with the file name bar.html.
-s
-s Parameters will not output error and progress information.
$ curl -s https://www.example.com
Once an error occurs in the above command, no error message will be displayed. If no error occurs, the operation result will be displayed normally.
If you want curl to not produce any output, you can use the following command.
$ curl -s -o /dev/null https://google.com
-S
-S The parameter specifies that only error information is output, and is usually -sused with it.
$ curl -s -o /dev/null https://google.com
There is no output from the above command unless an error occurs.
-u
-u The parameters are used to set the user name and password for server authentication.
$ curl -u 'bob:12345' https://google.com/login
The above command sets the user name boband password to 12345, and then converts it to an HTTP header Authorization: Basic Ym9iOjEyMzQ1.
curl can recognize the username and password in the URL.
$ curl https://bob:[email protected]/login
The above command can recognize the username and password in the URL and convert them to the HTTP header in the previous example.
$ curl -u 'bob' https://google.com/login
The above command only sets the user name, after execution, curl will prompt the user to enter the password.
-v
-v The whole process of parameter output communication is used for debugging.
$ curl -v https://www.example.com
--trace Parameters can also be used for debugging, and raw binary data will be output.
$ curl --trace - https://www.example.com
-x
-x The parameter specifies the proxy for the HTTP request.
$ curl -x socks5://james:[email protected]:8080 https://www.example.com
The above command specifies myproxy.com:8080 the socks5 proxy through which the HTTP request is sent.
If the proxy protocol is not specified, the default is HTTP.
$ curl -x james:[email protected]:8080 https://www.example.com
In the above command, the requested proxy uses the HTTP protocol.
-X
-X The parameter specifies the method of the HTTP request.
$ curl -X POST https://www.example.com
The above command pair https://www.example.com issues a POST request.
=================================================================================================
curl website development guide
Ruan Yifeng finishing
Curl is a command line tool, which is used to make network requests, then get and extract data, and display it on the "standard output" (stdout).
It supports multiple protocols, the following examples explain how to use it for website development.
1. View the source code of the webpage
Add the URL directly after the curl command, and you can see the source code of the web page. Let's take the website www.sina.com as an example (choose this website mainly because its webpage code is short):
$ curl www.sina.com
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://www.sina.com.cn/">here</a>.</p>
</body></html>
If you want to save this web page, you can use the `-o` parameter, which is equivalent to using the wget command.
$ curl -o [file name] www.sina.com
Two, automatic jump
Some URLs are redirected automatically. Using the `-L` parameter, curl will jump to the new URL.
$ curl -L www.sina.com
Type the above command, the result will automatically jump to www.sina.com.cn.
Three, display header information
The `-i` parameter can display the header information of the http response, together with the web page code.
$ curl -i www.sina.com
HTTP/1.0 301 Moved Permanently
Date: Sat, 03 Sep 2011 23:44:10 GMT
Server: Apache/2.0.54 (Unix)
Location: http://www.sina.com.cn/
Cache-Control: max-age=3600
Expires: Sun, 04 Sep 2011 00:44:10 GMT
Vary: Accept-Encoding
Content-Length: 231
Content-Type: text/html; charset=iso-8859-1
Age: 3239
X-Cache: HIT from sh201-9.sina.com.cn
Connection: close<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://www.sina.com.cn/">here</a>.</p>
</body></html>
The `-I` parameter is to display only the header information of the http response.
Four, display the communication process
The `-v` parameter can display the entire process of an http communication, including port connection and http request header information.
$ curl -v www.sina.com
* About to connect() to www.sina.com port 80 (#0)
* Trying 61.172.201.195... connected
* Connected to www.sina.com (61.172.201.195) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.21.3 (i686-pc-linux-gnu) libcurl/7.21.3 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
> Host: www.sina.com
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 301 Moved Permanently
< Date: Sun, 04 Sep 2011 00:42:39 GMT
< Server: Apache/2.0.54 (Unix)
< Location: http://www.sina.com.cn/
< Cache-Control: max-age=3600
< Expires: Sun, 04 Sep 2011 01:42:39 GMT
< Vary: Accept-Encoding
< Content-Length: 231
< Content-Type: text/html; charset=iso-8859-1
< X-Cache: MISS from sh201-19.sina.com.cn
< Connection: close
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://www.sina.com.cn/">here</a>.</p>
</body></html>
* Closing connection #0
If you feel that the above information is not enough, then the following command can view a more detailed communication process.
$ curl --trace output.txt www.sina.com
or
$ curl --trace-ascii output.txt www.sina.com
After running, please open the output.txt file to view.
Five, send form information
There are two methods for sending form information, GET and POST. The GET method is relatively simple, as long as the data is appended to the URL.
$ curl example.com/form.cgi?data=xxx
The POST method must separate the data from the URL, and curl will use the --data parameter.
$ curl -X POST --data "data=xxx" example.com/form.cgi
If your data has not been form-encoded, you can let curl encode it for you. The parameter is `--data-urlencode`.
$ curl -X POST--data-urlencode "date=April 1" example.com/form.cgi
Six, HTTP verbs
The default HTTP verb for curl is GET, and other verbs can be supported by using the `-X` parameter.
$ curl -X POST www.example.com
$ curl -X DELETE www.example.com
Seven, file upload
Suppose the file upload form is as follows:
<form method="POST" enctype='multipart/form-data' action="upload.cgi">
<input type=file name=upload>
<input type=submit name=press value="OK">
</form>
You can upload files with curl like this:
$ curl --form upload=@localfilename --form press=OK [URL]
Eight, Referer field
Sometimes you need to provide a referer field in the http request header to indicate where you are jumping from.
$ curl --referer http://www.example.com http://www.example.com
Nine, User Agent field
This field is used to indicate the device information of the client. The server sometimes returns web pages in different formats for different devices based on this field, such as the mobile version and the desktop version.
The User Agent of iPhone4 is
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7
curl can be simulated like this:
$ curl --user-agent "[User Agent]" [URL]
十、cookie
Use the `--cookie` parameter to make curl send cookies.
$ curl --cookie "name=xxx" www.example.com
As for the specific cookie value, it can be obtained from the `Set-Cookie` field of the http response header.
`-c cookie-file` can save the cookie returned by the server to a file, `-b cookie-file` can use this file as cookie information for subsequent requests.
$ curl -c cookies http://example.com
$ curl -b cookies http://example.com
11. Add header information
Sometimes it is necessary to add a header information to the http request. The `--header` parameter can do this.
$ curl --header "Content-Type:application/json" http://example.com
12. HTTP authentication
Some domains require HTTP authentication, then curl needs to use the `--user` parameter.
$ curl --user name:password example.com