Article directory
01. Introduction to libcurl
Regarding the introduction of libcurl, I wrote a blog. If you are not sure, you can turn to it and read: 2019 Configure Http protocol, libcurl third-party library for POST communication
02. libcurl test
After completing the method in the above blog, we can write a small project to test the effect. Let's take Baidu as an example to test the content of this website : https://www.baidu.com/ .
Citing an example step from the official website as the original template
/***************************************************************************
* _ _ ____ _
* Project ___| | | | _ \| |
* / __| | | | |_) | |
* | (__| |_| | _ <| |___
* \___|\___/|_| \_\_____|
*
* Copyright (C) 1998 - 2021, Daniel Stenberg, <[email protected]>, et al.
*
* This software is licensed as described in the file COPYING, which
* you should have received as part of this distribution. The terms
* are also available at https://curl.se/docs/copyright.html.
*
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
* copies of the Software, and permit persons to whom the Software is
* furnished to do so, under the terms of the COPYING file.
*
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
* KIND, either express or implied.
*
***************************************************************************/
/* <DESC>
* Shows how the write callback function can be used to download data into a
* chunk of memory instead of storing it in a file.
* </DESC>
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <curl/curl.h>
//定义一个结构体，记录每次回调的内容与数据长度
typedef struct MemoryStruct {
char *memory;
size_t size;
}MemoryStruct;
//定义一个全局静态函数，响应服务器的数据
/* 参数说明
* void* contents: 此参数为服务器响应的源数据，即服务器发过来的数据（原封不变的数据）
* size_t size: 此参数为发送的多少个块儿（类似fwrite函数中的第二参数）
* size_t nmemb: 此参数为发送的块中的数量大小（类似fwrite函数中的第三参数）
* void* userp: 此参数为与外部交互的指针通常为（FILE*），以实际为参考，看下面用法
*/
static size_t WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp)
{
size_t realsize = size * nmemb; //此次回调的数据总大小
struct MemoryStruct *mem = (struct MemoryStruct *)userp; //外部申请空间、内部追加数据方式
//为结构体成员memory动态申请空间
char *ptr = realloc(mem->memory, mem->size + realsize + 1);
if(!ptr) {
/* out of memory! */
printf("not enough memory (realloc returned NULL)\n");
return 0;
}
mem->memory = ptr;
memcpy(&(mem->memory[mem->size]), contents, realsize);
mem->size += realsize;
mem->memory[mem->size] = 0;
return realsize; //只要realsize不为0，回调就要一直被调用
}
int main(void)
{
CURL *curl_handle;
CURLcode res;
struct MemoryStruct chunk;
struct curl_slist* headers = NULL; //http报文头
chunk.memory = malloc(1); /* will be grown as needed by the realloc above */
chunk.size = 0; /* no data at this point */
curl_global_init(CURL_GLOBAL_ALL);
/* init the curl session */
curl_handle = curl_easy_init();
/* specify URL to get */
curl_easy_setopt(curl_handle, CURLOPT_URL, "http://www.baidu.com/");
/* send all data to this function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
/* we pass our 'chunk' struct to the callback function 此参数为回调的第4参数*/
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)&chunk);
/* some servers don't like requests that are made without a user-agent
field, so we provide one */
curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, "libcurl-agent/1.0");
/* get it! */
res = curl_easy_perform(curl_handle);
/* check for errors */
if(res != CURLE_OK) {
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
}
else {
/*
* Now, our chunk.memory points to a memory block that is chunk.size
* bytes big and contains the remote file.
*
* Do something nice with it!
*/
char* buf = chunk.memory;
FILE* fptr = fopen(strFilePath,"wb");
if(!fptr)/*strFilePath为文件路径（完整且加名字的路径）*/
{
printf("fopen is failed......\n");
return 0;
}
size_t bRet = fwrite(buf,1,Chunk.size,fptr);
if(bRet >0)
{
printf("write Content to file is success!");
return 0;
}
fclose(fptr);
printf("%lu bytes retrieved\n", (unsigned long)chunk.size);
}
/* cleanup curl stuff */
curl_easy_cleanup(curl_handle);
free(chunk.memory);
chunk.memory = NULL;
/* we're done with libcurl, so clean it up */
curl_global_cleanup();
return 0;
}
03. Description of fwrite append mode
The C function fopen can set a variety of read and write modes for the opened file. It is necessary to explain what the append mode of fwrite is. Many articles on the Internet are not clear. They only say that a is for appending, and w cannot be appended. In fact, Lots of misunderstandings, need clarification here (pass case test ok):
1. w : Indicates that when fopening a file, the information of the original file (if it exists) will be cleared and rewritten. If the file is not fclosed, multiple fwrites will also be appended to the end of the file and will not overwrite the previous fwrite Content.
2. a : When fopening a file, the information of the original file (if it exists) will be retained, and will be appended to the end for writing, each time fwrite is written to the end of the file.
Other instructions:
w: Text writing, write only
w+: Readable
and writable wb: Binary writing, writing only
wb+: Binary writing or reading, readable and writable
a: Additional writing to fopen files, in text form
ab: Additional write to fopen file, binary format
ab+: Additional write or read to fopen file, binary format
04. Summary
Regarding libcurl, I wrote an introduction to the basics before, but in the actual application later, there were many bugs in the process of use, which caused the program to close abnormally. I always thought that there was a problem in this communication, and I was confused for 2~3 weeks. I didn’t get it done, and finally I had no choice. I downloaded the source code and found that the callback function must be a static function outside the class, otherwise it will cause the callback address to be different every time, leading to an illusion that the fourth one in the callback function is passed The parameter address is changing, resulting in the content is always wrong, I would like to record my experience using libcurl in this article.