Concurrent use transfer_pool batch download

TBOX provide highly encapsulated upper transport pool for the client concurrent batch download, upload, file copy, socket pass-through data transmission, etc. io demand.

Post to transfer_pool transport pool all tasks are asynchronous concurrent processing. If you do not ask, then concurrency, such as a client application, then use the global tb_transfer_pool () is sufficient.

Global transfer_pool use the global tb_aicp (), and considering that most client applications, concurrency is not very high, so the demand for resources made some restrictions, generally more suitable for the occasion concurrent less than 100.

If circumstances require high concurrency, it can tb_transfer_pool_init, tb_aicp_init manually create: concurrency transfer_pool and aicp, designated aicp need to improve concurrency requirements.

If you want to transfer each io more flexible control can be used directly async_transfer, do your own top management on the line.

// 传输信息回调函数
static tb_bool_t tb_demo_transfer_done_func(tb_size_t state, tb_hize_t offset, tb_hong_t size, tb_hize_t save, tb_size_t rate, tb_cpointer_t priv)
{
    // 计算进度
    tb_size_t percent = 0;
    if (size > 0) percent = (tb_size_t)((offset * 100) / size); else if (state == TB_STATE_OK) percent = 100; // 打印信息 tb_trace_i("save[%s]: %llu, rate: %lu bytes/s, percent: %lu%%, state: %s", priv, save, rate, percent, tb_state_cstr(state)); // 继续下载 return tb_true; } // 投递几个http下载任务 tb_transfer_pool_done(tb_transfer_pool(), "http://www.xxxx.com/file1", "/tmp/a", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); tb_transfer_pool_done(tb_transfer_pool(), "http://www.xxxx.com/file2", "/tmp/b", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); // 投递几个https下载任务 tb_transfer_pool_done(tb_transfer_pool(), "https://www.xxxx.com/file3", "/tmp/c", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); tb_transfer_pool_done(tb_transfer_pool(), "https://www.xxxx.com/file4", "/tmp/d", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); // 投递几个文件copy任务 tb_transfer_pool_done(tb_transfer_pool(), "/tmp/file1", "/tmp/e", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); tb_transfer_pool_done(tb_transfer_pool(), "/tmp/file2", "/tmp/f", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); // 投递几个文件copy任务 tb_transfer_pool_done(tb_transfer_pool(), "/tmp/file1", "/tmp/g", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); // 在windows下投递文件copy任务 tb_transfer_pool_done(tb_transfer_pool(), "c://xxxx/file2", "c://tmp/h", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); // 投递tcp socket传输任务 tb_transfer_pool_done(tb_transfer_pool(), "sock://127.0.0.1", "/tmp/i", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); // 投递udp socket传输数据到其他对方tcp socket服务器上 tb_transfer_pool_done(tb_transfer_pool(), "sock://127.0.0.1/?udp=", "sock://xxx.xxx.xxx.xxx", 0, 0, tb_demo_transfer_done_func, tb_null, tb_null)); /*! 投递异步io传输任务到传输池:iurl => ourl * * @param pool 传输池,默认可以使用全局的tb_transfer_pool(),也可以自己创建私有的pool * @param iurl 输入url * @param ourl 输出url * @param offset 源端开始传输的数据offset,一般只要传0就行了 * @param rate 指定限速值,传0为默认无限速 * @param done 传输信息回调函数,间隔1s调用一次 * @param ctrl 控制参数设置回调函数,在开始传输前,对io的传输做一些参数控制 * @param priv 用户传入的私有数据 * * @return 投递成功:tb_true,投递失败:tb_false */ tb_bool_t tb_transfer_pool_done(tb_transfer_pool_ref_t pool, tb_char_t const* iurl, tb_char_t const* ourl, tb_hize_t offset, tb_size_t rate, tb_async_transfer_done_func_t done, tb_async_transfer_ctrl_func_t ctrl, tb_cpointer_t priv);

Wherein the transmission parameter control using the callback function, reference may be provided TBOX spider demo program.

Reproduced in: https: //www.cnblogs.com/huobumingbai/p/3963908.html

Guess you like

Origin blog.csdn.net/weixin_34301132/article/details/93727666