How to configure openai to return data, maintain streaming output, and load in segments
You can use the stream parameter of the OpenAI API to achieve streaming output, and you can use the max_tokens parameter to control the length of each returned data. The following is a sample code to demonstrate how to use the OpenAI API to achieve streaming output and load in parts:
python
import openai
# 连接 OpenAI API openai.api_key = "YOUR_API_KEY"
# 设置 OpenAI API 参数
model_engine = "davinci"
# 模型引擎
prompt = "你好，我需要帮助进行自然语言处理任务。" # 请求语句
temperature = 0.5 # 温度参数
max_tokens = 100 # 每次最大输出长度 # 获取 OpenAI 返回数据
completions = openai.Completion.create( engine=model_engine, prompt=prompt, temperature=temperature, max_tokens=max_tokens, n=1, stream=True ) # 分段获取数据
for i, text in enumerate(iter(completions.choices), start=1):
print(f"============== 第 {
i} 段输出 ==============")
print(text.text)
In this example, we streamset the parameter to True, so that the OpenAI API will transmit the data back to the caller in real time, and will return a generator object ( iter(completions.choices)) that allows us to fetch the data piece by piece and print it out. At the same time, we use max_tokensparameters to control the length of the data returned each time, which can also effectively avoid the problem of insufficient memory caused by too long returned data. Note that here YOUR_API_KEYneeds to be replaced with the API key you applied for at OpenAI. [Request time: 14 seconds]
openai returns the stream, whether it is UDP or TCP, and receives the stream data code DEMO
The OpenAI GPT-3 API returns an HTTPS stream using the TCP protocol.
If you need to process the data stream returned from the OpenAI GPT-3 API, you can use the requests library in Python to process it.
The following is a sample code that can receive and process the data stream returned by the OpenAI GPT-3 API:
python
import requests
def process_stream(stream):
for line in stream.iter_lines():
if line: # 处理返回的数据
print(line.decode('utf-8')) # 发起 API 请求并处理返回的数据流
response = requests.post(url, headers=headers, data=data, stream=True)
process_stream(response.iter_lines())
In the code above, we use requests.postthe method to make a stream=TruePOST request with the parameter and save the response to responsethe variable . Next, we call process_streamthe function to process the stream of data returned from the API. In process_streamthe function , we use iter_linesthe iterator to process the data stream line by line and convert each line to a UTF-8 encoded string. In this simple example, we just print the data to the console, but you can write corresponding logic to process the data according to actual needs.
The PHP code example for receiving stream data is as follows:
php
<?php $ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'STREAM_URL');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_WRITEFUNCTION, function($curl, $data)
{
// 处理获得的数据
echo $data;
return strlen($data); });
curl_exec($ch);
curl_close($ch); ?>
Among them, STREAM_URLrepresents the stream URL address of OpenAI. This code will incrementally write data to the screen.
Correspondingly, the Java code example for receiving stream data is as follows:
java
import java.net.*;
import java.io.*;
public class StreamReceiver {
public static void main(String[] args) throws Exception
{
URL streamURL = new URL("STREAM_URL");
try (
BufferedReader reader = new BufferedReader( new InputStreamReader(streamURL.openStream())))
{
String line;
while ((line = reader.readLine()) != null)
{
// 处理获得的数据 System.out.println(line);
}
}
}
}
Similarly, STREAM_URLit represents the stream URL address of OpenAI. This code will output the data step by step to the command line interface
PHP code example:
<?php // 请求OpenAI接口获取stream数据，并转发到H5页面端
$openAIUrl = 'https://api.openai.com/v1/engines/davinci-codex/completions'; $headers = array( 'Content-Type: application/json', 'Authorization: Bearer <YOUR_API_KEY>' );
$data = array( 'prompt' => 'Hello, world!', 'max_tokens' => 5, 'temperature' => 0.7 );
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $openAIUrl);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$output = fopen('php://output', 'w');
curl_setopt($ch, CURLOPT_WRITEFUNCTION, function ($ch, $chunk) use ($output) {
fwrite($output, $chunk);
return strlen($chunk); });
curl_exec($ch);
fclose($output); // 将stream数据转发给H5页面端
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
header('Connection: keep-alive');
header('Access-Control-Allow-Origin: *');
fpassthru($output);
Java code example:
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import javax.servlet.AsyncContext;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
@WebServlet(asyncSupported = true, urlPatterns = {
"/stream"})
public class StreamServlet extends HttpServlet
{
private static final ExecutorService executor = Executors.newCachedThreadPool();
@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// 创建异步上下文
AsyncContext asyncContext = request.startAsync(); // 提交任务到线程池
executor.submit(() -> {
try {
// 请求OpenAI接口获取stream数据
URL url = new URL("https://api.openai.com/v1/engines/davinci-codex/completions");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestProperty("Authorization", "Bearer <YOUR_API_KEY>"); conn.setDoOutput(true);
OutputStream out = conn.getOutputStream();
out.write("{\"prompt\": \"Hello, world!\", \"max_tokens\": 5, \"temperature\": 0.7}".getBytes());
out.flush();
out.close();
InputStream in = conn.getInputStream();
response.setContentType("text/event-stream");
response.setHeader("Cache-Control", "no-cache");
response.setHeader("Connection", "keep-alive");
response.setHeader("Access-Control-Allow-Origin", "*");
byte[] buffer = new byte[1024];
int count;
while ((count = in.read(buffer)) != -1) {
// 将stream数据转发给H5页面端
OutputStream outputStream = asyncContext.getResponse().getOutputStream();
String data = new String(buffer, 0, count);
outputStream.write(("data: " + data + "\n\n").getBytes());
outputStream.flush(); } in.close(); }
catch (IOException e) {
e.printStackTrace(); }
finally {
asyncContext.complete(); } }); } }
Python code example:
import requests from flask
import Flask, Response app = Flask(__name__) @app.route('/stream')
def stream(): # 请求OpenAI接口获取stream数据
url = 'https://api.openai.com/v1/engines/davinci-codex/completions'
headers = {
'Content-Type': 'application/json', 'Authorization': 'Bearer <YOUR_API_KEY>' }
data = {
'prompt': 'Hello, world!', 'max_tokens': 5, 'temperature': 0.7 } res = requests.post(url, headers=headers, json=data, stream=True)
# 将stream数据转发给H5页面端
def generate():
for chunk in res.iter_content(chunk_size=1024):
if chunk: yield 'data: ' + chunk.decode() + '\n\n'
return Response(generate(), mimetype='text/event-stream', headers={
'Cache-Control': 'no-cache', 'Connection': 'keep-alive', 'Access-Control-Allow-Origin': '*' })
if __name__ == '__main__':
app.run()
Note: The above code is for reference only, and the specific implementation needs to be modified appropriately according to your own business needs.
How to enable front-end Html to display mardown format
To display Markdown formatting in front-end HTML, you can use a library or tool called a Markdown parser. Here is a simple example:
1. First, import a Markdown parser library, such as marked.js, through CDN or download it to a local folder.
html <script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
- Add a textarea tag on the page to enter text in Markdown format, and add a div tag to display the parsed HTML content.
javascripthtml <textarea id="mdInput"></textarea> <div id="htmlOutput"></div>
- Get the value of the textarea in JavaScript, pass it to the Markdown parser, and insert the output HTML code into the div.
javascript
var markdown = document.getElementById('mdInput').value;
var html = marked(markdown);
document.getElementById('htmlOutput').innerHTML = html;
In this way, the text in Markdown format can be displayed in the browser.
Similar to the ChatGPT website, the stream (Stream) processing technology in Node.js is used to convert the output into markdown format.
Streaming is a technique that is useful when working with large amounts of data, allowing data to be processed in chunks without waiting for the entire dataset to be loaded. In the ChatGPT website, a method similar to the following code may be used to convert the output stream to markdown format:
javascript
const markdown = require('markdown').markdown;
const stream = require('stream');
const outputStream = new stream.Writable({
write(chunk, encoding, callback) {
const text = chunk.toString();
const markdownText = markdown.toHTML(text);
process.stdout.write(markdownText);
callback(); },
});
// Stream output to markdown formatter
originalOutputStream.pipe(outputStream);
In this code example, markdownthe library is used to convert input plain text to Markdown format. A Writablewritable stream for receiving output, converting it to Markdown format and sending it to the standard output stream ( process.stdout.write()). Finally originalOutputStreamconnect the original output stream ( ) to the new writable stream ( outputStream), and pipe output to standard output through that stream. This is just a simple example, in reality, the ChatGPT website may use more complex techniques to stream and convert the output to Markdown format