Produced | OSC Open Source Community (ID: oschina2013)
ChatGPT is a large language model chat robot released by OpenAI, which can generate text based on user input, including code. However, researchers at the University of Quebec in Canada found that the code generated by ChatGPT often has serious security problems, and it does not actively inform users of these problems. It will only be answered if the user asks if the generated code is safe.
The researchers published the paper "How Secure is Code Generated by ChatGPT?" on arXiv, analyzing the security of the code generated by ChatGPT. They said the results were concerning because some of the code generated by ChatGPT did not meet the minimum security mark. And even if ChatGPT knows that the code it generates is not safe, it will not actively inform the user unless the user asks it.
From the description of the paper, the researchers let ChatGPT generate 21 programs and scripts using four programming languages: C, C++, Python and Java. These programming tasks were carefully selected so that each exhibited a specific security vulnerability, such as memory corruption, denial of service, and flaws related to deserialization and improperly implemented encryption.
It turned out that only 5 of the 21 programs ChatGPT generated on the first try were safe. After further prompting to correct its wrong steps, the large language model managed to generate 7 more secure applications, but this is only "safe" in relation to the specific vulnerability being evaluated, not to say that the final code does not have any other risks. Vulnerabilities exploited.
Part of ChatGPT's problem, the researchers note, is that it doesn't take into account the attacker's code execution model. It will repeatedly tell users that security issues can be avoided by "not entering invalid data", but this is not feasible in the real world. However, it appears to be aware of and acknowledge critical vulnerabilities in the code it suggests.
"Obviously, it's just an algorithm. It doesn't know anything, but it can identify unsafe behavior," said Raphaël Khoury, one of the paper's co-authors. He mentioned that the initial ChatGPT response to security concerns suggested only With valid input, this is clearly unreasonable. It provided useful guidance only when later asked to refine the question.
But since prompting ChatGPT to fix a problem requires familiarity with specific vulnerabilities and coding techniques, when we know the correct prompt for ChatGPT to fix a bug, we may already know how to solve the problem.
The researchers also pointed to the ethical inconsistency of ChatGPT's code-generating behavior. For example, it will refuse to generate offensive code, but it will generate vulnerable code. They cited an example of a Java deserialization vulnerability, "The chatbot generated vulnerable code and offered advice on how to make it more secure, but said it couldn't generate a more secure version of the code."
------
We have created a high-quality technical exchange group. When you are with excellent people, you will become excellent yourself. Hurry up and click to join the group and enjoy the joy of growing together. In addition, if you want to change jobs recently, I spent 2 weeks a year ago collecting a wave of face-to-face experience from big factories. If you plan to change jobs after the festival, you can click here to claim it !
recommended reading
··································
Hello, I am DD, a programmer. I have been developing a veteran driver for 10 years, MVP of Alibaba Cloud, TVP of Tencent Cloud. From general development to architect to partner. Along the way, my deepest feeling is that we must keep learning and pay attention to the frontier. As long as you can persevere, think more, complain less, and work hard, it will be easy to overtake on curves! So don't ask me if it's too late to do what I do now. If you are optimistic about something, you must persevere to see hope, not to persevere only when you see hope. Believe me, as long as you stick to it, you will be better than now! If you have no direction yet, you can follow me first, and I will often share some cutting-edge information here to help you accumulate capital for cornering and overtaking.