Some software developers Now let artificial intelligence Help write their code. They found that artificial intelligence is as flawed as humans.
Last June, GitHub, Subsidiary Microsoft Provide tools for hosting and collaborative code, Released A beta version of a program that uses AI to help programmers.Start typing commands, database queries or requests to API, and then call the program Co-pilot, Will guess your intentions and write the rest.
Alex NakaA data scientist at a biotechnology company, he signed up for the Copilot test. He said that this program was very helpful and it changed the way he works. “It allows me to spend less time jumping to the browser to find API documentation or examples on Stack Overflow,” he said. “It feels a bit like my job has changed from a code generator to a code discriminator.”
But Naka discovered that errors would penetrate his code in different ways. “Sometimes I miss a subtle error when I accept one of the suggestions,” he said. “And it’s difficult to track this down, perhaps because the mistakes it made seem to be different from the mistakes I made.”
The risk of AI generating incorrect codes can be staggeringly high.New York University researchers recently Analysis code generated by Copilot And found that for some security-critical tasks, the code contains security vulnerabilities about 40% of the time.
This number is “a little higher than I expected,” said Brendan Dolan-Gavitt, Professor of New York University who participated in the analysis. “But the training method Copilot received is not actually written code-it just generates that kind of text based on the given prompts.”
Despite these flaws, Copilot and similar artificial intelligence tools may herald a radical change in the way software developers write code. People are increasingly interested in using artificial intelligence to help automate more mundane tasks. But Copilot also highlighted some shortcomings of today’s artificial intelligence technology.
When analyzing the code available for the Copilot plug-in, Dolan-Gavitt discover It includes a list of restricted phrases. These are obviously to prevent the system from blurting off offensive information or copying well-known code written by other people.
Erge de MooreGitHub research vice president and one of the Copilot developers said that security has been an issue from the beginning. He said that the percentage of defective code cited by New York University researchers is only related to the subset of code that is more likely to have security flaws.
De Moore invented Code QL, A tool used by researchers at New York University to automatically identify errors in code. He said that GitHub recommends that developers use Copilot with CodeQL to ensure their work safety.
The GitHub program is built on the AI model developed by Open artificial intelligence, A well-known artificial intelligence company engaged in cutting-edge work Machine learningThe model is called Codex, and is composed of a large human Neural Networks Train to predict the next character in text and computer code. The algorithm extracts billions of lines of code stored on GitHub—not all perfect—to learn how to write code.
OpenAI has built its own AI coding tool on top of Codex, which can Perform some amazing coding skillsIt can convert typed instructions (such as “create an array of random variables between 1 and 100, and then return the largest one”) into working codes in multiple programming languages.
Another version of the same OpenAI program, called GPT-3, can Generate coherent text on a given topic, But it can also ruminate Offensive or biased language Learned from the dark corners of the Internet.
Copilot and Codex have Cause some developers to want to know If the AI may automatically stop working. In fact, as Naka’s experience has shown, developers need considerable skills to use the program because they often have to review or adjust their suggestions.