“The amazing thing about these large language models is that they know how the world works just by reading everything they can find,” said Chris Manning, A professor at Stanford University, specializing in artificial intelligence and language.
But GPT and its kind are essentially very talented statistical parrots. They learn how to recreate the words and grammatical patterns in the language. This means they can blurt out nonsense, Very inaccurate facts, And Hate language Scratched from the darker corners of the network.
Amnon SashuaProfessor of Computer Science at the Hebrew University of Jerusalem, is the co-founder of another startup that builds AI models based on this method. He knew a little bit about artificial intelligence commercialization and sold his last company, Move eyes, The first to use artificial intelligence to help cars find things on the road, Intel In 2017, it was 15.3 billion U.S. dollars.
Shashua’s new company, AI21 laboratoryIt emerged from the stealth state last week and developed an artificial intelligence algorithm called Jurassic-1, which demonstrated amazing English and Hebrew language skills.
In the demonstration, Jurassic-1 can generate text paragraphs on a given topic, design attractive headlines for blog posts, write simple computer code, and so on. Shashua said that this model is more complex than GPT-3, and he believes that future versions of the Jurassic may be able to build a common-sense understanding of the world from the information it collects.
Other efforts to recreate GPT-3 reflect the linguistic diversity of the world and the Internet. In April, the researchers Huawei, Chinese technology giant, Announce details PanGu-alpha (written as PanGu-α), a Chinese language model similar to GPT. May, NevilleA Korean search giant said it has developed its own language model called HyperCLOVA, which can “speak” Korean.
Jie TangTsinghua University professor leads the team Beijing Institute of Artificial Intelligence With the help of the government and industry, another Chinese language model called Wudao (meaning “Enlightenment”) was developed.
The Wudao model is much larger than any other model, which means that its simulated neural network is distributed on more cloud computers. Increasing the size of the neural network is the key to making GPT-2 and -3 more capable. Wudao can also process images and texts, and Tang has established a company to commercialize them. “We believe this can be the cornerstone of all artificial intelligence,” Tang said.
The capabilities of these new AI programs seem to prove this enthusiasm, but the race to commercialize such language models may also be faster than efforts to increase guardrails or limit abuse.
Perhaps the most pressing concern about AI language models is that they may be abused. Because these models can generate compelling text on a topic, some people worry that they can easily be used to generate fake comments, spam or fake news.
“I would be surprised if disinformation operators at least don’t invest a lot of effort to test these models,” said Mika Mather, Georgetown University research analyst, once researched Possibility of disseminating misinformation by language models.
Musser said research has shown that it is impossible to use artificial intelligence to capture false information generated by artificial intelligence. It is unlikely that there will be enough information in a tweet for a machine to determine whether it was written by a machine.
More problematic biases may also be lurking in these huge language models.Research shows that language models trained on Chinese Internet content Will reflect the review system Shaped that content. These programs also inevitably capture and reproduce subtle and public prejudices about race, gender, and age, including hate speech and ideas, in the language they use.
Similarly, these large language models may fail in surprising or unexpected ways, adding Posi Liang, Another computer science professor and principal researcher at Stanford University A new center Committed to researching the potential of powerful general AI models (such as GPT-3).