Sep 14, 2023 05:42 PM
Verbal nonsense reveals limitations of AI chatbots
https://www.eurekalert.org/news-releases/1001380
INTRO: The era of artificial-intelligence chatbots that seem to understand and use language the way we humans do has begun. Under the hood, these chatbots use large language models, a particular kind of neural network. But a new study shows that large language models remain vulnerable to mistaking nonsense for natural language. To a team of researchers at Columbia University, it’s a flaw that might point toward ways to improve chatbot performance and help reveal how humans process language.
In a paper published online today in Nature Machine Intelligence, the scientists describe how they challenged nine different language models with hundreds of pairs of sentences. For each pair, people who participated in the study picked which of the two sentences they thought was more natural, meaning that it was more likely to be read or heard in everyday life. The researchers then tested the models to see if they would rate each sentence pair the same way the humans had.
In head-to-head tests, more sophisticated AIs based on what researchers refer to as transformer neural networks tended to perform better than simpler recurrent neural network models and statistical models that just tally the frequency of word pairs found on the internet or in online databases. But all the models made mistakes, sometimes choosing sentences that sound like nonsense to a human ear.
“That some of the large language models perform as well as they do suggests that they capture something important that the simpler models are missing,” said Dr. Nikolaus Kriegeskorte, PhD, a principal investigator at Columbia’s Zuckerman Institute and a coauthor on the paper. “That even the best models we studied still can be fooled by nonsense sentences shows that their computations are missing something about the way humans process language.” (MORE - details, no ads)
How do robots collaborate to achieve consensus?
https://www.eurekalert.org/news-releases/1001528
INTRO: Making group decisions is no easy task, especially when the decision makers are a swarm of robots. To increase swarm autonomy in collective perception, a research team at the IRIDIA artificial intelligence research laboratory at the Université Libre de Bruxelles proposed an innovative self-organizing approach in which one robot at a time works temporarily as the “brain” to consolidate information on behalf of the group. Their paper was published Sept. 13 in Intelligent Computing, a Science Partner Journal. In the paper, the authors showed that their method improves collective perception accuracy by reducing sources of uncertainty... (MORE - details, no ads)
Artificial Intelligence: AI may outperform most humans at creative thinking task
https://www.eurekalert.org/news-releases/1001219
INTRO: Large language model (LLM) AI chatbots may be able to outperform the average human at a creative thinking task where the participant devises alternative uses for everyday objects (an example of divergent thinking), suggests a study published in Scientific Reports. However, the human participants with the highest scores still outperformed the best chatbot responses... (MORE - details, no ads)
https://www.eurekalert.org/news-releases/1001380
INTRO: The era of artificial-intelligence chatbots that seem to understand and use language the way we humans do has begun. Under the hood, these chatbots use large language models, a particular kind of neural network. But a new study shows that large language models remain vulnerable to mistaking nonsense for natural language. To a team of researchers at Columbia University, it’s a flaw that might point toward ways to improve chatbot performance and help reveal how humans process language.
In a paper published online today in Nature Machine Intelligence, the scientists describe how they challenged nine different language models with hundreds of pairs of sentences. For each pair, people who participated in the study picked which of the two sentences they thought was more natural, meaning that it was more likely to be read or heard in everyday life. The researchers then tested the models to see if they would rate each sentence pair the same way the humans had.
In head-to-head tests, more sophisticated AIs based on what researchers refer to as transformer neural networks tended to perform better than simpler recurrent neural network models and statistical models that just tally the frequency of word pairs found on the internet or in online databases. But all the models made mistakes, sometimes choosing sentences that sound like nonsense to a human ear.
“That some of the large language models perform as well as they do suggests that they capture something important that the simpler models are missing,” said Dr. Nikolaus Kriegeskorte, PhD, a principal investigator at Columbia’s Zuckerman Institute and a coauthor on the paper. “That even the best models we studied still can be fooled by nonsense sentences shows that their computations are missing something about the way humans process language.” (MORE - details, no ads)
How do robots collaborate to achieve consensus?
https://www.eurekalert.org/news-releases/1001528
INTRO: Making group decisions is no easy task, especially when the decision makers are a swarm of robots. To increase swarm autonomy in collective perception, a research team at the IRIDIA artificial intelligence research laboratory at the Université Libre de Bruxelles proposed an innovative self-organizing approach in which one robot at a time works temporarily as the “brain” to consolidate information on behalf of the group. Their paper was published Sept. 13 in Intelligent Computing, a Science Partner Journal. In the paper, the authors showed that their method improves collective perception accuracy by reducing sources of uncertainty... (MORE - details, no ads)
Artificial Intelligence: AI may outperform most humans at creative thinking task
https://www.eurekalert.org/news-releases/1001219
INTRO: Large language model (LLM) AI chatbots may be able to outperform the average human at a creative thinking task where the participant devises alternative uses for everyday objects (an example of divergent thinking), suggests a study published in Scientific Reports. However, the human participants with the highest scores still outperformed the best chatbot responses... (MORE - details, no ads)
