Today 01:29 AM
(This post was last modified: Today 01:29 AM by C C.)
https://www.eurekalert.org/news-releases/1130075
INTRO: Artificial intelligence chatbots need to work on their social judgment, recent events suggest. At one end of the spectrum, they’re facing lawsuits for recommending dangerous actions. At the other end, the models can be so nice they’re considered sycophantic.
The problem could get worse as AI bots work more with humans, such as handling customer complaints, says Yan Leng, assistant professor of information, risk, and operations management at the McCombs School of Business at The University of Texas at Austin.
But help may be on the way. In new research, Leng has devised a sort of personality test — more precisely, a behavioral audit — for large language models (LLMs), the technology that drives products such as ChatGPT.
By understanding an LLM’s existing tendencies, an organization can decide whether an available model already fits its values and usage scenarios. If not, it might need to fine-tune a model before putting it to use.
Leng compares her framework to trying to understand a person through their actions and thought processes. “For a human, we would have our values, and our values would dictate how we make decisions, so we try to have that for LLMs as well,” she says... (MORE - no ads)
PAPER: http://dx.doi.org/10.1287/isre.2024.0857
INTRO: Artificial intelligence chatbots need to work on their social judgment, recent events suggest. At one end of the spectrum, they’re facing lawsuits for recommending dangerous actions. At the other end, the models can be so nice they’re considered sycophantic.
The problem could get worse as AI bots work more with humans, such as handling customer complaints, says Yan Leng, assistant professor of information, risk, and operations management at the McCombs School of Business at The University of Texas at Austin.
But help may be on the way. In new research, Leng has devised a sort of personality test — more precisely, a behavioral audit — for large language models (LLMs), the technology that drives products such as ChatGPT.
By understanding an LLM’s existing tendencies, an organization can decide whether an available model already fits its values and usage scenarios. If not, it might need to fine-tune a model before putting it to use.
Leng compares her framework to trying to understand a person through their actions and thought processes. “For a human, we would have our values, and our values would dictate how we make decisions, so we try to have that for LLMs as well,” she says... (MORE - no ads)
PAPER: http://dx.doi.org/10.1287/isre.2024.0857