Mar 3, 2024 08:13 AM
https://www.sciencealert.com/something-s...-star-trek
EXCERPTS: The art of speaking to AI chatbots is continuing to frustrate and baffle people.
A study attempting to fine-tune prompts fed into a chatbot model found that, in one instance, asking it to speak as if it were on Star Trek dramatically improved its ability to solve grade-school-level math problems.
"It's both surprising and irritating that trivial modifications to the prompt can exhibit such dramatic swings in performance," the study authors Rick Battle and Teja Gollapudi at software firm VMware in California said in their paper.
The study, first reported by New Scientist, was published on February 9 on arXiv, a server where scientists can share preliminary findings before they have been validated by careful scrutiny from peers.
[...] "Intuition tells us that, in the context of language model systems, like any other computer system, 'positive thinking' should not affect performance, but empirical experience has demonstrated otherwise," they said. This would suggest it's not only what you ask the AI model to do, but how you ask it to act while doing it that influences the quality of the output... (MORE - missing details)
EXCERPTS: The art of speaking to AI chatbots is continuing to frustrate and baffle people.
A study attempting to fine-tune prompts fed into a chatbot model found that, in one instance, asking it to speak as if it were on Star Trek dramatically improved its ability to solve grade-school-level math problems.
"It's both surprising and irritating that trivial modifications to the prompt can exhibit such dramatic swings in performance," the study authors Rick Battle and Teja Gollapudi at software firm VMware in California said in their paper.
The study, first reported by New Scientist, was published on February 9 on arXiv, a server where scientists can share preliminary findings before they have been validated by careful scrutiny from peers.
[...] "Intuition tells us that, in the context of language model systems, like any other computer system, 'positive thinking' should not affect performance, but empirical experience has demonstrated otherwise," they said. This would suggest it's not only what you ask the AI model to do, but how you ask it to act while doing it that influences the quality of the output... (MORE - missing details)