Dec 12, 2024 07:22 PM
RELATED: AI's mysterious ‘black box’ problem, explained
- - - - - - - - - - - - -
Unlocking the ‘black box’: scientists reveal AI’s hidden thoughts
https://www.eurekalert.org/news-releases/1068008
INTRO: Deep neural networks are a type of artificial intelligence (AI) that imitate how human brains process information, but understanding how these networks “think” has long been a challenge. Now, researchers at Kyushu University have developed a new method to understand how deep neural networks interpret information and sort it into groups. Published in IEEE Transactions on Neural Networks and Learning Systems, the study addresses the important need to ensure AI systems are accurate and robust and can meet the standards required for safe use.
Deep neural networks process information in many layers, similarly to humans solving a puzzle step by step. The first layer, known as the input layer, brings in the raw data. The subsequent layers, called hidden layers, analyze the information. Early hidden layers focus on basic features, such as detecting edges or textures—like examining individual puzzle pieces. Deeper hidden layers combine these features to recognize more complex patterns, such as identifying a cat or a dog—similar to connecting puzzle pieces to reveal the bigger picture.
“However, these hidden layers are like a locked black box: we see the input and output, but what is happening inside is not clear,” says Danilo Vasconcellos Vargas, Associate Professor from the Faculty of Information Science and Electrical Engineering at Kyushu University. "This lack of transparency becomes a serious problem when AI makes mistakes, sometimes triggered by something as small as changing a single pixel. AI might seem smart, but understanding how it comes to its decision is key to ensuring it’s trustworthy.”
Currently, methods for visualizing how AI organizes information rely on simplifying high-dimensional data into 2D or 3D representations. These methods let researchers observe how AI categorizes data points—for example, grouping images of cats close to other cats while separating them from dogs. However, this simplification comes with critical limitations.
“When we simplify high-dimensional information into fewer dimensions, it’s like flattening a 3D object into 2D—we lose important details and fail to see the whole picture. Additionally, this method of visualizing how the data is grouped makes it difficult to compare between different neural networks or data classes,” explains Vargas.
In this study, the researchers developed a new method, called the k* distribution method, that more clearly visualizes and assesses how well deep neural networks categorize related items together... (MORE - details, no ads)
- - - - - - - - - - - - -
Unlocking the ‘black box’: scientists reveal AI’s hidden thoughts
https://www.eurekalert.org/news-releases/1068008
INTRO: Deep neural networks are a type of artificial intelligence (AI) that imitate how human brains process information, but understanding how these networks “think” has long been a challenge. Now, researchers at Kyushu University have developed a new method to understand how deep neural networks interpret information and sort it into groups. Published in IEEE Transactions on Neural Networks and Learning Systems, the study addresses the important need to ensure AI systems are accurate and robust and can meet the standards required for safe use.
Deep neural networks process information in many layers, similarly to humans solving a puzzle step by step. The first layer, known as the input layer, brings in the raw data. The subsequent layers, called hidden layers, analyze the information. Early hidden layers focus on basic features, such as detecting edges or textures—like examining individual puzzle pieces. Deeper hidden layers combine these features to recognize more complex patterns, such as identifying a cat or a dog—similar to connecting puzzle pieces to reveal the bigger picture.
“However, these hidden layers are like a locked black box: we see the input and output, but what is happening inside is not clear,” says Danilo Vasconcellos Vargas, Associate Professor from the Faculty of Information Science and Electrical Engineering at Kyushu University. "This lack of transparency becomes a serious problem when AI makes mistakes, sometimes triggered by something as small as changing a single pixel. AI might seem smart, but understanding how it comes to its decision is key to ensuring it’s trustworthy.”
Currently, methods for visualizing how AI organizes information rely on simplifying high-dimensional data into 2D or 3D representations. These methods let researchers observe how AI categorizes data points—for example, grouping images of cats close to other cats while separating them from dogs. However, this simplification comes with critical limitations.
“When we simplify high-dimensional information into fewer dimensions, it’s like flattening a 3D object into 2D—we lose important details and fail to see the whole picture. Additionally, this method of visualizing how the data is grouped makes it difficult to compare between different neural networks or data classes,” explains Vargas.
In this study, the researchers developed a new method, called the k* distribution method, that more clearly visualizes and assesses how well deep neural networks categorize related items together... (MORE - details, no ads)