New Theory Cracks Open the Black Box of Deep Learning

C C Offline

EXCERPT: [...] Like a brain, a deep neural network has layers of neurons — artificial ones that are figments of computer memory. When a neuron fires, it sends signals to connected neurons in the layer above. During deep learning, connections in the network are strengthened or weakened as needed to make the system better at sending signals from input data — the pixels of a photo of a dog, for instance — up through the layers to neurons associated with the right high-level concepts, such as “dog.” After a deep neural network has “learned” from thousands of sample dog photos, it can identify dogs in new photos as accurately as people can. The magic leap from special cases to general concepts during learning gives deep neural networks their power, just as it underlies human reasoning, creativity and the other faculties collectively termed “intelligence.” Experts wonder what it is about deep learning that enables generalization — and to what extent brains apprehend reality in the same way.

[...] Naftali Tishby, a computer scientist and neuroscientist from the Hebrew University of Jerusalem, presented evidence in support of a new theory explaining how deep learning works. Tishby argues that deep neural networks learn according to a procedure called the “information bottleneck,” which he and two collaborators first described in purely theoretical terms in 1999. The idea is that a network rids noisy input data of extraneous details as if by squeezing the information through a bottleneck, retaining only the features most relevant to general concepts. Striking new computer experiments by Tishby and his student Ravid Shwartz-Ziv reveal how this squeezing procedure happens during deep learning, at least in the cases they studied. Tishby’s findings have the AI community buzzing....


Possibly Related Threads…
Thread Author Replies Views Last Post
  How bodies get smarts + New link to an old model could crack mystery of deep learning C C 0 26 Oct 12, 2021 05:24 PM
Last Post: C C
  10 ways to apply machine learning in Earth & space sciences + Edge of chaos C C 0 45 Jun 29, 2021 11:08 PM
Last Post: C C
  Machine learning accelerates cosmological simulations C C 0 27 May 5, 2021 12:24 AM
Last Post: C C
  Machine learning is shaking up the world of earthquake prediction C C 0 89 Feb 15, 2021 05:51 AM
Last Post: C C
  Machine learning tool predicts early symptoms of schizophrenia in patient's relatives C C 0 70 Jan 26, 2021 11:32 PM
Last Post: C C
  Machine learning predicts kids at risk of not getting vaccinated C C 0 219 May 7, 2019 06:17 PM
Last Post: C C
  Bina48 Takes University Philosophy Class in Hopes of Learning What Love Is Yazata 2 498 Dec 29, 2017 11:26 PM
Last Post: RainbowUnicorn
  Rise of the racist robots – how AI is learning all our worst impulses C C 1 298 Aug 9, 2017 08:40 AM
Last Post: RainbowUnicorn
  Accelerating AI learning + Machine gun robot may lead uprising one day C C 0 372 May 9, 2017 10:45 PM
Last Post: C C
  How computer games satisfy basic needs + Machines learning to disambiguate people C C 1 302 Jan 14, 2017 03:02 PM
Last Post: Ben the Donkey

Users browsing this thread: 1 Guest(s)