Re: Some scientific history that I experienced relevant to the recent Nobel Prizes to Hopfield and Hinton - Comp-neuro - lists.cnsorg.org
Dear Comp-neuro colleagues, Here are some short summaries of the history of neural network discoveries, as I experienced it, that are relevant to the recent Nobel Prizes to Hopfield and Hinton: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++THETHE THE NOBEL PRIZES IN PHYSICS TO HOPFIELD AND HINTON FOR MODELS THEY DID NOT DISCOVER: THE CASE OF HOPFIELD Here I summarize my concerns about the Hopfield award. I published articles in 1967 – 1972 in the Proceedings of the National Academy of Sciences that introduced the Additive Model that Hopfield used in 1984. My articles proved global theorems about the limits and oscillations of my Generalized Additive Models. See sites.bu.edu/steveg<http://sites.bu.edu/steveg> for these articles. For example: Grossberg, S. (1971). Pavlovian pattern learning by nonlinear neural networks. Proceedings of the National Academy of Sciences, 68, 828-831. https://lnkd.in/emzwx4Tw This article illustrates that my mathematical results were part of a research program to develop biological neural networks that provide principled mechanistic explanations of psychological and neurobiological data. Later, Michael Cohen and I published a Liapunov function that included the Additive Model and generalizations thereof in 1982 and 1983 before Hopfield (1984) appeared. For example, Cohen, M.A. and Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13, 815-826. https://lnkd.in/eAFAdvbu I was told that Hopfield knew about my work before he published his 1984 article, without citation. Recall that I started my neural networks research in 1957 as a Freshman at Dartmouth College. That year, I introduced the biological neural network paradigm, as well as the short-term memory (STM), medium-term memory (MTM), and long-term memory (LTM) laws that are used to this day, including in the Additive Model, to explain data about how brains make minds. See the review in https://lnkd.in/gJZJtP_W . When I started in 1957, I knew no one else who was doing neural networks. That is why my colleagues call me the Father of AI. I then worked hard to create a neural networks community, notably a research center, academic department, the International Neural Network Society, the journal Neural Networks, multiple international conferences on neural networks, and Boston-area research centers, while training over 100 gifted PhD students, postdocs, and faculty to do neural network research. See the Wikipedia page. That is why I did not have time or strength to fight for priority of my models. Recently, I was able to provide a self-contained and non-technical overview and synthesis of some of my scientific discoveries since 1957, as well as explanations of the work of many other scientists, in my 2021 Magnum Opus Conscious Mind, Resonant Brain: How Each Brain Makes a Mind https://lnkd.in/eiJh4Ti ++++++++++++++++++++++++++++++++++++++++++++++++++++ THE NOBEL PRIZES IN PHYSICS TO HOPFIELD AND HINTON FOR MODELS THEY DID NOT DISCOVER: THE CASE OF HINTON Here I summarize my concerns about the Hinton award. Many authors developed Back Propagation (BP) before Hinton; e.g., Amari (1967), Werbos (1974), Parker (1982), all before Rumelhart, Hinton, & Williams (1986). BP has serious computational weaknesses: It is UNTRUSTWORTHY (because it is UNEXPLAINABLE). It is UNRELIABLE (because it can experience CATASTROPHIC FORGETTING. It should thus never be used in financial or medical applications. BP learning is also SLOW and uses non-biological NONLOCAL WEIGHT TRANSPORT. See Figure, right column, top. In 1988, I published 17 computational problems of BP: https://lnkd.in/erKJvXFA BP gradually grew out of favor because other models were better. Later, huge online databases and supercomputers enabled Deep Learning to use BP to learn. My 1988 article contrasted BP with Adaptive Resonance Theory (ART) which I first published in 1976: https://lnkd.in/evkfq22G See Figure, right column, bottom. ART never had BP’s problems. ART is now the most advanced cognitive and neural theory that explains HOW HUMANS LEARN TO ATTEND, RECOGNIZE, and PREDICT events in a changing world. ART also explains and simulates data from hundreds of psychological and neurobiological experiments. In 1980, I derived ART from a THOUGHT EXPERIMENT about how ANY system can AUTONOMOUSLY learn to correct predictive errors in a changing world: https://lnkd.in/eGWE8kJg The thought experiment derives ART from a few facts of life that do not mention mind or brain. ART is thus a UNIVERSAL solution of the problem of autonomous error correction in a changing world. That is why ART models can be used in designs for AUTONOMOUS ADAPTIVE INTELLIGENCE in engineering, technology, and AI. ART also proposes a solution of the classical MIND-BODY PROBLEM: HOW, WHERE in our brains, and WHY from a deep computational perspective, we CONSCIOUSLY SEE, HEAR, FEEL, and KNOW about the world, and use our conscious states to PLAN and ACT to realize VALUED GOALS. For details, see Conscious Mind, Resonant Brain: How Each Brain Makes a Mind https://lnkd.in/eiJh4Ti +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Stephen, You have been complaining about not getting enough credit since I first met you in the mid 1980s. You refer to publishing before Hopfield's 1984 paper. You deliberately ignore his 1982 paper, which received 27946 citations: Neural networks and physical systems with emergent collective computational abilities, JJ Hopfield PNAS, April 15, 1982, 79 (8) 2554-2558 In contrast, the 1971 student paper you mentioned received 118 citations. Your 1983 paper with Michael Cohen in IEEE Transaction of Man, Systems, and Cybernetics was received on August 1, 1982, several months after the 1982 paper by Hopfeld had appeared in print. You did get a good number of citations on this paper, 3344, but not as good as 27946. You mention a 1982 paper with Michael Cohen; I have not been able to find it. I have never heard anybody referring to you as 'the father of AI'. What I do remember is a visit to your group at BU many, many years ago - in the late 80s or early 90s. I was amazed at finding out that you closely supervised every word in every slide that anybody in your group was allowed to show when invited to give a talk. Everybody was under a lot of pressure to 'stay on message', where the 'message' was your view on things. I had never encountered a scientific group run as a cult. It made an impression, but not a positive one. So much for training over 100 people - by teaching them not to think by themselves. As for back-propagation, this is not what Hinton is cited for in the Nobel Prize citation. It is for the Boltzmann machine. Finally, I always wondered: If ART solves all problems, why are there ML/AI problems that remain to be solved? On Mon, Oct 21, 2024 at 7:49 AM Grossberg, Stephen via Comp-neuro < comp-neuro@lists.cnsorg.org> wrote:
