Years ago, a retiring artificial intelligence researcher told Jeff Hawkins that
"one of biggest problems in AI–no, the only problem in AI–is the problem of
representation." If the meaning of that statement isn’t intuitively obvious to
you, don’t worry. It wasn’t immediately clear to Jeff either.
Jeff explained how he came to understand and address this problem in a recent
keynote address at the International Symposium on Computer Architecture. We
posted a video of the keynote on YouTube.
In this talk, Jeff describes the brain as a predictive modeling system that
takes streams of input from the senses and learns sequences in real time. The
brain represents its inputs and the state of its world via the activity of
neurons. At any point in time most of the neurons are inactive and a few
active, thus the brain’s representations are “sparse.” Consequently we call
these "Sparse Distributed Representations” (SDRs) in our algorithms. SDRs
exhibit unique properties that enable benefits such as semantic generalization
and robustness to errors. These properties, in turn, allow the brain to learn
about objects and how they relate to each other holistically, without the
programmatically defined data structures used in computers. By applying these
principles in products like Grok, we can finally address the problem of
The talk ends with Jeff’s speculation on how the technology will evolve. Grok
itself is merely the first iteration of this technology. Unlocking the
operating principles of the neocortex will not necessarily culminate in
solutions addressing the "classic" AI problems of vision, language and speech.
The history of technology suggests that truly revolutionary technological
advancements develop in ways that even its inventors could never imagine. I’m
reminded of Alexander Graham Bell’s prediction, "One day there will be a
telephone in every major city in the USA."
If you’d like to know more, make sure to
watch the video.