I have written a few posts where I make the point that most of the important knowledge that is needed to build intelligent agents is not learned —because it cannot be learned differently, and it cannot be susceptible for incremental, approximate and individual learning from observations. It is that simple…

(last edited August 29, 2022) Commonsense (background) knowledge, at least the kind of knowledge that we fetch and rely upon in the process of language understanding: (i) cannot be learned by processing vast amounts of text because that knowledge is never explicitly stated in the text — and you cannot…

A while back a 2-day online workshop on compositionality and AI was organized by Gary Marcus and Raphael Milliere, with additional contributions form Allyson Ettinger and Paul Smolensky (one of the pioneers of tensor algebra in NNs). As often, and although compositionality touches on many subjects in AI, natural language…

Introduction In the previous post I argued that most of the knowledge that an intelligent agent needs to perform most higher-level cognitive tasks (language understanding, planning, reasoning, etc.) is not learned, but is acquired. Unlike skills (say playing guitars), that are individually learned, and learned differently due to our unique individual…

It’s very simple … Suppose someone uttered the following sentence: (1) Carlos has a Greek statue in every room in his house. A 4-year old understands the above sentence as “In every room in his house, Carlos has a Greek statue” — that is, the 4-year old somehow knows that “a Greek statue” does…

In the Beginning In the very first sentence in his introduction of Logic, Immanuel Kant states the following:

Great article. It is all about how (and who) measures 'success' - in large language models (LLMs) like GPT-3 (I have a short article that talks about this here https://www.sorcero.com/blog/measuring-progress-natural-language-understanding) and a longer article in The Gradient that shows why the brute force data-driven approach will never achieve true language understanding - here https://thegradient.pub/machine-learning-wont-solve-the-natural-language-understanding-challenge/.

edited 1/2/2022 Generalization and High-Level Representations Yoshua Bengio, a prominent machine learning (ML) researcher who is also considered to be one of the godfathers of deep learning (DL) has recently been quoted as saying: Deep learning, as it is now, has made huge progress in perception, but it hasn’t delivered yet on systems that…

It seems obvious (Or does it?) Let’s consider a general rule that — clearly all of us, and since the early years of childhood — we come to learn, namely the metaphysical fact shown in the picture below.