The debate about the energy greediness of large AI models is raging. Recently, an AI ethics researcher at Google was dismissed because she had pinpointed the upward spiral of exploding training data sets. The fact is that the numbers make one’s head swim. In 2018, the BERT model made the headlines by achieving best-in-class NLP performance with a training dataset of 3 billion words.
My top 5 language AI books
Language AI is one of the most challenging areas of artificial intelligence, one where mainstream AI is far from coming near human-level performance, because it needs world knowledge to be solved (AI complete).
The shortcomings of modern machine learning approaches can be explained by the low efficiency of artificial neural networks. Because natural evolution is mainly driven by efficiency, I developed a strong interest for biologically inspired natural language understanding, hence the following book recommendations.
#1
By Jeff Hawkins and Sandra Blakeslee
This book published in 2004 describes the core concepts in Jeff Hawkins’ cortical theory. Hawkins defines intelligence as the capacity to anticipate the future based on past experiences. The brain acts as a memory system that stores patterns and makes predictions out of current experiences based on memorized patterns. In order to be truly intelligent, machines must recreate this memory-prediction