By Lambert Strether of Corrente.
Patient readers, Water Cooler was a little out of balance today, because there was more interesting business news than usual (a sign of shifts in the zeitgeist, perhaps). I will add more material here and there shortly. –lambert UDPATE All done!
Bird Song of the Day
From Buenos Aires, Argentina. Alert readers Flora and A Different Chris commented yesterday that they emjoyed the Bird Song of the Day. I too enjoy beginning writing on, er, a positive note. Thanks also to Doc Octogon, Amfortas the hippie, Patrick, and ambrit for discussing the habits of this interesting creature.
Multimodal Neurons in Artificial Neural Networks
March 4, 2021
22 minute read
We’ve discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually. This may explain CLIP s accuracy in classifying surprising visual renditions of concepts, and is also an important step toward understanding the associations and biases that CLIP and similar models learn.
Fifteen years ago, Quiroga et al. discovered that the human brain possesses multimodal neurons. These neurons respond to clusters of abstract concepts centered around a common high-level theme, rather than any specific visual feature. The most famous of these was the “Halle Berry” neuron, a neuron featured in both Scientific American and The New York Times, that responds to photographs, sketches, and the text “Halle Berry” (but not other names).