Background reading/watching

• Principles of Neural Science
• Nancy Kanwisher online course
• Christos Papadimitriou on-line course ("Computation and the Brain")
• Sanjoy Dasgupta’s course (“Neurally-inspired unsupervised learning")
• Reviews on Spike-timing dependent plasticity

(Tentative) Schedule

(2/14) Overview and planning: Nancy Lynch

• Goals, overview of topics
• Planning topics, assigning papers/books to present
• Review of learning rules:
  • Short-term vs. long-term learning: Setting up firing patterns vs. changing edge weights.
  • Local learning rules, vs. global optimization
  • Supervised and unsupervised learning
  • Hebbian learning in rate-based networks and SNNs, Oja, Other rules? Hopfield network material?
Source:
• Neural Networks---A Systematic Introduction, Chapters 5, 13, 14.
• Oja, Principal Components, Minor Components, and Linear Neural Networks
• Oja, Neural Networks, Principal Components, and Subspaces
What was presented:
• Overview of brain algorithms.
• Assignment of papers.
• Nancy's meeting note

(2/21) Short-term learning: Cameron Musco

To read:
• Hitron, Lynch, Musco, Parter. Random Sketching, Clustering, and Short-Term Memory in Spiking Neural Networks. ITCS, January 2020.
What was presented:
• Overview of the renaming paper by Cameron

(2/28) Oja's rule, analysis: Brabeeba Wang and Chi-Ning Chou

To read:
• Chou, Wang. ODE-Inspired Analysis for the Biological Version of Oja's Rule in Solving Streaming PCA with For-All-Time Guarantee.
What was presented:
• Overview of the Oja's rule by Chi-Ning and Brabeeba
• Haft and Hemmen. Theory and implementation of infomax filters for the retina. 1998.
• Biological plausibility and inplausibility of Oja's rule. BCM rule review.
• Oja, Principal Components, Minor Components, and Linear Neural Networks
• Cengiz Pehlevan, Tao Hu, Dmitri B. Chklovskii. A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data. 2015

(3/6) Coronavirus. Postpone meeting for a week and all the subsequent meetings are conducted over Zoom.

(3/13) Valiant's work: Nishanth Dikkala

To read:
• Basic neuroid model and how it represents and learns concepts, for long-term learning.
• Limitations on memory capacity.
• Circuits of the Mind
What was presented:
• Slides
• Basic neuroid model and algorithms to solve memorization task
• Inductive learning will be spilled over to next week

(3/20) Valiant, cont'd: Nishanth Dikkala, Quanquan

To read:
• Circuits of the Mind. Continued.
• Valiant. Memorization and association on a realistic neural model. Neural Computation, 2006
What was presented:
• Slides
• Learning linear threshold function and numerical calculation
• A model to support memorization and association using random graph theory

(3/27) Valiant, cont'd. Papadimitriou. Quanquan, Jiajia

To read:
• Valiant. The Hippocampus as a Stable Memory Allocator for Cortex. Neural Computation, 2012
• Papadimitriou, Vempala. Cortical Learning via Prediction. JMLR 2015
What was presented:
• Hippocampus slide
• predictive join slide
• Finish Valiant's memorization and association paper
• Finish hippocampus as a stable memory allocator

(4/3)Papadimitriou papers: Shankha

To read:
• Maass, Papadimitriou, Vempala, Legenstein. Brain Computation: A Computer Science Perspective.
• Papadimitriou, Vempala, Mitropolsky, Collins, Maass. Brain Computation by Assemblies of Neurons. BioRxiv.
• Legenstein, Maass, Papadimitriou, Vempala. Long Term Memory and the Densest K-Subgraph Problem. ITCS 2018
What was presented:
• Finish Papadimitriou's papers

(4/10) Papadimitriou papers: Papadimitriou

To read:
• Slides
• Papadimitriou, Vempala. Random Projection in the Brain and Computation with Assemblies of Neurons. ITCS 2019
What was presented:
• Slides

(4/17) Other Papadimitriou work: Brabeeba

To read:
• Pokorny, Ison, Rao, Legenstein Papa., Maass. STDP forms associations between memory traces in networks of spiking neurons
• Papa. On the optimality of grid cells. ArXiv 2016
What was presented:
• Slides

(4/24) Connections with Artificial Neural Networks: Lili Su, Jiajia

To read:
• Relating gradient descent algorithms to biological algorithms.
• Simulating gradient descent with local algorithms.
Main papers
• Shanshan Qin, Nayantara Mudur, Cengiz Pehlevan. Supervised Deep Similarity Matching.
• Pehlevan, Chklovskii: Neuroscience-inspired online unsupervised learning algorithms and the journal version
Other papers
• Pehlevan. A Spiking Neural Network with local learning rules derived from nonnegative similarity matching. ArXiv 1902.01429v2, 2019
• Pehlevan, Sengupta, Chklovskii. Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks? Neural Computation 30, 2018
• Giovannucci, Minden, Pehlevan, Chklovskii. Efficient principal subspace projection of streaming data through fast similarity matching. ArXiv 1808.02083v1, 2018
• Minden, Pehlevan, Chklovskii. Biologically plausible online principal component analysis without recurrent neural dynamics. ArXiv 1810.06966v2, 2018
What was presented:
• Slides

(5/1) Learning of logically structured concepts: Nancy Lynch, Frederik, Shankha

To read:
Main papers
• Lynch and Mallmann-Trenn. Learning Hierarchically Structured Concepts.
Background
• Mhaskar, Liao, Poggio. Learning functions: When is deep better than shallow?. arXiv 2016
• Zhou, Bau, Oliva, Torralba. Interpreting deep visual representations via network dissection. arXiv 2017.
• Hubel and Wiesel. Receptive fields of single neurons in the cat's striate cortex. J. Physiology 1959.
• Hubel and Wiesel. Receptive fields, binocular interaction, and functional architecture in the cat's visual cortex. J. Physiology 1962.
• Felleman and van Essen. Distributed hierarchical processing in the primate cerebral cortex. Cerebral cortex, 1991.
Language
• Friederici. Language in our brain
• Stern, Andreas, Klein. A Minimum Span-Based Neural Constituency Parser.
What was presented:
• Language Slides
• Hierachical Slides

(5/8) Language continued: Frederik; Mathematics and biology of the retina: Brabeeba Wang, Chi-Ning-Chou

To read:
• Baccus, SA, Meister, M (2002) Fast and slow contrast adaptation in retinal circuitry. Neuron 36:909–919.
• Hosoya, T, Baccus, SA, Meister, M (2005) Dynamic predictive coding by the retina. Nature 436:71–77.
• Gollisch, T, Meister, M (2010) Eye smarter than scientists believed: Neural computations in circuits of the retina. Neuron 65:150–164.
• Ozuysal, Y. & Baccus, S. A. Linking the computational structure of variance adaptation to biophysical mechanisms. Neuron 73, 1002–1015 (2012).
What was presented:
• Language Slides
• Retina Slides