Compact deep neural network models of the visual cortex

Nature News · Feb 25, 2026 · Collected from RSS

Full Article

Data availabilityRaw data including spike timing for all recording sessions are available97. Processed responses for model training and evaluation as well as stimulus images are available on GitHub (https://github.com/cowleygroup/V4_compact_models). Source data are provided with this paper.Code availability All spike sorting was performed using custom Matlab software available on GitHub (https://github.com/smithlabvision/spikesort). Model weights and code are available on GitHub (https://github.com/cowleygroup/V4_compact_models). ReferencesYamins, D. L. & DiCarlo, J. J. Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19, 356–365 (2016).Article CAS PubMed Google Scholar Kriegeskorte, N. Deep neural networks: a new framework for modeling biological vision and brain information processing. Annu. Rev. Vis. Sci. 1, 417–446 (2015).Article PubMed Google Scholar Heeger, D. J. Half-squaring in responses of cat striate cells. Vis. Neurosci. 9, 427–443 (1992).Article CAS PubMed Google Scholar Rust, N. C., Schwartz, O., Movshon, J. A. & Simoncelli, E. P. Spatiotemporal elements of macaque V1 receptive fields. Neuron 46, 945–956 (2005).Article CAS PubMed Google Scholar Pillow, J. W. et al. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454, 995–999 (2008).Article ADS CAS PubMed PubMed Central Google Scholar Vintch, B., Movshon, J. A. & Simoncelli, E. P. A convolutional subunit model for neuronal responses in macaque V1. J. Neurosci. 35, 14829–14841 (2015).Article CAS PubMed PubMed Central Google Scholar Ustyuzhaninov, I. et al. Digital twin reveals combinatorial code of non-linear computations in the mouse primary visual cortex. Preprint at bioRxiv https://doi.org/10.1101/2022.02.10.479884 (2022).Maheswaranathan, N. et al. Interpreting the retinal neural code for natural scenes: from computations to neurons. Neuron 111, 2742–2755 (2023).Article CAS PubMed PubMed Central Google Scholar David, S. V., Hayden, B. Y. & Gallant, J. L. Spectral receptive field properties explain shape selectivity in area V4. J. Neurophysiol. 96, 3492–3505 (2006).Article PubMed Google Scholar Yamins, D. L. et al. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl Acad. Sci. USA 111, 8619–8624 (2014).Article ADS CAS PubMed PubMed Central Google Scholar Cadieu, C. et al. A model of V4 shape selectivity and invariance. J. Neurophysiol. 98, 1733–1750 (2007).Article PubMed Google Scholar Bahri, Y., Dyer, E., Kaplan, J., Lee, J. & Sharma, U. Explaining neural scaling laws. Proc. Natl Acad. Sci. USA 121, e2311878121 (2024).Article MathSciNet CAS PubMed PubMed Central Google Scholar Liu, Z. et al. A ConvNet for the 2020s. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 11966–11976 (IEEE, 2022).Kaplan, J. et al. Scaling laws for neural language models. Preprint at arXiv https://doi.org/10.48550/arXiv.2001.08361 (2020).Schrimpf, M. et al. Integrative benchmarking to advance neurally mechanistic models of human intelligence. Neuron 108, 413–423 (2020).Article CAS PubMed Google Scholar Hinton, G., Vinyals, O. & Dean, J. Distilling the knowledge in a neural network. Preprint at arXiv https://doi.org/10.48550/arXiv.1503.02531 (2015).Han, S., Mao, H. & Dally, W. J. Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. In Proc. 4th International Conference on Learning Representations (ICLR, 2016).Butts, D. A. Data-driven approaches to understanding visual neuron activity. Annu. Rev. Vis. Sci. 5, 451–477 (2019).Article PubMed Google Scholar Pierzchlewicz, P. et al. Energy guided diffusion for generating neurally exciting images. In Proc. Advances in Neural Information Processing Systems 36 (eds Oh, A. et al.) (NeurIPS, 2023).Bashivan, P., Kar, K. & DiCarlo, J. J. Neural population control via deep image synthesis. Science 364, eaav9436 (2019).Article CAS PubMed Google Scholar DiMattina, C. & Zhang, K. Adaptive stimulus optimization for sensory systems neuroscience. Front. Neural Circuits 7, 101 (2013).Article PubMed PubMed Central Google Scholar Pillow, J. W. & Park, M. in Closed Loop Neuroscience (ed. El Hady, A.) Ch. 1 (Elsevier, 2016).Seung, H. S., Opper, M. & Sompolinsky, H. Query by committee. In Proc. Fifth Annual Workshop on Computational Learning Theory 287–294 (Association for Computing Machinery, 1992).Cowley, B. & Pillow, J. W. High-contrast “gaudy” images improve the training of deep neural network models of visual cortex. In. Proc. Advances in Neural Information Processing Systems 33 (eds Larochelle, H. et al.) (NeurIPS, 2020).Frankle, J. & Carbin, M. The lottery ticket hypothesis: finding sparse, trainable neural networks. In Proc. 7th International Conference on Learning Representations 8954–8995 (ICLR, 2019).He, Y., Zhang, X. & Sun, J. Channel pruning for accelerating very deep neural networks. In Proc. IEEE International Conference on Computer Vision 1398–1406 (IEEE, 2017).Wang, T. et al. Large-scale calcium imaging reveals a systematic V4 map for encoding natural scenes. Nat. Commun. 15, 6401 (2024).Article ADS CAS PubMed PubMed Central Google Scholar Du, F., Núñez-Ochoa, M. A., Pachitariu, M. & Stringer, C. Towards a simplified model of primary visual cortex. Preprint at bioRxiv https://doi.org/10.1101/2024.06.30.601394 (2024).Kamali, F., Suratgar, A. A., Menhaj, M. & Abbasi-Asl, R. Compression-enabled interpretability of voxelwise encoding models. PLoS Comput. Biol. 21, e1012822 (2025).Article ADS PubMed PubMed Central Google Scholar Cowley, B. R., Williamson, R. C., Acar, K., Smith, M. A. & Yu, B. M. Adaptive stimulus selection for optimizing neural population responses. In Proc. Advances in Neural Information. Processing Systems 30 (eds Guyon, I. et al.) (NeurIPS, 2017).Willeke, K. F. et al. Deep learning-driven characterization of single cell tuning in primate visual area V4 unveils topological organization. Preprint at bioRxiv https://doi.org/10.1101/2023.05.12.540591 (2023).Szegedy, C. et al. Intriguing properties of neural networks. In Proc. 2nd International Conference on Learning Representations (ICLR, 2014).Guo, C. et al. Adversarially trained neural representations may already be as robust as corresponding biological neural representations. In Proc. 39th International Conference on Machine Learning (eds Chaudhuri, K. et al.) 8072–8081 (PMLR, 2022).Berardino, A., Ballé, J., Laparra, V. & Simoncelli, E. Eigen-distortions of hierarchical representations. In Proc. Advances in neural information processing systems 30 (eds Guyon, I. et al.) (NeurIPS, 2017).Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Commun. ACM https://doi.org/10.1145/3065386 (2017).DiCarlo, J. J., Zoccolan, D. & Rust, N. C. How does the brain solve visual object recognition? Neuron 73, 415–434 (2012).Article CAS PubMed PubMed Central Google Scholar Dapello, J. et al. Simulating a primary visual cortex at the front of cnns improves robustness to image perturbations. In Proc. Advances in Neural Information Processing Systems 33 (eds Larochelle, H. et al.) (NeurIPS, 2020).Federer, C., Xu, H., Fyshe, A. & Zylberberg, J. Improved object recognition using neural networks trained to mimic the brain’s statistical properties. Neural Netw. 131, 103–114 (2020).Article PubMed Google Scholar Kornblith, S., Norouzi, M., Lee, H. & Hinton, G. Similarity of neural network representations revisited. In Proc. 36th International Conference on Machine Learning (eds Chaudhuri, K. & Salakhutdinov, R.) 3519–3529 (PMLR, 2019).Majaj, N. J., Hong, H., Solomon, E. A. & DiCarlo, J. J. Simple learned weighted sums of inferior temporal neuronal firing rates accurately predict human core object recognition performance. J. Neurosci. 35, 13402–13418 (2015).Article CAS PubMed PubMed Central Google Scholar Cadena, S. A. et al. Deep convolutional models improve predictions of macaque V1 responses to natural images. PLoS Comput. Biol. 15, e1006897 (2019).Article CAS PubMed PubMed Central Google Scholar Conway, B. R. The organization and operation of inferior temporal cortex. Annu. Rev. Vis. Sci. 4, 381–402 (2018).Article PubMed PubMed Central Google Scholar Pasupathy, A., Popovkina, D. V. & Kim, T. Visual functions of primate area V4. Annu. Rev. Vis. Sci. 6, 363–385 (2020).Article PubMed PubMed Central Google Scholar Heeger, D. J. Normalization of cell responses in cat striate cortex. Vis. Neurosci. 9, 181–197 (1992).Article CAS PubMed Google Scholar Carandini, M. & Heeger, D. J. Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13, 51–62 (2012).Article CAS Google Scholar Coen-Cagli, R., Kohn, A. & Schwartz, O. Flexible gating of contextual influences in natural vision. Nat. Neurosci. 18, 1648 (2015).Article CAS PubMed PubMed Central Google Scholar Burg, M. F. et al. Learning divisive normalization in primary visual cortex. PLoS Comput. Biol. 17, e1009028 (2021).Article CAS PubMed PubMed Central Google Scholar Ruff, D. A. & Cohen, M. R. A normalization model suggests that attention changes the weighting of inputs between visual areas. Proc. Natl Acad. Sci. USA 114, E4085–E4094 (2017).Article ADS CAS PubMed PubMed Central Google Scholar Verhoef, B.-E. & Maunsell, J. H. Attention-related changes in correlated neuronal activity arise from normalization mechanisms. Nat. Neurosci. 20, 969–977 (2017).Article CAS PubMed PubMed Central Google Scholar Ungerleider, L. G., Galkin, T. W., Desimone, R. & Gattass, R. Cortical connections of area V4 in the macaque. Cereb. Cortex 18, 477–499 (2008).Article PubMed Google Scholar Semedo, J. D. et al. Feedforward and feedback interactions between visual cortical areas use different population activity patterns. Nat. Commun. 13, 1099 (2022).Article ADS CAS PubMed PubMed Central Google Scholar Jun, N. Y. et al. Coordinated multiplexing of information about separate objects in visual cortex