{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,3]],"date-time":"2024-08-03T00:27:44Z","timestamp":1722644864001},"reference-count":61,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T00:00:00Z","timestamp":1722556800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T00:00:00Z","timestamp":1722556800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2024,9,1]]},"abstract":"Abstract<\/jats:title>\n Auto-encoders are capable of performing input reconstruction, denoising, and classification through an encoder-decoder structure. Spiking Auto-Encoders (SAEs) can utilize asynchronous sparse spikes to improve power efficiency and processing latency on neuromorphic hardware. In our work, we propose an efficient SAE trained using only Spike-Timing-Dependant Plasticity (STDP) learning. Our auto-encoder uses the Time-To-First-Spike (TTFS) encoding scheme and needs to update all synaptic weights only once per input, promoting both training and inference efficiency due to the extreme sparsity. We showcase robust reconstruction performance on the Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets with significantly fewer spikes compared to state-of-the-art SAEs by 1\u20133 orders of magnitude. Moreover, we achieve robust noise reduction results on the MNIST dataset. When the same noisy inputs are used for classification, accuracy degradation is reduced by 30%\u201380% compared to prior works. It also exhibits classification accuracies comparable to previous STDP-based classifiers, while remaining competitive with other backpropagation-based spiking classifiers that require global learning through gradients and significantly more spikes for encoding and classification of MNIST\/Fashion-MNIST inputs. The presented results demonstrate a promising pathway towards building efficient sparse spiking auto-encoders with local learning, making them highly suited for hardware integration.<\/jats:p>","DOI":"10.1088\/2634-4386\/ad5c97","type":"journal-article","created":{"date-parts":[[2024,6,27]],"date-time":"2024-06-27T22:26:11Z","timestamp":1719527171000},"page":"034005","update-policy":"http:\/\/dx.doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Efficient sparse spiking auto-encoder for reconstruction, denoising and classification"],"prefix":"10.1088","volume":"4","author":[{"ORCID":"http:\/\/orcid.org\/0000-0001-5464-8468","authenticated-orcid":true,"given":"Ben","family":"Walters","sequence":"first","affiliation":[]},{"given":"Hamid","family":"Rahimian Kalatehbali","sequence":"additional","affiliation":[]},{"given":"Zhengyu","family":"Cai","sequence":"additional","affiliation":[]},{"given":"Roman","family":"Genov","sequence":"additional","affiliation":[]},{"given":"Amirali","family":"Amirsoleimani","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-5832-4054","authenticated-orcid":true,"given":"Jason","family":"Eshraghian","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-7975-3985","authenticated-orcid":true,"given":"Mostafa","family":"Rahimi Azghadi","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2024,8,2]]},"reference":[{"key":"ncead5c97bib1","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1038\/s41746-023-00811-0","article-title":"Self-supervised learning for medical image classification: a systematic review and implementation guidelines","volume":"6","author":"Huang","year":"2023","journal-title":"npj Digit. Med."},{"key":"ncead5c97bib2","first-page":"pp 7059","article-title":"Fully spiking variational autoencoder","volume":"vol 36","author":"Kamata","year":"2022"},{"key":"ncead5c97bib3","first-page":"pp 653","article-title":"Denoising autoencoders for unsupervised anomaly detection in brain MRI","volume":"vol 172","author":"Kascenas","year":"2022"},{"key":"ncead5c97bib4","doi-asserted-by":"publisher","DOI":"10.1121\/2.0001227","article-title":"Application of autoencoder to traffic noise analysis","volume":"39","author":"Czy\u017cewski","year":"2019","journal-title":"Proc. Meetings Acoust."},{"key":"ncead5c97bib5","doi-asserted-by":"publisher","first-page":"621","DOI":"10.3389\/fnins.2019.00621","article-title":"Synthesizing images from spatio-temporal representations using spike-based backpropagation","volume":"13","author":"Roy","year":"2019","journal-title":"Front. Neurosci."},{"first-page":"pp 227","article-title":"Hardware implementation for spiking neural networks on edge devices","year":"2022","author":"Nguyen","key":"ncead5c97bib6"},{"key":"ncead5c97bib7","doi-asserted-by":"publisher","first-page":"6548","DOI":"10.3390\/s23146548","article-title":"EdgeMap: an optimized mapping toolchain for spiking neural network in edge computing","volume":"23","author":"Xue","year":"2023","journal-title":"Sensors"},{"first-page":"pp 1731","article-title":"HATS: histograms of averaged time surfaces for robust event-based object classification","year":"2018","author":"Sironi","key":"ncead5c97bib8"},{"first-page":"pp 5633","article-title":"End-to-end learning of representations for asynchronous event-based data","year":"2019","author":"Gehrig","key":"ncead5c97bib9"},{"key":"ncead5c97bib10","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.127010","article-title":"Neuromorphic imaging and classification with graph learning","volume":"565","author":"Zhang","year":"2024","journal-title":"Neurocomputing"},{"key":"ncead5c97bib11","first-page":"pp 16495","article-title":"Dynamic spiking graph neural networks","volume":"vol 38","author":"Yin","year":"2024"},{"key":"ncead5c97bib12","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1109\/MSP.2019.2931595","article-title":"Surrogate gradient learning in spiking neural networks: bringing the power of gradient-based optimization to spiking neural networks","volume":"36","author":"Neftci","year":"2019","journal-title":"IEEE Signal Process. Mag."},{"key":"ncead5c97bib13","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/S0925-2312(01)00658-0","article-title":"Error-backpropagation in temporally encoded networks of spiking neurons","volume":"48","author":"Bohte","year":"2002","journal-title":"Neurocomputing"},{"key":"ncead5c97bib14","doi-asserted-by":"publisher","first-page":"1016","DOI":"10.1109\/JPROC.2023.3308088","article-title":"Training spiking neural networks using lessons from deep learning","volume":"111","author":"Eshraghian","year":"2023","journal-title":"Proc. IEEE"},{"key":"ncead5c97bib15","first-page":"pp 1","article-title":"ReckOn: a 28nm sub-mm2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales","volume":"vol 65","author":"Frenkel","year":"2022"},{"article-title":"ETLP: event-based three-factor local plasticity for online learning with neuromorphic hardware","year":"2023","author":"Quintana","key":"ncead5c97bib16"},{"key":"ncead5c97bib17","doi-asserted-by":"publisher","first-page":"2","DOI":"10.3389\/fnsyn.2012.00002","article-title":"Spike-timing-dependent plasticity: a comprehensive overview","volume":"4","author":"Markram","year":"2012","journal-title":"Front. Synaptic Neurosci."},{"key":"ncead5c97bib18","doi-asserted-by":"publisher","first-page":"10464","DOI":"10.1523\/JNEUROSCI.18-24-10464.1998","article-title":"Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength and postsynaptic cell type","volume":"18","author":"Bi","year":"1998","journal-title":"J. Neurosci."},{"key":"ncead5c97bib19","doi-asserted-by":"crossref","first-page":"717","DOI":"10.1109\/JPROC.2014.2314454","article-title":"Spike-based synaptic plasticity in silicon: design, implementation, application and challenges","volume":"102","author":"Rahimi Azghadi","year":"2014","journal-title":"Proc. IEEE"},{"key":"ncead5c97bib20","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1016\/j.neunet.2017.12.005","article-title":"STDP-based spiking deep convolutional neural networks for object recognition","volume":"99","author":"Kheradpisheh","year":"2018","journal-title":"Neural Netw."},{"key":"ncead5c97bib21","doi-asserted-by":"publisher","first-page":"99","DOI":"10.3389\/fncom.2015.00099","article-title":"Unsupervised learning of digit recognition using spike-timing-dependent plasticity","volume":"9","author":"Diehl","year":"2015","journal-title":"Front. Comput. Neurosci."},{"key":"ncead5c97bib22","doi-asserted-by":"publisher","first-page":"1083","DOI":"10.1109\/JETCAS.2023.3328926","article-title":"Spike timing dependent gradient for direct training of fast and efficient binarized spiking neural networks","volume":"13","author":"Cai","year":"2023","journal-title":"IEEE J. Emerg. Sel. Top. Circuits Syst."},{"key":"ncead5c97bib23","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1016\/j.neunet.2019.09.005","article-title":"Rethinking the performance comparison between SNNS and ANNS","volume":"121","author":"Deng","year":"2020","journal-title":"Neural Netw."},{"first-page":"01","article-title":"A general error-modulated STDP learning rule applied to reinforcement learning in the basal ganglia","year":"2011","author":"Bekolay","key":"ncead5c97bib24"},{"key":"ncead5c97bib25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pcbi.1000180","article-title":"A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback","volume":"4","author":"Legenstein","year":"2008","journal-title":"PLoS Computat. Biol."},{"key":"ncead5c97bib26","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1016\/j.neucom.2021.06.027","article-title":"An autonomous learning mobile robot using biological reward modulate STDP","volume":"458","author":"Lu","year":"2021","journal-title":"Neurocomputing"},{"first-page":"pp 4725","article-title":"End to end learning of spiking neural network based on R-STDP for a lane keeping vehicle","year":"2018","author":"Bing","key":"ncead5c97bib27"},{"article-title":"What is the other 85 percent of V1 doing","year":"2006","author":"Olshausen","key":"ncead5c97bib28","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780195148220.003.0010"},{"article-title":"Efficient-Sparse-Spiking-Auto-Encoder-for-Reconstruction-Denoising-and-Classification","year":"2024","author":"Walters","key":"ncead5c97bib29","doi-asserted-by":"crossref","DOI":"10.1088\/2634-4386\/ad5c97"},{"article-title":"Training deep spiking auto-encoders without bursting or dying neurons through regularization","year":"2021","author":"H\u00fcbotter","key":"ncead5c97bib30"},{"first-page":"pp 293","article-title":"Training spiking autoencoders by truncated BPTT under trade-offs between simulation steps and reconstruction error","year":"2022","author":"Shimmyo","key":"ncead5c97bib31"},{"key":"ncead5c97bib32","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2021.712667","article-title":"Spiking autoencoders with temporal coding","volume":"15","author":"Com\u015fa","year":"2021","journal-title":"Front. Neurosci."},{"key":"ncead5c97bib33","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1016\/j.neunet.2018.05.018","article-title":"Representation learning using event-based STDP","volume":"105","author":"Tavanaei","year":"2018","journal-title":"Neural Netw."},{"first-page":"pp 1","article-title":"E-STDP: a spatio-temporally local unsupervised learning rule for sparse coded spiking convolutional autoencoders","year":"2023","author":"Kotariya","key":"ncead5c97bib34"},{"key":"ncead5c97bib35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pcbi.1004566","article-title":"Mirrored STDP implements autoencoder learning in a network of spiking neurons","volume":"11","author":"Burbank","year":"2015","journal-title":"PLoS Computat. Biol."},{"first-page":"pp 263","article-title":"Sparsity enables data and energy efficient spiking convolutional neural networks","year":"2018","author":"Bhatt","key":"ncead5c97bib36"},{"key":"ncead5c97bib37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pcbi.1002250","article-title":"A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields","volume":"7","author":"Zylberberg","year":"2011","journal-title":"PLoS. Biol."},{"key":"ncead5c97bib38","doi-asserted-by":"publisher","first-page":"1065","DOI":"10.3390\/s21041065","article-title":"Using a low-power spiking continuous time neuron (SCTN) for sound signal processing","volume":"21","author":"Bensimon","year":"2021","journal-title":"Sensors"},{"key":"ncead5c97bib39","doi-asserted-by":"publisher","first-page":"2937","DOI":"10.1109\/TCSII.2021.3065827","article-title":"A new SCTN digital low power spiking neuron","volume":"68","author":"Bensimon","year":"2021","journal-title":"IEEE Trans. Circuits Syst. II"},{"key":"ncead5c97bib40","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1016\/j.neuroscience.2017.06.032","article-title":"STDP allows close-to-optimal spatiotemporal spike pattern detection by single coincidence detector neurons","volume":"389","author":"Masquelier","year":"2018","journal-title":"Neuroscience"},{"first-page":"pp 4288","article-title":"Efficient implementation of STDP rules on spinnaker neuromorphic hardware","year":"2014","author":"Diehl","key":"ncead5c97bib41"},{"key":"ncead5c97bib42","doi-asserted-by":"publisher","first-page":"742","DOI":"10.1038\/s42256-021-00387-y","article-title":"Sparsity provides a competitive advantage","volume":"3","author":"Frenkel","year":"2021","journal-title":"Nat. Mach. Intell."},{"key":"ncead5c97bib43","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0088326","article-title":"Tunable low energy, compact and high performance neuromorphic circuit for spike-based synaptic plasticity","volume":"9","author":"Rahimi Azghadi","year":"2014","journal-title":"PLoS One"},{"key":"ncead5c97bib44","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1007\/BF00275835","article-title":"Input-output relationship of the leaky-integrator neuron model","volume":"8","author":"Scharstein","year":"1979","journal-title":"J. Math. Biol."},{"first-page":"pp 1","article-title":"Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks","year":"2019","author":"Shrestha","key":"ncead5c97bib45"},{"key":"ncead5c97bib46","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1109\/TNANO.2013.2250995","article-title":"Immunity to device variations in a spiking neural network with memristive nanodevices","volume":"12","author":"Querlioz","year":"2013","journal-title":"IEEE Trans. Nanotechnol."},{"key":"ncead5c97bib47","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.126682","article-title":"A new pre-conditioned STDP rule and its hardware implementation in neuromorphic crossbar array","volume":"557","author":"Tao","year":"2023","journal-title":"Neurocomputing"},{"key":"ncead5c97bib48","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.neunet.2020.11.005","article-title":"Necessary conditions for STDP-based pattern recognition learning in a memristive spiking neural network","volume":"134","author":"Demin","year":"2021","journal-title":"Neural Netw."},{"key":"ncead5c97bib49","doi-asserted-by":"publisher","first-page":"812","DOI":"10.3389\/fnins.2019.00812","article-title":"Unsupervised learning on resistive memory array based spiking neural networks","volume":"13","author":"Guo","year":"2019","journal-title":"Front. Neurosci."},{"key":"ncead5c97bib50","doi-asserted-by":"publisher","first-page":"1569","DOI":"10.1007\/s00521-023-09135-2","article-title":"Unsupervised character recognition with graphene memristive synapses","volume":"36","author":"Walters","year":"2023","journal-title":"Neural Comput. Appl."},{"key":"ncead5c97bib51","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1016\/j.neunet.2019.09.007","article-title":"A biologically plausible supervised learning method for spiking neural networks using the symmetric STDP rule","volume":"121","author":"Hao","year":"2020","journal-title":"Neural Netw."},{"article-title":"Memory via temporal delays in weightless spiking neural network","year":"2022","author":"Hazan","key":"ncead5c97bib52","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2202.07132"},{"key":"ncead5c97bib53","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2021.638474","article-title":"Neural coding in spiking neural networks: a comparative study for robust neuromorphic systems","volume":"15","author":"Guo","year":"2021","journal-title":"Front. Neurosci."},{"key":"ncead5c97bib54","doi-asserted-by":"publisher","first-page":"3601","DOI":"10.1109\/TCAD.2020.3013049","article-title":"FSpiNN: an optimization framework for memory-efficient and energy-efficient spiking neural networks","volume":"39","author":"Putra","year":"2020","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"ncead5c97bib55","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"},{"first-page":"1519","article-title":"LISNN: improving spiking neural networks with lateral interactions for robust object recognition","year":"2020","author":"Cheng","key":"ncead5c97bib56"},{"key":"ncead5c97bib57","first-page":"12022","article-title":"Temporal spike sequence learning via backpropagation for deep spiking neural networks","volume":"33","author":"Zhang","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"first-page":"pp 172","year":"2019","author":"Opie\u0142ka","key":"ncead5c97bib58"},{"key":"ncead5c97bib59","doi-asserted-by":"publisher","first-page":"1255","DOI":"10.1007\/s11063-021-10680-x","article-title":"BS4NN: binarized spiking neural networks with temporal coding and learning","volume":"54","author":"Kheradpisheh","year":"2022","journal-title":"Neural Process. Lett."},{"key":"ncead5c97bib60","doi-asserted-by":"publisher","first-page":"4837","DOI":"10.1109\/TCSI.2021.3126555","article-title":"How to build a memristive integrate-and-fire model for spiking neuronal signal generation","volume":"68","author":"Kang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. I"},{"key":"ncead5c97bib61","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1038\/s41467-023-44365-x","article-title":"Mosaic: in-memory computing and routing for small-world spike-based neuromorphic systems","volume":"15","author":"Dalgaty","year":"2024","journal-title":"Nat. Commun."}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad5c97","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad5c97\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad5c97","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad5c97\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad5c97\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad5c97\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad5c97\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad5c97\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T09:32:39Z","timestamp":1722591159000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ad5c97"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,2]]},"references-count":61,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,8,2]]},"published-print":{"date-parts":[[2024,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/ad5c97","relation":{},"ISSN":["2634-4386"],"issn-type":[{"type":"electronic","value":"2634-4386"}],"subject":[],"published":{"date-parts":[[2024,8,2]]},"assertion":[{"value":"Efficient sparse spiking auto-encoder for reconstruction, denoising and classification","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2024 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2024-03-11","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-06-27","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-08-02","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}
