Elvin Hajizada
ABSTRACT
Intelligent robots need to recognize objects in their environment. This task is conceptually different from the typical image classification task in computer vision. Robots need to recognize particular object instances, not classes of objects, which makes these tasks simpler. However, these instances need to be recognized under different viewing angles, poses, and lighting conditions reliably. Moreover, for many application, robots need the capability to learn new objects quickly, e.g., in an interactive session with the user, and adapt object representations if conditions change and mistakes are made. This scenario creates a demand for object learning that (1) is continual, i.e. new objects can be added at any time without causing forgetting, (2) requires a small amount of data that can be acquired in a short interactive session with the user, and (3) stays open to later adaptation. Deep neural networks trained with slow gradient-based backpropagation, despite of their excellent performance on image processing tasks, are not well-suited for interactive robotic learning tasks. We thus explore smaller neural architectures and increase autonomy of the learning process by a neuronal state machine (NSM). The NSM regulates learning in the network and enables continual adaptation of the learned object prototypes. We implement this model as a spiking neural network in Intel’s neuromorphic research chip Loihi and test it on a custom event-based camera dataset generated in a simulated 3D environment. Our spiking neuronal network uses simple feature extraction layers and a single plastic layer that stores visual patterns using online, on-chip local learning rules. This network reaches 96.55±2.02% of testing accuracy on sets of 8 3D objects with 8 different views per object in interactive, on-demand learning experiments; it demonstrates up to x300 energy efficiency and better or on par latency compared to other online learning methods. This work contributes to neuronal-network based machine learning for robots with a small power footprint and interactive learning capability.
- Sandro Baumgartner, Alpha Renner, Raphaela Kreiser, Dongchen Liang, Giacomo Indiveri, and Yulia Sandamirskaya. 2020. Visual Pattern Recognition with on On-chip Learning: towards a Fully Neuromorphic Approach. In 2020 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 1–5.Google Scholar
Cross Ref
- Christian Brandli, Raphael Berner, Minhao Yang, Shih-Chii Liu, and Tobi Delbruck. 2014. A 240 × 180 130 db 3 μs latency global shutter spatiotemporal vision sensor. IEEE Journal of Solid-State Circuits 49, 10 (2014), 2333–2341.Google ScholarCross Ref
- Berk Calli, Arjun Singh, James Bruce, Aaron Walsman, Kurt Konolige, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. 2017. Yale-CMU-Berkeley dataset for robotic manipulation research. The International Journal of Robotics Research 36, 3 (2017), 261–268.Google ScholarDigital Library
- Berk Calli, Aaron Walsman, Arjun Singh, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M. Dollar. 2015. Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model Set. IEEE Robotics Automation Magazine 22, 3 (2015), 36–52. https://doi.org/10.1109/MRA.2015.2448951Google ScholarCross Ref
- Gert Cauwenberghs and Tomaso Poggio. 2000. Incremental and decremental support vector machine learning. Advances in neural information processing systems 13 (2000).Google Scholar
- Kuilin Chen and Chi-Guhn Lee. 2020. Incremental few-shot learning via vector quantization in deep embedded space. In International Conference on Learning Representations.Google Scholar
- Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. 2006. Online passive aggressive algorithms. (2006).Google Scholar
- Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, 2018. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro 38, 1 (2018), 82–99.Google ScholarCross Ref
- Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Prasad Joshi, Andrew Lines, Andreas Wild, and Hong Wang. 2018. Loihi: A Neuromorphic Manycore Processor with On-Chip Learning. IEEE Micro PP (01 2018), 1–1. https://doi.org/10.1109/MM.2018.112130359Google Scholar
- Mike Davies, Andreas Wild, Garrick Orchard, Yulia Sandamirskaya, Gabriel A Fonseca Guerra, Prasad Joshi, Philipp Plank, and Sumedh R Risbud. 2021. Advancing neuromorphic computing with Loihi: A survey of results and outlook. Proc. IEEE 109, 5 (2021), 911–934.Google ScholarCross Ref
- Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).Google Scholar
- Giulia D’Angelo, Adam Perrett, Massimiliano Iacono, Steve Furber, and Chiara Bartolozzi. 2022. Event driven bio-inspired attentive system for the iCub humanoid robot on SpiNNaker. Neuromorphic Computing and Engineering 2, 2 (2022), 024008.Google ScholarCross Ref
- Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, and Marcus Rohrbach. 2019. Uncertainty-guided Continual Learning with Bayesian Neural Networks. (2019), 1–16. arxiv:1906.02425http://arxiv.org/abs/1906.02425Google Scholar
- Sebastian Farquhar and Yarin Gal. 2018. Towards robust evaluations of continual learning. arXiv preprint arXiv:1805.09733(2018).Google Scholar
- Jérémy Fix, Nicolas Rougier, and Frédéric Alexandre. 2011. A dynamic neural field approach to the covert and overt deployment of spatial attention. Cognitive Computation 3, 1 (2011), 279–293.Google ScholarCross Ref
- Bernd Fritzke. 1994. A growing neural gas network learns topologies. Advances in neural information processing systems 7 (1994).Google Scholar
- Guillermo Gallego, Tobi Delbruck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew Davison, Jörg Conradt, Kostas Daniilidis, 2019. Event-based vision: A survey. arXiv preprint arXiv:1904.08405(2019).Google Scholar
- Wulfram Gerstner, Marco Lehmann, Vasiliki Liakoni, Dane Corneil, and Johanni Brea. 2018. Eligibility traces and plasticity on behavioral time scales: experimental support of neohebbian three-factor learning rules. Frontiers in neural circuits 12 (2018), 53.Google Scholar
- Suman Ghosh, Giulia D’Angelo, Arren Glover, Massimiliano Iacono, Ernst Niebur, and Chiara Bartolozzi. 2022. Event-driven proto-object based saliency in 3D space to attract a robot’s attention. Scientific reports 12, 1 (2022), 1–14.Google Scholar
- Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. 2013. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211(2013).Google Scholar
- Raul Grieben, Jan Tekülve, Stephan KU Zibner, Jonas Lins, Sebastian Schneegans, and Gregor Schöner. 2020. Scene memory and spatial inhibition in visual search. Attention, Perception, & Psychophysics(2020), 1–24.Google Scholar
- Tyler L Hayes, Kushal Kafle, Robik Shrestha, Manoj Acharya, and Christopher Kanan. 2020. Remind your neural network to prevent catastrophic forgetting. In European Conference on Computer Vision. Springer, 466–483.Google ScholarDigital Library
- Tyler L Hayes and Christopher Kanan. 2020. Lifelong machine learning with deep streaming linear discriminant analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 220–221.Google ScholarCross Ref
- Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. 2019. Learning a unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 831–839.Google ScholarCross Ref
- Massimiliano Iacono, Giulia D’Angelo, Arren Glover, Vadim Tikhanoff, Ernst Niebur, and Chiara Bartolozzi. 2019. Proto-object based saliency for event-driven cameras. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 805–812.Google ScholarDigital Library
- Elia Kaufmann, Antonio Loquercio, Rene Ranftl, Alexey Dosovitskiy, Vladlen Koltun, and Davide Scaramuzza. 2018. Deep drone racing: Learning agile flight in dynamic environments. In Conference on Robot Learning. PMLR, 133–145.Google Scholar
- Saeed Reza Kheradpisheh, Mohammad Ganjtabesh, Simon Thorpe, and Timothée Masquelier. 2017. STDP-based spiking deep convolutional neural networks for object recognition. Neural Networks 99 (12 2017). https://doi.org/10.1016/j.neunet.2017.12.005Google ScholarDigital Library
- James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114, 13(2017), 3521–3526.Google ScholarCross Ref
- Stephan Kirstein, Heiko Wersing, and Edgar Körner. 2005. Rapid online learning of objects in a biologically motivated recognition architecture. In Joint Pattern Recognition Symposium. Springer, 301–308.Google ScholarDigital Library
- Hee-kyoung Ko, Martina Poletti, and Michele Rucci. 2010. Microsaccades precisely relocate gaze in a high visual acuity task. Nature neuroscience 13, 12 (2010), 1549–1553.Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012).Google Scholar
- Brenden Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua Tenenbaum. 2011. One shot learning of simple visual concepts. In Proceedings of the annual meeting of the cognitive science society, Vol. 33.Google Scholar
- Robert Legenstein, Dejan Pecevski, and Wolfgang Maass. 2008. A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol 4, 10 (2008), e1000180.Google ScholarCross Ref
- Timothée Lesort, Hugo Caselles-Dupré, Michael Garcia-Ortiz, Andrei Stoian, and David Filliat. 2019. Generative models from the perspective of continual learning. In 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.Google ScholarCross Ref
- Timothée Lesort, Vincenzo Lomonaco, Andrei Stoian, Davide Maltoni, David Filliat, and Natalia Díaz-Rodríguez. 2020. Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges. Information fusion 58(2020), 52–68.Google Scholar
- Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck. 2008. A 128 × 128 120 dB 1 5μs latency asynchronous temporal contrast vision sensor. IEEE journal of solid-state circuits 43, 2 (2008), 566–576.Google Scholar
- Viktor Losing, Barbara Hammer, and Heiko Wersing. 2015. Interactive online learning for obstacle classification on a mobile robot. In 2015 international joint conference on neural networks (ijcnn). IEEE, 1–8.Google Scholar
- Viktor Losing, Barbara Hammer, and Heiko Wersing. 2018. Incremental on-line learning: A review and comparison of state of the art algorithms. Neurocomputing 275(2018), 1261–1274.Google ScholarDigital Library
- Jie Luo, Andrzej Pronobis, Barbara Caputo, and Patric Jensfelt. 2007. Incremental learning for place recognition in dynamic environments. In 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 721–728.Google ScholarCross Ref
- Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. 2021. Online Continual Learning in Image Classification: An Empirical Survey. 1 (2021), 1–64. arxiv:2101.10423http://arxiv.org/abs/2101.10423Google Scholar
- CD Manning, P Raghavan, and H Schütze. 2008. Xml retrieval. In Introduction to Information Retrieval.Cambridze University Press.Google Scholar
- Timothée Masquelier and Simon Thorpe. 2007. Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity. PLoS computational biology 3 (03 2007), e31. https://doi.org/10.1371/journal.pcbi.0030031Google Scholar
- Michael McCloskey and Neal J Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation. Vol. 24. Elsevier, 109–165.Google Scholar
- Stefan Milz, Georg Arbeiter, Christian Witt, Bassam Abdallah, and Senthil Yogamani. 2018. Visual slam for automated driving: Exploring the applications of deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 247–257.Google ScholarCross Ref
- M. Mozafari, S. R. Kheradpisheh, T. Masquelier, A. Nowzari-Dalini, and M. Ganjtabesh. 2018. First-Spike-Based Visual Categorization Using Reward-Modulated STDP. IEEE Transactions on Neural Networks and Learning Systems 29, 12(2018), 6178–6190. https://doi.org/10.1109/TNNLS.2018.2826721Google ScholarCross Ref
- German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. Neural Networks 113(2019), 54–71.Google ScholarDigital Library
- Giulia Pasquale, Carlo Ciliberto, Francesca Odone, Lorenzo Rosasco, and Lorenzo Natale. 2017. Are we Done with Object Recognition? The iCub robot’s Perspective. Robotics and Autonomous Systems 112 (09 2017). https://doi.org/10.1016/j.robot.2018.11.001Google Scholar
- Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake Richards, and Richard Naud. 2020. Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits. (03 2020). https://doi.org/10.1101/2020.03.30.015511Google Scholar
- José Pérez-Carrasco, Bo Zhao, Carmen Serrano, Begoña Acha, Teresa Serrano-Gotarredona, Shoushun Chen, and Bernabé Linares-Barranco. 2013. Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate-Coding and Coincidence Processing. Application to Feed Forward ConvNets. IEEE Transactions on Pattern Analysis and Machine Intelligence 35 (11 2013), 2706 – 2719. https://doi.org/10.1109/TPAMI.2013.71Google ScholarDigital Library
- Hang Qi, Matthew Brown, and David G Lowe. 2018. Low-shot learning with imprinted weights. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5822–5830.Google ScholarCross Ref
- Henri Rebecq, Daniel Gehrig, and Davide Scaramuzza. 2018. ESIM: an open event camera simulator. In Conference on Robot Learning. PMLR, 969–982.Google Scholar
- Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2001–2010.Google ScholarCross Ref
- Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter. 2022. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning. PMLR, 91–100.Google Scholar
- Jonas Ruesch, Manuel Lopes, Alexandre Bernardino, Jonas Hornstein, José Santos-Victor, and Rolf Pfeifer. 2008. Multimodal saliency-based bottom-up attention a framework for the humanoid robot icub. In 2008 IEEE International Conference on Robotics and Automation. IEEE, 962–967.Google ScholarCross Ref
- Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016. Progressive neural networks. arXiv preprint arXiv:1606.04671(2016).Google Scholar
- Catherine D Schuman, Thomas E Potok, Robert M Patton, J Douglas Birdwell, Mark E Dean, Garrett S Rose, and James S Plank. 2017. A survey of neuromorphic computing and neural networks in hardware. arXiv preprint arXiv:1705.06963(2017).Google Scholar
- Thomas Serre, Lior Wolf, Stanley Bileschi, Maximilian Riesenhuber, and Tomaso Poggio. 2007. Robust object recognition with cortex-like mechanisms. IEEE transactions on pattern analysis and machine intelligence 29, 3(2007), 411–426.Google ScholarDigital Library
- Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, Vol. 2017-Decem. 2991–3000. arxiv:1705.08690Google Scholar
- Sumit Bam Shrestha and Garrick Orchard. 2018. SLAYER: Spike Layer Error Reassignment in Time. In NeurIPS.Google Scholar
- Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. Advances in neural information processing systems 30 (2017).Google Scholar
- Ghada Sokar, Decebal Constantin Mocanu, and Mykola Pechenizkiy. 2021. SpaceNet: Make Free Space for Continual Learning. Neurocomputing 439(2021), 1–11. https://doi.org/10.1016/j.neucom.2021.01.078 arxiv:2007.07617Google Scholar
- Kenneth Stewart, Garrick Orchard, Sumit Bam Shrestha, and Emre Neftci. 2020. Online few-shot gesture learning on a neuromorphic processor. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 10, 4(2020), 512–521.Google ScholarCross Ref
- Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence.Google ScholarDigital Library
- Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning. PMLR, 6105–6114.Google Scholar
- Vadim Tikhanoff, Angelo Cangelosi, Paul Fitzpatrick, Giorgio Metta, Lorenzo Natale, and Francesco Nori. 2008. An open-source simulator for cognitive robotics research: the prototype of the iCub humanoid robot simulator. In Proceedings of the 8th workshop on performance metrics for intelligent systems. 57–61.Google ScholarDigital Library
- Gido M. van de Ven, Hava T. Siegelmann, and Andreas S Tolias. 2020. Brain-inspired replay for continual learning with artificial neural networks. Nature Communications 11, 1 (2020). https://doi.org/10.1038/s41467-020-17866-2Google Scholar
- Eleni Vasilaki, Nicolas Frémaux, Robert Urbanczik, Walter Senn, and Wulfram Gerstner. 2009. Spike-based reinforcement learning in continuous state and action space: when policy gradient methods fail. PLoS Comput Biol 5, 12 (2009), e1000586.Google ScholarCross Ref
- Roberto A Vazquez, Bernard Girau, and Jean-Charles Quinton. 2011. Visual attention using spiking neural maps. In The 2011 International Joint Conference on Neural Networks. IEEE, 2164–2171.Google ScholarCross Ref
- QingXiang Wu, T Martin McGinnity, Liam Maguire, Rongtai Cai, and Meigui Chen. 2013. A visual attention model based on hierarchical spiking neural networks. Neurocomputing 116(2013), 3–12.Google ScholarCross Ref
- Xiaohui Xie and H Sebastian Seung. 2004. Learning in neural networks by reinforcement of irregular spiking. Physical Review E 69, 4 (2004), 041909.Google ScholarCross Ref
- Lu Yu, Bartlomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. 2020. Semantic drift compensation for class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6982–6991.Google ScholarCross Ref
- Bianca Zadrozny and Charles Elkan. 2002. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. 694–699.Google ScholarDigital Library
- Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. 2020. Maintaining discrimination and fairness in class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13208–13217.Google ScholarCross Ref
Index Terms
- Interactive continual learning for robots: a neuromorphic approach
Recommendations
Continual Learning of Visual Concepts for Robots through Limited Supervision
HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot InteractionFor many real-world robotics applications, robots need to continually adapt and learn new concepts. Further, robots need to learn through limited data because of scarcity of labeled data in the real-world environments. To this end, my research focuses ...
Effects of Noise on Leaky Integrate-and-Fire Neuron Models for Neuromorphic Computing Applications
Computational Science and Its Applications – ICCSA 2022AbstractArtificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), ...
Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges
Highlights- State of the art on continual / lifelong learning and its implications for robotics.
AbstractContinual learning (CL) is a particular machine learning paradigm where the data distribution and learning objective change through time, or where all the training data and objective criteria are never available at once. The evolution ...
Comments