Event-Based Gesture Recognition through a Hierarchy of Time-Surfaces for FPGA

• Main Authors: Ricardo Tapiador-Morales, Jean-Matthieu Maro, Angel Jimenez-Fernandez, Gabriel Jimenez-Moreno, Ryad Benosman, Alejandro Linares-Barranco
• Format: Article
• Language: English
• Published: MDPI AG 2020-06-01
• Series: Sensors
• Subjects: dynamic vision sensors; event-based; synchronous digital VLSI; HDL; FPGA; pattern recognition
• Online Access: https://www.mdpi.com/1424-8220/20/12/3404

Neuromorphic vision sensors detect changes in luminosity taking inspiration from mammalian retina and providing a stream of events with high temporal resolution, also known as Dynamic Vision Sensors (DVS). This continuous stream of events can be used to extract spatio-temporal patterns from a scene. A time-surface represents a spatio-temporal context for a given spatial radius around an incoming event from a sensor at a specific time history. Time-surfaces can be organized in a hierarchical way to extract features from input events using the Hierarchy Of Time-Surfaces algorithm, hereinafter HOTS. HOTS can be organized in consecutive layers to extract combination of features in a similar way as some deep-learning algorithms do. This work introduces a novel FPGA architecture for accelerating HOTS network. This architecture is mainly based on block-RAM memory and the non-restoring square root algorithm, requiring basic components and enabling it for low-power low-latency embedded applications. The presented architecture has been tested on a Zynq 7100 platform at 100 MHz. The results show that the latencies are in the range of 1 <inline-formula> <math display="inline"> <semantics> <mi mathvariant="sans-serif">μ</mi> </semantics> </math> </inline-formula>s to 6.7 <inline-formula> <math display="inline"> <semantics> <mi mathvariant="sans-serif">μ</mi> </semantics> </math> </inline-formula>s, requiring a maximum dynamic power consumption of 77 mW. This system was tested with a gesture recognition dataset, obtaining an accuracy loss for 16-bit precision of only 1.2% with respect to the original software HOTS.