Share This Post

Featured News

A neuro-inspired artificial peripheral nervous system for scalable electronic skins

A neuro-inspired artificial peripheral nervous system for scalable electronic skins

Abstract

The human sense of touch is essential for dexterous tool usage, spatial awareness, and social communication. Equipping intelligent human-like androids and prosthetics with electronic skins—a large array of sensors spatially distributed and capable of rapid somatosensory perception—will enable them to work collaboratively and naturally with humans to manipulate objects in unstructured living environments. Previously reported tactile-sensitive electronic skins largely transmit the tactile information from sensors serially, resulting in readout latency bottlenecks and complex wiring as the number of sensors increases. Here, we introduce the Asynchronously Coded Electronic Skin (ACES)—a neuromimetic architecture that enables simultaneous transmission of thermotactile information while maintaining exceptionally low readout latencies, even with array sizes beyond 10,000 sensors. We demonstrate prototype arrays of up to 240 artificial mechanoreceptors that transmitted events asynchronously at a constant latency of 1 ms while maintaining an ultra-high temporal precision of <60 ns, thus resolving fine spatiotemporal features necessary for rapid tactile perception. Our platform requires only a single electrical conductor for signal propagation, realizing sensor arrays that are dynamically reconfigurable and robust to damage. We anticipate that the ACES platform can be integrated with a wide range of skin-like sensors for artificial intelligence (AI)–enhanced autonomous robots, neuroprosthetics, and neuromorphic computing hardware for dexterous object manipulation and somatosensory perception.

INTRODUCTION

Electronic skins (e-skins) are essential for sensing human-machine-environment interactions, with applications in advanced collaborative anthropomorphic robots (1) and neuroprosthetics (24). To perform the sensing, e-skins typically consist of numerous tactile sensor elements distributed over a large area substrate that is preferably soft (5), conformable (6), stretchable (710), and lightweight (11). Similar to biological skins, the ideal e-skin should be highly responsive and capable of resolving millisecond-precise tactile stimuli to facilitate rapid discrimination (12, 13). This enables emerging dexterous robots the ability to react quickly to highly localized and transient contact events, such as a prick from a needle or slippage of an object. The sensors of the ideal e-skin should also readily scale to thousands in number and be distributed with variable spatial densities depending on the sensing needs while having minimal wiring requirements (14). Furthermore, an e-skin that remains functional while being subject to physical harm enables continuous sensor feedback for safe robotic motor controls and decision making. Although several pioneering efforts have been made to achieve some of these desirable traits, there is currently no technology that encompasses all of the stated attributes.

One major reason is because a vast majority of tactile sensor arrays are currently interfaced via time-divisional multiple access (TDMA), where individual sensors are sampled sequentially and periodically to reconstruct a two-dimensional (2D) map of pressure distribution. Although the serial readout nature of TDMA allows conductor traces to be shared across multiple sensors, which simplifies the wiring of large arrays, it consequently leads to a decline in readout rates as the number of sensors in the array increases. Existing solutions to boost the sampling rate of TDMA-based e-skins include use of high-speed electronic components (15, 16), intelligent subsampling (17), and data compression techniques (18). Unfortunately, such approaches present challenges to scaling to the thousands of sensors needed to sensorize large exterior areas of a robot and may necessitate impractically large amounts of computation and power (16). Similarly, intelligent sampling and data compression techniques often depend on having a priori knowledge of the tactile stimuli, which limits broad applicability.

A promising alternative to TDMA is event-based signaling. Unlike TDMA, event-based sensors are not periodically polled by a central electronic controller. Instead, data are transmitted by individual sensors only when necessary (19, 20), similar to biological mechanoreceptors. By capitalizing on the temporal sparsity of tactile signals, event-based representations of touch have been shown to use the available communication bandwidth more effectively in comparison with TDMA-based solutions, thus yielding shorter readout latencies (16, 20, 21). Some vision and auditory sensors have also applied such event-driven approaches (22) using asynchronous protocols such as Address Event Representation (AER) (23). However, these systems require monolithic integrations of numerous electrical traces on silicon substrates, which are challenging to do for large-area, distributed skin-like sensing using flexible and stretchable substrates. Hybrid event-based packet forwarding methods are an interesting approach (16, 20), but such data packets require time stamping and rearrangements at the receiver because they are transmitted using communication protocols that do not guarantee fixed latencies.

Moreover, e-skins are expected to make frequent physical contact with the environment. Thus, it is imperative for the e-skin system to withstand mechanical damages, such as abrasion and cuts, with minimal loss of functionality or repair downtime. The typical n × m sensor matrix (n, rows; m, columns) has limited mechanical robustness because mechanical damage to the electrical trace would disable all the transducers on the entire row/column. Recognizing this limitation, researchers have introduced alternate architectures with redundant components and connections to improve damage resilience of e-skins (24, 25). For instance, sensor nodes configured in a mesh network can reconfigure their routing tables to avoid damaged connections (24). However, individual nodes in such a network require sufficient computational power to handle the overhead of data packet routing and damage repair protocols, thus constraining the level of miniaturization possible. Reconfiguration of larger networks will also incur substantial downtime, potentially impairing sensing when it is needed the most (e.g., during damage events).

In contrast, the human somatosensory system overcomes many of these limitations by coupling action potential signal representations with extensive arrays of nerve fibers in the peripheral nervous system. The ultra-high density of nerve bundles allows mechanoreceptors to use dedicated bioelectronic pathways to the somatosensory cortex (26). In this manner, the latency of human somatosensory system is largely invariant with the number of receptors in the body and limited only by the propagation speed of action potentials or “spikes” (12). When responding to tactile contact events, mechanoreceptors asynchronously transduce ensembles of spikes that represent information through spatiotemporal patterns (12, 13). Unlike TDMA, these spikes are propagated in parallel to the somatosensory cortex (Fig. 1, right) with submillisecond resolution (12). Tactile information is conveyed in a myriad of ways, including spike frequency, spike latency, and phase (27). Damage resilience is excellent because the connections are physically discrete and unaffected by the loss of a particular mechanoreceptor or its axon, enabling continuous functionality of the rest of the system.

Fig. 1 The ACES architecture for large arrays with single-wire transmission capability.

Illustration of ACES artificial receptors on e-skin (left) that independently and asynchronously transduce tactile events into pulse signatures, analogous to biological action potentials, or spikes (right). (A) ACES receptors generate tactile events with spatiotemporal structures (dashed lines) that encode the stimulation sequence. (B) Pulse signatures are combined and propagated via a single conductor. (C) Decoders match pulses in time. A strong match with correlation |T| exceeding a predefined decoder threshold (shaded box) indicates the presence of an event by the particular receptor. The decoded events preserve the spatiotemporal structure (dashed lines) of receptor activation with ultra-high temporal precision, resembling the spike patterns that enable rapid discrimination in the somatosensory system.

Inspired by the human nervous system and motivated by the limitations of existing e-skin technology, we developed a new communication architecture for e-skins that can support thousands of spatially distributed sensors, each capable of asynchronous transmission that requires only a single common conductor for signaling. We term this platform Asynchronously Coded Electronic Skin (ACES). By using a spread spectrum technique, ACES enables an artificial electronic version of the peripheral nervous system for e-skins by multiplexing signals from many sensors to a single receiver (Fig. 1, left).

In our ACES platform, each sensor, referred to as an ACES “receptor,” captures and transmits stimuli information asynchronously as “events” using electrical pulses spaced in time (Fig. 1A and fig. S1A). The temporal arrangement of the pulses, which we refer to as an ACES pulse signature, is unique to each receptor. The spread spectrum (28) nature of the pulse signatures permits multiple sensors to transmit without specific time synchronization, propagating the combined pulse signatures to the decoders via a single electrical conductor (Fig. 1B). The ACES platform is inherently asynchronous due to its robustness to overlapping signatures and does not require intermediate hubs used in existing approaches to serialize or arbitrate the tactile events (19, 20). The ACES signature was designed to be transmitted in 1 ms, similar in duration to a biological action potential (12). At the receiving end, the decoder identifies the transmitting receptor by correlating the received pulses against the known temporal arrangement of pulses for each receptor’s signature. An event from a particular receptor was deemed to be detected if the number of matched pulses exceeded predefined thresholds (Fig. 1C).

RESULTS

Performance of ACES signaling scheme

To replicate the functional role of the biological nerve, the ACES signaling scheme is capable of propagating pulse signatures (events) from thousands of ACES receptors while preserving the relative time differences between the events. To demonstrate the concept, we developed a prototype system using off-the-shelf components to characterize the performance of our ACES communication architecture (Fig. 2A). Using a physical network of 240 ACES receptors, we established that the temporal resolution, defined as the minimum resolvable time difference between two pulse signatures, is <60 ns (Fig. 2B). The transmission latency is also constant, dependent only on the duration of the pulse signature and, importantly, not on the number of receptors. An exceptionally high temporal precision is potentially achievable even for arrays with >10,000 sensors (fig. S1B), despite an increase in temporal uncertainty with an increasing number of overlapping pulse signatures (Fig. 2B), determined using Simulation Program with Integrated Circuit Emphasis (SPICE) simulations (see Materials and Methods).

Fig. 2 Architecture and performance characterization of ACES.

(A) Multiple ACES receptors (blue blocks) are connected to a decoder (orange block) via a single electrical conductor. The red variable resistor represents the resistive sensors used in the current prototype. (B) Temporal precision of decoded events versus number of overlapping pulse signatures for different numbers of pulses per signature (W). Shaded regions indicate SD. (C) The influence of decoder threshold on the detection error tradeoff graph (W = 10). Dashed lines represent simulation results. Error bars indicate 95% confidence bounds for simulated results. (D) Influence of pulse width on DSIR. (E) Effect of W on DSIR (pulse width = 60 ns). Simulations were used to derive (D) and (E).

To establish the reliability of ACES signaling, we observed that by using a decoder threshold of 6 for signatures with 10 pulses (W = 10), a false positive probability of 2.3 × 10−5 and missed detection probability of 3.1 × 10−4 could be achieved even with 240 pulse signatures overlapping temporally (Fig. 2C). Using Monte Carlo simulation, which realistically predicts the performance of our experimental prototype (fig. S1C), we further show that false positive and missed detection probabilities increased to 2.4 × 10−2 and 2.1 × 10−3, respectively, for 1000 overlapping pulse signatures (Fig. 2C). However, the likelihood of signatures overlapping in the thousands was estimated to be rare because of the known sparse nature of tactile stimuli (13, 20). Moreover, we could further improve decoding accuracy by an order of magnitude when the duration of the voltage pulses was halved (Fig. 2D), characterized using the decoded signal-to-interference ratio (DSIR) metric (see Materials and Methods). In addition, we observed that, although larger values of pulse signature weight (W) improved DSIR for <100 overlapping signatures, the opposite was true if a high amount of signature overlap was expected (Fig. 2E). Hence, a value of W = 10 appears to be optimal from our experiments.

A major concern of electronic communications in real-world scenarios is the reliability of the system when exposed to electromagnetic interference (EMI). Hence, we evaluated the effects of two common sources of EMI on ACES: (i) the 13.56-MHz band from radio frequency identification (RFID) devices and (ii) the 2.4-GHz band from Wi-Fi, Bluetooth, and microwave ovens. Using only basic shielding, the effects of 13.56-MHz waves did not significantly impair the signal quality of ACES (fig. S1D). Similarly, the EMI of 2.45-GHz waves had little effect on signal quality (fig. S1E). This is also expected given that 2.45-GHz frequency is beyond the bandwidth of the operational amplifier used. Thus, ACES can function reliably in real-world scenarios without the need for extensive shielding.

Neuromimetic tactile representations using ACES

The ACES signaling scheme is uniquely suitable for encoding biomimetic somatosensory representations (13) because it can propagate the sparse stimuli events from numerous receptors with high temporal precision. We designed a set of biomimetic models using our ACES platform to mimic the fast-adapting (FA), slow-adapting (SA), and temperature afferents by integrating flexible tactile and temperature sensors that communicate through ACES pulse signatures (Fig. 3A). Similar to their biological counterparts (13), ACES-FA receptors respond only to dynamic skin deformations (fig. S2A, see Materials and Methods for details) but are insensitive to static forces, whereas ACES-SA receptors respond to static pressure by producing events at frequencies that increases with greater force amplitudes (Fig. 3B and fig. S2B).

Fig. 3 ACES receptor responses using integrated multimodal sensors.

(A) Schematic of a multimodal ACES sensor array, where biomimetic models convert tactile stimuli to events that are concurrently propagated. (B) Top: Estimation of time-varying force intensities based on ACES-FA and ACES-SA receptor output. Center and bottom: Decoded response of ACES-FA and ACES-SA receptors to a force profile. The ACES-SA receptor modulates pressure intensity by event frequency, whereas the ACES-FA receptor transmits FA+ and FA− events whenever it increases and decreases in pressure are experienced, respectively. (C) Photograph of the flexible multimodal sensor, wrapped onto a finger of a robotic hand grasping a cup of hot coffee. Inset: Photograph of the sensor. (D) Raw events generated from pressure-sensitive (top) and temperature-sensitive (bottom) ACES receptors during the grasp. (E) Derived pressure and temperature profiles from (D). (F) Snapshots of pressure and temperature distribution before and after grasp onset.

The ACES-SA receptors enable accurate reconstruction of applied forces by extracting the inter-event intervals (Fig. 3B). However, ACES-SA receptors alone could not accurately reproduce transient stimuli, such as a prick from a lancet lasting 1 ms, because the stimulus duration was below the interval needed to encode the force intensity. Instead, we reliably detected the applied impulse using the ACES-FA receptors because they transmit events immediately due to an increase (FA+) or decrease (FA−) in pressure. Our system allows for ACES-FA receptors to also encode for a decrease in stimuli magnitude, as opposed to biological FA receptors that do not discriminate between increase or decrease in force (12). However, ACES-FA output may not accurately track pressure intensities due to the dynamic nature of the threshold used (Fig. 3B and fig. S2A). Combining both FA and SA behaviors, e-skins based on ACES have the versatility to react to a wide temporal range of tactile stimuli.

To demonstrate a multimodal sensing e-skin using ACES, we developed an e-skin integrated with flexible pressure- and temperature-sensitive transducers (Fig. 3C and fig. S2C). We used pressure transducers that have heterogeneous transduction profiles by altering the Young’s modulus of microstructured elastomers (29). This increased the dynamic range of pressure transduction (fig. S2, D to F). We also designed resistive temperature sensors that are most sensitive between 20° and 50°C (fig. S2, G and H), which is similar to the cold receptor afferents in human skin (30). Akin to biological cold receptors, our ACES thermoreceptor transmitted events at a reduced frequency as the surrounding temperature rose above 25°C (fig. S2I).

Combining both transducers on a single sheet of flexible substrate, we show that our ACES platform could perform simultaneous multimodal sensing. We demonstrate that, when an ACES-equipped prosthetic hand grasped a hot cup of coffee, simultaneous detection of thermotactile sensations could be achieved (Fig. 3, C to F).

Slip detection via spatiotemporal ACES events

The ability to preserve the spatiotemporal profile of tactile stimuli allows rapid detection of object slippage, which is essential for grasp stability during in-hand object manipulation (31). Inspired by the simplicity and computational efficiency of optical flow algorithms in event-based vision sensors (32), we implemented a spiking convolutional network that computes the magnitude and direction of slippage in an event-driven manner (Fig. 4A). As an object is pulled out of grasp, ACES-FA receptors are triggered in particular spatiotemporal sequences that could indicate the onset of slippage. By comparing each received event with other events correlated in space and time (see Materials and Methods), our network obtained movement estimates immediately upon slip onset and accurately identified the downward movement of the object (Fig. 4B). The system was also capable of detecting the slippage of a needle with minimal latency (Fig. 4C). Despite the fair accuracy of the speeds derived compared with a high-speed camera, the spike timing patterns can indicate and enable control systems to react with low latencies (<10 ms).

Fig. 4 Spatiotemporal patterns in the ACES-FA receptor response enable rapid slip detection.

(A) Schematic of the event-based algorithm for movement estimation. The input consists of spatiotemporal events from ACES-FA receptors. Each intermediate node computes local movement estimates for every event occurring within its sensitive region. The global movement estimate is computed by pooling responses from intermediate nodes. (B) Detection of slip as a disc is pulled out of grasp. A sudden loss of string tension indicates slip onset. Estimates of movement and direction are shown in the bottom and inset panels, respectively. Blue trace indicates speed derived from optical tracking (see Materials and Methods). (C) Detecting slippage of a needle.

Speed and force invariant classification of texture

Humans typically recognize surface textures by sliding their fingers over them laterally. Studies suggest that the humans can discriminate textures independent of sliding speed (33)—a feat that is difficult to replicate in robotic tactile sensing because controlled tangential speeds are needed to derive stable frequency-based descriptors (21, 34, 35). We propose that our motion estimation technique (Fig. 4A) can be applied to improve texture recognition when sliding speeds are inconsistent.

We constructed a dataset by sliding an array of ACES-FA receptors over gratings of various spatial density by hand, thus ensuring that the sliding speed and normal forces varied within and between trials (Fig. 5, A and B). This substantially increased the problem complexity, because the variability in tangential speed meant that a larger grating pitch did not always trigger receptors less frequently (Fig. 5B). Next, we extracted frequency component information by compiling the distribution of ACES-FA event time intervals into a feature vector for classification by a neural network (see Materials and Methods). Our event-based motion estimation technique was also used to approximate the tangential speed of the recording. We observed that discriminating between the grating sizes was about 14% more accurate when the estimated tangential speed was included as a feature (Fig. 5C). The improvement agrees with our expectation, because the approximated sliding speed tracked the actual speed fairly well. In addition, we observed a decrease in recognition performance from 88 to 50% when temporal resolution of the events decreased from 1 to 40 ms, indicating the importance of having tactile sensors with high temporal fidelity. The sparsity of the ACES-FA representation allowed for exceptional timing precision while consuming only a fraction of the data rates produced by a comparable frame-based implementation (Fig. 5D). Moreover, frequency domain information was derived directly from the inter-event time intervals, unlike frame-based systems where complex operations such as Fourier transformations are typically required (36).

Fig. 5 Feature computation and classification of grating pitch.

(A) Features extracted from an example trial where an ACES-FA receptor array is moved over a grating pattern with 4-mm pitch. Red trace in top panel indicates estimated tangential speed using ACES, whereas the dotted blue trace is the ground truth speed derived from frequency components (see Materials and Methods). Bottom panel is a 2D histogram of event frequencies over time. Color codes correspond to bin counts C normalized by total counts ΣC. Magenta box highlights the 100 × 1 vector of bin counts for classification. (B) Selected trials illustrating outputs from five different grating sizes. (C) Classification accuracy of grating sizes obtained when sampled at different rates. Error bars denote SD. (D) Comparison of classification accuracy achievable (with speed estimates) based on data rate consumed. For event-based output, the vertical line in the box plot is the median data rate, the caps indicate 1st to 99th percentiles, and the sides of the box indicate 25th to 75th percentiles of distribution.

Rapid perception of local curvature and object hardness

Dexterous manipulation tasks often necessitate the ability to swiftly perceive object local curvature and hardness from mechanosensory stimuli. In humans, manipulation tasks are typically executed as action phases delimited by mechanical contact events (13). Meissner corpuscles, with their FA responses of submillisecond precision, play a crucial role in rapid and reliable contact classification to ensure seamless transitions between action phases. Using an array of 69 ACES-FA receptors (Fig. 6A) coupled with a spike-based classification technique (see Materials and Methods), we demonstrated the ability to classify various local curvatures up to 10 times faster (<7 ms for 97% accuracy) than a 100–frame per second (fps) conventional sensor array (Fig. 6, B and C). A marked improvement in classification speed was also observed when the ACES-FA receptor output was used to distinguish geometrically identical shapes of different hardness (Fig. 6, D to F). Our results highlight the importance and effectiveness of temporal features in rapid tactile discrimination, which concur with earlier studies in both biological and artificial somatosensory systems (12, 37).

Fig. 6 Classification results from the indentation of objects with various geometry and hardness.

(A) A photograph of the sensor array used in the experiment, where classification speed and accuracy were compared using outputs from ACES-FA and ACES-SA receptors, for rapid indentation of (B) three soft shapes, (C) three rigid shapes, (D) two sharp cones of different hardness, (E) two broad cones of different hardness, and (F) two domes of different hardness. Shapes are shown in insets at top left.

ACES enables flexible arrangements of sensor array

Wiring simplicity in e-skins is critical, especially when routing wires along nonuniform surfaces or curvatures. Our ACES pulse–based communication approach markedly simplifies the problem because it allows signals from all receptors to be transmitted through a single, common electrical conductor. Here, we demonstrate nine ACES-FA receptors that could be dynamically rearranged to form multiple spatial formations as long as contact with a sheet of conductive fabric was maintained (Fig. 7). Thus, our ACES platform enables highly irregular sensor arrangements that accommodate nonuniform geometries.

Fig. 7 Flexible and dynamic reconfiguration of ACES receptor placement.

Photographs of multiple ACES receptors dynamically arranged into three different formations on a sheet of conductive fabric. The plots below each photograph indicate the corresponding FA events decoded when pressure is applied on each formation. Colored boxes highlight the receptors that responded.

Robustness of ACES networks to physical damage

E-skins are vulnerable to wear and tear because they constantly rely on physical contact to sense the environment. Our ACES platform enables sensor arrays to have high signaling redundancy when interfaced to a common electrical conductor, which translates to exceptional robustness against physical damage. We exemplify this level of resilience by showing that an array of 16 ACES-SA receptors retained full functionality even when the substrate was cut at multiple sites (Fig. 8, A and B, and movie S1). In comparison, a conventional row-column multiplexed tactile sensor array experiencing similarly large damage could not sense at most locations (Fig. 8, C and D, and movie S2). Unlike network-based approaches (24), ACES networks do not require reconfiguration when damaged and hence experience zero down time (movie S1).

Fig. 8 Damage resilience of ACES architecture compared with a conventional row-column multiplexed array.

(A) Photographs show the array before (top left) and after damage (bottom left), where the stretchable polyurethane substrate embedded with ACES-SA receptors was cut at three different locations. The corresponding pressure distributions reconstructed from ACES receptor outputs are shown to the right of each image. (B) Schematic of signal propagation path on damaged substrate of ACES receptor array. (C) Photographs of the four-by-four multiplexed sensor array before and after being cut. The plots to the right of each photograph indicate the sensor readout corresponding to the physical state of the sensor. (D) Schematic representation of the multiplexed sensor array. Each sensor element is represented by a variable resistor. Dashed lines indicate traces affected by the damage.

DISCUSSION

ACES is designed to be a communication technique analogous to nerve bundles in biological skin. Toward this goal, our ACES electronic peripheral nervous system architecture presents several advances in e-skin technology. First, ACES uses a novel spread-spectrum technique to asynchronously multiplex signals from a large number of transducers while achieving near-constant latency despite increasing the number of transducers. Therefore, larger, full-body e-skins with thousands of spatially distributed sensors can be realized without compromising system responsiveness to stimuli.

Second, being event based, our ACES platform enables tactile signals to be captured with ultra-high temporal fidelity, because the sampling time is not constrained to a central “clock.” This enables precise time recording of tactile events that enable rapid slip detection and shape and material hardness classifications and also provides movement estimates that enhance speed-invariant texture discrimination. Third, ACES allows signals from all receptors to propagate via a single shared conductor, empowering roboticists and sensor system designers with outstanding levels of flexibility when spatially arranging sensors on the e-skin. Furthermore, the high redundancy of physical connections when a planar conductor is used ensures damage robustness of ACES sensor arrays—the system is shown to operate uninterrupted even when multiple mechanical cuts were being made to the substrate.

Having tactile signals that are millisecond precise may appear excessive and unnecessary, especially if the time frames for physical motion of the intended robot are not particularly fast. However, recent works in both biological and artificial tactile sensing have shown that significantly more information is present in the temporal domain of tactile signals than the rate of movement suggests (12, 13). For instance, vibratory signal above 100 Hz are often used to characterize slip events (15), whereas the high-frequency signal components generated when sliding tactile sensors over surfaces typically describe texture (21, 38). In the human skin, there are specialized mechanoreceptors that respond to transient stimuli from 50 to 500 Hz (13). Therefore, harnessing tactile signals with millisecond precision will enhance texture recognition and dexterous manipulation capabilities while ensuring safety in human-robot interactions. Moreover, the event-based nature of ACES allows e-skins to achieve impressive temporal resolutions at a fraction of the data rates needed by existing systems.

ACES appears to have similarities with AER, which is an asynchronous protocol used in neuromorphic devices (23). However, ACES and AER are fundamentally dissimilar and serve different niches. First, AER is a point-to-point protocol, simulating a network of interconnected neurons. Events from a neuron are routed to another particular neuron by matching the address with a routing table. It is thus possible to have bidirectional communication in AER. Conversely, ACES is a many-to-one protocol, where there is only one receiver in the network. Therefore, information can only flow from the sensors to the receiver. ACES is thus more suitable for sensing applications, whereas AER may be used in both sensing and computational networks.

Second, AER is time multiplexed, and arbitration is necessary to ensure proper sharing of the address bus. Most implementations use flow control to ensure that the receiver is ready to receive the next event. AER thus have very low transmission error rates but require additional logic for arbitration and handshaking. Most systems today use field programmable gate arrays (FPGAs) for AER interfaces, which leads to larger and more complex devices. In contrast, ACES can “fire-and-forget” with no need for any arbitration or flow control. However, ACES is not error free, and one potential disadvantage is that it is impossible for the transmitter or receiver to know whether a loss of sensing event has occurred. Nevertheless, the simplicity of ACES enables simple transmitter implementations and exceptional timing precision of stimuli events. Tiny sensor nodes embedded in elastic substrates can be used to implement the ACES platform at relatively high density and sensing speed, which are especially important for e-skin applications where real-time motions and mechanical compliance are critical.

Third, most implementations of AER require numerous wires to realize, from four wires per sensor in serial AER (39) and up to 32 wires in parallel AER (40). Although this enables AER systems to have superior throughput (typically millions of events/s) compared with ACES (hundreds of thousand events/s), it requires high-density interconnects fabricated in silicon. For e-skins, replicating such patterns over large areas on flexible and stretchable substrates remains impractical. Therefore, in e-skin applications, our ACES’s single-shared conductor approach is highly desirable.

Last, our ACES platform can complement AER, rather than being mutually exclusive. For example, we can describe ACES as the nerve bundles from mechanoreceptors (sensors) to the spinal cord (communications bus), where events received can then be transported via AER from the spinal cord to the brain (central microprocessor) of the robot. Such a hierarchy permits compliant and damage robust e-skin patches to be wrapped over the exterior of the device, complemented by high-throughput AER networks at the robot interior where the use of rigid components are more palatable.

The use of one microcontroller per receptor in current prototypes is currently more complex when compared with n × m type cross-bar sensor arrays, inevitably imposing upper limits to the network in terms of spatial density and power savings. However, similar limitations apply to any sensor array that uses local computation and communication. For instance, Roboskin (41) on the iCub has 12 elements per 390-mm2 triangular tile (32.5 mm2 per receptor), whereas Hex-o-skin (42) has 3 elements per 665-mm2 module (221 mm2 per receptor). With a density of 9 mm2 per receptor (fig. S3A), ACES compares favorably with the aforementioned systems in terms of achievable spatial density.

Our prototype consumes about 7 mW per receptor at 3.3 V (see Materials and Methods). Similar e-skin architectures consume from 0.2 to 55 mW per sensor (4143). It is important to note that our reported power consumption should be seen as an upper bound, because these microcontroller-based prototypes were meant to be an early demonstration of ACES using off-the-shelf components. We anticipate smaller-sized receptors with much lower power consumption to be achievable through the use of application-specific integrated circuitry (ASIC). ASICs each representing several ACES receptors per chip are another viable option for regions of e-skin where extremely high densities of receptors are desirable.

A key benefit of ACES-enabled e-skins is the unique combination of high mechanical robustness and wiring simplicity. The damage robustness of an ACES network stems from the physical redundancy of connections from each receptor to the receiver. When a planar conductor, such as a conductive fabric or stretchable electrodes, is used to link the receptors to the receiver, each receptor has multiple direct connections to the receiver. The ACES sensor network thus retains full functionality as long as there exists at least a physical connection between each receptor to the receiver. Damage to the conductive substrate can be interpreted as merely a change in the shape of the substrate and will not affect the system functionality (movie S1).

The fire-and-forget nature of ACES pulse signatures also ensures that individual receptors are plug and play. Multiple patches of ACES e-skins are easily combined by connecting the substrates together. Similarly, gross damage that disconnects several receptors from the system will not affect the functionality of the remaining intact connected network. Although the receiver may not immediately realize the loss of the disconnected receptors, additional protocols built on top of ACES can be implemented to identify missing receptors. For instance, disconnected SA receptors may be detected if no signatures from these receptors are received after a specified timeout period. The most severe damage that could occur would be the shorting of the signal carrier to the supply voltage or ground traces, which would render the entire network inoperable. This affects any other wired communication systems, and the vulnerability could be minimized through proper design of the system encapsulation.

The assembly of thousands of microcontrollers in a network may appear to be complex in comparison with n × m matrix–type sensor arrays. Although this is true for rectangular and uniformly distributed sensor arrays, the complexity of fabricating matrix-type arrays rapidly increases if stretchable e-skins with nonuniform densities are desired. In contrast, the bulk of the manufacturing complexity for ACES-based networks is the assembly of common electronic components, for which established volume manufacturing processes are available. Hence, in terms of manufacturing complexity, ACES-based networks can be implemented more conveniently for e-skins with nonuniform shapes and densities.

The mechanoreceptor models implemented in this paper are relatively simplistic when compared with the more sophisticated biological models reported (44). However, our goal is not to reproduce biologically accurate tactile responses but to capitalize on ACES’s high temporal resolution for efficient information transfer. ACES events, similar to biological action potentials, are slow to propagate individually (1 ms), but these events can be propagated in parallel. If each event were to represent a single bit of a digitally encoded resistance value, it would require eight events (assuming 8-bit resolution) to be transmitted consecutively, resulting in an 8-ms latency. Theoretically, this latency can be reduced to 2.015 ms using ACES-SA encoding, where two events spaced 15.36 μs (60 ns × 256) apart effectively encode the same 8-bit resistance value. Moreover, current research efforts suggest that classification tasks can be rapidly performed even without knowledge of actual pressure values by relying on relative time differences of events from a population of receptors (12, 13, 37). ACES-FA encoding is designed to meet these requirements. Nevertheless, the programmable nature of individual ACES receptors ensures that receptor models with increased efficiency and/or biologically relevant mechanoreceptor output can be simulated in the future.

By using our ACES architecture, one potential disadvantage is the increase in computational complexity required in the receiver/decoder. Similar to the human brain, concentrating power-hungry operations at the receiver makes practical sense, because this design allows the sensors (transmitters) to be realized with low-cost and low-power hardware.

In summary, our neuro-inspired ACES platform enables highly scalable and ultra-fast somatosensory perception, with damage robustness and sensor placement flexibility that rival wireless solutions. We anticipate that our ACES architecture will help advance human-machine-environment interactions for autonomous anthropomorphic robots (2), naturalistic embodiment of neuroprostheses (2, 4, 45), high-performance brain-machine interfaces (46, 47), and soft machines (48).

MATERIALS AND METHODS

Objectives and study design

Our objectives were to show that (i) the ACES architecture have the capacity to multiplex asynchronously transmitted events from numerous sensors and reconstruct the output at the receiver and (ii) the high temporal resolution of tactile events afforded by our technique can be uniquely applied to solve various challenges in e-skin applications.

Design of ACES pulse signatures

We designed a set of electrical voltage pulse codes that can be asynchronously added with a low probability of false decoding. Each pulse signature consists of W voltage pulses spaced apart at specific time instances τ = {τ1, τ2, … τw–1}. At the receiving end, pulses are received at time instances τ′ = {τ1′, τ2′ … τv′}. The decoder finds the intersection T = (ττ′) where the cardinality |T| denotes the correlation strength. If |T| exceeds a predefined decoder threshold, then an event is deemed to have been received (Fig. 1C).

An ideal set of pulse signatures is one that has minimal autocorrelation and cross-correlation. There should also be enough unique signatures in the set to identify all receptors in the array. A family of pulse signatures may be characterized by three parameters: F, the maximum number of voltage pulses that fit within the duration of a signature; W, the weight or the number of pulses per signature; and L, the maximum allowable interference (autocorrelation and cross-correlation) between two signatures.

For a pulse signature with a duration of Ts = 1 ms with voltage pulses lasting Tp = 100 ns each, the maximum number of voltage pulses that fit within the duration of a signature is calculated as followsF=TsTp=10,000

Parameters W and L are closely related to the error performance of the decoder. Under ideal conditions, all W voltage pulses from the target signature should match the template; thus W is the maximum signal strength. L is the maximum interference (cross-correlation) allowed between signatures of the same family. If N nontarget signatures overlap, the amount of interference could be as high as N × L. Error performance will thus degrade if N × L is much larger than W.

There needs to be as many unique pulse signatures as there are ACES receptors. To accommodate for the thousands of ACES receptors, the pulse signatures used in this paper have L = 2 such that the number of signatures (C) will follow the inequality (49)C≤(F−1)(F−2)W(W−1)(W−2)

Therefore, with F = 10,000 and W = 10, the array could accommodate up to 138,847 receptors, which should be sufficient for whole-body robot sensor skins (50). Several techniques for finding these codes have been documented (51).

The voltage pulses in a pulse signature can either be positive or negative, which allows each signature to take on several variants. For instance, ACES-FA receptors use two variants of the same signature to indicate an increase or decrease in pressure.

Implementation of an ACES receptor array

Each receptor consists of a resistive sensor, a microcontroller, and several passive components to perform signal conditioning (Fig. 2A). A potential divider circuit converts sensor resistance to voltage values. The voltage is then sampled at 10 KHz with 10-bit resolution by the analog-to-digital converter (ADC) on the ATtiny20 microcontroller (Microchip Technology, United States). The values are sent to firmware models to mimic the fast (FA) or slow (SA) adaptation behavior of receptors found in the human skin.

The ACES-FA model generates an event whenever a measured voltage has changed more than 50 mV since the last transmitted event. A positive/negative pulse signature is transmitted for a pressure increase/decrease event, respectively. After transmitting the event, a new voltage baseline is set (fig. S2A). To mimic SA mechanoreceptors, the model generates events at intervals proportional to the 8-point averaged ADC value. For Fig. 3, each bit corresponds to a 100-μs interval. Together with a 1-ms pulse signature duration, a maximum ADC value of 1023 will correspond to a 103.3-ms interval between events. Receptors with the SA-mimicking model only transmit positive versions of the pulse signature.

The pulse signature is generated by toggling a digital pin at specific time intervals. A capacitive high-pass filter is used, ensuring that only the high-frequency components of the signal are transmitted, in the form of voltage pulses (fig. S1A). An inverted summing circuit, constructed using an OPA354 operational amplifier (Texas Instruments, United States) is used to combine pulses from the multiple receptors. The resultant signal is digitized with 8-bit resolution at 125 MHz using an oscilloscope (PicoScope 3406D). Decoding is performed offline (MATLAB). Voltage thresholds V+ and V− (fig. S1A) are heuristically defined in software to be 40% of the amplitude of a single pulse, above and below 0 V, respectively. We found this to reliably distinguish a pulse from background noise.

For the purpose of controlled experimentation, we developed a modular prototype system consisting of a board with 80 ACES receptors. The transducer arrays, namely, a flexible 80-element transducer for slip detection and grating classification (Fig. 3C and fig. S3B) and a rigid 69-element transducer for local curvature and hardness classification (Fig. 6A and fig. S3C), are separate entities that can be attached to the board for data collection. An additional prototype of 25 receptors connected by serpentine interconnects (fig. S3A) was also developed for the purposes of movie S1.

Effects of EMI on signal

We induced EMI of 13.56 MHz using a commercial RFID card reader (RF430FRL15xH by Texas Instruments). We evaluated the condition when the antenna was placed 1 cm above the summing amplifier and when the antenna was placed 1 cm above a 15 cm by 15 cm conductive fabric used for propagating the ACES signatures. We also evaluated the situation when a grounded aluminum foil sheet was placed between the antenna and the system for the aforementioned antenna placements. The signal-to-noise ratio, computed as the ratio of peak-to-peak voltage of the pulse signature to the peak-to-peak voltage of background noise, was measured for all four scenarios. The 2.4-GHz interference was projected from a TG.30 antenna (Taoglas Antenna Solutions) driven by a SMB100A signal generator (Rohde & Schwarz). The antenna was placed 20 cm above the ACES system without shielding.

Determining DSIR and verification of timing precision

The physical hardware test setup consisted of an array of 240 receptors, each programmed with a unique pulse signature. Each trial began with an external digital edge signal that was broadcasted to all receptors, triggering a request for transmission. Upon receiving the trigger, each receptor transmits its pulse signature after a random delay of less than 1 ms, thus ensuring that all the 240 transmitted signatures will overlap at varying temporal offsets between trials. For trials involving less than 240 receptors, the excluded receptors were programmed to ignore the trigger.

Receptors (16 of 240) have recording probes attached to their transmission pins. Signals from these probes served as the ground truth on the actual time of the pulses transmitted. The digital signals from these 16 probes, as well as the combined pulse signatures from the 240 receptors, were digitized at 125 MHz simultaneously by a mixed signal oscilloscope (PicoScope 3406D), thus ensuring that all channels were synchronized in time.

We defined the DSIR metric to characterize the ease of an event detection in the presence of interference. The interference value is taken to be the root mean square value of the decoder correlation |T| during the 1-ms transmission window of the corresponding receptor. The last 100 ns of the decoder output was excluded because it corresponded to the detection of the correct pulse signature and should not be considered as interference. DSIR was then computed as the ratio between the signature weight W and the interference value. The DSIR was computed for the decoders responsible for the 16 probed receptors. For each network size, 1000 trials were conducted, and the reported DSIR was averaged from the 16 decoder outputs across all 1000 trials.

Timing precision was obtained as the difference in time between the start of last pulse transmission, as obtained from the attached probe, and the transmission time, as determined from the decoder output. The measured time difference is thus independent of the mechanical inertia of the transducer and processing time of the receptor model. The reported timing precision in Fig. 2A was obtained as an average across all the 1000 trials for each network size.

Monte Carlo simulations

Monte Carlo simulation was used to obtain estimates of DSIR for network sizes above 240 receptors. For each trial involving a simulated array of N receptors, we additively combined N pulse signatures at random offsets of <1 ms. The resultant signal was then decoded using the same software decoder as the physical experiments. One thousand Monte Carlo trials were performed for each network size.

SPICE simulations

The temporal precision of an ACES receptor array was limited mainly by the duration of a single voltage pulse. As more receptors are added to the array, the capacitance of the electrical conductor in which the pulses propagate also increases. The increased capacitance results in reduced phase margin of the op-amp feedback loop and causes ringing in the output (fig. S4A). The ringing can be reduced by increasing the feedback capacitance (CF in fig. S4B) to improve stability. However, pulse width also increases as a consequence. Hence, a SPICE simulation (Cadence Spectre) was used to determine how the pulse width changes with increasing number of receptors. The simulated circuit is shown in fig. S4B. The input from each receptor was modeled as a voltage source with a square wave. The edges of the waveform were high-pass–filtered to obtain the waveform of the voltage pulse. A transient simulation was run for N = 200 to 16,000. For each N, the value of CF was swept to find the minimum CF that has acceptably low levels of ringing (i.e., the overshoot does not exceed quantization threshold set at 40% of pulse amplitude). Last, the resultant pulse width was taken as the length of time in which the voltage remained above quantization threshold. The temporal precision of the system was interpreted as 0.5× pulse width.

ACES receptor response to pressure and prick

The analog output channel of a load cell (Mark-10) was connected to the ADC pin of an ACES-FA receptor and an ACES-SA receptor, as well as a recording channel on an oscilloscope, to serve as ground truth (PicoScope). The summed signal from both receptors was simultaneously recorded using a separate channel on the same oscilloscope. Two mechanical stimuli were presented, the first being a finger press, and the second being a prick by a lancet using a lancing device. The load cell reading in Fig. 3B was smoothed using a moving average filter of eight points (OriginLab 2017). Reconstruction traces were computed offline using MATLAB.

Slip detection

An object (an acrylic disk with a diameter of 1 cm in Fig. 4B and a needle with a diameter of 0.8 mm in Fig. 4C) was held vertically between two flat surfaces of a bench vise. A pressure-sensing array with 80 sensors (fig. S3B), interfaced with an array of ACES-FA receptors, was pasted on one of the surfaces between the vice and the object. A thread connected the object to a load cell (Mark-10 5i). A separate piece of thread, connected to the opposite side of the load cell, was pulled to make the object slip out of the vice. The combined signal output from the 80 ACES receptors was sampled together with the analog output of the load cell at 125 MHz using an oscilloscope (PicoScope 3406D). Computation of slip detection was processed offline in MATLAB.

The computation of local movement estimates (Fig. 4A) was generally as follows:

1) For an event from a particular receptor A that occurred at t0, look for prior events from receptors within distance D of receptor A that occurred at tprior where t0 − Δt < tprior < t0.

2) For each prior event, compute movement magnitudeMagnitude=Dt0−tprior

3) Movement direction for each prior eventDirection=a tan 2(dy,dx)where dx and dy are the x and y components of the distance D.

4) By averaging the magnitude and direction for each prior event, the local movement estimate at receptor A’s location was obtained.

The global movement estimate was obtained as the moving average (exponential kernel of 5-ms time constant) for all the local movement estimates. For Fig. 4 (B and C), the time interval (Δt) was 5 ms and the distance (D) was 2 mm. Ground-truth information was obtained using an optical camera recording at 240 fps to track a marked spot on the thread during the experiment.

Grating classification

Rectangular holes of 40 × X mm were laser-cut from 5-mm-thick acrylic sheets, where X denotes the pitch of the grating. A pattern consists of multiple holes were spaced X mm apart. Five grating pitches were used (X = 2 to 6 mm in 1-mm increments). Each pattern was at least 25 cm in length. An 80-element transducer array (fig. S3B) was mounted on a 5-mm-thick piece of polyurethane rubber for compliance (Ecoflex 00-30, Smooth-On). To achieve realistic stimulus conditions, we moved the array over the gratings by hand at varying tangential speeds and normal forces. Fifty trials were collected per pattern, each lasting at least 1 s. To extract frequency domain information, we compiled inter-event time intervals for each ACES-FA receptor into a histogram of 100 bins (1 to 100 ms in 1-ms bins). The histograms were computed from moving time windows of 100-ms duration in steps of 10 ms each. Thus, a 1-s trial generates 91 histograms. Speed estimates were computed using the same technique as the slip detection with Δt = 20 ms and D = 2 mm and averaged within each time window. A secondary speed estimate was computed by multiplying the event frequency corresponding to the highest bin count with the grating pitch used. This second estimate serves as the ground truth (blue dashed lines in Fig. 5, A and B).

We classified individual histograms for grating pitch using a multilayer perceptron with 40 hidden neurons and 5 output neurons (nntool, MATLAB). Input to the network was a 100 × 1 vector of bin counts from the inter-event time histogram (magenta box in Fig. 5A). For the case with speed estimates, a 101 × 1 vector was used, where the first 100 elements were the bin counts and the last element was the estimated speed. In all cases, the scaled conjugate gradient backpropagation (trainscg) algorithm was used for training. One randomly selected trial per grating was excluded for each training instance and used for test. Results reported in Fig. 5C were compiled from 20 training instances. To simulate reduced temporal resolution, we binned time stamps of the events at discrete values that are multiples of the reported temporal resolution.

To compare data rates (Fig. 5D), we assumed a single frame to consume 12 bytes (80 bits for 80 receptors + 1 byte escape character + 1 byte delimiter). An event was assumed to be 1 byte long (7 bits address, 1 bit data). Statistics for the event-based representation were obtained by counting the number of received events in all the 100-ms time windows acquired.

Local curvature and hardness classification

The shapes used for indentation are (i) a spherical dome of 10-mm radius, (ii) a broad right circular cone of 7.5-mm height and 10-mm base radius, and (iii) a sharp right circular cone of 7.5-mm height and 2.5-mm base radius. Each shape is 3D printed twice, once using a rigid RGD810 VeroClear material and a duplicate using the softer FLX980 TangoBlackPlus material (fig. S5A). A 3D printer (Objet260 Connex) was used to produce the shapes. (See PolyJet Materials Data Sheet for the material properties.)

The indentations were performed using a mechanized z-axis stage (Newmark) with force feedback from the force gauge (Mark-10 5i) at a speed of 50 mm/s. The depth of indentation of the softer shapes were fixed at 2800 μm, whereas the hard shapes were indented to depths that produced the same forces as their softer counterparts (~9 N for the sphere, ~2 N for the broad cone, and ~0.3 N for the sharp cone). The shapes were indented onto the center of a circular array of 69 sensors interfaced with an array of ACES-FA receptors covered with a 1-mm-thick elastomer sheet (fig. S5B). Each shape was indented 10 times, with a 5-s interval between indentations to allow the deformable material to recover. Data from the force gauge and the ACES receptor array were recorded simultaneously at 125 MHz using an oscilloscope (PicoScope). All the trials were repeated with the sensors interfaced with ACES-SA receptors to obtain the pressure distribution.

The ACES-SA receptors were programmed to have an inter-event interval of 10 μs per ADC bit. Therefore, each receptor will generate events at a rate of about 100 Hz and will thus be sufficient to model a conventional 100-fps pressure sensor array.

The output from ACES-FA receptors were classified on the basis of the Van Rossum spike distance measure (52), commonly used to quantify the similarity between spatiotemporal spike patterns. Events from each receptor are convolved with a double exponential kernel of 5-ms decay, thus yielding a continuous trace. Traces from the same receptor can then be compared between trials using the Euclidean distance. The distance between any two trials was then calculated as the sum of the differences between individual traces. Last, a K-nearest neighbor algorithm (53) (with K = 5) was used to classify the trials based on the summed distances.

To classify the output from ACES-SA receptors, we first reconstructed traces of pressure intensity over time for each receptor based on the time interval between events. The traces were then decimated at 10-, 20-, and 40-ms intervals to mimic 100-, 50-, and 25-fps sensors, respectively. Similar to ACES-FA receptors, the Euclidean distance was used to compare the pressure traces from the same receptor between trials, and the sum of differences between individual traces defines the distance between any two trials. The K-nearest neighbor algorithm (K = 5) was then used to classify the trials based on their distances.

Free-form receptor arrangements

To demonstrate the flexible arrangement of ACES receptors on a single electrical conductor, we developed ACES receptors with their own battery power source. The receptors and the decoding circuit were connected only by a stretchable conductor (knit jersey conductive fabric, Adafruit). Pressure was applied by pressing a conductive rod onto the receptors. The conductive rod provides a charge return path, such that charges from the environment can flow back to the receptors by coupling with the human operating the prototype. The same effect could have been achieved through the use of a grounded conductive encapsulant. However, this approach was omitted to achieve better clarity of the demonstration.

Robustness against severe damage

Battery-powered ACES receptors, connected together with a stretchable conductive fabric (knit jersey conductive fabric, Adafruit), were encapsulated in stretchable silicone rubber (Ecoflex 00-30, Smooth-On). A stretchable coat of silver ink (PE873, DuPont) and encapsulant (PE73, DuPont) was applied over the rubber via screen printing and grounded to provide the charge return path.

To construct the conventional cross-bar multiplexed sensor array used in the comparison, we fabricated two flexible printed circuit boards (PCBs) to form the row and column traces. A piezoresistive layer (Velostat, 3M) was sandwiched between the PCBs. Each intersection between a row and a column formed a pressure-sensitive element. Traces from the PCBs were connected to an ATmega328 microcontroller (Atmel). Software running on the microcontroller polled each sensor element sequentially to obtain the pressure distribution of the array. Figure 8D illustrates the circuitry used. Because of the simplicity of the readout circuit, some cross-talk will be expected (54). Nevertheless, because the array is relatively small, the cross-talk did not affect the results significantly.

A ring-shaped acrylic object was pressed onto the sensor arrays to deliver the stimulus. We cut the sensor arrays using a pair of scissors to cause damage.

Power consumption measurements

Using a source measurement unit (Keithley 2450), we measured the power consumption of a prototype board of 80 ACES receptors at 3.3-V operating voltage. The receptors were programmed with ACES-SA behavior, and results were an average from a 10-s observation. We did not observe significant differences in power consumption when ACES-FA behavior was implemented.

Fabrication of transducers

Details of material fabrication and characterization are available in texts S1 and S2. Figure S2F was obtained using an automated microindenter (FemtoTools).

SUPPLEMENTARY MATERIALS

robotics.sciencemag.org/cgi/content/full/4/32/eaax2198/DC1

Text S1. Pressure sensor fabrication and characterization

Text S2. Temperature sensor fabrication and characterization

Fig. S1. Additional characterization of ACES signaling.

Fig. S2. Characterization of transducers.

Fig. S3. Example prototypes of ACES sensor arrays.

Fig. S4. SPICE circuit used for simulation.

Fig. S5. Setup for local curvature and hardness classification.

Movie S1. A typical 5 × 5 cross-bar sensor array subjected to physical damage.

Movie S2. Robustness of an ACES sensor array to physical damage.

REFERENCES AND NOTES

  1. T. Asfour, T. Asfour, J. Schill, H. Peters, C. Klas, J. Bücker, C. Sander, S. Schulz, A. Kargov, T. Werner, V. Bartenbach, ARMAR-4: A 63 DOF torque controlled humanoid robot, in Proceedings of the 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), (2015) pp. 390–396.

  2. C. Schürmann, M. Schöpfer, R. Haschke, H. Ritter, A high-speed tactile sensor for slip detection, in Springer Tracts in Advance Robotics (Springer, 2012), vol. 76, pp. 403–415.

  3. W. W. Lee, S. L. Kukreja, N. V. Thakor, A kilohertz kilotaxel tactile sensor array for investigating spatiotemporal features in neuromorphic touch, in Proceedings of the 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS) (2015), pp. 1–4.

  4. O. Oballe-Peinado, J. A. Hidalgo-Lopez, J. A. Sanchez-Duran, J. Castellanos-Ramos, F. Vidal-Verdu, Architecture of a tactile sensor suite for artificial hands based on FPGAs, in Proceedings of the 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) (2012), pp. 112–117.

  5. C. Bartolozzi, C. Bartolozzi, P. M. Ros, F. Diotalevi, N. Jamali, L. Natale, M. Crepaldi, D. Demarchi, Event-driven encoding of off-the-shelf tactile sensors for compression and latency optimisation for robotic skin, in Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017), pp. 166–173.

  6. F. Bergner, P. Mittendorfer, E. Dean-Leon, G. Cheng, Event-based signaling for reducing required data rates and processing power in a large-scale artificial robotic skin, in Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2015), 17 December 2015, pp. 2124–2129.

  7. C. Bader, F. Bergner, G. Cheng, A Robust and Efficient Dynamic Network Protocol for a large-scale artificial robotic skin, in Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2019), pp. 1600–1605.

  8. D. Um, V. Lumelsky, Fault tolerance via component redundancy for a modularized sensitive skin, in Proceedings of the1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C) (1999), pp. 722–727.

  9. S. S. Baishya, B. Bäuml, Robust material classification with a tactile skin using deep learning. IEEE International Conference on Intelligent Robots and Systems, pp. 8–15 (2016).

  10. J. A. Fishel, G. E. Loeb, Sensing tactile microvibrations with the BioTac—Comparison with human sensitivity, in Proceedings of the 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) (2012), pp. 1122–1127.

  11. P. M. Ros, M. Crepaldi, C. Bartolozzi, D. Demarchi, Asynchronous DC-free serial protocol for event-based AER systems, in Proceedings of the 2015 IEEE International Conference on Electronics, Circuits, and Systems (ICECS), pp. 248–251.

  12. D. B. Fasnacht, A. M. Whatley, G. Indiveri, A serial communication infrastructure for multi-chip address event systems, in Proceedings of the 2008 IEEE International Symposium on Circuits and Systems (2008), 648–651.

  13. G. Cannata et al., Modular Skin for Humanoid Robot Systems. CogSys 2010 Conf. Proc. 231500, 231500 (2010).

  14. R. A. M. Chandra, S. Chauhan, thesis, Uttar Pradesh Technical University, Lucknow, India (2015).

Acknowledgments: We thank T. Sun for microfabrication assistance, H. Godaba for help in micromechanical testing, and Z. Goh for help with some figure illustrations. Funding: B.C.K.T. acknowledges the support from the Singapore National Research Fellowship (NRFF2017-08), NUS Start-up Grant, and the Singapore National Robotics Programme (NRP). B.C.K.T. and J.S.H. acknowledges grant support from Institute for Health Innovation and Technology and the N.1 Institute for Health, NUS. Author contributions: W.W.L., J.S.H., K.A.N., and B.C.K.T conceived the idea. B.C.K.T. directed the research activities. W.W.L. and B.C.K.T. designed and led the experiments (with inputs from Y.J.T., H.Y., S.L., M.H., H.H.S., and B.X.). W.W.L., Y.J.T., H.H.S., and B.C.K.T. contributed to data analysis and interpretation. W.W.L. and B.C.K.T. wrote the paper, and all authors provided feedback. Competing interests: The National University of Singapore has filed a patent application on this work. Data and materials availability: All data needed to evaluate the conclusions in this paper are present in the paper or the Supplementary Materials. Data and software code can be made available by materials transfer agreement upon reasonable request.

Science Robotics

Share This Post

Leave a Reply