Share This Post

Featured News

Optical lace for synthetic afferent neural networks

Optical lace for synthetic afferent neural networks

INTRODUCTION

Many biological systems have adapted to functionally interact with the external environment by developing an entwined brain-to-body relationship. This complex arrangement allows the latter to perceive the world and the former to interpret it in a closed-loop manner. As robots become more physically complex and cognitively advanced, we can take inspiration from animal nervous systems to create the necessary groundwork for controlling robotic artifacts. Through sight, hearing, smell, taste, touch, and even interpretation of magnetic and electric fields, animals and plants feel a sense of exteroception. Proprioception, knowledge of position and state of their own body parts, is achieved through some combination of these sensory organs as well; animals, in particular, use vibrations and forces felt through mechanoreceptors under skin (1) and through deformation and stress measurements within muscles and tendons (2, 3). From this information, they perform complex tasks, such as maneuvering when blind or altering their behavior when injured.

Presently, autonomous robots mainly use visual and tactile detectors placed at end effectors for exteroception (4) and sensors at joints and the center of mass for proprioception (4) to complete complex tasks. A more distributed sensor network, as in the neural circuitry of animals, would allow robots to interact at higher tactile resolutions and to measure mechanical damage (i.e., bent or broken limbs) accumulated over time. The importance of distributed, volumetric sensing is even more pronounced in the field of soft robotics, where every part of the machine deforms. In these robots, the stretchable sensors distributed throughout the body have only recently been used for feedback control (412).

These stretchable sensors typically use electrical properties [e.g., resistance (4, 13), capacitance (14, 15), voltage (16), and current (17)] in planar arrays (or “skins”) to localize surface deformations (1721). Compared with electricity, light carries information faster and with higher data densities through changes in intensity, phase, polarization, or wavelength (22). It can also be carried in multiple directions simultaneously and is less susceptible to electromagnetic interference (22). As a result, optical sensors provide an excellent alternative to electrical ones (7, 2328). Optical skins have used light loss through bent fibers (2931), fiber Bragg gratings (24), and frustrated total internal reflection (3235) to measure planar shape change but have not been used to directly measure complex, volumetric deformation due to their two-dimensional (2D) fabrication limitations. Stretchable light guides sensitive to tensile strain and external forces through Beer’s law can be easily embedded into a volume but lack the spatial differentiation of skins (7, 23, 2528). The optical sensor network that we present here uses an entirely different principle—frustrated total internal reflection—to measure deformation through light coupled from the physical interaction of neighboring light guides (Fig. 1A, left). By measuring these coupling interactions, we can differentiate between simultaneous multiple and singular deformations throughout a volume.

Fig. 1 Fabrication and design overview.

(A) Examples of OL without supporting structure showing light coupling (circled in red) in a 2D array (left) and a 3D geometry (right). (B) 3D-printed soft scaffolding with embedded channels for elastomeric light guide cores (left). Eight compression tests of the 3D-printed lattice with 1 SD above and below the average in gray (right). (C) Schematic cross section showing the light guides with close-up of the mechanoreceptor (left). LED illuminating the straight input core and light coupling to an output core when deformed (right).

Here, we introduce a platform, which we call optical lace (OL), for creating arbitrary 3D grids of soft, stretchable light guides for spatially continuous deformation sensing (Fig. 1A). This 3D sensory array provided functions similar to those of the afferent neural network in organisms. We distributed these light guide networks throughout the volume of a 3D-printed soft scaffold. To determine the extensive capabilities of this system, we performed mechanical analysis, optical simulation, x-ray imaging, and sensor characterization of our system. We concluded with devices composed of OL that show (i) exteroception that can locate multiple finger presses simultaneously and (ii) proprioception that monitors volumetric structural deformation.

RESULTS

System design

Using a commercially available high-resolution, rapid 3D-printing technique [continuous liquid interface production (CLIP) (36); Carbon M1], we created a scaffold from elastomeric material (EPU 40; fig. S1; see the “System fabrication” section in Materials and Methods) to house a four-cored OL (Fig. 1B, left). We labeled the network as OL1,3, where the notation “1,3” represents one powered input core and three unpowered output cores. Because the 3D-printed structural framework we created is of variable stiffness (to demonstrate arbitrary deformable interfaces), we co-designed the OL1,3 to be at the top of the scaffold, where it is softest (Fig. 1B, right), so force can be best transferred to the sensor in the exteroceptive case.

Our demonstrative structure deformed through the bending and buckling of the struts in a 3D-printed lattice; we programmed the stiffness gradients by varying the strut thickness (t). For example, by making the struts thinner, they bent and buckled more easily, creating a softer section. Equations 1 and 2 show rectangular beam bending and buckling forces (37)Fbuckle=π2Et412l2(1)Fbend=3δEt412l3(2)where E is the elastic modulus, δ is the deflection, and l is the length of a strut. For our printed geometry, this quartic dependence on thickness results in large behavior differences of the softer top layer, t = 0.75 mm, which buckles at Fcomp ≈ 6 N, and stiffer bottom layer, t = 1.125 mm, which bends until Fcomp ≈ 12 N (Fig. 1B, right) and does not exhibit buckling behavior upon compression. This complex response to force is ideal for touch interfaces because a finger press provides sufficient force to completely deform the top, whereas the bottom layer provides substantial resistance to deformation. The difference in stiffness also increases the sensitivity of the OL by transferring forces directly to the mechanosensor, the air gap separating the input and output light guides (Fig. 1C, left). We fixed the cores in place within this area of interest using lattice-work guide channels printed into the scaffold.

Optical array design and lace fabrication

After printing, we threaded the polyurethane light guides [Stretch Magic, Pepperell Braiding Co., MA (23); diameter dout = 1 mm for outputs and din = 1.5 mm for the input; see the “System fabrication” section in Materials and Methods] into the lattice-work channels that loosely held the cores in place. The channel wall’s porosity, ϕair = Aair/(Aair + AEPU) = 0.53, allowed our cores to guide more light by increasing the numerical aperture (NA = 0.174 without porosity and NA = 0.880 with porosity) and critical angle (θcrit = 83.42° without porosity and θcrit = 54.02° with porosity) due to the increased difference in refractive index between the air and printed polyurethane scaffolding and polyurethane cores [ncore = 1.52 (23); nair = 1.0; nEPU = 1.51 (fig. S2); nporous = 1.24 at λ = 850 nm].

The interconnected guide channels allowed the contact and the coupling of light between input and output light guides. The optically powered input core leaked light into the unpowered outputs only when deformations caused the light guides to contact (Fig. 1C, right). In the OL presented here, the outputs were U shaped, lined up next to each other, and optically separated from the input light guide with a small air gap (0.4 mm) in the resting state (Fig. 1C). This configuration allowed the outputs to be parallel to the input core so that line (not point) contact occurred and increased the dynamic signal range through the increasing coupling length with compression.

We measured the power from all the output light guides to interpret the position of localized deformations with photodiodes (see the “Electronics setup for reading signals” section in Materials and Methods). In the exteroceptive structures, we designed the receptive field of each mechanosensor to be comparable with the size of a typical fingertip measured by a ruler, w1 = 6.4 mm (fig. S3), so that a press centered on the output would produce only one signal for simple processing. To measure the location of touch continuously over a distance, we placed the outputs close together (w2 = 8.35 mm; fig. S3) so that the receptive fields overlapped, allowing a single finger press to cause signal in multiple mechanosensors at once. In the case where multiple outputs are contacted, the touch position can be calculated using the ratio of intensities in the neighboring light guides.

Experimental validation

To visualize the scaffold deformation and core contact, we used x-ray micro–computed tomography (μ-CT) to image the OL1,3 (fig. S4; see the “μ-CT scan” section in Materials and Methods). The 3D reconstructions show that the variable stiffness scaffolding deformed as designed, with the stiff scaffold undisturbed even with full compression of the soft segment (Fig. 2A, left). In addition, because of the high compliance of the soft scaffold, external forces resulted in highly localized deformation to ensure that only one or two receptive fields were activated with one touch. We also confirmed that the coupling length (i.e., physical contact between input and output cores) does increase to its maximum length, 3.5 mm, at high forces. Under the large deformation applied (e.g., 5 mm), light remained coupled; however, the CT scan reveals that extreme deformations caused the input to roll around the output core instead of simply contacting from above (Fig. 2A, middle right).

Fig. 2 Experimental validation.

(A) Sliced x-ray μ-CT scan reconstructions of light guide–lattice composites with contact between cores labeled. Bottom: Magnified isometric and side view of CT reconstructions showing the contact and orientation of the cores at the deformation site under high deformation. (B) Normalized simulated and experimental signals coupled from input to output with varying coupling length and constant contact width (~0.3 mm). Error bars indicate 1 SD above and below average.

We ran simulations on the lace (COMSOL Ray Optics; see the “Optical simulation” section in Materials and Methods), which, as expected, indicated a linear increase in the normalized coupled signal transmitted from the input as the coupling length increases (Fig. 2B). We modeled the light guides as lossless cylinders (d = 1 mm) with ncore = 1.52, cladded by air, nair = 1.00. We set them to contact over a straight section of varied length, 0.1 mm < L < 4.5 mm (Fig. 2B), on the output core. We also set the cores to overlap by 0.050 to 0.55 mm to simulate a changing contact width when they are deformed into each other. From the measured power of light at the end of the output, we see that a minority fraction of the input light (<12%; fig. S5) coupled. This ensures that multiple signals can be read simultaneously from a single input. The correlation of signal intensity to core contact made it possible to measure force or extent of deformation in addition to position. In addition, our experimental results from testing two straight, cylindrical light guides coupled with a constant contact width and varying length, L, matched well to the simulated results (Fig. 2B and fig. S6). The signals in Fig. 2B were normalized to their signal at a 4-mm coupling length.

Although the change in coupling length produces most of the signal, the contact width is also a control variable (fig. S7). Very small contact widths led to a small contact area and low signal; too large implied significant deformation of the cores, causing more light loss to the environment (Supplementary Text and fig. S7). We experimentally confirmed that our sensor network functions mainly because of changes in contact length that can be seen in Fig. 2A, where even at high deformations there is limited change in core shape.

Sensor characterization

We characterized the force response of an OL1,3 by attaching an infrared (IR) light-emitting diode (LED) to the end of the input core and photodiodes to the ends of one leg of each output core to read the coupled light intensity (see the “Force and positional accuracy measurements” section in Materials and Methods). To measure the force response, we used a z-axis stage to repeatedly press a force gauge into the centers of each of the three outputs between 1.5 and 3.5 N at 0.5-N intervals and recorded the signal (fig. S8; see the “Force and positional accuracy measurements” section in Materials and Methods). The signals were normalized to the maximum possible value that the electronics can produce (Fig. 3A; see the “Electronics setup for reading signals” section in Materials and Methods). The minimum and maximum indentation needed for a reliable, linear signal response was ~1 and 3 mm, respectively, with the current scaffold. The signal output was highly repeatable, with an average SD, SD¯=1n∑1nSDavg signal=0.02, in normalized output signal across the n = 300 to 400 total data points. The low SD of the signal implies a force resolution (~0.3 N) comparable with the skin in the human hand [i.e., humans can perceive 7% change in applied force between 2.5 and 10 N (38)] and is better than typical resistive or capacitive soft sensors [~10% variation of signal (9, 11)]. The minimum indentation required for this particular OL1,3 is much larger for our sensor than for the human hand [OL, 1 mm; human hand, 10 to 500 μm (39, 40)] and electronic soft sensors that are typically on the scale of a few millimeters thick (4, 13, 14, 18, 19). The design of our scaffold OL1,3 had a reliable minimum readable force of 1.5 N and a maximum of 5 N [for comparison, resistive and capacitive soft sensors can measure minimum forces around 0.01 N (9, 11) and the human hand 0.001 N (1)], and we could tune the sensitivity and dynamic force range via lattice geometry. The lace itself, without being encapsulated in a lattice, had a minimum detectable force of 0.06 N for the particular fiber mechanical properties used in this study.

Fig. 3 Sensor characterization.

(A) Normalized averaged signal from each output when pressed more than 20 times with increasing force reported with 1 SD above and below. (B) Fastest (black square) and slowest (green triangle) of eight sensor responses to a ~0.1-ms impulse that impacts at time = 0 (marked by a red vertical line). (C) Signal from all three outputs as the OL1,3 is pressed every millimeter along the input from the middle of the left to right outputs. (D) Average calculated versus actual position of eight samples of presses with a dotted line showing where the points would lie if 100% accurate (ground truth). Error bars indicate 1 SD above and below.

To assess the frequency response of our sensors, we shot a 4.5-mm-diameter projectile into an OL1,3 at ~0.23 km s−1 (movie S1; see the “Impulse measurement” section in Materials and Methods). While collecting sensor data, we visualized the impact using high-speed camera (Phantom LAB 310) data for ground truth. The OL embedded in the 3D-printed framework could measure an impulse between 0.2 and 2.5 ms long from six samples, whereas the actual impact was about 0.1 ms (Fig. 3B and fig. S9). From high-speed video, we estimated that the OL can measure deformation rates of at least 46 kHz: frequency=velocity of projectiledeformation observed=230,000 mm s−15 mm. The sensor response, however, is slightly delayed from the time of impact by 0.7 to 26.4 ms. The inconsistent time between analog readouts suggests that the error is due to the microcontroller because the Arduino can read an analog signal at about 4 kHz and the operational amplifier (OPAMP) used will produce an attenuated signal above 6 kHz. Another source of delay is in the finite time it takes for the compression wave to collapse the elastomer matrix.

To measure the positional accuracy of the OL1,3, we pressed the sensor every millimeter between left and right outputs (Fig. 3C) with the same z-axis stage and force gauge (see the “Force and positional accuracy measurements” section in Materials and methods). We took the maximum value from each output during the press to calculate position and plotted it against the actual position measured by a ruler. From Fig. 3C, we see the overlapping receptive fields of each mechanosensor because there are signals from both neighboring outputs when the sensor was pressed directly between, but not when pressed directly above, their centers. From the ratio of the output intensities, we interpolated the position of touch between two outputs using the empirically determined (fig. S10) Eq. 3xcalc=xbase+5.09∗(IRightILeft+IRight)0.34(3)where xbase is the center of the closest output to the left and I is the maximum signal during a press of either the left or right neighboring output (fig. S10). We fit the ratio to the true position, for one set of data, to find the nonlinear equation above (fig. S10). When only one output core has a signal, the ratio is zero, so the system assumes that the press is directly above the center of that output. This introduces greater error (maximum error, Errmax = 2.7 mm; average error, Err = 0.71 mm) near the centers of the receptive fields where there is no overlap (i.e., near the center of an output; Fig. 3C). Although the data were taken with presses of around 4 to 5 N, this model does not require previous knowledge of the magnitude of applied force because it relies on the ratio of intensities and not their absolute value. It also has submillimeter average positional accuracy [Err = 0.71 mm; SD = 0.69 mm; Fig. 3D], which is comparable with the spatial resolution of touch sense in a human hand [2 to 3 mm (41)]. Typical electronic soft sensors are often shown to have spatial resolution on the 0.1-mm scale (2, 17) but, like OL, can likely have much higher upper limits in positional accuracy with improved manufacturing methods.

Exteroception

To demonstrate the exteroceptive abilities of this system, we fabricated a musical instrument (OL1,15) with 15 output cores over a single, 12.5-cm-long input line (Fig. 4A) using the same core geometries as the OL1,3 shown previously. Because of the light propagating through the input in one direction, the coupled light primarily travels in the same direction and thus preferentially down one leg of the contacting output (Fig. 1C). We took advantage of this directionality and placed an LED (IR on the right, visible on the left) on both sides of the input core. In Fig. 4, we used a blue LED to visualize the pathway for data collection; however, we took data using an IR light source, which would not have been visible in our illustration. We threaded the right leg of the output to the left side to read the coupled IR light with a photodiode (Fig. 4A, output: data, and fig. S11) and the remaining leg to the front of the scaffold as a visual indicator of ground truth (Fig. 4A, output: visual, and fig. S11). The IR signals were interpreted by an Arduino microcontroller communicating with the software package “Processing” on a laptop computer (fig. S12) to produce auditory signals based on position of touch in real time (movie S2). The calculated position is also plotted on a number line from 0 to 14, representing each of the 15 outputs. The program also measured the intensity of the signal to play notes of varying volumes (movie S2). Figure 4 (B to D) shows the calculated positions for a press directly above, between, and over multiple outputs, along with a representation of the press in the same position.

Fig. 4 Exteroception.

(A) Musical instrument with 15 output cores. A red LED for visual feedback and the photodiodes for each output core are located on the left side. On the right side, we placed a blue LED to show light to be read by the photodiodes. Direct visual and computed locations of press positions (B) directly on an output, (C) between outputs, and (D) over four outputs simultaneously.

Proprioception

To demonstrate a proprioceptive use case, we collected data from an OL1,12 (Fig. 5A; see the “System fabrication” section in Materials and Methods) to reconstruct the 3D deformation state of a cylinder, with embedded stiffness gradients, undergoing uniaxial compression (Fig. 5B and movies S3 and S4). We varied the stiffness within the cylinder by varying the strut thickness (Fig. 5B and fig. S13; from bottom to top, tstiff = 1 mm, tsoft = 0.75 mm, tmedium = 0.875 mm). By adding a stiff spiral (Fig. 5A, scaffold) with a powered waveguide core (Fig. 5A, input), we increased the sensitivity of the OL and swept out a larger volume for sensory information. The variable stiffness lattice and added stiffness from the spiral allowed the cylinder to measure forces from 0.5 to 22 N. Figure 5A shows this spiral input core, illuminated using a blue and green LED, and the 12 output cores positioned normal to it and along its length.

Fig. 5 Proprioception.

(A) Computer-aided design (CAD) model of each component of the cylinder and the completed device with three different stiffness sections. (B) Actual structure, (C) FEA simulated model with strain, and (D) computer-reconstructed cylinder shown during compression. (E) Average displacement error in millimeters between calculated and measured versus normalized displacement for each stiffness section with 1 SD above and below over nine single-axis compression tests.

To calculate the compression of each section, we obtained 18 sets of simulated force versus localized compression for every 2.2 mm of height from finite element analysis (FEA) of the scaffold and selected the 12 that correspond to the innervated area (see the “Proprioception accuracy measurements” section in Materials and methods). We measured the force in each section of the cylinder using a linear model of signal intensity to force and calculated the displacement of each section with a compression versus force model fit to the simulated FEA model (table S2). The model and our reconstructed data are shown in Fig. 5 (C and D). For model validation, we compared the calculated strains with those measured using a digital image correlation (DIC) extensometer during single-axis compression tests. The average magnitudes of error over nine tests were 0.033, 0.33, and 0.24 mm with an SD of ±0.01, 0.24, and 0.25 mm (Fig. 5E) for the stiff, medium, and soft sections, respectively. These errors are a little larger but on the same order of magnitude as human perception of skin indentation in the hand, ~0.1 mm (40), and can be decreased (0.021, 0.029, and 0.19 mm) if using a model fit to DIC data (movie S4).

Limits of the OL

Innervating large and tortuous 3D structures will require knowledge of the power transfer limits of the OL, which has inherent length limitations due to optical losses through and between fibers. We modeled these losses using Beer’s law under two specific use cases: (i) The first one defined the OL length as if we are looking for only a few signals at one time over as large an area as possible. This situation is useful in exteroception where knowing when and where a robot has been contacted is important. In this case, presently, the input could be ~80 cm long for the material and electronics system that we have chosen (see Supplementary Text and fig. S14). (ii) The second defined how long the OL can be if we want to interpret all signals simultaneously. This situation is useful for proprioception where constantly measuring local curvatures globally would be important. In this case, presently, the input can be 31 cm long with 37 outputs (see Supplementary Text and fig. S14).

DISCUSSION

In this work, we demonstrated a 3D mechanosensor network, which we call OL, cointegrated with variable compliance structures that performs similar functions as, but does not mimic the mechanisms of, an afferent nervous system. We demonstrated the OL’s ability to repeatably sense localized, volumetric deformations within a 3D-printed polyurethane lattice. Whereas the OL itself could detect 0.06-N forces, the OL embedded in the lattice could detect 1.5 N with submillimeter average positional accuracy (0.71 mm) comparable with human skin.

Although we only demonstrate two simple examples of computer interfaces with extero- and proprioception, the OL and scaffold can be tuned for sensitivity, spatial resolution, and accuracy as well as designed to be responsive to both the direction and type (tension, compression, bending, and twisting) of deformation. Changing the scaffolding to be selectively stiff to better transfer forces to the mechanosensor, removing the air gap so that the sensor starts at a nonzero value, and adding mechanical amplifiers (like hair on animals or feathers on birds; fig. S15) could decrease the measurable force to ~0.035 N (corresponds to a ~0.5-mm displacement).

The minimum detectable force of the OL without the scaffold could be further reduced by lowering the elastic modulus of the core, allowing more contact area with smaller forces. Higher positional accuracy, especially between outputs, could be increased by changing the output geometries to be flatter and overlapping (fig. S16). We could also increase both the spatial resolution and accuracy of the OL by narrowing the widths of the outputs. Last, to measure direction and multiple types of deformations, we may combine multiple OLs (fig. S17 and movie S5) and even use different geometries altogether (fig. S18 and movie S6). Larger volumes can be innervated by a single input using improved light guide materials, higher power LEDs, or more sensitive photodetectors. Fewer accompanying electronics may be required if we take inspiration from biology to combine signals from individual mechanosensors without losing information (42).

In the future, we would like to take advantage of the higher information density that can be carried through optical systems to create integrated sensorimotor networks. These networks could combine not only deformation sensing but also temperature (43), humidity (44), and chemical monitoring. We may also better incorporate the cores through directly 3D-printing the light guides (45). By mimicking the inherent link between sensory and motor signals in animals, which are used not only for action execution but also for action “selection” (46), we may be able to use OL to simplify controls and electronics and improve the efficiency of robots.

MATERIALS AND METHODS

Materials testing

To characterize the mechanical properties of the 3D-printed material, we performed uniaxial tensile tests according to the ISO (International Organization for Standardization) 37 method with a type 4 dumbbell sample geometry. The test was run at 200 mm min−1 on a Z010 Zwick Roell using a 10-kN load cell with pneumatic grips pressurized at 85 psi. The graph in fig. S1 shows the averaged data with SD of 11 samples. The data show slightly exaggerated strains due to the sample slightly slipping out of the grips. All compression tests were run on the Zwick with a 10-kN load cell between two parallel plates. The sample below was a solid cylinder (diameter, 29 mm; height, 12.5 mm) and was compressed at a strain rate of 12 mm min−1. The lattices in Fig. 1 were compressed at 10 mm min−1.

Measuring signal change due to varying contact length and width

With a constant coupling length rig (fig. S15), the contact width between cores was changed by spacers that limited the amount of compression between cores. We see that at first, the output signal increased as the compression and coupling length increased, but then dropped off (fig. S15). This may be caused by loss of higher-order modes that can be carried by the light guide due to a large change in core diameter and increased loss due to a change in the angle at which rays hit the core surface, as previously noted by Harnett et al. (23). The initial increase in signal may imply that more light couples as the contact area increases due to greater contact width.

System fabrication

We designed the variable stiffness scaffolding using Fusion 360 (Autodesk Inc.; fig. S3) and stereolithographically 3D-printed the OL scaffolding from a commercial EPU material on a Carbon M1 printer. We designed the top ~8 mm to have a thinner strut thickness, tsoft = 0.75 mm, compared with the rest of the scaffold, tstiff = 1.125 mm, which created an area that was sensitive to external touch. Within the scaffolding were lattice-walled channels to hold the light guides with thin, tclad = 0.4 mm, walls to minimize the effect on the mechanical properties of the structure. The scaffold was fully cured after being in an oven for 8 hours at 120°C after excess resin was removed in an isopropyl alcohol wash.

We threaded a thin monofilament string (Darice Jewelry Designer, 8 lb) through each channel in the printed scaffold and super-glued the end to a commercial, clear, soft, polyurethane core (Stretch Magic, Pepperell) with a refractive index of 1.52 (23) and diameter dout = 1.0 mm for the outputs and din = 1.5 mm for the input. We then pulled the cores into place with the string.

For the proprioception cylinder, the output U size and spacing changed to fill the available space to be innervated. The output U’s were 6.8 mm wide and spread out to be every 9 mm along the input so that each one represented about 2.5 mm of height on the cylinder. They were parallel to the compression axis because this caused the inputs and outputs to be pressed together during compression. The U’s were also curved to match the spiral so that the output and input core line contact increased with more compression. The input was designed as a spiral as opposed to a zigzag or s-curve to avoid any sharp curves that would introduce more light loss.

μ-CT scan

We designed and printed a rig to hold the CT sample in the deformed state for the duration of the scan, as shown in fig. S4. The sample was scanned on an Xradia Zeiss Versa XRM-520 run at 100 kV/9 W with 0.7-s exposure time and 36.7-μm resolution. We reconstructed the scan into a 3D part in the software Aviso (Thermo Fisher Scientific) after removing the bar that caused the deformation from the slices. We then removed the background noise and testing rig before slicing the reconstruction to show the cores in Fig. 2A.

Electronics setup for reading signals

We designed and 3D-printed elastomeric parts to hold the photodiodes (380 to 1100 nm; SFH 229 from OSRAM Licht AG) and IR LEDs (peak, 875 nm; TSHA4400 from Vishay Intertechnology Inc.) in contact with the cores using friction fits. The setup used to take data for the OL1,3 is shown in fig. S9. All data for force and positional accuracy were taken with photodiodes amplified with an OPAMP (LM324A; Texas Instruments Inc.) that had a 2-megohm gain and a 4700-pF capacitor acting as a low-pass filter with a maximum signal value of 800. The signal from the photodiodes was sent through a simple current-to-voltage (I/V) converter circuit (fig. S19) and read through the analog pins on an Arduino microcontroller (Arduino MEGA 2560) at a baud rate of 9600.

All the photodiodes for the exteroceptive sensor used a 2-megohm resistor (R1), whereas the proprioceptive sensor used 2-, 4.99-, or 10-megohm resistors as listed in table S1. The IR LEDs have 33-ohm resistors (R2). The circuits were powered by two wall outlets for +V = 5 V and –V = −5 V. Data were transferred to Processing about 50 times per second.

Optical simulation

We modeled a straight input and single output in COMSOL’s Ray Optics package as lossless, cylindrical, 1-mm-diameter light guides. Walls were set up at both ends of each core to measure the light intensity through those areas. Input light was set up through the “Release from Grid” module, entering from one of the input ends in a uniform density cone of 400 rays with a cone angle of π/9 and a total power of 1 W to simulate the LED. Rays leaving the cores were suppressed. The mesh had a minimum element size of 0.009 mm, a maximum of 0.25 mm with a curvature factor of 0.1, and a maximum element growth rate of 1.1. The simulation was solved using the ray tracing solver over a path length of 0 to 300 mm with a step size of 0.1 mm. We ran a parametric sweep of the coupling length from 0.1 to 4.5 mm with a step size of 0.1. To simulate varying levels of core deformation, we also ran a parametric sweep of the overlap between the two cores from 0.05 to 0.55 mm in steps of 0.1 (fig. S7).

Impulse measurement

To measure an impulse, we first removed the capacitor (fig. S19, C1), which was inserted as a low-pass filter. We recorded 4002 points of data on an Arduino set at a baud rate of 2,000,000, which took roughly 1 s. The actual amount of time it took to get 4002 analog readings varied from 904 to 1093 ms. We shot seven sensors with 0.177-caliber bb pellets as the projectile and measured the signal from the impact using a high-speed camera (Phantom LAB 310) set to take 23,000 frames per second and a flashing LED to sync the data to the video (fig. S20). From the high-speed video, we determined that the projectile took ~0.00009 s (two frames) to pass through the entire sensor. Movie S1 shows one test, with the high-speed video brightened and contrast increased for better viewing.

Force and positional accuracy measurements

We mounted a digital force tester to a vertical lift stage and attached a 3D-printed finger phantom to its end (fig. S9). We attached the OL1,3 to a delrin block and a ruler to the acrylic sheet under it so that we could measure presses every millimeter along the length (fig. S9). To apply a force, we lowered the vertical lift stage to press the finger phantom into the sample until the desired force was read. From the collected data, we used the maximum value from each output for each press, defined as a chain of nonzero signal values, and normalized them by 800, the maximum value possibly read from the sensor (Fig. 3, A and C). To find the nonlinear portion of Eq. 3, we divided the actual position, xtrue, by the ratio of neighboring output signals, x=IRightILeft+IRight, to get a multiplication factor. We then plotted the factor by the ratio and fit a curve in Excel (Microsoft Inc.) (fig. S10). The factor was multiplied by the ratio and added to xbase to get the final equation.

Proprioception accuracy measurements

We placed the cylinder between two parallel plate attachments of the Z010 Zwick Roell tensile tester using a 10-kN load cell and ran the compression test at 10 mm min−1 while acquiring data from the sensors. We filtered the data with an exponential filter using a smoothing constant of 0.9 on an Arduino. To measure the localized deformations experimentally, we marked the front face of the cylinder with a silver Sharpie every 2.5 mm, tracked the dots using a DIC extensometer with Vic-Snap and VIC-2D (Correlated Solutions Inc.), and converted the pixel positions of the dots to displacements. From the sensor data, we correlated the signal magnitude to force with a linear model. We then created our model using MATLAB to fit a sixth-degree polynomial to the localized displacement and force values from the FEA simulation done in ANSYS (movie S7). We found the error of the calculated displacements using error = abs(dispcalc − dispDIC). Because the simulation can only solve up to 35% compression, we show two models in the supplementary movies; movie S3 shows the FEA model, and movie S4 shows a model based on experimental data up to 35% compression.

SUPPLEMENTARY MATERIALS

robotics.sciencemag.org/cgi/content/full/4/34/eaaw6304/DC1

Text

Fig. S1. Scaffold material properties and mechanical simulation material model.

Fig. S2. Index of refraction measurement.

Fig. S3. Dimensions of the OL1,3.

Fig. S4. μ-CT sample.

Fig. S5. COMSOL optical simulation results.

Fig. S6. Experimental setup for coupling length measurements.

Fig. S7. Experimental setup and results for coupling width measurements.

Fig. S8. Setup to take force and position data in Fig. 3.

Fig. S9. Other clear impact testing sensor signals.

Fig. S10. Curve fit to find nonlinear multiplication factor for Eq. 3.

Fig. S11. CAD and images of setup for exteroceptive demo.

Fig. S12. Schematic of information flow.

Fig. S13. Compression tests of the individual sections in the proprioceptive demo.

Fig. S14. Schematic of theoretical length model use cases.

Fig. S15. Increase sensitivity with feather.

Fig. S16. CAD examples of ways to increase accuracy and resolution.

Fig. S17. OL3,15.

Fig. S18. Different geometry OL5,11.

Fig. S19. Schematic for electronics circuit.

Fig. S20. Setup for impact test.

Table S1. Proprioceptive sensor circuit gains.

Table S2. Linear parameters used in proprioceptive demo to relate signal to force using F = F0 + signal/S1*S2.

Reference (47)

Movie S1. High-speed impact.

Movie S2. Exteroception.

Movie S3. Proprioception with FEA model.

Movie S4. Proprioception with experimental model.

Movie S5. OL3_15.

Movie S6. Multiple deformation sensing in a block.

Movie S7. FEA simulation.

Acknowledgments: R.F.S. thanks J.J. Vericella for being a continued inspiration, posthumously. Funding: The project described was supported by the National Center for Advancing Translational Sciences, NIH award number TL1-TR-002386. The content is solely the responsibility of the author(s) and does not necessarily represent the official views of the NIH. The authors acknowledge Air Force Office of Scientific Research contract number FA9550-18-1-024 and Office of Naval Research Department of Defense contract number N00014-17-1-2837 for financial support and technical advice. The x-ray μ-CT was performed on an instrument purchased using NIH grant number S10OD012287. The index of refraction data was measured at the Cornell NanoScale Facility, a member of the National Nanotechnology Coordinated Infrastructure, which is supported by the NSF (grant number ECCS-1542081). Last, this work made use of the Cornell Center for Materials Research Shared Facilities, which are supported through the NSF Materials Research Science and Engineering Centers program (DMR-1719875). Author contributions: P.A.X. conceived the OL, designed and conducted experiments, analyzed data, and wrote the manuscript. A.K.M. and H.B. made and ran ANSYS and COMSOL mechanical and optical simulations, respectively. C.A.A. conducted experiments and analyzed data. L.Z. edited the manuscript. R.F.S. supervised the research, designed and ran experiments, and edited the manuscript. Competing interests: R.F.S. is a cofounder of Organic Robotics Corporation that licenses some patents covered here, including publication no. US-2019-0056248-A1, as well as 62/592,073 PCT/US18/63172 and 62/642,407 PCT/US18/63173. The other authors declare no competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper or the Supplementary Materials.

Science Robotics

Share This Post

Leave a Reply