Optical Neuron Modeling

Design & Testing

Libraries and Modules

Two open source libraries were used in our final product. The MCP3008 ADC Python library from Adafruit was used to interface with the MCP3008 ADC over four-pin software SPI. Additionally, he open-source library "pi_ina219" written by Chris Ballard was used to fix conflicting GPIO instantiations by the ADC library and Adafruit.GPIO module.

Several modules were imported in our final product as well. The module "sys" was used to point to the paths where our ADC libraries existed and append them to the in-built Python path. "RPi.GPIO" was imported in order to use the GPIO pins of the Raspberry Pi. The modules "MCP3008" and "Adafruit_MCP3008" worked with the Adafruit ADC library to interface with the MCP3008 ADC chip. "Matplotlib" and "Pyplot" were imported to graph and then save the image of each neuron's action potential. The "Collections" module was used as a high-performance double-sided array. Finally, the standard modules "os" and "math" were used to manipulate operating system functionality and acess basic mathematical functions respectively.

ADC Functionality

As we quickly came to realize, the Raspberry Pi did not come with any analog functionality. As such, we purchased a low-cost 8-channel 10-bit analog to digital converter from Adafruit: the MCP3008.

The MCP3008 has two modes of functionality, hardware SPI and software SPI. Our original intention was to utilize hardware SPI as it is slightly faster and more efficient. However, the hardware SPI pins required by the MCP3008 (SCK, MOSI, MISO, CE0) were already used by the TFT. As such, we implemented a software SPI connection instead. Due to our pin availabilities, the MCP3008 library was modified to use pins 12, 16, 20, and 21 for CLK, CS, MOSI, and MISO respectively. Figure 6 below shows the schematic of our connection between the Raspberry Pi header and MCP3008 chip.

Figure 6: ADC Wiring

The ADC was tested by running the "SimpleTest.py" example code provided by the Adafruit team. This code simply iterates through each ADC channel and prints their values every half second on the command terminal. A sample output of this code can be seen in Figure 7 below. Following this, our ADC was up and ready to go!

Figure 7: ADC SimpleTest Output

Plotting Graphs

In order to plot the graphs of our neuron action potentials, we used the open-source modules "Collections", MatplotLib", and "Pygame". "Collections" was used to create an array that acted as a rolling FIFO. The FIFO included 50 elements that represented the voltages of a singular neuron at any point in time. At the end of each iteration of code, an element was popped off the left end of the array, and a new value was appended to the start of the array (an artificial FIFO). Three FIFOs were created corresponding to each of the three neurons

We utilized the module "Matplotlib" in order to plot these FIFO data streams on our Raspberry Pi. First, we created a constant range of values from 0 to 50 to resemble time. After iterating through our equations and appending a new element to our FIFOs, we use the 'plot' function to create graphs of each array vs. time. Functions such as 'xlabel' and 'ylabel' as well as keywords such as 'b', 'r', and 'g' were used to modify the graph's axes and colors respectively. Finally, the plot was saved as a .png file into the Raspberry Pi's memory.

Our final step involved using "Pygame" to create our multimedia application on top of the SDL libraries. We initialized a Pygame screen with the correct size of our monitor (960x720 for desktop. 320x240 for TFT) and loaded a Pygame image as our saved Matplotlib figure. Finally, we appended this image to our screen every iteration through our code for a smooth animation experience.

Neural Math

The primary mathematics for a neuron is set by the below equation

Figure 8: Differential Equation for Leaky Integrate and Fire

The neuron fires when the voltage V _m reaches a certain threshold voltage V _thresh . At this point, the neuron spikes. The integrated voltage is directly affected by the stimulating current of the neuron. After a spike occurs, the membrane voltage is set back to resting potential. After the membrane is set to resting potential, its voltage remains constant until the end of the refractory period. As a result, the differential equation can be defined by the time since the end of the refractory period. We can then solve the differential equation to characterize membrane voltage. First, we define the time constant τ as R _m C _m. We define t as the time since the end of the refractory period. Finally, we define I _stim as the current generated by the stimulus. The solution to the differential equation is Figure 9: Differential Equation Solution for Leaky Integrate and Fire

Implementation in Code

The aforementioned differential equations were implemented in code as follows. We first have several settable parameters. These parameters all effect the final output graph. These settable parameters include tau, the membrane time constant, cap, the membrane capacitance, V _thresh , the threshold voltage, and V _rest , the resting potential of the neuron. In addition, each of the neurons has a settable excitatory gain and an inhibitory gain. The excitatory gain is multiplied by the stimulus value when the neuron is excited by light and the inhibitory gain is multiplied when it is inhibiting other neurons.

After parameters are set, we run the actual calculations. The same calculations happen in parallel for all neurons. We first set the refractory time, t _ref , to 0. This means that the neuron is not currently in the refractory period. If the analog readings of the neuron are less than a threshold, we assume that the stimulus current is automatically leaked. For this reason, the membrane voltage and the integral value are set to the resting potential. If t _ref is nonzero, then we are in the refractory period. In this case, we decrement t _ref

If the neuron’s value is greater than the threshold, we begin including the stimulus readings into the integral. If t _ref is nonzero, we are still in the refractory period, so this value is decremented. In addition, the integral and voltage membrane values are set to the resting potential. The LED used to display neuron spiking is set to output low. Otherwise, if t _ref is zero, we are no longer in the refractory period. We increment the value of t _refEnd , which is the time since the end of the refractory period. We then add to the integral as in the aforementioned equation. Additionally the stimulus current is changed to include lateral inhibition with the equation below:

Figure 10: Stimulus for Neuron Adjusted for Lateral Inhibition

At each iteration, the integral is updated. In addition, the membrane potentials are updated as in the first equation. If the membrane voltage surpasses the set voltage threshold, a spike is recorded. In this case, t _ref is set to 5 to indicate the start of the refractory period. The membrane voltage is set to equal the threshold voltage, and the neuron’s spiking LED is set to output high. If t _refEnd is greater than 40, we reset the membrane potential back to the resting potential, as the integration never led to the neuron firing and a spike was not detected. This step is essentially a forced leaking action for the neuron. With that, we have a complete working set of equations to model our neurons!

Box Construction

For constant testing of our optical neurons, we built a box with sliding doors. The sliding doors were used to gradually expose the LEDs to light. With two sliding doors, we were able to choose which neurons were exposed to light as well as cause individual neurons to fire. This allowed for better demonstration of effects such as lateral inhibition.

Figure 11: Side View of Box The sides of the box were made with 6 ½ x 3 ¾ x ¾ “ blocks of plywood. The bottom of the box was made with a 4 ½ x 6 ½ x 3/16 “ block of wood. The back of the box was made with a 3 ¾ x 4 ½ ” x 3/16” block of wood. All of these components were attached to each other with screws. In order to make the sliding doors, we attached ½ ” x 6 ½ ” x 1” blocks of wood on top of the sides. We created ¼ ” grooves along the length of these blocks. The sliding doors were two 3 ½ x 3 ¼ x 3/16 “ blocks of wood.



Figure 12: Top View of Box These blocks of wood slide into the grooves on the top of the box to block out light from a hanging lamp. These blocks were designed to block out light from two neurons at a time. We controlled the space between the sliding doors to show how exposure to light affects the neurons. We created a hole in the box to pass the piCobbler ribbon cable through, and attached the raspberry Pi to the outside of the box using Velcro. We also drilled holes for 3 LEDs to be seen on the outside of the box. These LEDs were used to identify spiking of the left, right, and center neurons. The side view of our box can be seen in figure 11 to the right. Figure 12 shows the top view of the box with the sliding doors.

TFT Functionality

The piTFT essentially ran as a plug-and-play system on the Raspberry Pi. In order to get it working, we simply had to export the desktop monitor's framebuffer to our TFT screen and ensure that all environment variables were changed accordingly. Figure 13 below shows our working code on the TFT screen.

Figure 13: Working display of TFT on Box

Startup on Power

Our final step involved creating a simple Bash script so that our program would run on bootup of the Raspberry Pi. This was done by including the python command to run the program in the rc.local file, and setting the command to run in the background.

Figure 14: Final Schematic of Circuit

Results & Conclusions

Final Product

Figure 15: Isolated Red Neuron Figure 16: Isolated Blue Neuron Figure 17: Isolated Green Neuron Overall, we were able to successfully model integrate and fire neurons and observe behavior that resembled the action potentials of real integrate and fire optical neurons. Based on our settings, red neurons had the highest gain, followed by green neurons, and finally blue neurons. The blue neuron was the center neuron, while the red and green neurons were to the side of the center neuron. Figures 15, 16, and 17, show the action potentials of these neurons when they were isolated from the other neurons. This was done by covering light from the other two neurons. We can see that all three neurons quickly reached threshold, spiked, and had a brief refractory period. For all of the neurons, the charging resembled RC charging as we would expect from leaky integrate and fire neurons. In addition, as set by their excitatory gains, the blue neurons converged the slowest, followed by the green neurons, and finally the red neurons.

Figure 18: Red and Center Neurons Illuminated Figure 19: Green and Center Neurons Illuminated We then checked the results of pairs of neurons at a time. Specifically, we shined light on only the red and blue neurons, and then only the green and blue neurons. From this, we were able to clearly observe the effects of lateral inhibition on the neuron. As compared to when in isolation, these neurons fired slower. In addition, the green neuron had a larger inhibitory gain than the red neuron. As seen in the figures, the blue neuron fired slower as a result of the inhibitory gain from the green neuron than that of the red neuron. Figure 18 and 19 contains images of pairs of neurons charging and firing.

Finally, we tested the effects of illumination on all of the neurons. We observed all 3 neurons charging and spiking. Lateral inhibition was much stronger in this case, especially for the center neuron. The center neuron spiked much less frequently when all neurons were active. Sometimes, if charging was too slow, the neuron would completely fail to spike as well. This often was a result of the ADC not reading the LED as strongly as previously measured. Figure 20 contains an image of the potentials when all three neurons were illuminated. Figure 20: Voltage over Time for Neuron < Figure 20: All Neurons Illuminated

Issues Faced

We faced several issues during the design and testing of this project. Our first issue involved conflicts between Adafruit's MCP3008 ADC and the Raspberry Pi's GPIO instantiation. This conflict essentially involved resulted from a dependency issue of Python3 and Adafruit's GPIO calls. The fix was relatively simple through the use of CHris Ballard's 'pin_ina219' library.

Our second issue involved fixing our strangely slow plotting speed. When first running Matplotlib on the TFT screen, our update times were extremely slow (>2 seconds per new data point). We initially assumed that this was because our entire program was running sequentially, with a new data point being processed only after graphing had finished. However, even after using the "threading" module to separate our graphing and ADC reads, we still were not able to improve efficiency. The saving grace ironically came when we changed our kernel from preemptive real-time Wheezy to standard Jessie. For some reason, this completely fixed our update errors. We can only assume that some portion of the preemptive kernel was messing with our Matplotlib module's savefig function and causing latency issues.

Our final issue involved the seemingly quick sensitivity loss of our LEDs. When connected into our circuit, the LEDs would initially act as perfect photodiodes, with varying analog signals depending on the amount of light shown. However, over a week, their sensitivity would decrease and require them to be replaced with another LED. They would still work perfectly fine as standard LEDs but no longer could work as a photodiode. This was especially confusing as the amount of voltage generated was far less than their threshold voltage and little current was flowing through the LED. Ultimately, we assume that there are some inherent material characteristics that prohibit the LEDs we were using to act photovoltaicly for long periods of time.

Future Work

We first would improve the performance of our system. Our display updates approximately once every half second, which is slow with respect to neuron behavior. In the future, we will update our program to use fixed point calculations instead of floating point to speed computation time. We will also implement better threading so that calculations can truly be done in parallel. In addition, we will change the platform of the program from python to C in order to optimize and be able to use a real time kernel. Finally, matplotlib is overall slow at graphing. We will graph in a way that is more efficient than re-plotting the entire time series every time.

In addition, we will add functionality to the device. Many of the settable parameters are only settable in code. In the future, we will allow them to be settable by potentiometer so that users of the device can see the effects of the parameters on the action potentials. For instance, changing the capacitance and time constant changes the time for the neurons to reach threshold. These changes would add significant value to the device.

Finally, we will implement a way to expand the array in order to see the effects of optical illusions on neurons. We will be able to show grid patterns and see how these patterns affect neuron responses.

Youtube Video

Check out our video to see the optical neurons in action!

Appendix

Acknowledgements

We would like to thank Professor Joe Skovira and all of the ECE 5725 TAs for their dedication and guidance in helping us finish our device. We would also like to thank Professor Bruce Land for the inspiration to design this device.

Parts List

Most of the cost of this device came from costs not included in the budget. The Raspberry Pi Kit, including the case, the Raspberry Pi, the piTFT, and the piCobbler ribbon cable do not count towards the budget. In addition, costs of wires and LEDs are negligible, so they are not included in the budget. We also considered the cost of the wood used to build the device to be negligible. The remaining costs are the cost for the solder boards and the cost of the ADC.

References

Datasheets

• MCP3008 Raspberry Pi SPI ADC

Figures

• Figure 1
• Figure 2:From Wikipedia page for Biological Neuron Models
• Figure 3:From Wikipedia page for Cable Theory
• Figure 4: Credit to EPFL school of computer and communication sciences
• Figure 5: From Wikipedia page for Lateral Inhibition
• Figure 6
• Figure 7
• Figure 8:From Wikipedia page for Biological Neuron Models
• Figure 9:Credit to EPFL school of computer and communication sciences
• Figure 10
• Figure 11
• Figure 12
• Figure 13
• Figure 14
• Figure 15
• Figure 16
• Figure 17
• Figure 18
• Figure 19
• Figure 20

Tasks Carried Out By Team Members

• Adarsh: Developed python software and soldered protoboard
• Samir: Tuned system and created box for final product

• © Untitled. All rights reserved.