Pi Resistor Sorter

Background

The color code indicates the value of the resistors. Due to this feature, Computer Vision is a potential approach to tell the value. There are more than one color standards of color ring resistor such as 4-band, 5-band or 6-band type. For our project, 4-band resistor is used for testing. The background color of our testing resistor is light yellow. The tradition method of measuring the resistor is electrical measurement, but this electrical measuring only tells the actual value of the resistor. The value of what this resistor supposes to be is hidden. Also compared with electrical measurement, the computer vision detection can tell the tolerance of the resistor.

Design and Testing

High Level Design

1. Convey Belt Movement – Control the movement of the convey belt, this convey belt moves the resistor under camera before sorting. After sorting, convey belt move the resistors to sorted plate.
2. Resistor Recognition – Detect the color of the resistor rings, and calculate the value of resistor based on the color of rings and its order.
3. Plate Movement – Rotate specific angles according to the result of the computer vision measurement.

Hardware Design


Sorting roller
This device is for sorting the resistors on the conveyor belt. With this roller, we can put the resistors simutaniously. But it did not work. Because the resistors are not perfect cylinder and there are only two points that can actually contact with the sorting wheel and the conveyor. And once there is a tiny un-smooth part in any of those two parts. It will make the direction of the resistor change and lead to the result that the resistors will be stucked in the slot between the the wheel and the conveyor




Turntable
This part can store the different resistors. When the resistors fall from the belt, they will fall into the 6 divided areas.




Support structure
Foudamental support of the whole machine




Roller for the conveyor
This is one of the rollers for the belt. The servos provide power to the conveyor through these rollers


Software Design

OpenCV Library
OpenCV (Open Source Computer Vision Library) is a library of programming functions mainly aimed at real-time computer vision. We chose OpenCV-python to detect the color of the resistor. The color images are represented as three-dimensional arrays, we use Numpy to finish detection assignment.

Color Detection


GRB color model              HSV color model
RGB HSV
Red, Green, Blue are initials of the three additive primary colors. RGB color model is the most popular color model, in which red, green and blue light are added together in various ways to reproduce a broad array of colors. The computer vision, the shadows part will most likely have very difficult characteristics than the parts without shadows.

The HSV model describes colors in terms of hue, saturation, and value. The hue indicates the primary color, and value indicates the brightness of the color. The change of the shadow only influences the value attribute, and the hue attribute will not change so much. Hence the HSV model is better for measuring the image where shadows always change.



Value Detector

Working flow of value detector

Pre-processing
UnSharp mask: The UnSharp mask adds the detail of resistor that lets us recognize the its color features. Sharpening filter emphasizes the differences between adjacent light and dark sample points in an image.

Color Detection
Color Area detection is the method we used to find the color ring of the resistor. After the UnSharp mask filtering, the color feature of resistor will focus on specific areas. In this stage, we transfer the RGB color model to HSV color model. Set specific color bound for different color, and filter the image. A 2D Numpy array is shown after color filtering operation.

Morphology
Erosion: Due to the limit of real life light environment, the color noise is unavoidable. Erosion is one of the basic operations in the area of morphology. This operation erodes away the boundaries of regions of foreground pixels. Pi resistor sorter uses the erosion algorithm to filter noises on the image.

Calculation
Using the data gathered from color detection algrithom, we could find how many rings on the resistor target. When we were able to calculate the value of resistor based on these color data. The algrithom we implemented is based on counting the points and calculate the median location of the resistor. The value of resistor is depending on the color ring and its location.
According to our test, the result of color detection changes for different frame. To improve the accuracy and tolerance of resistors sorting, we marked a resistor value as valid only when three repeat results happen.

Edge detection
The above image processing took long take to finish, to guarantee the accuracy, we implemented an edge detection algrithom to test if there was a resistor coming. The edge detection algorithm we implemented is canny detection. MinVal and MaxVal were set to fit number to find the edge between resistor and white background page.


GUI Design
GUI is used to convey the information of Pi resistor sorter. Since Pi owns operating system, we chose to run Pi as a website server, thus other electronic devices can access it using website. Compared with using normal PiTFT, the webpage is more convenient and intuitive.

The web framework is python micro framework library, flask. This framework can handle http request as python functions. Our web page is straightforward, which enable users to see the information of sorting when users access Pi’s IP address. The History window shows the sorting history combined with resistor value and the time. The Statistics window shows the counting number of different resistors.

Database
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. In this project, we chose Redis due to its lightweight feature. When resistor sorting python scrip detect a valid resistor value, the history information of this resistor will add to Redis. The web scrip accesses the database, and puts the resistor data on web page from Redis.



Result

---

We successfully detect the color feature using OpenCV. The result of color detection based on threshold is shown as following image.



Color detection using OpenCV

We chose the resistor with apparent color feature as our samples (such as green, blue, red, black, brown). The result shows it’s accurate to sort these resistors based on our color model.


Resistor sorter testing result

The website shows the basic information of Pi resistor sorter. History column tells the users the resistor id, value and its adding time. -1 in the Resistor value column means the detection is failed.

History column for resistor sorting results
The website will count and sort the resistor history. Users can click on the statistics button to see the information.

Statistic of sorting results
Tolerance Test


We chose 4 kinds of resistor to do sorting test. The result is shown as following table. Pi resistor sorter still has some limits for sorting resistor.

Failure Case Analyze
Mechanical Case. Based on our observed, the convey belt cannot move with constant speed. The stop location of resistor was variable. The failure case is caused by the wrong stop location of the resistor.

Computer Vision. There are some overlap between the bound of brown and red. The change of environment light caused misdetection of red and brown. The Pi resistor sorter cannot recognize the value of the resistor in this case.

---

Conclusions

Our project proved that computer vision is a potential method for resistor sorting. Pi resistor sorter performs partly of the function it was designed for. Pi resistor sorter can tell the resistors with specific color feature as we expected. The website GUI and movement control meets the goals we set early. But our design has some limits for mechanical staffs. And the result of color detection is influenced by the changeable environment.

Early our expectation is to design a machine based on convey belt model that can sort many resistors at a time. The performance of mechanical roller did not meet our expectation since the irregular shape of resistors. Hence, we have to put the resistor one by one to finish the detection assignment. Moreover, the movement of convey belt did not work as us supposed to be. The unstable movement sometimes causes the resistor cannot stop to the right location after Pi detects the edge of the resistor.

Our Pi resistor sorter performs bad for detecting some specific colors like gray, silver and white. Also since the color bound of orange and red is very closed, sometimes the Pi resistor sorter will take the orange color as red. The mechanical design also limits the accuracy. The paper material we used to build for convey belt works poorly after long time testing. Sometimes the movement of convey paper will not on the same horizon, which means the focus will change.



---

Further Working

Mechanical Design
We can improve the conveyor by using a true belt. Then we can maximize the use of this device by making the sorting and feeding devices, which means that we can change the part to sort different staffs at anytime we want. Like the candies screws and nuts (But this version one is still mainly used for the resistors and the detachable parts are the potential uses detachable



Replace servo motor with stepper motor
Stepper motors have a large number of poles, magnetic pairs. Servo motor often has 4 to 12 in total. To move the stepper motor to specific position, it requires a position encoder, which map an angle to specific time. Our implementation is based on servo motor based on position encoder. The servo rotation location is not very accurate based on our result. The design of the stepper motor provides a constant holding torque without the need for the motor to be powered. The torque of a stepper motor at low speeds is greater than a servo motor. Hence, we think the stepper motor is better for using for plate rotation control in our design.



Stable computer vision Environment
To color detection algrithom is sensitive to the light environment. In this project, we did a lot to make the light environment stable like blocking the environment light and adding LED rings around the camera. However, due to the limits of convey belt, the focus for camera will change. A black box environment and conveyor belts with gear may be a potential solution.

Sorting Algrithom
Now the algrithom of sorting is combined with edge detection and direct color detection. This algorithm is a tradeoff between speed and accuracy. A better way for color detection is detecting the edge of resistor first, and then analyze image and color detection only inside the edge of resistor. The background color of testing resistor is light yellow, which is closed to the background color.

Appendix:
Code Listing
code.py





Special thanks to Cornell Rapid Prototyping Lab (RPL) providing us 3-D printer environment.

Labor Division
Sichen Liu:
Computer Vision, Webpage GUI, Movement Control, and Hardware circuit setup.

Ruichen Tan:
Hardware design and build, program optimization, web production

References
OpenCV library
Servo Datasheet
Flask
Color Model
UnSharp Masking
Mathematical Morphology
Redis

Acknowledges
We would like to thank Prof. Joesph Skovira for his advice for this project. He provided many useful suggession for system deisgn and color detection algorithm. In addition, we would like to thank all TAs for answering our questions.

---

Project group memeber

Ruichen Tan

rt446@cornell.edu

Sichen Liu

sl2947@cornell.edu

---







---