The main office area of the office in the basement has been being painted over the last week or so. I have been relegated to the dining room table. When I moved back down and was shuffling things around, I found the the lights Gene made for the bean sorting project. I am going to get them sent off to him so he can start sending me some images. We might get lucky and have our original setup work, but I think that is pretty unrealistic. We will definitely have to make modifications quite a few times until we get the whole lighting designed tweaked to the point it works with the falling beans. That is not to mention the fact that we have not even started at all on the mirror setup to see both sides of each bean.
Tag: Bean sorter
I was able to advance on the bean sorter program sufficiently to send the computer and camera off to Gene so he can start working on the lighting. I am amazed that we are continue to move forward. This project is not moving along at lightening speed, but with Gene’s efforts and great mechanical skill and knowledge we make steady progress. Hopefully, he will be able to take some images of dropping beans and it will allow us to see the spread of the beans and whether I believe I can see them well enough in the images to do the calculations needed to sort them properly. The next big challenge is two-fold: 1) getting the beans to fall as straight as possible and 2) getting the mirror set up. After that, we will attack the lighting.
The new $243 plus tax computer arrived today and I have to admit that it is great. I loaded Linux Mint, OpenCV, Boost, Qt Creator (only for the IDE), the Wt libraries, downloaded the bean sort code from Subversion and had it compiling in three hours or so and that includes building OpenCV, Boost, and the Wt libraries from source.
Tomorrow I hope to get the thing taking pictures via the USB 3.0 port so it will be ready to ship to Gene early next week. The biggest challenge is keeping Kiwi out of the way. Maybe that says something about how hot the thing runs.
A couple of days ago, I broke down and bought a refurbished laptop for the bean sorting project. It surely seems to be a smoking good deal at $243.09 plus tax. I have been working to get the thing running on a Raspberry Pi and that works fine, but is way more hassle than we need during the development stage. It was necessary to hook up a keyboard, a mouse, a monitor, and the camera which, during the development stage, needs to be moved around a lot. It is just easier to do it on a laptop.
The other really big benefit is that, for the Raspberry Pi, I needed to do my development and testing on my home laptop, commit the code to a repository, switch the camera from the laptop to the Raspberry Pi and rebuild the code on the command on the RPi. That was a hassle. Now I will be able to put the entire development environment on the laptop and send it off to the mechanical designer (Gene). We can get all the lighting and controls developed with a full blown computer, then switch over to whatever cheap embedded computer we pick when everything is working. In the meantime, when I get some new code for Gene to consume, it will be way easier for him to build it and test it out with duplicate systems.
Gene outdid himself with his first pass at the bean feeder. I got this video as a text this morning and was very impressed. It will take some more work, but it is doing all the really critical things we need it to do–singulating the beans, dropping them off the end in single file with separation. He will refine the design and set up to start taking pictures of the beans as the drop. The main takeaway for me is that now I am the short stick again and will need to start blasting away.
This opt-output rack I bought from China cost less than $17. They used to cost an order of magnitude more than that. It is kind of sad we do not make them so much anymore in the US, but it is a very good thing they are available at so a low price and that we do not have to make them as low cost items with thin profit margins. This is the last piece of hardware I needed for hooking up the control to our coffee bean sorting project, but I have had two other projects take priority (involving clinical trials and compliance issues–that is a good thing because the trials and compliance issues would not be needed if the product did not work). So I am going to have to sit and just look at this fun new toy for a month or two before I can hook it up and make it do its thing.
Gene made the camera mount for our camera stand. This should drop write into our prototype. Progress continues. I know he is really busy, so I appreciated this a lot. We are moving forward on several fronts right now. I think I am going to be the bottle neck fairly quickly so I have decided to continue down that path of working out the UI so we can get an embedded bean-drop controller setup for both Montana and Texas before I continue on to any more control stuff. Kelly is going to help us by developing and implementing a market survey that, if this thing looks at all promising, we will continue each year while it is still interesting.
The indicator light tower for the bean sorting project arrived today. Really nice, but really cheap, too. I hope it works when I hook it up. Honestly, I had gotten pretty burnt out on all the user interface programming I was doing and, combined with a ton of stuff going on at my day job and a trip to Canada, I was getting a little weary and was planning to take a break. Then, some new hardware came in the mail and I realized I was not going to be able to hook it up and get it going until I finish with some UI enhancements, bug fixes, and robustification.
There is nothing like new toys for motivation for an engineer and this definitely qualifies. So, today, I am reinvigorated and will dive back into the UI so I can move on to the fun stuff. In the meantime, we are putting together a marketing survey of a large group of potential users of this product. Actually, there is value in making this thing (for me anyway) outside of having a market for it. I am learning a ton, having fun and putting together the structure I need to complete other projects of a similar nature with my buddy John.
After about a gazillion fits and restarts associated with reading the manual and getting syntax and basic concepts wrong, I have finally gotten the bean sorter web interface up and running. The idea is to be able to control the bean sorter vision system from a cell phone, tablet, or PC. I have been going around in circles for about two weeks now, but it appears I am ready to move on to the next thing. Here is a shot of my phone accessing the site from outside the LAN. I am glad to be done with this. The next step is to not just load a previously captured image, but to allow users to capture images by clicking a link.
Kiwi and I continue to program hard on our Raspberry Pi and coffee bean project. You can see from the image at the right that we made a small breakthrough. The capture control is now on the Raspberry Pi as well as the ability to display the last image via the web. I suppose it is not such a big deal, but since I have generally worked on image processing algorithm development and the analytics that go along with that, I am not such a great web programming. It has been frustrating but fun to work through getting the syntax and understanding the idiosyncrasies of embedded web servers and programming tools. Nevertheless, I have the main part of the start of the struggle behind me 
and look forward to returning focus to the application at hand. The camera is set up with the calibration target underneath. The next step will be to create a way for the web server to talk to the camera control program so a user can capture an image on demand, calibrate the system, run a bunch of beans, look at logs to assure everything is working right, etc. etc. I have done more of that kind of thing than web programming, but I am sure there is a lot of minutiae that I have forgotten.
After a struggle that took way longer than it should, I was able to get the Raspberry Pi ready for development. It is no exposed to the outside world so our partners in Texas and our mechanical guru in Montana can access the Raspberry Pi from their cell phones, tablets or other connected devices. The web page is just a place holder for other stuff we will do it, but everyone needs a login page.
You can say a lot of things about this image–it is blurry, it is too dark, it manifests the starry night problem, etc., etc. Still, it is our first image out of the bean sorter cam connected to a Raspberry Pi. I am going to do some infrastructure stuff to be able to pull stuff down easily from the embedded computer, but I will be moving on to work on the lights Gene sent me within a few days. Of course those days extend out quite a bit because I have a day job. Nevertheless, one has to take their satisfaction when they can get it and this is satisfaction any engineer might understand.
Today was a good day at work. It all had to do with sorting and measuring spuds on a conveyor, but that is a story for when we see each other face to face. The other reason it is a good day is the image below. I spoke prematurely when I said I had everything ready to go with the bean sorting development environment for the Raspberry Pi. I was wrong. It turns out the stuff I had on my development computer was incompatible with the stuff on the Raspberry Pi and it took me until about 15 minutes ago to get it all sorted out. Hopefully, I have a shot at getting the camera going on the RPi and maybe even getting started in on controlling the lights we need for the project with the RPi. Another fun filled weekend!
A couple more hours and remote debug is up and running. Develop on my desktop, deploy and debug over wifi to the Raspberry Pi. It took about 14 hours all told, but it was interesting and all worth the investment. Now it is on to getting the camera working on the RPi.
I got up to my office about 7:00 AM this morning and have been programming steadily since then. Well, I call it programming. Really what I was doing was trying to figure out how to get Raspberry Pi programs I write and build on my laptop (that I use as a desktop) to cross compile with Qt Creator so they will run on the Raspberry Pi which is what we started with on our coffee bean sorting project because it is cheap and we are cheap. I finally got it all to work about 12 hours later. I am wildly happy to have the bulk of this out of the way. Now I can bet back to thinking about coffee beans. Now the program I compiled previously on the Raspberry Pi should be fundamentally easier to debug.
The one good part about all this is that when I am programming I am generally not eating and the time flies. I did a pretty good job of staying on my diet.
Things are happening fast and furious with the bean sorter project. The pre-prototype lights Gene made for me arrived. They are just perfect. He had them all hooked up so all I have to do is flip a switch. In addition, two light controllers arrived in a separate package. Both of the are Pulse Width Modulated dimmers, but one is controlled manually with a dial and the other is controlled digital via a computer–in this case, a RaspberryPi. Can hardly wait for the weekend to see if I can get the control part of this thing going.
Yesterday, I spent my spare time on creating a camera calibration for our bean sorter project. The purpose of the calibration is to convert measurements of beans in captured images from pixel units to mm units. Images are made up of pixels, so when measurements are performed we know how big things are in terms of pixels. Something might be 20 pixels wide and 17.7 pixels high (subpixel calculations is a topic for another day). Knowing the width of something in an image is pretty worthless because the real world width ( e.g. in millimeters) of that object will vary greatly based on magnification, camera angle and a bunch of other stuff. That is a big problem if the camera moves around a lot.
Fortunately, in our case, the camera will be in a fixed location and the distance to the falling beans will always be the same. That allows us to make some fixed calculations to convert pixel units to millimeters. To that end, we put a “calibration target” in the cameras field of view at the 
position where through which the beans will fall. In our case that calibration target is a checkerboard pattern with squares of a known size. If we take a picture of the checkerboard pattern, then find the location of each square in the image in pixels, and store that information away.
Notice the red marks at each intersection of squares in the checkerboard–those are the found pixel positions (e.g. 133.73 pixels from the top of the image and 214.5 pixels from the left edge of the image). We can then convert the positions and sizes of found beans in the image from pixel units to mm units by using equations derived from the know mm sizes of the squares and the found position of the squares in the image as measured in pixel units. I used to have to hand write the equations to do this, but now there are open source libraries for this, so I was able to do the whole thing in an evening.
Our bean sort project has heated back up–literally and figuratively. Gene finished the prototype light stands and shipped them out. Here is a picture of them working. They heat up a ton when they are under constant current, so I have ordered a manual PWM light driver (with a dial for brightness) and an PWM driver that I should be able to control with one of the DAC’s on the Raspberry Pi. All of that should arrive within the next few days which means I am the one who is holding everyone up because there are so many things that I have to do before I can test them. I can hardly wait to get them in my hands.
In the mean time, I am working on the pixel to world coordinate calibration model for the beans. I hope to have that done sometime today–by Thursday at latest. Then I can work on controlling the lights and capturing images with the Raspberry Pi.
Some good news and some good news arrived yesterday. The first is that my participation in the sickle cell disease diagnostic project is wrapping up. I will still be on call for the machine vision elements of the project, but I will not be tasked with the day to day programming any longer. The second is a good friend (Gene C.) I have known since I was a child has agreed to work with me on a side project. We are going to make a “cheap but good” coffee bean inspection machine. There are lots of machines that do that, but none of them are particularly cheap in the way we want our machine to be cheap. We hope to do this for another friend who lives in Dallas.
I bought two lights I plan to use for the project. One of them is a back light and one of them is a ring light. I am pretty sure we will not be able to use these in our finished instrument, but they will certainly help me with development of lighting and optics. I still need to buy (at least) a few m12 mount lenses and a cheap USB microscope. I already have a camera with the wrong lens, but it has allowed me to start writing the program I will use to do image processing and classification algorithm development. I got it to take pictures before I went to bed last night.