After about a gazillion fits and restarts associated with reading the manual and getting syntax and basic concepts wrong, I have finally gotten the bean sorter web interface up and running. The idea is to be able to control the bean sorter vision system from a cell phone, tablet, or PC. I have been going around in circles for about two weeks now, but it appears I am ready to move on to the next thing. Here is a shot of my phone accessing the site from outside the LAN. I am glad to be done with this. The next step is to not just load a previously captured image, but to allow users to capture images by clicking a link.
Category: Technology Page 3 of 9
Kiwi and I continue to program hard on our Raspberry Pi and coffee bean project. You can see from the image at the right that we made a small breakthrough. The capture control is now on the Raspberry Pi as well as the ability to display the last image via the web. I suppose it is not such a big deal, but since I have generally worked on image processing algorithm development and the analytics that go along with that, I am not such a great web programming. It has been frustrating but fun to work through getting the syntax and understanding the idiosyncrasies of embedded web servers and programming tools. Nevertheless, I have the main part of the start of the struggle behind me 
and look forward to returning focus to the application at hand. The camera is set up with the calibration target underneath. The next step will be to create a way for the web server to talk to the camera control program so a user can capture an image on demand, calibrate the system, run a bunch of beans, look at logs to assure everything is working right, etc. etc. I have done more of that kind of thing than web programming, but I am sure there is a lot of minutiae that I have forgotten.
After a struggle that took way longer than it should, I was able to get the Raspberry Pi ready for development. It is no exposed to the outside world so our partners in Texas and our mechanical guru in Montana can access the Raspberry Pi from their cell phones, tablets or other connected devices. The web page is just a place holder for other stuff we will do it, but everyone needs a login page.
The old laptop Lorena used as her main computer died yesterday, so in the middle of my day job and bean project, I got to build her a new computer. I have to admit I had quite a bit of fun with it. It is based around a Kangaroo $99 computer with an Atom processor, bluetooth, wifi, etc. The computer is that is the smart phone shaped thing between the coffee cup and the terabyte hard drive. We have the following things connected:
- Monitor
- Mouse
- Keyboard
- 1 TB drive
- Webcam
I ordered a cheap USB speaker and will get a microphone so we can use the system for Skype. It hooked up to the wireless internet with no problem. We installed Xubuntu on it which is a lightweight version of Ubuntu. It is actually pretty snappy and just what Lorena needs to browse the web. I kind of wish I had one myself. I will just have to be satisfied with my Raspberry Pi.
Got the bean sorter program running on the Raspberry Pi. It was WAY less trouble than I expected. I am not sure whether it is because I have more skills than before or they have just made it all easier. Probably a little of both. Next task is to disconnect the camera from my development computer and try to get it going on the RPi.
Update: Recognizing the camera, but taking black images so far. Giving up for the night.
The other thing that arrived today is my new diet Logitech H800 Bluetooth/Wireless headset. It allows me to connect it to my computer with their little custom wireless dongle and to my telephone via Bluetooth. The reason it is a diet headset is that it allows me to not be tethered to my desk while I am talking on the phone or via skype. That is a FINE thing because I get in extra steps and it allows me to either eat more or drop weight more or both.
Things are happening fast and furious with the bean sorter project. The pre-prototype lights Gene made for me arrived. They are just perfect. He had them all hooked up so all I have to do is flip a switch. In addition, two light controllers arrived in a separate package. Both of the are Pulse Width Modulated dimmers, but one is controlled manually with a dial and the other is controlled digital via a computer–in this case, a RaspberryPi. Can hardly wait for the weekend to see if I can get the control part of this thing going.
Our bean sort project has heated back up–literally and figuratively. Gene finished the prototype light stands and shipped them out. Here is a picture of them working. They heat up a ton when they are under constant current, so I have ordered a manual PWM light driver (with a dial for brightness) and an PWM driver that I should be able to control with one of the DAC’s on the Raspberry Pi. All of that should arrive within the next few days which means I am the one who is holding everyone up because there are so many things that I have to do before I can test them. I can hardly wait to get them in my hands.
In the mean time, I am working on the pixel to world coordinate calibration model for the beans. I hope to have that done sometime today–by Thursday at latest. Then I can work on controlling the lights and capturing images with the Raspberry Pi.
Gene and I continue to make progress on our bean inspection project. Here is the first pass at measuring bean size as beans drop past the camera. This includes finding the bean in the image, calculating its contour and measuring how big it is in the image. The next step is to convert the bean size in the image measured in pixel units to the size in millimeters. I am half way into that.
The other thing I did this weekend was load up a Raspberry Pi with the latest Raspbian OS and got it running on the network. Right now, I am doing all my work on my Linux PC, but the idea is to move it over to the Raspberry Pi as soon as it is a little further up the development path because that is a much cheaper computer. There are some other options that might be even better (cheaper and faster), but I have a Raspberry Pi, so that is where we will start.
Gene is not sitting still either. He has built me up some prototype lighting, but I will save that for a post of its own.
I bought a global shutter camera from Ameridroid for Gene’s and my new project. It is a pretty amazing little camera, especially for the price. It is a USB 3.0 so it runs fast. I do not have the lens I need for the application we are doing so I ordered a three lens kit (need it anyway). I hope to be able to start testing beans falling past the camera before the end of the holiday, but that might be a little ambitious.
The other really good thing about this camera compared to the ov5640 cameras I have been using is that the Korean company, WithRobot, that makes the camera provides great, freely available libraries to control all the things the camera does. If I can get the camera control into our proto-type program, we will have made a major step in getting to the point where we can actually start developing a product.
The camera for the project Gene and I are working on arrived. It is pretty amazing. It cost literally 20% of the cost of what a camera with similar features would have cost only five years ago. I am dying to try it out, but in my normal bonehead manner of operation, I did not get the USB 3.0 cable I need to make it work. I was highly confident I had the right cable. It turns out I have a gazillion cables that are rapidly in the process of becoming obsolete and a gazillion more that ARE obsolete.
Yesterday I bought a machine vision camera for the project my buddy Gene and I are doing to build a (semi-)cheap little machine to inspect coffee beans. We need something called a global shutter camera because the beans will be in motion when we capture their images. In the past a camera like this would have cost in the $1000 range. Over the years they dropped to $300-$400. Yesterday, I paid $135 for this camera–quantity 1–and that included shipping. If this is coupled with a Raspberry Pi and OpenCV (~$200 with a power supply, heat sink, and other necessary stuff), it is possible to build a vision system that is faster (by a lot) and smarter (by a lot) than the vision systems we used to sell when I started at Intelledex in 1983 for $30k (~$74k in today’s dollars). The upshot is that it is now possible to do tasks for cheap that no one would have ever thought possible. There are large categories of machine vision problems that companies are accustomed to paying through the nose to solve. That is truly not necessary anymore if one is smart enough to put the pieces together. I hope we are smart enough.
A very cool new toy arrived at our door a couple of days ago. It is a camera copy stand that will hold a cellphone or other arbitrary item that fits a certain form factor. It will be invaluable in the bean sorting project, but I will be able to use it for a lot of other machine vision image capture testing tasks. We have made a lot of progress on the project and expect to start buying the actual cameras and lighting we hope to use within the next few weeks. The next difficult task is to drop the beans past a camera while we synchronize the capture and lighting of the scene so the bean is not blurry and we can see the surface of both sides very well. There are more tricky little things we need to do after that, but we cannot move forward until we know that is possible. As soon as that is figured out, I will see if Gene can help me set up a more permanent test fixture to systematically drop beans past the cameras so we can gather a test set from which to build a classifier.
The last time Bob and Gena were here, Bob dropped off his spotting scope for us to try for a little while. We knew we had pretty bad optics on all our cameras, but now we are beginning to understand how bad they really are. The scope he brought by is nothing short of amazing and WAY too addictive. We have all been taking turns. Now if I could only figure out how to take a picture through the thing. I can hardly wait until the eagle comes back.
Some good news and some good news arrived yesterday. The first is that my participation in the sickle cell disease diagnostic project is wrapping up. I will still be on call for the machine vision elements of the project, but I will not be tasked with the day to day programming any longer. The second is a good friend (Gene C.) I have known since I was a child has agreed to work with me on a side project. We are going to make a “cheap but good” coffee bean inspection machine. There are lots of machines that do that, but none of them are particularly cheap in the way we want our machine to be cheap. We hope to do this for another friend who lives in Dallas.
I bought two lights I plan to use for the project. One of them is a back light and one of them is a ring light. I am pretty sure we will not be able to use these in our finished instrument, but they will certainly help me with development of lighting and optics. I still need to buy (at least) a few m12 mount lenses and a cheap USB microscope. I already have a camera with the wrong lens, but it has allowed me to start writing the program I will use to do image processing and classification algorithm development. I got it to take pictures before I went to bed last night.
My buddy (the brilliant) Andrew B. posted the following image on his Twitter feed along with a link to the article from which it came. Those who work in this arena will understand. I get angsty about whether I have chosen the right model. Most of the time, it turns out that, if I did not chose the best one, I got pretty close. Thanks Andrew.
Google, in the minds of many, is pretty much the devil. I have started to come around to that opinion myself. There are lots of people who believe this. The problem is there has never really been any other option. After the most recent spate of ignorance and draconian overreach when Google fired a guy for expressing an idea that is objectively true, I decided to see if I could bail out of use of the Google search engine because, in my opinion, they censors what is good and true to promulgate evil. I found out that is kind of a hard thing to do. There are some searches for my work at which Google is better than anything else available. That led me to the idea that I need to minimize my Google searches to those for which there is no other alternative.
The good news is that I found a solution. I wrote earlier (here) about what I believe to be malfeasance on the part of Mozilla in their treatment of a very good man. That man, Brendan Eich, got fired from Mozilla for unconscionable pretexts. Because he believes in and supports “traditional marriage” (just marriage for non-double-speak people), he got fired from Mozilla. He immediately started a competing browser company named Brave. I tried that browser when it first came out, but it really was not ready for prime time. I tried it again today because I was so feed up with Google and Mozilla and I have to say that if it is not yet ready for prime time, it is very, very close. In addition, they have a way to set my browser (I use DuckDuckGo to avoid Google for the vast bulk of my searches) and Google only for those where it works best–stuff where they cannot impose their extreme hard left-wing bias. Brave facilitates all of this and I am grateful. Check it out. Don’t get swallowed by the Borg.
We found the power cable to the charger for Christian’s Nikon D90 Camera. We set it up to take pictures of stuff going on in the yard, the area just above the Chehalis river, the town behind it, the hills behind that, and finally Mount Rainier behind that. We cannot see the river, but that draws a ton of wildlife so we absolutely benefit from its presence. We kind of have not figured out the camera yet, but eventually I will find the time to figure it out and start to take some photos.
We had a long talk with Christian about which zoom lens we should get.
He said, “What do you want to do with it?”
We said, “We want to take pictures of the mountain, stuff like deer, flowers, and birds in the yard, and birds like the bald eagle that was on the tree by the river yesterday morning.”
We got it all figured out, but we cannot afford it all for awhile, so in the meantime, I bought some binoculars with image and video capture capability. I am sure they are not great, but they are much better than what we have now and we can just use them as binoculars. Here is what we are getting. I will leave you with one more deer picture we took at dinnertime out on the deck this afternoon.
The internet came up today. Based on my previous experience with DSL, I have to admit I was stunned the whole process was so painless and even inexpensive. We plugged it in, filled in a few lines on the screen, set up the wireless router part of the whole deal and were up and running. Not only that, I checked the upload and download speeds and they were as advertised. I was pretty sad because we do not have access to cable internet on our property. We are not so far from town, but in a little pocket where it almost certainly does not pay to run the cable out here. I should have known that since the many years that have passed since we previously had DSL, they have gotten their act together. I know some of the speed and quality issues have to do with how close one is located to a hub and we are pretty close so that might be why everything works so well. On the other hand, we ARE close. That was a dodged bullet.
I will have one more work week in Texas after today. I enjoy my job and the people where I work a lot and it was agonizing to turn in my notice. Part of the job I love the most is the requirement to create sophisticated machine vision and video analytics applications with cheap USB cameras and ARM embedded computers that run embedded Linux, usually Debian. We prototype a lot of the stuff on Raspberry Pi’s which is great because there is such a big user community it is easy to quickly get answers about just about anything. There are four cameras in the article accompanying this post that range in value between $20 and $50.
All of the cameras work just fine right out of the box for the purpose for which they were design–that is generally streaming video with camera controlling the capture gain and offset. Conversely, it reduces the repeatability and precision of most machine vision application if the offset, gain and lighting controls are not managed by the application. So, it has been part of my job to dive into the driver code far enough to figure out how to set the registers that need to be set to control cheap cameras well enough to work in accord with the stringent requirements of many machine vision applications. That takes a lot of patience and, although it is not exactly rocket science, it is very rewarding when the last piece of minutiae is chased down and the stuffs starts working.
One thing I have learned is that this “big data” thing is here to stay, at least in my world of machine vision, embedded computing and video analytics. There are tons of things you can almost do deterministically that become tractable when enough data and machine learning are thrown at them. I am loving working with Weka and R and the machine learning functionality in the OpenCV library because they open up new vistas, not to mention I can more frequently say, “I think I can do that” and not squint my eyes and wonder whether I am lying.