The last couple of weeks has been nothing but programming around the clock. We are getting ready for a product release and a trade show at work so I have been going through the weekends at my computer, but in addition to that, I decided to do some major upgrades to the GRIME2 software I did before and during my doctoral research. The game plan is to write one more serious journal article, then move on to start working on the use of the GRIME-AI software package being developed by my friend and fellow PhD student, John Stranzl. I probably won’t get to start doing that until January or February of 2025, God willing, but that is the plan.
Category: Software Page 1 of 6
My University of Nebraska professor, Troy, just sent out his “first day of school” picture for his 26th year of school. Because of the advent of Artificial Intelligence, he decided he needed some formal education in computer and data science. He started an M.S. in Data Science about a year and a half ago and this is the last class of the first half of the program. He is picking up a ton of skills like Python, R, and a bunch of database and visualization stuff. It would not hurt me to take a few classes like that. I am sure I would enjoy them, but I am leaning toward something, maybe a little less practical and little more esoteric like something in Art, History, or Philosophy.
My entire Mexican experience informed me that the only coffee available for general consumption, even at the finest dining establishments was Instant Nescafe. I was wrong and Tío Lauro very graciously showed me the error of my ways. In his trip to Veracruz he drank the spectacular local product that is some of the best coffee in the world. In addition, he filled a sorely needed deficiency in my working life. As is the wont of many programmers, it is difficult for me to work without a mug (read mug, not cup) of coffee at my desk. Lauro arrived in Monterrey with a much needed mug of the Café de la Parroquia from Veracruz. I am forever indebted.
The current incarnation of the GRIME2 (v0.4.0.0) ground-based camera water level measurement program was a port of much earlier code that was updated and pushed to GitHub on September 12, 2020. The earliest date I can find for the code that was ported was v0.1j.0 committed (to SVN) and released on July 30, 2009. My PhD adviser at University of Nebraska-Lincoln (UNL) and I had a fairly long impromptu meeting this afternoon about the direction we are taking with this software and work. I have decided I am going to disaffiliate from UNL (to a certain extent–never not going to love my experience there) and focus my efforts on research associated with the GaugeCam GRIME Lab. That, in some sense is a difference without a distinction. What is really needed right now that I can contribute is
- publication of research that has already been performed,
- extension of the GRIME2 software package to handle much smaller calibration targets,
- integration of GRIME2 functionality into GRIME-AI (Artificial Intelligence),
- use of GRIME-AI to answer interesting hydrology, ecology, and agricultural questions,
- and publications of those results in appropriate journals.
So, that is my current plan moving forward, as part of GaugeCam and the GRIME Lab. Part of the reason for that is so that we can investigate the possibility of developing a user base and commercializing GRIME-AI as an independent entity, unencumbered by the lumbering bureaucracy of an institution of higher learning. That is why I am pleased that we had the foresight to license GRIME2 with and Apache 2 license and solely in my name long before I stared working with ANY academic institution.
I have used Linux exclusively at work for seven out of the last eight years and at home for around 15 years now. Xubuntu is my distribution of choice because it is lightweight and uncomplicated. Also, it is a pain in the neck to change distribution although I might be forced into it as there are rumors that Xubuntu will no longer be supported for the long term. I hope they are just rumors. I bring this up because of the Blue Screen of Death (BSOD) pandemic that hit Windows computers worldwide. Linux and Mac users are crowing loudly about not getting hit–although if I were a Mac user, I would be too ashamed to admit that to crow about it. I am absolutely a huge Open Source software fan, with a free (as in beer), open source software project for measuring water level with cameras called GaugeCam GRIME2 that I maintain on GitHub. I hope the reason Linux was not hurt by this problem is that is just better because it is Open Source.
We flew to Oregon last week to get together with all my siblings for the first time since our parents funeral, seven years ago. Part of the trip was set aside for a meeting with two colleagues of mine with whom I have worked for almost 40 years (Frank Evans) and around 30 years (Dr. Mark Singer). We got a meeting room in Tualatin, Oregon to discuss the technical aspects of work they do for me as contractors in my day job. It is a gift to be able to work with them. In our field, there are none better both technically and in terms of good will. I am writing this as a marker by which to honor and remember them–we are all old and live far apart–them in Oregon and me in Texas, so this might be the last time. I hope not.
Dr. J is exactly half-way through to his PhD at University of Nebraska-Lincoln. His research software is turning into something amazing indeed. This is an example of his (very preliminary) instance segmentation. Even though it is very early in the process, you can see this is going to be a powerful addition to his already powerful GRIME-AI software suite of tools. The software already downloads and merges images and scalar data according to a user specified, GUI-selectable recipe that includes data cleaning and image triage. Can’t wait to see the finished product.The software is part of the GaugeCam GRIME Lab research initiative at University of Nebraska.
Troy flew to Tuscaloosa, Alabama yesterday to attend a water conference. He will give a presentation on the work we do at the GRIME Lab. The main focus of the lab is to drive complexity of the use of ground-based imagery to answer hydrological questions. The above, fairly simple, graphic describes it well. The image on the left is pretty hard to set up and maintain, but reduce the complexity of the image processing task because of the vision targets in the image. The image on the right is way easier to set up because nothing has to be installed or maintained in front of the camera, but the processing is way harder because there are no physical references for real-world unit calibration or camera motion in the scene. We are going to be able to watch Troy’s presentation online this afternoon.
The volunteer Hydrology work I do with University Nebraska-Lincoln is starting to solidify now that I no longer have a formal position at UNL. Last night a researcher in Brisbane, Australia met online to work out the use of the GRIME2 Command Line Interface (CLI) libraries for measurement of water level at what could be over 100 separate locations if we can demonstrate capability. Then today, I met online with a graduate student who we hope will be able to extend the libraries to handle much smaller calibration targets for the purpose of crowd-sourcing imagery and placing small calibration targets at the top of garden variety staff gauges. What is great about this is that these people are starting to adopt and extend the libraries without my involvement other than to provide guidance to get them going and to help them when they get stuck.
The first “product” we developed when the GaugeCam project started back in 2009 at NCSU was called GRIM. That stood for GaugeCam Remote Image Manager. The name did not evoke a sense of positivity, so in the spirit of G.R.O.S.S. from Calvin and Hobbes (i.e. Get Rid Of Slimy GirlS), we added Educational and changed the software name to GRIME. A MUCH better name for people mucking around in the mud. Today, Troy sent me some of the accuracy research results for the project. We (and she) think it is Kelly’s handwriting, but we are really not sure. At the top of the page you can see GRIM VERSION: 0.1j. I looked at the appendix of my dissertation that holds the release notes for all the GRIME software I wrote and it says that it was released on July 30, 2009. It is cool that we have a record that shows we were doing the research from that far back.
We pretty much had to trash our exercise room to get it done, but Christian and I got our new projector installed, hooked up to a NUC (mini-computer) and pointed at the screen on the back wall. The screen was the biggest part of the project, but it was truly a marvel of engineering. The combination of the projector and the screen are nothing short of amazing in terms of image quality, especially with respect to how much we had to pay for the thing. This is actually going to be a great place to even do things like take video conferences.
My PhD adviser, Troy, and I have decided to spin back up the GaugeCam software as an open source software project with a commercial friendly license for the GRIME2 program. GRIME2 stands for GaugeCam Remote Image Manager (Educational) version 2. It is planned as kind of a Swiss army knife for hydrology research. So far, it only has functionality for measurement of water level using a target in the water, but the code to do machine learning with a variety of image and non-image features has been tested out and works well as represented by this paper.
I am a big fan of Linux and, more particularly, a big fan of Xubuntu Linux. This is not a review of Xubuntu but an acknowledgement of my biennial struggle about when to replace the previous Long Term Support (LTS) release, Xubuntu 18.04 with the new release 20.04 that is scheduled to drop on April 23. I do not think I have made it more than half a day before I bite the bullet and make the change. This will be a pretty straightforward deal with the computer I use for my PhD work and our home computers, but my day job machine is another story. We have lots of heavy lifting going on and the setup of a development computer of any stripe (Linux, Windows, or apple) for the kind of work I am doing is often a challenge and generally takes at least half a day if I am really lucky. I could just install over the top of 18.04, but that kind of defeats the fun and I like the idea of completely cleaning out cruft every now and then, so I plan to wipe the hard drive. I will continue to run into stuff I forgot for a few more days or even weeks. This is true even though I write down all the stuff I need. New thing just keep popping up.
I am all Linux all the time at both work and home. For the first time ever, I have no reason to ever boot to Windows and we avoid allowing the plague known as apple into the house. The guys at work mostly use Ubuntu, but I really like Xubuntu, mostly because its use changes only minimally. I always look forward to a new release and this one is no exception.
A couple of days ago, I broke down and bought a refurbished laptop for the bean sorting project. It surely seems to be a smoking good deal at $243.09 plus tax. I have been working to get the thing running on a Raspberry Pi and that works fine, but is way more hassle than we need during the development stage. It was necessary to hook up a keyboard, a mouse, a monitor, and the camera which, during the development stage, needs to be moved around a lot. It is just easier to do it on a laptop.
The other really big benefit is that, for the Raspberry Pi, I needed to do my development and testing on my home laptop, commit the code to a repository, switch the camera from the laptop to the Raspberry Pi and rebuild the code on the command on the RPi. That was a hassle. Now I will be able to put the entire development environment on the laptop and send it off to the mechanical designer (Gene). We can get all the lighting and controls developed with a full blown computer, then switch over to whatever cheap embedded computer we pick when everything is working. In the meantime, when I get some new code for Gene to consume, it will be way easier for him to build it and test it out with duplicate systems.
I got a book for Christian at Christmas time. While he was with us, I read the preface. I liked it so much, I bought for Kindle on my phone. The premise is that the business model represented by Google will be supplanted by a business model with block-chain money that is fundamentally more secure and monetarily stable. I love the way the author, George Gilder, writes and I believe he is write on this. This is not the first time he rightly called a sea change like this would represent.
One of the things that was particularly interesting about it was that the technology underlying the functionality that will cause the change is precisely what Christian has studied and researched for the last four years for his PhD program: Information Theory
Kelly sent this photo of her and Christian yesterday. They had a pretty good time in San Francisco over the New Year. More than anything, I think they are pretty tired. Lorena and I had a quiet day at home because I was still suffering from the residual of a cold. We DID have a big steak with a grilled onion, and a baked potato. Also, I spent most of the day working on the bean sorting project. Gene has made really big progress, so I need to get an application going so that he can have a way to see what the camera captures as the beans drop. That will allow him to develop the lighting. It is a little more complicated than just creating a capture application. We really need to find a way for me to upgrade is program (running on a Raspberry Pi) over the internet. I have done this before, so it will be great to get a little more experience at doing this sort of thing.
It has been a month or so since I have provided an update on the Retirement PhD/GaugeCam projects–they are kind of tied to each other. Everything is not complete (I think we are about halfway there), but everything that is complete has been up and running for over a month on two separate computers: 1) A Raspberry Pi and 2) a plain vanilla Ubuntu Server. The plan for over the holidays was to jump back in and see if I could get the project to the point where we could do a beta deployment, but that was not to happen–suffered a minor setback due to a cold. So, now I am talking to my buddy John H. down in Arizona (well, up actually, if we are talking altitude and not global direction). Hopefully, I can get him on the project, partially because he is a profoundly better embedded programmer than me and partially because it is a lot more fun to do this kind of thing with a friend. We are still on a long slow approach to entry into a PhD program, but it is still on track. We have added two new sets of functionality that will be required for the GaugeCam work and John will be perfect for that if he has time to do it.
If you click on the image (or here), you can see a live version of the new GaugeCam web server. Right now, the program is running on a Raspberry Pi in our house. The reason it is live web page is so professors at University of Nebraska Lincoln and North Carolina State University can critique the design and view adjustments as new features are implemented. There is on-going scholarly work associated with this software. One refereed journal article has already been published (click here) and two more are in the works. Click here to see an article on the early phases of the work on the web site of a commercial camera company (full disclosure–I wrote the article). The goal is to accommodate a variety of sensors that produce 2d and 3d images as well as point measurements and hyper-spectral images. Hopefully, this effort will continue on into my retirement both as a way to contribute and maybe even earn a PhD for this and previous work on similar projects.
I am still not full convinced I can or should do this PhD thing, but all the pieces are still in motion. I expect I should be able to make a decision to move forward or not by late winter or sometime in the spring. In the meantime, I continue to make progress on the GaugeCam reboot as Open Source software (free as in liberty and free as in beer). The video speaks for itself. We hope to present a journal article and make the beta software available for download by spring. The video kind of speaks for itself
One positive outcome of this new effort to get a PhD in my retirement years is the impetus it will give to the spinning back up of the GaugeCam project as a wholly open source project (free as in freedom and free as in beer, as they say). I will be doing the heavy lifting on the software end of the project, taking over the server part and maintaining and improving the client parts. I have started on a very basic web server to show the graphs of the water height and point to our blog and downloads and documentation. I have a bit of a learning curve on this, but am on my way (see above).
Now I just have to figure out a way to include the bean inspection in all this.