Tag Archives: Information Theory

Christian’s job is to think

Christian at IHOP Tempe, AZ May 1, 2017We saw the April 30, 2017 whiteboard (below) when we first arrived at his apartment on our move to Centralia, WA. Christian hopes he is on the verge of his first (semi) important, first author publication in a major academic journal. The work for that paper is pretty much done. He has to refine the verbiage and get past the whole scholarly review thing which is never a sure thing, but he has something that is pretty solid. The two whiteboard’s below are two consecutive days of work. I thought he spent a lot of time at the computer, but that is not really how he works. He looks at the whiteboard and then he just thinks. His job, his professor tells him, is to think. He has a second paper in mind. He hopes it will be better than the first. The first is solid–something that needed to be worked out. The second, however, is something that might be a true innovation. Something new, not yet considered, that contributes to the field. We hope so, but it is hard to know. Even after a paper like that is published, its importance might not even be know in the lifetime of the author. Truly interesting stuff. AND the whiteboards look really cool.

April 30, 2017 – Christian’s Whiteboard
Whiteboard at Christian's apartment ASU April 2017

May 1, 2017 – Christian’s Whiteboard
Whiteboard at Christian's apartment ASU May 2017

Christian presents his research

Christian at Logan Airport, Boston on MIT Lincoln Lab visitIf I have the time right, Christian, at this very moment, is presenting his Information Theory research at MIT Lincoln Laboratory. This is the first time he has done this kind of formal presentation (with a tie and all that). It is the culmination of a full year of research in a brand new (to Christian) area of Mathematics and Electrical Engineering. Lorena and I are on pins and needles waiting to hear how it went. If all goes well, this should eventually turn into a refereed conference paper and, with expanded research and content, possibly even a refereed journal article.

The next step after will be his “quals” presentation which will be this same research work but to his doctoral committee back at Arizona State. I am not sure what happens after that, but it probably has something to do with preparation for “prelims” or “comprehensive exams” which are usually pretty challenging.

Betty Blonde #378 – 12/28/2009
Betty Blonde #378
Click here or on the image to see full size strip.

Intuition at work

Christian talks on a semi-regular basis about intuition with respect to the work he does in Mathematics. He started talking like that shortly after he entered his junior year in his Applied Math degree at North Carolina State when he moved from solely applied math to more theoretical stuff, first in an introductory class titled Foundations of Advanced Mathematics and then on to Mathematical Analysis (I think that was an introduction to Real Analysis) and other more theoretical work in Abstract Algebra, etc., etc.

Yesterday the subject of mathematical intuition came up at work. A fellow who did his PhD in the same intensely mathematical area of engineering as Christian talked with me a little bit about what Christian was up against. He said something to the effect that the math of Information Theory (Christian’s area) is very complex, but no more so than other areas of higher math. The problem, he said was that intuitions, for him, in that area of math were much more difficult than the other areas he had studied. We did not have time to get to the underlying reasons for that but now it has me curious. That idea was intimated in the book I read on Information Theory, some of it having to do with the way entropy is defined differently in Information Theory than in Physics. It will be interesting to understand the why of that a little more.

Betty Blonde #297 – 09/07/2009
Betty Blonde #297
here or on the image to see full size strip.

Woo-hoo! Being as Communion arrives

Being as Communion: A Metaphysics of Information by William DembskiI pre-ordered William Dembski’s new book, Being as Communion: A Metaphysics of Information a few weeks back and it arrived today. I am really looking forward to reading it but I am still only part way into John R. Pierce’s Introduction to Information Theory: Symbols, Signals and Noise. Christian convinced me to read the Pierce book so I could have a better background to understand the Dembski book. I love the quote that came on the “Thank you for your pre-order!” card. Part of it says:

In Being as Communion, Dr. Dembski challenges the oft-made claim that mind is a myth and that everything about us–including our thoughts, our ethics, and our decisions–are ultimately the products of unguided material processes. Dembski provocatively argues that the opposite is true: In light of modern information theory, it is materialism, not mind that is “myth.”

With Christian taking his first graduate level class in Information Theory next semester, I had better pick up the pace or I will be left far behind.

Betty Blonde #198 – 04/20/2009
Betty Blonde #198
here or on the image to see full size strip.

Trying to figure out information theory

An Introduction to Information Theory: Symbols, Signals and NoiseI usually wimp out when it comes to learning hard mathematical stuff like what is required to have a working understanding of Information Theory. In this case, though, I am glad I let Christian convince me to give it a shot because it appears to be fundamental to things like how the brain works, intelligent design, statistical inference, cryptography, quantum computing and a ton of other stuff related to my work and/or are my avocational interests. I looked around for a decent introductory book that did not get so bogged down in the math that the big picture did not emerge. John R. Pierce’s book An Introduction to Information Theory: Symbols, Signals and Noise seemed to be an almost universal choice to meet this criteria.

I am half way through the first chapter of the book. It has become abundantly clear that a full understanding of Information Theory is not really possible without an engagement with the math at a deep level. Nevertheless, a review at Amazon made the following observation about the book that makes me think I am on the right path. I might need to read one or more additional books to arrive at the working understanding I want, but this will definitely get me started at a level that does not discourage me from taking the next steps. Here is an excerpt from the review:

The book is geared towards non-mathematicians, but it is not just a tour. Pierce tackles the main ideas just not all the techniques and special cases. Perfect for: anyone in science, linguistics, or engineering.

Another thing that is abundantly clear is that Christian, in his current position with his current major professor and research sponsor, has an exceptional opportunity to get a strong grounding in the area of Information Theory and that such a grounding will serve him very well whether in whatever technical research pursuit he chooses when he finishes this degree. His first research project is the solution of a difficult problem that engages specifically with the material about which I am reading, but with mathematical rigor beyond the scope of the book.

If the material is not too tedious for a general blog like this, I plan to write about it more because it is so interesting. I am early in the book and engaged with topic of entropy as it is used in the field of Information Theory. Entropy has a very specific definition in this context and is different from entropy as that word is used in thermodynamics or statistical mechanics. The bigger deal for me is that I can see it has important ramifications for even the work I do in my day job.

Betty Blonde #183 – 03/30/2009
Betty Blonde #183
here or on the image to see full size strip.