Information theory began as the study of information transmissions over a wire. Notable contributors were Claude Shannon and Harry Nyquist.
As a software developer, I relate the practical application of information theory to the skill of computer programming. The fundamentals of programming build on the basic blocks of sequence, branch, and loop. Any computer program ever written is some combination of these three basics.
Definitions
- A sequence is one or more actions performed sequentially.
- A branch tests a condition for true or false and then selects a sequence based on the test’s outcome.
- A loop combines sequence and branch by testing a condition to determine whether to repeat the sequence or exit the loop.
In an interview with Quanta Magazine, Michigan State’s Christoph Adami explains his information theory of life. He offers this definition as a starting point:
- Information is “the ability to make predictions with a likelihood better than chance”.
Before “life” can capture “information” there must be an alphabet, some set of symbols. In the proper order, this alphabet can represent instructions for biological processes. With this paradigm shift, we can look at evolution as a process of countless repetitions of trial and error. The results of these lessons learned are written in a repository of information about how to live in a certain environment: critical information, such as how to convert sugar into energy, how to flee predators, or how to self-replicate.
Our DNA is an encyclopedia about the world we live in and how to survive in it.
When the environment changes, past information is no longer helpful because it can’t predict any better than chance. This usefulness of information represents fitness, the ability to survive and thrive in face of selection pressure.
Each bit of information represents a lesson learned, at the cost of staggering improbability. Once in the realm of information, the lesson can be copied. Replication takes the information from scarcity to abundance.
But which came first, replication or metabolism? One thought Adami offers is that the original bit of information likely formed where other information appeared “for free.” In other words, the abundance of certain chemicals and thermal energy near volcanic vents means that metabolism is within easy reach, so that only the problem of replication has to be solved. Once replication existed, organisms could move into other environments to solve the question of metabolism.
Wrapping up my summary of Adami’s interview, I leave the reader with these questions: Do these two thoughts connect? Does the information theory of life reflect any of the concepts of sequence, branch, and loop? Let me hear from you in the comment section below!
I thought this was an interesting addition to the intersection of DNA and information theory. It refers to a project known as “Analysis using Denoising Autoencoders of Gene Expression.” Signal-to-noise is a key concept in information theory.
If you’re interested in diving deeper into the science behind the signal, Slashdot just posted an article on information theory