Information Theory and Entropy

“The universe always falls upward in entropy” PW Atkins The 2nd Law, Energy, Chaos and Form

“I thought of calling it ‘information’, but the word was overly used, so I decided to call it ‘uncertainty.’ When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, ‘You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one knows what entropy really is, so in a debate you will always have the advantage.” Claude Shannon

Information Theory might have more to do with entropy than it does information. In fact according to Decoding the Universe by Charles Seife Information and Entropy are twins.

Entropy on the surface is the great riddle of the universe the one that cannot be unequivocally explained away. Why in a universe where things dissolve towards disorder as a rule, one find so much complexity?

Entropy explained through the second law of thermodynamic is a measure of the configurations that correspond to a specific state. In an enclosed system entropy would be a measure of molecular disorder within a macro system. Such systems spontaneously evolve towards thermodynamic equilibrium, the state with maximum entropy.

The Second Law and Entropy Principle

” an isolated system’s entropy never decreases. These systems spontaneously evolve towards thermodynamic equilibrium, the state with maximum entropy. Non-isolated systems may lose entropy…”

Entropy Principle

“Natural processes are accompanied by an increase in the entropy of the universe.”

The earth is an isolated pocket where information isn’t dissolving into chaos as but growing. There is as yet undefinable force is in opposition to the second law. An unnamed force of organization . This tendency toward of organization can be studied in information systems as complex as the world economy.

“The man who follows the crowd will usually get no further than the crowd. The man who walks alone is likely to find himself in places no one has ever been before. Creativity in living is not without its attendant difficulties, for peculiarity breeds contempt. And the unfortunate thing about being ahead of your time is that when people finally realize you were right, they’ll say it was obvious all along. You have two choices in life: You can dissolve into the mainstream, or you can be distinct. To be distinct is to be different. To be different, you must strive to be what no one else but you can be. ” -Alan Ashley-Pittis

This quote can be found in the online resource An Introduction to Information Theory and Entropy

Entropy Example

An illustration of entropy would be a classroom full of students taking a test. The teacher announces that the students must keep their eyes on the paper and not talk. The room is organized and order reigns. Then the teachers leaves the room. A few whispers begin and then someone laughs and eventually the whole the whole room is talking. The room is marching towards greater entropy as the students embodying information grow disorganized.

The teacher keeps the order, the alpha male or female holds the room together and holds entropy at bay. Is there a pecking order among the non living and is there something in nature that acts as the alpha? A star could be considered the alpha of a solar system holding the planets in order. So could gravity be the organizer of the solar system and motion be the entropy?