Intelligence

1 Comment

This is a brief review of the online course "Machine Learning Foundations: A Case Study Approach" from the University of Washington. I must disclose that although I worked through the examples in the course, I could not complete the assessments since I was not taking the course for credit.

The course was run by Carlos Guestrin, Amazon Professor of Machine Learning Computer Science and Engineering and Emily Fox, Amazon Professor of Machine Learning Statistics. Carlos is the co-founder and CEO of Dato, Inc (formerly GraphLab, Inc).

The ML (Machine Learning) blackbox used in the course was from GraphLab as well as the iPython Notebook programing language. These modules were downloaded to one's own computer which made it possible to play along and make coding mistakes. The fundamental modelling  design is illustrated in the diagram below -- using predicting house prices.

House-prices-ML

The examples used were:

  • Regression:  Predicting house prices
  • Classification:  Analyzing consumer sentiment
  • Clustering and Similarity:  Retrieving documents
  • Recommending Products
  • Deep Learning:  Searching for images

This course provided an excellent introduction into the world of machine learning. As the open source tools of ML become more sophisticated and easier to use, it opens the doors for anyone with an interest in data analysis or modelling for mind-blogging applications. Below is a video mashup of Carlos and Emily discussing the future of ML.

What is the bottom line about machine learning? It doesn't require any coding, it uses a tool-kit of statistical methods and the data is split into 'training data' and 'test data' where the training data is used to fit into a regression or a nearest-neighbor (or any other) statistic by iteration until there is convergence in the error and then this is evaluated by using the test data. The modelling is theory-free and only requires good data.

Tao-to-MS2

If you have read the previous post The Tao of Intelligence then you can ignore this one. I am trying an experiment to see how much the title of a blog post can influence the traffic to the post. However the change of title is not a trivial one. How else could a ssRNA molecule of 3569 nucleotides result in the building of an elegant icosahedral virus particle (pictured above) and orchestrate a molecular dance of grace and poise utilising the twists and turns and foldings of its RNA genome?  All of this in the absence of a Designer — just by following its ‘path’ — the force of intelligence.

Tao-to-MS2There has been a lapse in recent posts to this blog due to deaths in the immediate family. They have served as a reminder of one’s own mortality and ‘resets’ one’s priorities.

This post is a concluding one on the topic: ‘the equation for intelligence’ — F = T . I have noticed that whenever I try to explain it at any social occasion there is either a glazed over look in the eyes of the listener or a furtive check of their smartphones for a nonexistent recent text message. Understandably most people read the news or browse the blogs to confirm their preconceptions of their world view (aided by being in a Google bubble). Rarely are they willing to undergo any questioning or change in their status quo. I think that by the age of about 25 you’ve formed a world view and you may make small changes as you go along but almost never make a paradigm shift.

It is in this context that ‘the equation for intelligence’ appeals to me. Having a Taoist predisposition to seeing the universe as self-organising and naturally flowing along intelligence lines, the universal law of intelligence proposed by Wissner-Gross  seemed imminently sensible. Having spent more than 40 years as a molecular virologist I have marvelled as to how fundamental life processes follow this knife-edge path between chaos (disorder) and stability (stagnation).  One of my favourite examples of this is the life cycle of a simple bacterial virus MS2. For the purposes of the story-telling I shall simplify some details but hopefully preserve the elegance of the life-cycle.

MS2 Phage Life-Cycle

The host of this virus is the common E.coli bacteria found in vast quantities in your gut (and faeces).  MS2 is one of simplest viruses consisting of 180 molecules of a coat protein, one copy of a maturation protein and an single-stranded RNA molecule consisting of 3569 nucleotides. The virus forms an icosahedral shell with the RNA inside, (see the right-handed side of the diagram above). This  RNA codes for four proteins — the maturation  (A-protein), the coat protein (CP), a lysis protein (which overlaps the coat protein) and the replicase protein (RdRp) for making RNA copies (see gene order and diagram below). The expression of these genes - their timing and the quantities produced is orchestrated in the most elegant way. The reading of genes on an RNA molecule by ribosomes and the protein translating system occurs from left to right (this true of all messenger RNA’s including our own).

Leviviridae_genome

When the viral RNA molecule enters the bacteria the first gene is the maturation gene (protein-A) which would normally be the first gene expressed by the ribosomes but its 5’-end is hidden within a RNA secondary structure so the first gene to be read is the coat protein which makes sense since the virus needs 180 copies. The start of the maturation protein gene is only accessible in RNA freshly replicated (before it can fold on itself) — only a few copies are made per RNA (only one copy is actually required). As the ribosomes travel along the RNA, the gene downstream to the coat protein is the replicase gene — this enzyme is necessary to make RNA copies but only a few copies are necessary since one replicase enzyme molecule can make hundreds of RNA copies. To shutdown this gene dimers of the coat protein bind to the start of the replicase gene and block further ribosomes from binding — shutting down the making of more copies of replicase.

Copies of the coat protein continue to be made — 180 are required per virus particle. Finally the lysis protein is expressed and this is controlled by ‘slip back’ by the ribosomes to the start of the lysis protein gene within the coat protein gene (the gene within a gene). Ribosomes as they travel along the RNA are ‘noisy’ and can fall off or slip —particularly at a slippery part of the RNA near the start of the lysis gene at about a 5% frequency.  This ensures that lysis expression occurs late in the life cycle (you don’t want cells bursting open before sufficient virus accumulates). The lysis protein forms pores in the cell wall of the host and the bacteria breaks open releasing hundreds of progeny virus particles that can infect more bacteria.

The key points of this story were to illustrate the elegant self-organising mechanisms that regulate the interplay between gene expression and RNA secondary structure and that the expression of the lysis gene depended on noisy ribosomes (entropic increase in the possible futures) to complete the life cycle of the virus. An example of “intelligence acting to maximise the future freedom of action — F = T ”. Or from a Taoist point of view the life-cycle follows the Tao "the path" or "the way” — the universal principle that underlies everything from the creation of galaxies to the interaction of human beings.

I am surprised that since the TED Talk by Alex Wissner-Gross there hasn’t been more positive discussions on the implications of the Equation for Intelligence (F = T  Sτ) — in other words why people haven’t been blown away by it. Most of the discussion has been from people working on Artificial Intelligence (an oxymoron if there ever was one) and in general they don’t like it.

However Michael Schaef, a physicist, has been blown away by the force of intelligence and I have paraphrase some of his comments below:

Is it a Break-Thru Like Chaos Theory?

I believe, this simple equation will change the way we see and understand the world. It may lead to a better future for mankind. I remember in the 80ies, when I studied physics, Chaos Theory emerged into physics. It provided a scientific explanation for the behaviour of complex systems. Once you understand that small differences in the initial conditions of non linear systems can yield to wildly diverging outcomes (butterfly effect),  chaos theory is simple and obvious. The Equation of Intelligence has similar qualities. Once you understand it, it is so obvious that you wonder why it was not formulated much earlier. As physicist, I really like the simplicity, beauty and generality of the equation. I also like the idea that we now can describe intelligence as a force that changes the physical world.

Hierarchy of Intelligent Systems

Let me point out, that intelligent systems are often hierarchical. Cells form plants, animals and humans. Humans form families, tribes, nations. The entire biotope of the earth is an intelligent system. Think about mankind. Each individual person has a model of the world, and the interaction of all humans form an implicit model of the world of mankind. Evolution, with survival of the fittest, is a form of intelligent system. Any believe system or memes, like a religion, atheism, capitalism or communism have a build-in set of models of the world and create forces in the world. It would be really interesting to analyse how good different believe systems are in terms of maximising the future freedom of the system. In terms of force to change the world and in terms of direction. There could be memes that have a lot of force but are misguided and die out.

Evolution on Steroids

Because intelligent systems actively increase their chance of survival, intelligent systems are systems with an accelerated evolution. "Blind" evolution depends on random variations and the power of selection. Intelligent systems can boost the chances of survival by using force to move to states with higher chance of survival.

 

Self-Organising Systems

Self-organising systems are examples of the force of intelligence — such as a city, a beehive, an ant colony, a slime mould, computer games and software.

Self_oraganizing_map_cartography

Cartographical representation of a self-organising map (U-Matrix) based on Wikipedia featured article data (word frequency). Distance is inversely proportional to similarity. The "mountains" are edges between clusters. The red lines are links between articles. (Click to enlarge)

Finally Michael Scharf concludes:

The simple formula

F = T

explains how intelligence applies forces to the world that maximises its future freedom action. If we understand and apply it, we might be able to act more intelligently, which means we may put us in a better state for the future. It might provides us with cues on how to act more intelligently as individuals, as families, as communities, as companies, as countries and as mankind.

 

Quantum-Mechanics

How does quantum mechanics at the microscopic level, at the level of atoms and electrons, play out on the macroscopic level of people, the world and ourselves? And how does the equation for intelligence (F = T ∇ Sτ) apply? Here are some of my thoughts.

In the weirdness of quantum mechanics every possible state that can exist does exist. There are a multiverse of possibilities— but like in quantum mechanics an electron can have an upward or downward spin, determination makes them either up or down. The determination (actualisation) limits the possibilities. As you move from the subatomic level to the next level — from electrons to atoms to molecules to organelles to cells to organs to bodies, determinations are made at each level. It is not a contradiction of the quantum multiverse — it is that the universe we are in, is the one where the uncertainty has been removed and we are its manifestation. The multiverse still exists but ours has converged into the one we are in.

What about the force of intelligence — the entropic growth? In your daily life each time you make a decision or a choice it opens other possibilities. You can picture yourself walking along a stoney beach where stepping on one particular stone will influence what the possibilities are for the next step and so on and so on as you walk along the beach. You don’t necessarily know which stones you will step on but they present themselves as you walk along. If you keep an open-mind as to the possible futures and if you have an open-hearted attitude to life you will be behaving in an intelligent manner.

I am reminded of the Taoist poem (#16):

Being open-hearted, you will act royally.

Being royal, you will attain the divine.

Being divine, you will be at one with the TAO.

In other words you will act in accordance with the force of intelligence.

A further comment on the Alexander Wissner-Gross TED talk and his Entropica  model — the model is not an entirely satisfactory exposition of the force of intelligence. The examples of walking upright or using tools seem unrealistic since the computer model would need to have been programmed for all possible actions and the maximisation of their diversity. In the real world you could not possibly know these parameters — all possible actions and their diversities. In my opinion Entropica has the ‘ghost in the machine’ of the programmer. The equation (F = T ∇ Sτ) makes the profound statement that intelligence is a force that acts so as to maximise the future freedom of action and keeps the options open. This is how the entropic increase of intelligence occurs.

In the words of Wissner-Gross: “Intelligence should be viewed as a physical process that tries to maximise future freedom of action and avoids constraints in its own future.”

China-44th-floor

Continuing on the theme of the equation for intelligence (F = T ∇ Sτ)

The most memorial thing about my visit to China was not the Terra Cotta Warriors, the temples, the structured gardens or the Great Wall but the view from the window of the 44th floor the Regent Hotel room looking down to the centre of Shanghai. Seeing that network of highways and skyscrapers and traffic and movement and people doing things, there was a palatable feeling of a force from all that activity of possibilities being actualised  — I can now think of it as the ‘force’ of intelligence (F = T ∇ Sτ). Things were happening, choices were been made, options were being actualised  — an entropic increase in intelligence was occurring.

This is not just a China thing, I get a similar feeling whenever I visit New York — the city that never sleeps — exhausting but full of possibilities.

The equation for intelligence (F = T ∇ Sτ) may have more relevance to our daily lives than we would have thought.

By and large in our daily lives we do not concern ourselves with the weirdness of quantum mechanics although we marvel at the products of quantum mechanics — the lasers, the integrated circuits, our cellphones, our computers, GPS, etc. However when we do try to think about it, it makes our head hurt. We accept that without this understanding we would still be in the 19th century with the Newtonian physics of billiard balls and heat exchange engines.

The idea that the universe is becoming more intelligent by way of some entropic accumulation as outlined in the equation of intelligence is intuitively reassuring. This means that there is a force that is moving the universe towards greater intelligence. This is all very well, but how does it affect us on a daily basis? For example we may not be able to explain gravity, another force in the universe, but we well know the consequences of it when we fall off a ladder. Is there an equivalent example of the force of intelligence? Yes I think there is.

This brings us to the Flynn Effect.

Jim Flynn

The Flynn effect is the substantial increase in average scores on intelligence tests all over the world. When IQ tests are initially standardized using a standardization sample the average result is set to 100. By convention, the standard deviation of the results is set to 15 points. When IQ tests are revised they are again standardized using a new standardization sample and the average result set to 100. However, if the new sample is tested using older tests in almost every case they score substantially above 100. The effect has been observed in most parts of the world at different rates. The Flynn effect is named for James R. Flynn, who did much to document it and promote awareness of its implications.

The effect's increase has been continuous and approximately linear from the earliest years of testing to the present. There are numerous proposed explanations of the Flynn effect such as improvements in early education (especially for girls), removal of lead paint, increased sophistication of tests, better test taking attitudes and adequate nutrition. But none of these explanations seem adequate to explain the effect. Flynn marveled at the magical constancy of the effect and wrote: “It’s as if some unseen hand is propelling scores upward”. Is it the ‘unseen hand’ or is it F = T ∇ Sτ ?

It is interesting to consider if there are other examples of the force of intelligence in our daily lives.

5 Comments

Intelligence-Equation
Alexander Wissner-Gross developed a single equation to explain intelligence, he calls it, "the closest thing to an E=MC2 for intelligence."  In the equation, F is a force that acts so to maximize future freedom of action. It acts with some strength, T with the diversity of possible accessible futures, S, up to some future time horizon, Tau. In short, intelligence maximizes future freedom of action and keeps options open. Intelligence not only correlates with the production of long-term entropy, but actually emerges directly from it.

BUT we have been taught to think of entropy as a bad thing. "Things fall apart; the centre cannot hold; anarchy is loosed upon the world" -- William Butler Yeats.

Throughout history, poets, philosophers and heretics of all kinds have tried to express the kind of openness of mind that leads to reliably good outcomes. In the 70s Fritjof Capra’s The Tao of Physics introduced a generation to the intersection of science and Eastern thought. Now, we can add Alexander Wissner-Gross, a computational physicist to that crowd.

As Anthony Kosner explains in his Forbes article, "It may be that it has taken us more than  century to acclimate to the intellectual disruptions set in motion by Einstein. In classical physics, entropy is the enemy, something that tears down what we build up. But in the quantum world, the chaos of entropy is the vitality of life. And it reveals itself not in grand historical revolutions or world wars, but in our casual, everyday activities. We cultivate open minds not because we are liberal or conservative, young or old, but because we understand intuitively that it is a matter of survival—physically, emotionally and intellectually—to maximize access to future possibilities."

Whether you agree or disagree with the idea that this explanation of intelligence is too good to be true, treat your intellect to the TED presentation by Alexander Wissner-Gross: