For many years Dr. Pollack’s lab has been working on
electronic and software systems, which can learn and develop
on their own in open- ended innovative ways. This is based
on understanding and mimicking natural co-evolution. However,
in nature, co-evolution refers to the contingent development
between species. For machine learning, co- evolution has
come to mean the search “arms-race” type phenomena, which
can lead multiple agents to develop through their own
interaction, without the need for an intelligent designer.
Most work in machine learning, for example using Neural
Networks, involves very careful design of data representations,
which are tuned to a carefully designed learning environment.
In co-evolution, the set-up is usually as a set of players
to a “game” who start with only the rules and must develop
strategy or tactics through interaction. Generally, this
interaction is a competition for limited resources such
as places in a fixed-sized population. His lab has had
some success, for example in optimization, such as discovering
the best sorting networks and cellular automata rules,
as well as in three generations of automatically designed
robots.
However, as they developed these co- evolutionary learning
algorithms, they discovered that despite many successes,
certain phenomena arise repeatedly to prevent continuous
innovation. These phenomena are familiar from economic
markets, and include winner-take-all monopolies, boom/bust
cycles, and stable mediocre oligarchies (groups of players
who tacitly collude to protect each other from further
innovation).
Dr. Pollack’s group has been developing theoretical incentive
frameworks in which self-interested adaptive agents can
keep learning, including the development of multi- objective
or Pareto Coevolution, the discovery of new dimensions
along which to compare evolving agents, and a central
metaphor “The Teacher’s Dilemma,” which replaces competition
with symmetric teacher- student interactions. The Teacher’s
Dilemma provides a scientific basis for rewarding teachers
in a different fashion than competition or altruism. This
leads to new mechanism designs in which self-interested
agents end up forming learning communities which don’t
suffer from the equilibria phenomena.
The first major practical application of this work has
been the development of scaleable peer-to-peer learning
environments for children. These are multi-player video
games, but the highest scores accrue to players who provide
appropriate challenges to each other, turning students
into each other’s teachers. Dr. Pollack’s group launched
the first online spelling bee www.spellbee.org
in 2004 and now have 25,000 members. Initial results show
that a majority of students adapt to the Teacher Dilemma
utility and many face gradually increasing challenges
from other students. They have just launched www.patternbee.org
and www.moneybee.org
in which students present each other with geometric and
algebraic problems.
