A QUICK INTRODUCTION TO GENETIC ALGORITHMS
Version:
Sun Mar 17 18:58:54 EDT 2013
GAs -- Introduction
Genetic Algorithms use the idea of genetic
reproduction and natural selection to solve problems. In the real
world, fitter members of the population survive longer and breed more,
passing their good qualities onto new members of the population.
Weaker members of the population die. Using GAs, we make strong
members correspond to good solutions to the problem, and weak members
to poor solutions.
GAs -- Concepts
individual - representation of a possible
problem solution as a coded sequence
population - set of individuals
fitness function - an evaluation function that defines the value of an individual
operators - crossover, mutation, replication, selection
GAs -- General Procedure
From a starting population keep repeating a cycle of actions consisting of applying operators to generate new members of the population, and weeding out the weaker members. After a while, the weak members will die out, and only strong ones remain. Pick the best as the answer.
1. define an initial population P
2. evaluate P
3. while P doesn't contain an acceptable solution
and you still want to continue looking...
3.1 replicate selected individuals
3.2 create new individuals by crossover and mutation
3.3 evaluate new offspring
3.4 select some individuals to produce a new population
GAs -- Search Space
• sample problem
To decide on the best type of restaurant to start. There are different types, depending on what sort of food, what sort of drink, and what sort of service.
• encoding
Use 0 or 1 for choices (i.e., features).
First choice is Hamburger (0) vs. Fish (1)
Second choice is CocaCola vs. Beer
Third choice is Quick&Casual vs. Slow&Formal
e.g., 000 = Hamburger/CocaCola/Quick&Casual
000 is a "feature string"
Encodings do not necessarily have to be binary.
• GAs good for large search spaces.
This example (unfortunately) is a small search space. There are 8 possible feature strings (i.e., a search space with possible 8 answers in it). Note that with larger search spaces it becomes infeasible to evaluate ALL of the possible answers. You have to search, but only have time to look at some of the space. Something has to control where to look. Unless there is a very good heuristic controlling the search, it will miss answers.
GAs -- Fitness
A fitness function evaluates each element of the population (individual) to determine how good an answer it is, i.e., how fit it is.
Assume that we want to determine the most effective combination of food, drink and service based on the income it brings. We know that each Hamburger meal means 4 cents more benefit than a Fish meal, a CocaCola drink means 2 cents more benefit than a Beer, while Quick&Casual service provides 1 cent more income per meal than Slow&Formal. Considering the Fish/Beer/Slow&Formal combination as the reference, we define our fitness function for each combination (individual) as the supplementary income in comparison to the reference.
For example the value of the function for the combination Fish/CocaCola/Quick&Casual (100) would be 0 + 2 + 1 = 3.
The fitness will influence which individuals will survive through to the next generation.
GAs -- Crossover
• The "crossover" operator is equivalent to gene splicing.
e.g., A crossed with B
Let A = 001 Let B = 110
Pick a (usually random) point in feature string at which to make
crossover. Say, between the first feature and the second. Put the
first part of A with the second part of B, and vice versa.
|
A 0 | 01 1 | 01 101
X ==> ==>
B 1 | 10 0 | 10 010
|
A and B, the strings to be crossed, can be selected in a variety of ways. For example, one could pick them at random, or one could make sure that A is always the best available, or that A and B are the two best available.
The hope is that the new string will inherit and combine the good features of the two parents, surpassing both of them.
The goal is to explore the space
without
having to look at ALL of it -- and to do it effectively in parallel.
The technique is to have the best answers have a high influence on the search.
GAs -- Mutation
• The "population" is the set of possible answers currently being considered.
e.g., { 011 111 000 100 }
• The "mutation" operator
is used occasionally to prevent the search from getting
"stale". It consists in modifying a randomly selected
feature of an individual.
Pick a random individual from population, e.g., 100. A mutation of the third feature would result in
100 ==> 101
The new population is:
{ 011 111 000 100 101 }
Mutation is usually don't only a little (e.g., < 10%).
GAs -- Replication/Reinforcement
One way to give really good individuals more influence is to
increase the number of each type of
element in proportion to fitness, by replication (i.e., add extra
copies). If crossovers are done between randomly selected individuals,
then the good ones will stand a better chance of being part of a
crossover as there are more of them, and therefore stand a better
chance of influencing the new population positively.
Note that to
maintain diversity there should be a non-zero possibility of a
low-fitness element being selected, so they should not be removed.
Assume we have defined the following ratings of individuals, based on the values of the fitness function previously defined:
Values of fitness function & Rating of individual
0 - 2 poor
3 - 5 fair
6 - 7 good
Accordingly, the rating of the individuals in the following population would be:
Population { 011 111 000 100 }
Fitness f p g f
Replication of individuals based on their rating yields:
Population { 011 011 111 000 000 000 100 100 }
Fitness f f p g g g f f
GAs -- Selection
Selection decides which individuals will survive through to the next
generation. Operators add new individuals to the population, but only
a fixed number of these individuals are supposed to survive. Selection
takes care of keeping the population size within predetermined limits
by using various criteria such as fitness and diversity of
individuals.
GAs -- Simple Example
Let's try to solve the restaurant problem introduced earlier.
• Pick a random initial population and evaluate it
Population - Fitness - Rating
001 - 6 - good
110 - 1 - poor
010 - 5 - fair
Fitness average of the initial population: 4.00
• Replicate selected members
New population - Fitness - Rating
001 - 6 - good
001 - 6 - good
001 - 6 - good
010 - 5 - fair
010 - 5 - fair
110 - 1 - poor
• Create new individuals
• Crossover (both individuals selected randomly)
(note that more frequent individuals stand more chance
of being selected)
|
0 | 01 010
X ==>
1 | 10 101
|
• Note that 101 is an individual we haven't seen yet
• Mutation (random selection of individual and feature)
010 ==> 011
• Evaluate offspring
New population - Fitness - Rating
001 - 6 - good
001 - 6 - good
001 - 6 - good
010 - 5 - fair
010 - 5 - fair
010 - 5 - fair (new)
011 - 4 - fair (new)
101 - 2 - poor (new)
110 - 1 - poor
Fitness average before doing selection: 4.44
• Do selection based on fitness and
by removing duplicates in order to preserve diversity (this is one technique).
New generation - Fitness
001 - 6
010 - 5
011 - 4
Fitness average of new generation: 5.00
• Replication
For this generation, as an example of what
is possible. we will favor the best element by using it in the
crossover, rather than by adding copies to the population.
Best element = 001
• Create new elements
• Crossover the
best string with a random string.
|
0 | 01 (best) 011
X ==>
0 | 11(random) 001
|
• Evaluate offspring
New population - Fitness
001 - 6
001 - 6 (new)
010 - 5
011 - 4
011 - 4 (new)
Fitness average before pruning: 5.00
• Do selection
New generation - Fitness
001 - 6
010 - 5
011 - 4
Fitness average for the currently
selected generation: 5.00
No new individuals have been added or
survived in this new generation. The fitness average is the same as in
the previous generation. Actually, in this case mutation is the only
operation likely to generate an element of higher fitness.
• Etcetera
Repeat until no significant change, or
until extremely profitable solution found.
• Solution
Get rich by offering 000 = Hamburger, CocaCola, Quick&Casual
GAs -- Large search spaces
The more features you encode in the strings the larger the search
space. Imagine all possible strings scattered close together all over
a floor. Consider fitter strings to be "taller". Many tall
strings together would form a peak. A single peak with a steady slope
from all parts of the space will result in a form of hill-climbing
(see figure). But there could be many, many peaks, and
isolated peaks (see figure).
Searching the peaks one at a time would take a very long time. The
members of the population act as samples all across that space.
Preference is given to "higher" strings. Thus the GA looks
for peaks, and tends to prefer strings at or near the peaks. But new
strings generated by crossover can take the search into new areas of
the search space. Making selections using some diversity measure
improves performance (see figure).
In addition to that, if we give a string with, say, a 1 in the first
position a high fitness, then we actually give a preferential status
to all strings with a 1 in the first
position. The fitness function just gives a fitness, it doesn't say
which feature caused it. We will tend to prefer that string for
crossovers. That 1 in the first position will tend to get in to other
strings. Similar arguments could be made for any feature.
Thus, the
GA mechanism, when it uses a string with a 1 in the first position, is
actually working with all strings of that form (i.e., 1** ). This gives the GA
search a sort of inherent parallelism, as each string you deal with in
a crossover, for example, is actually representing
lots of other strings as well. The same sort of argument can be made
for combinations of features (e.g., the substring *11 for example).
GAs -- Classifier Systems
Classifier systems are sets of rules.
Each rule recognizes some situation and
takes some action. The action is usually to produce some sort of
output, but it could be to trigger some real action (robotic agent).
Rules are thought of, and often represented as, situation-action pairs. The left hand side (LHS) does the situation recognition. The RHS acts.
The name "Classifier system" is used because a rule classifies a situation as one that the rule applies to. That is, a rule classifies situations into those that the rule knows how to respond to, and those that it can't deal with. This terminology for sets of rules seems only to be used with GAs.
• Task
To emulate a Frog.
• Classifier alphabet
{ 1 0 # }
Equivalent to: Yes, No, Don't care.
• Sample rules (i.e., a sample population)
An example of a rule is:
This is shown in the first row, below, by the string 1####10.
Notice that rules too can be represented by strings.
LHS RHS
---------------------------------------------------------------------
Moving, OnGround, Large, Far, Striped, Flee!, Pursue!
1 # # # # 1 0
1 0 0 0 # 0 1
1 0 0 0 1 0 0
When these rules are used, during frog
emulation, if more than one rule applies to the current situation, the
rule with the most specific matching LHS is chosen. For example, the
3rd rule is more specific than the 2nd, which is more specific than
the 1st.
• Frog Fitness
The fitness test we've used in the
restaurant example above gave a score to each string in the
population. For this example, each rule could be tested to see
whether it produced the right action when presented with a
situation. i.e., will the frog survive?!
One could also test a
set of rules by using them in froggy situations, and seeing whether they
did frog-like things. Each set would have the same number of rules
and crossover would be responsible for the exchange of rules between
different sets. Mutation should be able to change a rule into another
valid rule. Fitness could be evaluated by running each set of rules
independently to solve the same problems and assessing the result.
Emulated Frogs that eat well and don't get
eaten will get high fitness scores; those rule sets that don't eat
well but don't get eaten get medium scores; while those that get eaten
get very low fitness scores!
• The population
In our example, each string represents a
rule. The population is a set of strings. Each string represents a
candidate for a rule that should be included in the classifier system.
After we've applied the GA techniques for a while, the population
should be just those rules that work well to emulate an excellent frog.
GAs -- Using Symbols instead of Binary
The advantage of using strings, with each string of the same length,
is that we don't need to have special ways to pick the crossover point
-- anywhere in the string is OK, as long as it is the same place in
both. If we have a rule expressed normally (i.e., in symbols and using
only the predicates which do not have "Don't care" values),
we have to choose the crossover point carefully, taking the syntax
into account.
For example, the strings:
1000#01 and
1####10
are really:
IF Moving & ~OnGround & ~Large & ~Far THEN Pursue!
IF Moving THEN Flee!
How can we do a crossover?
For example, cutting at the forth gap between symbols gives:
IF Moving & ~OnGround | & ~Large & ~Far THEN Pursue!
X
IF Moving THEN Flee! |
producing:
IF Moving & ~OnGround
IF Moving THEN Flee! & ~Large & ~Far THEN Pursue!
Not quite what we'd like! We could tidy one up, I suppose, and ignore the other, to give:
IF Moving & ~Large & ~Far THEN Pursue!
So, how can we do a crossover? We could
make them the same length with some dummy tests, where
"dummy" equals #. i.e., we "normalize" them.
e.g.,
IF Moving & ~On-Ground & ~Large & ~Far & dummy THEN Pursue!
IF Moving & dummy & dummy & dummy & dummy THEN Flee!
An example crossover would be:
IF Moving & ~On-Ground | & ~Large & ~Far & dummy THEN Pursue!
X
IF Moving & dummy | & dummy & dummy & dummy THEN Flee!
producing:
IF Moving & ~On-Ground & dummy & dummy & dummy THEN Flee!
IF Moving & dummy & ~Large & ~Far & dummy THEN Pursue!
which reduces to:
IF Moving & ~On-Ground THEN Flee!
IF Moving & ~Large & ~Far THEN Pursue!
At least one of those would leave the poor frog hungry.
So, how can we do a crossover?
We can restrict the crossover point
very carefully by taking syntax into account. This can be a lot of
trouble.
We can normalize to make it work like the strings.
Or we can use a random crossover and check for
legal results: keeping just those.
Best??