WORCESTER POLYTECHNIC INSTITUTE

Computer Science Department

CS 4341 ❏ Artificial Intelligence

# A QUICK INTRODUCTION TO GENETIC ALGORITHMS

Version: Sun Mar 17 18:58:54 EDT 2013

## GAs -- Introduction

Genetic Algorithms use the idea of genetic reproduction and natural selection to solve problems. In the real world, fitter members of the population survive longer and breed more, passing their good qualities onto new members of the population. Weaker members of the population die. Using GAs, we make strong members correspond to good solutions to the problem, and weak members to poor solutions.

## GAs -- Concepts

individual - representation of a possible problem solution as a coded sequence

population - set of individuals

fitness function - an evaluation function that defines the value of an individual

operators - crossover, mutation, replication, selection

## GAs -- General Procedure

From a starting population keep repeating a cycle of actions consisting of applying operators to generate new members of the population, and weeding out the weaker members. After a while, the weak members will die out, and only strong ones remain. Pick the best as the answer.

1. define an initial population P

2. evaluate P

3. while P doesn't contain an acceptable solution
and you still want to continue looking...

3.1 replicate selected individuals

3.2 create new individuals by crossover and mutation

3.3 evaluate new offspring

3.4 select some individuals to produce a new population

## GAs -- Search Space

• sample problem

To decide on the best type of restaurant to start. There are different types, depending on what sort of food, what sort of drink, and what sort of service.

• encoding

Use 0 or 1 for choices (i.e., features).

First choice is Hamburger (0) vs. Fish (1)

Second choice is CocaCola vs. Beer

Third choice is Quick&Casual vs. Slow&Formal

e.g., 000 = Hamburger/CocaCola/Quick&Casual

000 is a "feature string"

Encodings do not necessarily have to be binary.

• GAs good for large search spaces.

This example (unfortunately) is a small search space. There are 8 possible feature strings (i.e., a search space with possible 8 answers in it). Note that with larger search spaces it becomes infeasible to evaluate ALL of the possible answers. You have to search, but only have time to look at some of the space. Something has to control where to look. Unless there is a very good heuristic controlling the search, it will miss answers.

## GAs -- Fitness

A fitness function evaluates each element of the population (individual) to determine how good an answer it is, i.e., how fit it is.

Assume that we want to determine the most effective combination of food, drink and service based on the income it brings. We know that each Hamburger meal means 4 cents more benefit than a Fish meal, a CocaCola drink means 2 cents more benefit than a Beer, while Quick&Casual service provides 1 cent more income per meal than Slow&Formal. Considering the Fish/Beer/Slow&Formal combination as the reference, we define our fitness function for each combination (individual) as the supplementary income in comparison to the reference.

For example the value of the function for the combination Fish/CocaCola/Quick&Casual (100) would be 0 + 2 + 1 = 3.

The fitness will influence which individuals will survive through to the next generation.

## GAs -- Crossover

• The "crossover" operator is equivalent to gene splicing.

e.g., A crossed with B

Let A = 001       Let B = 110

Pick a (usually random) point in feature string at which to make crossover. Say, between the first feature and the second. Put the first part of A with the second part of B, and vice versa.

```	     |
A  0 | 01              1 | 01              101

X         ==>              ==>

B  1 | 10              0 | 10              010
|
```

A and B, the strings to be crossed, can be selected in a variety of ways. For example, one could pick them at random, or one could make sure that A is always the best available, or that A and B are the two best available.

The hope is that the new string will inherit and combine the good features of the two parents, surpassing both of them.

The goal is to explore the space without having to look at ALL of it -- and to do it effectively in parallel.

The technique is to have the best answers have a high influence on the search.

## GAs -- Mutation

• The "population" is the set of possible answers currently being considered.

e.g., { 011 111 000 100 }

• The "mutation" operator is used occasionally to prevent the search from getting "stale". It consists in modifying a randomly selected feature of an individual.

Pick a random individual from population, e.g., 100. A mutation of the third feature would result in

100 ==> 101

The new population is:

{ 011 111 000 100 101 }

Mutation is usually don't only a little (e.g., < 10%).

## GAs -- Replication/Reinforcement

One way to give really good individuals more influence is to increase the number of each type of element in proportion to fitness, by replication (i.e., add extra copies). If crossovers are done between randomly selected individuals, then the good ones will stand a better chance of being part of a crossover as there are more of them, and therefore stand a better chance of influencing the new population positively.

Note that to maintain diversity there should be a non-zero possibility of a low-fitness element being selected, so they should not be removed.

Assume we have defined the following ratings of individuals, based on the values of the fitness function previously defined:

Values of fitness function & Rating of individual

```		      0 - 2      poor
3 - 5      fair
6 - 7      good```

Accordingly, the rating of the individuals in the following population would be:

```	Population    { 011  111  000  100 }
Fitness          f    p    g    f ```

Replication of individuals based on their rating yields:

```	Population     { 011  011  111  000  000  000  100  100 }
Fitness           f    f    p    g    g    g    f    f ```

## GAs -- Selection

Selection decides which individuals will survive through to the next generation. Operators add new individuals to the population, but only a fixed number of these individuals are supposed to survive. Selection takes care of keeping the population size within predetermined limits by using various criteria such as fitness and diversity of individuals.

## GAs -- Simple Example

Let's try to solve the restaurant problem introduced earlier.

• Pick a random initial population and evaluate it

Population - Fitness - Rating

001 - 6 - good
110 - 1 - poor
010 - 5 - fair

Fitness average of the initial population: 4.00

• Replicate selected members

New population - Fitness - Rating

001 - 6 - good
001 - 6 - good
001 - 6 - good
010 - 5 - fair
010 - 5 - fair
110 - 1 - poor

• Create new individuals

• Crossover (both individuals selected randomly)
(note that more frequent individuals stand more chance of being selected)

```	               |
0 | 01                          010
X              ==>
1 | 10	                     101
|```

• Note that 101 is an individual we haven't seen yet

• Mutation (random selection of individual and feature)

010 ==> 011

• Evaluate offspring

New population - Fitness - Rating

001 - 6 - good
001 - 6 - good
001 - 6 - good
010 - 5 - fair
010 - 5 - fair
010 - 5 - fair (new)
011 - 4 - fair (new)
101 - 2 - poor (new)
110 - 1 - poor

Fitness average before doing selection: 4.44

• Do selection based on fitness and by removing duplicates in order to preserve diversity (this is one technique).

New generation - Fitness

001 - 6
010 - 5
011 - 4

Fitness average of new generation: 5.00

• Replication

For this generation, as an example of what is possible. we will favor the best element by using it in the crossover, rather than by adding copies to the population.
Best element = 001

• Create new elements

• Crossover the best string with a random string.

```		           |
0 | 01 (best)                     011
X                    ==>
0 | 11(random)                    001
|```

• Evaluate offspring

New population - Fitness

001 - 6

001 - 6 (new)

010 - 5

011 - 4

011 - 4 (new)

Fitness average before pruning: 5.00

• Do selection

New generation - Fitness

001 - 6

010 - 5

011 - 4

Fitness average for the currently selected generation: 5.00

No new individuals have been added or survived in this new generation. The fitness average is the same as in the previous generation. Actually, in this case mutation is the only operation likely to generate an element of higher fitness.

• Etcetera

Repeat until no significant change, or until extremely profitable solution found.

• Solution

Get rich by offering 000 = Hamburger, CocaCola, Quick&Casual

## GAs -- Large search spaces

The more features you encode in the strings the larger the search space. Imagine all possible strings scattered close together all over a floor. Consider fitter strings to be "taller". Many tall strings together would form a peak. A single peak with a steady slope from all parts of the space will result in a form of hill-climbing (see figure). But there could be many, many peaks, and isolated peaks (see figure).

Searching the peaks one at a time would take a very long time. The members of the population act as samples all across that space. Preference is given to "higher" strings. Thus the GA looks for peaks, and tends to prefer strings at or near the peaks. But new strings generated by crossover can take the search into new areas of the search space. Making selections using some diversity measure improves performance (see figure).

In addition to that, if we give a string with, say, a 1 in the first position a high fitness, then we actually give a preferential status to all strings with a 1 in the first position. The fitness function just gives a fitness, it doesn't say which feature caused it. We will tend to prefer that string for crossovers. That 1 in the first position will tend to get in to other strings. Similar arguments could be made for any feature.

Thus, the GA mechanism, when it uses a string with a 1 in the first position, is actually working with all strings of that form (i.e., 1** ). This gives the GA search a sort of inherent parallelism, as each string you deal with in a crossover, for example, is actually representing lots of other strings as well. The same sort of argument can be made for combinations of features (e.g., the substring *11 for example).

## GAs -- Classifier Systems

Classifier systems are sets of rules.

Each rule recognizes some situation and takes some action. The action is usually to produce some sort of output, but it could be to trigger some real action (robotic agent).

Rules are thought of, and often represented as, situation-action pairs. The left hand side (LHS) does the situation recognition. The RHS acts.

The name "Classifier system" is used because a rule classifies a situation as one that the rule applies to. That is, a rule classifies situations into those that the rule knows how to respond to, and those that it can't deal with. This terminology for sets of rules seems only to be used with GAs.

To emulate a Frog.

• Classifier alphabet

{ 1 0 # }

Equivalent to: Yes, No, Don't care.

• Sample rules (i.e., a sample population)

An example of a rule is:

IF Moving THEN Flee!

This is shown in the first row, below, by the string 1####10.

Notice that rules too can be represented by strings.

```LHS                                              RHS
---------------------------------------------------------------------
Moving,   OnGround,   Large,   Far,   Striped,   Flee!,  Pursue!

1         #           #        #      #          1       0

1         0           0        0      #          0       1

1         0           0        0      1          0       0```

When these rules are used, during frog emulation, if more than one rule applies to the current situation, the rule with the most specific matching LHS is chosen. For example, the 3rd rule is more specific than the 2nd, which is more specific than the 1st.

• Frog Fitness

The fitness test we've used in the restaurant example above gave a score to each string in the population. For this example, each rule could be tested to see whether it produced the right action when presented with a situation. i.e., will the frog survive?!

One could also test a set of rules by using them in froggy situations, and seeing whether they did frog-like things. Each set would have the same number of rules and crossover would be responsible for the exchange of rules between different sets. Mutation should be able to change a rule into another valid rule. Fitness could be evaluated by running each set of rules independently to solve the same problems and assessing the result.

Emulated Frogs that eat well and don't get eaten will get high fitness scores; those rule sets that don't eat well but don't get eaten get medium scores; while those that get eaten get very low fitness scores!

• The population

In our example, each string represents a rule. The population is a set of strings. Each string represents a candidate for a rule that should be included in the classifier system. After we've applied the GA techniques for a while, the population should be just those rules that work well to emulate an excellent frog.

## GAs -- Using Symbols instead of Binary

The advantage of using strings, with each string of the same length, is that we don't need to have special ways to pick the crossover point -- anywhere in the string is OK, as long as it is the same place in both. If we have a rule expressed normally (i.e., in symbols and using only the predicates which do not have "Don't care" values), we have to choose the crossover point carefully, taking the syntax into account.

For example, the strings:

1000#01 and

1####10

are really:

IF Moving & ~OnGround & ~Large & ~Far THEN Pursue!

IF Moving THEN Flee!

How can we do a crossover?
For example, cutting at the forth gap between symbols gives:

IF Moving & ~OnGround | & ~Large & ~Far THEN Pursue!

X

IF Moving THEN Flee! |

producing:

IF Moving & ~OnGround

IF Moving THEN Flee! & ~Large & ~Far THEN Pursue!

Not quite what we'd like! We could tidy one up, I suppose, and ignore the other, to give:

IF Moving & ~Large & ~Far THEN Pursue!

So, how can we do a crossover? We could make them the same length with some dummy tests, where "dummy" equals #. i.e., we "normalize" them.

e.g.,
IF Moving & ~On-Ground & ~Large & ~Far & dummy THEN Pursue!

IF Moving & dummy & dummy & dummy & dummy THEN Flee!

An example crossover would be:

IF Moving & ~On-Ground | & ~Large & ~Far & dummy THEN Pursue!

X

IF Moving & dummy | & dummy & dummy & dummy THEN Flee!

producing:

IF Moving & ~On-Ground & dummy & dummy & dummy THEN Flee!

IF Moving & dummy & ~Large & ~Far & dummy THEN Pursue!

which reduces to:

IF Moving & ~On-Ground THEN Flee!

IF Moving & ~Large & ~Far THEN Pursue!

At least one of those would leave the poor frog hungry.

So, how can we do a crossover?
We can restrict the crossover point very carefully by taking syntax into account. This can be a lot of trouble.
We can normalize to make it work like the strings.
Or we can use a random crossover and check for legal results: keeping just those.
Best??