Inductive learning

Overview

Learning is essential for dealing with unknown environments.
Learning compensates for laziness or lack of time in designing agents.
Learning agents have four components:
- performance element--responsible for choosing actions
- learning element--makes changes to the performance element
- critic--judges the agent using an external performance standard
- problem generator--suggests (sub-optimal) actions that will provide learning experiences
There are three main types of learning:
- supervised--inputs and correct outputs are perceived by the agent
- reinforcement--inputs and reward/punishment perceived by the agent, not correct output
- unsupervised--inputs only told to agent; may learn to predict future based on previous percepts
Learning can be cast as a problem of coming up with a function
The difficulty of learning depends on the type of representation chosen:
- logical sentences
- belief networks
- neural networks
- decision trees
Decision trees are efficient for learning a sub-class of Boolean functions.
Ockham's razor suggests choosing the simplest hypothesis that matches the observed examples.
The information gain heuristic tends to find simple (ie, short) decision trees.
The performance of inductive learning algorithms is measured by their learning curve which shows prediction accuracy as a function of the number of observed examples.

The performance standard must be external or the agent could cheat and change it.

The type of learning possible depends on the kind of inputs and outputs available.
Inductive learning can be viewed as learning a function by selecting a function--the hypothesis--from a set of functions--the hypothesis class.

We have previously discussed decision trees, which can express Boolean functions.
Algorithms exist for learning in many other representation languages.
There is a tradeoff between expressiveness of the language and the speed and number of training examples needed to learn.

Today we will discuss the information gain test for choosing attributes to split on.
The information of an answer to a question depends on how many possible answers there are and how probable each of them is.
If there are n possible answers, with probabilities P(v_i), then the information of the answer is
```
	I(P(v_1), ..., P(v_n)) = SUM_i=1^n -P(v_i) log_2(P(v_i))
```
Suppose n=2, then with a fair coin the information of the answer is 1 bit. With a completely rigged coin, it is 0 bits.
The decision tree provides an answer to a yes/no question, so we can write the information of that answer as:
```
	I(p/(p+n), n/(p+n))
```
If we split on attribute A which has v possible values, then the i'th subtree has information:
```
	I(branch i) = I(p_i/(p_i+n_i), n_i/(p_i+n_i))
```

On average we still need this many bits to classify the example:

	Remainder(A) = 
	  SUM_i=1^v (p_i+n_i)/(p+n) I(p_i/(p_i+n_i), n_i/(p_i+n_i))

So we have gained the following number of bits of information from the attribute test:
```
	Gain(A) = I(p/(p+n), n/(p+n)) - Remainder(A)
```