# Perceptron

This module creates a layer of (an arbitrary amount of) perceptron (or rather perceptronish) nodes. You have many learning rules and activation types to choose from, and a variable or two to manipulate. The updates to the net can be done instantly at each tick, in batches, or partially from changes in this and the previous tick (called momentum update). Those different updates are called learning types. Finally this module has separate inputs for training and mere activation (calculating an output).

## Example XML definition

A simple example

<module class = "Perceptron" name = "Perceptron" />

## Parameters

Name | Description | Type | Default |
---|---|---|---|

class | The class name for the module; must be "Perceptron". | string | |

name | The name of this instance of the module. | string | |

rand_weights_min | Lower limit of the initial randomized weights. | float | -0.5 |

rand_weights_max | Upper limit of the initial randomized weights. | float | 0.5 |

learning_rate | The factor which to multiply the change with. | float | 0.1 |

learning_rate_mod | Learning rate modifier How to modify (decrease) the learning rate over time. sqrt has the formula 'learning_rate_now = learning_rate / (0.10 * sqrt(tick + 100));' and log has the formula 'learning_rate_now = learning_rate / (0.42 * ikaros::log(tick + 10));' | choices:none sqrt log | none |

bias | A bias is an extra node that has a fixed weight. This is the value of that node. 0 is not an allowed value. | float | 1.0 |

learning_type | Decides if the nodes should learn immediately, smeared out (momentum), or in batches. | choices:instant batch momentum | instant |

momentum_ratio | If momentum is used, this is the percentage of the learning taken from the previous tick. | float | 0.42 |

activation_type | What kind of activations the nodes should give. | choices:step sign sigmoid tanh | step |

learning_rule | Which learning rule? They correspond, in order, to the following formulas in the book 'Fundamentals of Artificial Neural Networks', by Hassoun: (3.1.2, 3.1.16, 3.1.29, 3.1.30, 3.1.35, 3.1.50). [NOTE: About 'rosenblatt_margin' and 'may'. These learning rules do not work with the 'step' and 'sigmoid' activation types. More specifically, these learning rules expect the targets to be 0/1 while rosenblatt_margin and may expect the targets to be -1/1. So, even if you do not use step/sigmoid, make sure the targets are -1/1 and nothing else]. [NOTE: About 'delta'. This learning rule only works with activation types 'tanh' and 'sigmoid']. | choices:rosenblatt rosenblatt_margin may alpha_lms mu_lms delta | rosenblatt |

batch_size | If learning type batch is used, how big should the batch be? That is, after how many ticks should the nodes be updated? | int | 42 |

step_threshold | If activation type step is used, how big should the threshold for activation to occur be? | float | 0.0 |

margin | For rosenblatt_margin and may learning rules. | float | 0.2 |

alpha | For alpha_lms learning rule. | float | 0.1 |

mu | For mu learning rule. | float | 0.1 |

beta | For delta learning rule. | float | 1.0 |

correct_average_size | From how many previous ticks to calculate the CORRECT output. | int | 42 |

normalize_target | The different activation types expect different target values, sometimes -1/1 and sometimes 0/1. If normalize_target is set to true the module tries to convert your target values to the expected values if they do not suit the activation type chosen. More specifically: with step/sigmoid the target will be set to 0.0 if it is 0.0 or less, otherwise set to 1.0, and with sign/tanh the target will be set to -1.0 if it is 0.0 or less, otherwise set to 1.0. | bool | false |

## Module Connections

### Inputs

Name | Description |
---|---|

INPUT | The inputs to the perceptrons to calculate the output from. An array of floats. |

T_INPUT | The inputs to the perceptrons to learn from. An array of floats. |

T_TARGET | The targets of the perceptrons (their desired output when training). This array is expected to be filled with 0/1 or -1/1, depending on which activation_type (step, sign, sigmoid or tanh) will be used. See the normalize_target parameter. Determines the amount of perceptrons the layer will have. |

TRAIN | Array with one single value. If this value is 0 in a certain tick, the module will not do any training, otherwise it (tries to) learn. |

### Outputs

Name | Description |
---|---|

OUTPUT | The output of the last input. This array will be the same size as the TARGET input. It will contain 0/1's or -1/1's depending on which activation_type is used. |

ERROR | The error from the last input and target. An array of floats, which is the root of the sum of the squared difference between the targets and outputs... |

CORRECT | Array with one float value. This is the percentage of how many examples that were correctly classified lately (how many examples it averages over depends on the parameter correct_average_size. |

## References

Mohamad H. Hassoun (1995). Fundamentals of Artificial Neural Networks. MIT Press.

## Author

Alexander Kolodziej

LUCS

## Files

Perceptron.h

Perceptron.cc

Perceptron.ikc