powered by NetLogo

WHAT IS IT?

This model applies a small neural network to the problem of recognizing emergency conditions. Learning emergency response rules requires recognizing several information inputs. In this case, there are two inputs, emergency-X and emergency-Y, which could also just be two emergency conditions of any types. The model is general enough to apply to any types of inputs, which could correspond to emergencies in different locales, two overwhelmed healthcare facilities etc, or any two types of information input.

There are several layers of neural networks here (including a “hidden layer.” If there were only one or two layer then the conditions and rules could not be learned.

The artificial neural network tries to recognize the combination of input (circles at the top), categorizing them correctly and signaling this in the output nodes (circles at the bottom). We feed the net many trials, and after each we tell it the answer. That is we run trials of different combinations over and over and the net learns which combination of alerts correctly signals an emergency. A “supervisor” knows if the answer is correct or not. Once the superviser tells the neural net if the answer is correct or not, the neural network adpats by changing is activation levels, until it correctly classifies all inputs correctly. Thicker lines are higher activation, like thicker pipes allow more flow. Red lines are positive activation, and white lines are negative, inhibitory activation.

This net learns under what conditions to classify an emergency. Specifically it learns to recognize only one, more than one, both, so that it correctly categorize an emergency and know how to respond. This recognition is necessary for learning condition-action rules, such as IF only one emergency condition, THEN can respond directly, without asking for outside help. But IF two emergency conditions THEN ask for help. It turns out it is harder for a net to learn to recognize “only one,” compared to recognizing both or >=1.

HOW IT WORKS

Initially the weights on the links of the networks are random. When inputs are fed into the network on the far left, those inputs times the random weights are added up to create the activation for the next node in the network. The next node then sends out an activation along its output link. These link weights and activations are summed up by the final output node which reports a value. This activation is passed through a sigmoid function, which means that values near 0 are assigned values close to 0, and vice versa for 1. The values increase nonlinearly between 0 and 1 with a sharp transition at 0.5.

To train the network a lot of inputs are presented to the network along with how the network should correctly classify the inputs. The network uses a back-propagation algorithm to pass error back from the output node and uses this error to update the weights along each link.

HOW TO USE IT

To use it press SETUP to create the network and initialize the weights to small random numbers.

Press TRAIN ONCE to run one epoch of training. The number of examples presented to the network during this epoch is controlled by EXAMPLES-PER-EPOCH slider.

Press TRAIN to continually train the network.

In the view, the larger the size of the link the greater the weight it has. If the link is red then its a positive weight. If the link is blue then its a negative weight.

To test the network, set INPUT-1 and INPUT-2, then press the TEST button. A dialog box will appear telling you whether or not the network was able to correctly classify the input that you gave it.

LEARNING-RATE controls how much the neural network will learn from any one example.

TARGET-FUNCTION allows you to choose which function the network is trying to solve.

THINGS TO NOTICE

Unlike the Perceptron model, this model is able to learn both OR and XOR (exclusive OR). It is able to learn XOR because the hidden layer (the middle nodes) in a way allows the network to draw two lines classifying the input into positive and negative regions. As a result one of the nodes will learn essentially the OR function that if either of the inputs is on it should be on, and the other node will learn an exclusion function that if both of the inputs or on it should be on (but weighted negatively).

However unlike the perceptron model, the neural network model takes longer to learn any of the functions, including the simple OR function. This is because it has a lot more that it needs to learn. The perceptron model had to learn three different weights (the input links, and the bias link). The neural network model has to learn ten weights (4 input to hidden layer weights, 2 hidden layer to output weight and the three bias weights).

THINGS TO TRY

Manipulate the LEARNING-RATE parameter. Can you speed up or slow down the training?

Switch back and forth between OR and XOR several times during a run. Why does it take less time for the network to return to 0 error the longer the network runs?

EXTENDING THE MODEL

Add additional functions for the network to learn beside OR and XOR. This may require you to add additional hidden nodes to the network.

Back-propagation using gradient descent is considered somewhat unrealistic as a model of real neurons, because in the real neuronal system there is no way for the output node to pass its error back. Can you implement another weight-update rule that is more valid?

NETLOGO FEATURES

This model uses the link primitives. It also makes heavy use of lists.

HOW TO CITE

I adapted this from:
- Rand, W. and Wilensky, U. (2006). NetLogo Artificial Neural Net model. http://ccl.northwestern.edu/netlogo/models/ArtificialNeuralNet. Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL.
- Wilensky, U. (1999). NetLogo. http://ccl.northwestern.edu/netlogo/. Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL.
That model used one output node, and was the second in the series of models devoted to understanding artificial neural networks. The first model is Perceptron. The code for that model was inspired by the pseudo-code which can be found in Tom M. Mitchell’s “Machine Learning” (1997).

In other publications, please use:
- Copyright 2006 Uri Wilensky. All rights reserved. See http://ccl.northwestern.edu/netlogo/models/ArtificialNeuralNet for terms of use.

COPYRIGHT NOTICE

Netlogo copyright 2006 Uri Wilensky. All rights reserved.

Permission to use, modify or redistribute this model is hereby granted, provided that both of the following requirements are followed:
a) this copyright notice is included.
b) this model will not be redistributed for profit without permission from Uri Wilensky. Contact Uri Wilensky for appropriate licenses for redistribution for profit.