In this unit we will introduce an important concept of neural network which is hidden layers.  While the details about middle layers involve a lot of advanced theories and calculation, we will only talk about it in a simple way.

What is middle layers?

• There are 3 types of layers in neural networks:
  • Input layer - in neural networks we must maintain one input layer to takes the inputs and perform some calculations through its neurons and the connections.  Then the outputs are transmitted to the next layers.
  • Output layer - in neural networks there must be always an output layer.  The output layer takes the signals which are passed in from the layers before it, and perform the calculations of the output values through its neurons.  
  • Hidden layer(s) - the hidden layers are placed in between the input and output layers.  These hidden layers are not visible to the external environment and these are private to the neural networks.  That is why they are called hidden layers.
    • Hidden layers process the data by applying complex and advanced mathematics.  These layers enable a neural network to learn complex tasks and achieve excellent performance.
    • The hidden layers make the neural network  a superior and powerful type of machine learning algorithms.
    • For large majority of problems hidden layers are required.
    • If there are a large number of hidden layers with large number of neurons, the neural network may take a very long time to do the calculation and produce the output.
    • A neural network which has multiple hidden layers is called a Deep Neural Network.

Below shown a neural network with 2 hidden layers.

One of the ways to realize the importance of hidden layers is to look into how computer vision operates.

Deep neural networks that consist of many hidden layers can achieve impressive results in image recognition including face recognition.

The first hidden layer of the neural network learn to detect short pieces of corners and edges in the image.  These features are easy to detect given the raw image but are not very useful by themselves to recognize the identity of the person in the image.

Then, the middle hidden layer combine the detected edges from the previous layers to detect parts of the face like the eyes, the nose and the ears.

Then, the final layer combines the detectors of the nose, eyes, etc. to detect the face of the person.

Objective:

- train the AlphAI robot to recognize 4 faces

See appendix A1

- Check how the hidden layers are shown on the screen.

- Switch on and connect the robot to your PC.  Press the <Connection> button.

- With the the AI parameters set up correctly, the above neural network will be shown.

- Fill in the names of the 3 participants in the table.  

- They should remember their reference numbers and the assigned output names.

- AlphAI will only communicate its decisions by the actions (of the wheels) shown on the screen.

- click the <reset learning> button once

- click the <learning> button

- ask one student to hold the robot in his hand so that it does not move even when its wheels are turning

- point at the face of a participant with the robot's camera, and press 5 times the labeled output (actions) as assigned in the label chart.  This action is used to identify the participant.

- repeat the above for the other 2 participant.  Remember to use correctly the assigned actions of the labeled output.

 Click the <self drive> button.  The robot will use the learned intelligence to recognize the faces

- test with the 1st participant

- test with the 2nd participant

- test with the 3rd participant

- Are the result correct?

- if the result is not good, you may add more training.

Can the robot recognize the faces correctly?

Has the robot gained intelligence after training?  How does you know?

Has learning been achieved?  How does you know?

How many input neurons are there?

How many output neurons are there?

Can you notice the hidden layers?  How many?

How many neurons are there in the 1st hidden layer?

How many neurons are there in the 2nd hidden layer?

What is the function of hidden layer?

Extended ACtivity

- use the same set up but extend the number of participants to 8.

- Can you do this?

Sensors (Input) - Camera 32 x 24

Sensors (Input) - 3 colors

Actions (Output) - Forward, Backward, Stop (this can be adjusted to accommodate more participants)

Labeled Output

AI Setting

Learning type = Supervised Learning

Algorithm = Neural Network

Hidden Layers

- 1st hidden layer with 100 neurons

- 2 hidden layer with 50 neurons

Visualization - set "keep space between neurons" for easy checking of values