In this unit we will introduce an important concept of neural network, which is hidden layers.  While the details about middle layers involve a lot of advanced theories and calculation, we will only talk about it in a simple way.

What is middle layers?

• There are 3 types of layers in neural networks:

• Input layer - in neural networks we must maintain one input layer to takes the inputs and perform some calculations through its neurons and the connections.  Then the outputs are transmitted to the next layers.
• Output layer - in neural networks there must be always an output layer.  The output layer takes the signals which are passed in from the layers before it, and perform the calculations of the output values through its neurons.  

• Hidden layer(s) - the hidden layers are placed in between the input and output layers.  These hidden layers are not visible to the external environment and these are private to the neural networks.  That is why they are called hidden layers.
  • The hidden layers' job is to transform the inputs into something that the output layer can use.
    • Hidden layers process the data by applying complex and advanced mathematics.  
    • These layers enable a neural network to learn complex tasks and achieve excellent performance.
    • The hidden layers make the neural network  a superior and powerful type of machine learning algorithms.
  • For large majority of problems hidden layers are required.
    • If there are a large number of hidden layers with large number of neurons, the neural network may take a very long time to do the calculation and produce the output.
    • A neural network which has multiple hidden layers is called a Deep Neural Network.

Below shown a neural network with 2 hidden layers.

One of the ways to realize the importance of hidden layers is to look into how computer vision operates.

Deep neural networks that consist of many hidden layers can achieve impressive results in image recognition including face recognition.

The first hidden layer of the neural network learn to detect short pieces of corners and edges in the image.  These features are easy to detect given the raw image but are not very useful by themselves to recognize the identity of the person in the image.

Then, the middle hidden layer combine the detected edges from the previous layers to detect parts of the face like the eyes, the nose and the ears.

Then, the final layer combines the detectors of the nose, eyes, etc. to detect the face of the person.

Objective:

- train the AlphAI robot to recognize faces

Follow Appendix A1 to set up the parameters.  

You should know how many people play in the activity.  Suggest 4 to 6 people.

- Check how the hidden layers are shown on the screen.

- Switch on and connect the robot to your PC.  Press the <Connection> button.

- With the the AI parameters set up correctly, the above neural network will be shown.

- click the <reset learning> button once

- click the <learning> button

- ask one student to hold the robot in his hand so that it does not move even when its wheels are turning

- point at the face of a participant with the robot's camera, and press 5 times the labeled output (actions) as assigned in the label chart.  This action is used to identify the participant.

Hints for successful face recognition

    - the face should cover 75-80% of the screen image

    - keep the same background for the same participant each time.  Use simple background.

    - move the camera a little bit each time when taking photo for the same participants.

- repeat the above for the other participants.  Remember to use correctly the output.

 Click the <self drive> button.  The robot will use the learned intelligence to recognize the faces

Hints for successful face recognition

    - the face should cover 75-80% of the screen image

    - keep the same background as the time of training for the same participant

    - move the camera a little bit to obtain better detection

- test with the 1st participant

- test with the 2nd participant

- test with the 3rd participant

- Are the result correct?

- if the result is not good, you may add more training.

Can the robot recognize the faces correctly?

Has the robot gained intelligence after training?  How does you know?

How many input neurons are there?

How many output neurons are there?

Can you notice the hidden layers?  How many?

How many neurons are there in the 1st hidden layer?

How many neurons are there in the 2nd hidden layer?

What is the function of hidden layer?

SUGGESTED ANSWER

Can the robot recognize the faces correctly?  Yes.

Has the robot gained intelligence after training?  How does you know?  Yes.  It has learned how to recognize faces.

How many input neurons are there?  32 x 24 x 3 colours = 2,304

How many output neurons are there?  Depend on number of players.

Can you notice the hidden layers?  How many?  There are 2 hidden layers.

How many neurons are there in the 1st hidden layer?  100

How many neurons are there in the 2nd hidden layer?  50

What is the function of hidden layer?  The hidden layers' job is to transform the inputs into something that the output layer can use.  For example, eye, ear, nose, etc.

Load from the demo Parameters

Select "Image Recognition"

Switch off all the defaulted actions (those in yellow and circled in red.)

Count how many people are playing in this face recognition activity (suggest 4 to 8).

Create the customized outputs with the " + " button with the names of the players.  For example, Jane and Adrian, etc.

In the AI tab, add "100 50" in the hidden neuron layers