Commit 4156233f authored by Nicolas Schuler's avatar Nicolas Schuler
Browse files

Update MockupFFNN.py

parent 0f8dd845
......@@ -17,7 +17,7 @@ Since the neurons are always the same number and feeding all the other neurons,
the inputData used must have the same dimension as the number of neurons.
Contains different activation functions that can be used to transform the
output data of a given layer, before feeding it to the next layer. Note that
outputData data of a given layer, before feeding it to the next layer. Note that
in this implementation we use the same function for all layers of a given
network.
......@@ -85,9 +85,9 @@ class Layer:
class Network:
"""Mockup of a neural network with layers (see Layer-class).
inputData():
input_data():
Sets the used inputData data to feed through the network.
Numpy copies the arrays it handles the inputData is unchanged
Numpy copies the arrays it handles, thus the inputData is unchanged
through different feeding processes.
Since the number of neurons is the same for each layer and we are
feeding all the other neurons, the inputData used must have the same
......@@ -97,8 +97,8 @@ class Network:
feedData():
Main function of the module. After a neural network has been created
and feed with the correct inputData data we can feed that data through
the different layers. The resulting output, after going through all
layers, can be called with outputData()
the different layers. The resulting outputData, after going through all
layers, can be called with output_data()
activateOutput():
Applies the given activation function to the provided
......@@ -113,33 +113,33 @@ class Network:
"""
def __init__(self, numberOfLayers=5, numberOfNeurons=10):
self.input = None
self.output = None
self.inputData = None
self.outputData = None
self.layers = [Layer(numberOfNeurons) for _ in range(numberOfLayers)]
def inputData(self, data):
self.input = data
def input_data(self, data):
self.inputData = data
def outputData(self):
return self.output
def output_data(self):
return self.outputData
def feedData(self, activationFunction="Sigmoid"):
inputData = np.array(self.input, copy=True, dtype="float64")
inputData = np.array(self.inputData, copy=True, dtype="float64")
for layer in self.layers:
output = np.array(range(10), dtype="float64")
for neuron in range(10):
# Multiply inputData with weights via dot product, then add bias of the given neuron
# Finally, calculate output value with the chosen activation function (exception: softmax)
output[neuron] = self.activateOutput(np.dot(inputData, layer.weights[neuron] + layer.biases[neuron]), activationFunction)
# Finally, calculate outputData value with the chosen activation function (exception: softmax)
output[neuron] = self.activateOutput(np.dot(inputData, layer.weights[neuron]) + layer.biases[neuron], activationFunction)
# For softmax we calculate the output in the end, since we need the entire output
# For softmax we calculate the outputData in the end, since we need the entire outputData
if (activationFunction == "Softmax"):
output = softmax(output)
inputData = output
self.output = output
self.outputData = output
def activateOutput(self, x, activationFunction):
if (activationFunction == "Identity"): return identity(x)
......@@ -170,33 +170,33 @@ if __name__ == "__main__":
network2 = Network(10, 10)
inputData = np.array(range(10), dtype="float64")
network.inputData(inputData)
network2.inputData(inputData)
network.input_data(inputData)
network2.input_data(inputData)
print("Input")
print(network.input)
print(network.inputData)
network.feedData()
print("\nIdentity")
network.feedData(activationFunction="Identity")
print(network.outputData())
print(network.output_data())
print("\nRectLinUnit")
network.feedData(activationFunction="RectLinUnit")
print(network.outputData())
print(network.output_data())
print("\nSigmoid")
network.feedData(activationFunction="Sigmoid")
print(network.outputData())
print(network.output_data())
print("\nSoftmax")
network.feedData(activationFunction="Softmax")
print(network.outputData())
print(network.output_data())
print("\nSoftmax")
network2.feedData(activationFunction="Softmax")
print(network2.outputData())
print(network2.output_data())
####################
# Exercise Answers #
......@@ -207,17 +207,17 @@ What effect do different activation functions have?
Since I only used positive values (between 0 and 1) as weights/biases, the
behavior of Identity and RectLinUnit are identical. Both functions keep
the output (effectively, since we have no negative values), thus each
consecutive layer has a larger and larger output.
the outputData (effectively, since we have no negative values), thus each
consecutive layer has a larger and larger outputData.
Sigmoid converts towards 1 (for x --> infinity). Since, again, we only operate
with positive values the relatively large values will convert towards 1. That is
true for every iteration, so the values of each layer will always be squashed to
1 and then be given to the next layer.
Finally, Softmax is calculated over the entire output. Like Sigmoid, its range is
between 0 and 1. But since the sum of the entire output (vector) for Softmax is = 1
we are always way below the 1 for the individual values. Example output after
Finally, Softmax is calculated over the entire outputData. Like Sigmoid, its range is
between 0 and 1. But since the sum of the entire outputData (vector) for Softmax is = 1
we are always way below the 1 for the individual values. Example outputData after
feeding: [0.10320238 0.16048054 0.07592815 0.09503466 0.13095142 0.07203039 0.11736508
0.06868714 0.08664903 0.08967121] = 1
......@@ -228,7 +228,7 @@ How did you initialize your weights and biases?
In the given exercise, the module does not learn/improve, so I just initialized the weights
and biases with random values between 0 and 1. I also did not initialize with only 0, since
then both weights and values would give the same output, regardless of inputData (since there is
then both weights and values would give the same outputData, regardless of inputData (since there is
no learning).
To my knowledge, neither method is ideal (random or zero), since especially the random
......@@ -237,6 +237,6 @@ functions like Sigmoid, which squash the given values in a small range (of [0,1]
On the other hand, RectLU can counter this (since we have no squashing down, as seen above).
So ideally, you would like to initialize the values of the weights in a way, that minimizes the
danger for something like that to occur. While I'm aware of this fact, I do not know what the
danger for something like that to occur. While I'm aware of that, I do not know what the
proper way would be to do so yet.
"""
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment