First look at the operating results:
Here is the overall project directory:
0. implement neural network overview
A layer neural networks, neurons, the weights and bias composed activation function. Each has one or more neurons, each neuron and the neural input / output connections that is weights.
It is important to emphasize that a neural network may be many hidden layers may also be a no, because the number of neurons each element may be different. However, the number of neurons in the input and output layer of neural number of input / output are equal.
In order to achieve that we need to define the following categories:
- Neuron: the definition of artificial neurons
- NeuralLayer: abstract class that defines a neuron layer.
- InputLayer: definition of the input layer of neural
- HiddenLayer: layer defined between the input layer and output layer
- OutputLayer: define the neural output layer.
- InputNeuron: input neurons defined neural network that appears.
- All classes ANN structure to a previously defined: NeuralNet.
In addition to these classes, we also need to define a IActivationFunction interface activation function. This is necessary because the activation function and similar methods, properties required as a neuron allocated. So you want to activate the function definitions of classes that need to implement IActivationFunction Interface:
- Linear
- Sigmoid
- Step
- HyperTan
Encoding the first chapter basically completed. In addition, we need to define two classes. For exception handling (NeuralException), one for generating a random number (RandomNumberGenerator). Finally, these classes into two packages, respectively.
1. Class neurons Neuron
Neurons class is the base class code in this chapter. According to the theory, the following properties of artificial neurons
- Entry
- Weights
- Bias
- Activation function
- Export
First define various properties of neurons:
public class Neuron {
//神经元相关的权重
protected ArrayList<Double> weight;
//神经元的输入
private ArrayList<Double> input;
//这个神经元的输出，由激活函数产生
private Double output;
//传递给激活函数的值
private Double outputBeforeActivation;
//输入的数量。如果为0，则表示神经元尚未初始化。
private int numberOfInputs = 0;
//神经元的偏差。除了第一层，其他都应该是1.0。
protected Double bias = 1.0;
//神经元的激活函数
private IActivationFunction activationFunction;
}
When instantiated neurons, specify the number and the activation function of the input data. Constructor as follows:
public Neuron(int numberofinputs,IActivationFunction iaf){
numberOfInputs=numberofinputs;
weight=new ArrayList<>(numberofinputs+1);
input=new ArrayList<>(numberofinputs);
activationFunction=iaf;
}Note that a bias define another weight. An important step is to initialize the neurons, that is, the weight assigned to the initial value. This is mainly the init () method is completed, the random number generated by a random number generator RandomNumberGenerator static class, assigned weights. Note: You need to prevent weight array bounds when setting weight values.
public void init(){
if(numberOfInputs>0){
for(int i=0;i<=numberOfInputs;i++){
double newWeight = RandomNumberGenerator.GenerateNext();
try{
this.weight.set(i, newWeight);
}
catch(IndexOutOfBoundsException iobe){
this.weight.add(newWeight);
}
}
}
}
Finally, the output value in the Calc () method:
public void calc(){
outputBeforeActivation=0.0;
if(numberOfInputs>0){
if(input!=null && weight!=null){
for(int i=0;i<=numberOfInputs;i++){
outputBeforeActivation+=(i==numberOfInputs?bias:input.get(i))*weight.get(i);
}
}
}
output=activationFunction.calc(outputBeforeActivation);
}First need to re-enter all the rights and achievements sum (bias take a final weight, i == Number-OfInputs), then the result is stored in the property outputBeforeActivation. Output neuron activation function calculated using this value.
Total code is as follows:
package neuralnet;
import java.util.ArrayList;
public class Neuron {
//神经元相关的权重
protected ArrayList<Double> weight;
//神经元的输入
private ArrayList<Double> input;
//这个神经元的输出，由激活函数产生
private Double output;
//传递给激活函数的值
private Double outputBeforeActivation;
//输入的数量。如果为0，则表示神经元尚未初始化。
private int numberOfInputs = 0;
//神经元的偏差。除了第一层，其他都应该是1.0。
protected Double bias = 1.0;
//神经元的激活函数
private IActivationFunction activationFunction;
public Neuron(){
}
public Neuron(int numberofinputs){
numberOfInputs=numberofinputs;
weight=new ArrayList<>(numberofinputs+1);
input=new ArrayList<>(numberofinputs);
}
public Neuron(int numberofinputs,IActivationFunction iaf){
numberOfInputs=numberofinputs;
weight=new ArrayList<>(numberofinputs+1);
input=new ArrayList<>(numberofinputs);
activationFunction=iaf;
}
public void init(){
if(numberOfInputs>0){
for(int i=0;i<=numberOfInputs;i++){
double newWeight = RandomNumberGenerator.GenerateNext();
try{
this.weight.set(i, newWeight);
}
catch(IndexOutOfBoundsException iobe){
this.weight.add(newWeight);
}
}
}
}
public void setInputs(double [] values){
if(values.length==numberOfInputs){
for(int i=0;i<numberOfInputs;i++){
try{
input.set(i, values[i]);
}
catch(IndexOutOfBoundsException iobe){
input.add(values[i]);
}
}
}
}
public void setInputs(ArrayList<Double> values){
if(values.size()==numberOfInputs){
input=values;
}
}
public ArrayList<Double> getArrayInputs(){
return input;
}
public double[] getInputs(){
double[] inputs = new double[numberOfInputs];
for (int i=0;i<numberOfInputs;i++){
inputs[i]=this.input.get(i);
}
return inputs;
}
public void setInput(int i,double value){
if(i>=0 && i<numberOfInputs){
try{
input.set(i, value);
}
catch(IndexOutOfBoundsException iobe){
input.add(value);
}
}
}
public double getInput(int i){
return input.get(i);
}
public double[] getWeights(){
double[] weights = new double[numberOfInputs+1];
for(int i=0;i<=numberOfInputs;i++){
weights[i]=weight.get(i);
}
return weights;
}
public ArrayList<Double> getArrayWeights(){
return weight;
}
public void updateWeight(int i, double value){
if(i>=0 && i<=numberOfInputs){
weight.set(i, value);
}
}
public int getNumberOfInputs(){
return this.numberOfInputs;
}
public void setWeight(int i,double value) throws NeuralException{
if(i>=0 && i<numberOfInputs){
this.weight.set(i, value);
}
else{
throw new NeuralException("Invalid weight index");
}
}
public double getOutput(){
return output;
}
public void calc(){
outputBeforeActivation=0.0;
if(numberOfInputs>0){
if(input!=null && weight!=null){
for(int i=0;i<=numberOfInputs;i++){
outputBeforeActivation+=(i==numberOfInputs?bias:input.get(i))*weight.get(i);
}
}
}
output=activationFunction.calc(outputBeforeActivation);
}
public void setActivationFunction(IActivationFunction iaf){
this.activationFunction=iaf;
}
public double getOutputBeforeActivation(){
return outputBeforeActivation;
}
}
2.NeuralLayer class
In this class, the neurons will be aligned in the same layer grouped. Because the need to pass values to one another, the need to define the connection between the layers. Attribute class defined as follows:
//这一层的神经元数量
protected int numberOfNeuronsInLayer;
//这一层的神经元
private ArrayList<Neuron> neuron;
//激励函数
protected IActivationFunction activationFnc;
//将值提供给此层的前一层
protected NeuralLayer previousLayer;
protected NeuralLayer nextLayer;
protected ArrayList<Double> input;
protected ArrayList<Double> output;
protected int numberOfInputs;The class is abstract, the whole layer can be instantiated class is InputLayer, HiddenLayer and Outp-utLayer. A class is created, another class constructor must be used, and these have similar class constructor.
The initialization and are calculated layer and neurons, they also realized the init () method and calc () method. Hey protected type of life, to ensure that only subclasses can invoke or override these methods.
All code is as follows:
package neuralnet;
import java.util.ArrayList;
public abstract class NeuralLayer {
//这一层的神经元数量
protected int numberOfNeuronsInLayer;
//这一层的神经元
private ArrayList<Neuron> neuron;
//激励函数
protected IActivationFunction activationFnc;
//将值提供给此层的前一层
protected NeuralLayer previousLayer;
protected NeuralLayer nextLayer;
protected ArrayList<Double> input;
protected ArrayList<Double> output;
protected int numberOfInputs;
public NeuralLayer(int numberofneurons){
this.numberOfNeuronsInLayer=numberofneurons;
neuron = new ArrayList<>(numberofneurons);
output = new ArrayList<>(numberofneurons);
}
public NeuralLayer(int numberofneurons,IActivationFunction iaf){
this.numberOfNeuronsInLayer=numberofneurons;
this.activationFnc=iaf;
neuron = new ArrayList<>(numberofneurons);
output = new ArrayList<>(numberofneurons);
}
public int getNumberOfNeuronsInLayer(){
return numberOfNeuronsInLayer;
}
public ArrayList<Neuron> getListOfNeurons(){
return neuron;
}
protected NeuralLayer getPreviousLayer(){
return previousLayer;
}
protected NeuralLayer getNextLayer(){
return nextLayer;
}
protected void setPreviousLayer(NeuralLayer layer){
previousLayer=layer;
}
protected void setNextLayer(NeuralLayer layer){
nextLayer=layer;
}
protected void init(){
if(numberOfNeuronsInLayer>=0){
for(int i=0;i<numberOfNeuronsInLayer;i++){
try{
neuron.get(i).setActivationFunction(activationFnc);
neuron.get(i).init();
}
catch(IndexOutOfBoundsException iobe){
neuron.add(new Neuron(numberOfInputs,activationFnc));
neuron.get(i).init();
}
}
}
}
protected void setInputs(ArrayList<Double> inputs){
this.numberOfInputs=inputs.size();
this.input=inputs;
}
protected void calc(){
if(input!=null && neuron!=null){
for(int i=0;i<numberOfNeuronsInLayer;i++){
neuron.get(i).setInputs(this.input);
neuron.get(i).calc();
try{
output.set(i,neuron.get(i).getOutput());
}
catch(IndexOutOfBoundsException iobe){
output.add(neuron.get(i).getOutput());
}
}
}
}
protected ArrayList<Double> getOutputs(){
return output;
}
protected Neuron getNeuron(int i){
return neuron.get(i);
}
protected void setNeuron(int i, Neuron _neuron){
try{
this.neuron.set(i, _neuron);
}
catch(IndexOutOfBoundsException iobe){
this.neuron.add(_neuron);
}
}
}3.ActivationFunction Interface
Before defining NeerualNetwork classes, look at the examples of Java code interface:
public interface IActivationFunction {
double calc(double x);
public enum ActivationFunctionENUM {
STEP, LINEAR, SIGMOID, HYPERTAN
}
}
Wherein Calc () method belonging to a particular class implements IActivationFunction activation function interface, e.g. Sigmoid function.
public class Sigmoid implements IActivationFunction {
private double a=1.0;
public Sigmoid(){
}
public Sigmoid(double value){
this.setA(value);
}
public void setA(double value){
this.a=value;
}
@Override
public double calc(double x){
return 1.0/(1.0+Math.exp(-a*x));
}
}
This is one example of polymorphism, i.e., the same function name, class and method for presenting different behavior, to produce flexible application.
4. Neural Network (NeuralNet) class
Finally, the definition of neural networks class. So far, we already know that the neural network in the nerve tissue layer neurons, and each neural network has at least two layers, one for receiving income, a processing output, and a variable number of hidden layer is used . Thus, in addition to neurons and having similar properties and classes outside NeuralLary, Neural will also have several properties, such as numberOfInputs, numberOfOutputs like.
private InputLayer inputLayer;
private ArrayList<HiddenLayer> hiddenLayer;
private OutputLayer outputLayer;
private int numberOfHiddenLayers;
private int numberOfInputs;
private int numberOfOutputs;
private ArrayList<Double> input;
private ArrayList<Double> output;
The constructor for this class more than the preceding class parameters:
public NeuralNet(int numberofinputs,int numberofoutputs,
int [] numberofhiddenneurons,IActivationFunction[] hiddenAcFnc,
IActivationFunction outputAcFnc)If the number of hidden layer is variable, we should also consider that there may be more hidden layer, or hidden layer 0, and each hidden layer, the number of hidden neurons is also variable. This variability is the best way to handle indicating the number of neurons in each hidden layer is a integer vector (parameter numberofhiddenlayers). Further, the need to activate the function defined for each hidden layer and output layer comprises a complete objective parameters that are needed and hiddenActivationFnc outputAcFnc.
Complete implementation as follows:
public NeuralNet(int numberofinputs,int numberofoutputs,
int [] numberofhiddenneurons,IActivationFunction[] hiddenAcFnc,
IActivationFunction outputAcFnc){
numberOfHiddenLayers=numberofhiddenneurons.length;
numberOfInputs=numberofinputs;
numberOfOutputs=numberofoutputs;
if(numberOfHiddenLayers==hiddenAcFnc.length){
input=new ArrayList<>(numberofinputs);
inputLayer=new InputLayer(numberofinputs);
if(numberOfHiddenLayers>0){
hiddenLayer=new ArrayList<>(numberOfHiddenLayers);
}
for(int i=0;i<numberOfHiddenLayers;i++){
if(i==0){
try{
hiddenLayer.set(i,new HiddenLayer(numberofhiddenneurons[i],
hiddenAcFnc[i],
inputLayer.getNumberOfNeuronsInLayer()));
}
catch(IndexOutOfBoundsException iobe){
hiddenLayer.add(new HiddenLayer(numberofhiddenneurons[i],
hiddenAcFnc[i],
inputLayer.getNumberOfNeuronsInLayer()));
}
inputLayer.setNextLayer(hiddenLayer.get(i));
}
else{
try{
hiddenLayer.set(i, new HiddenLayer(numberofhiddenneurons[i],
hiddenAcFnc[i],hiddenLayer.get(i-1)
.getNumberOfNeuronsInLayer()
));
}
catch(IndexOutOfBoundsException iobe){
hiddenLayer.add(new HiddenLayer(numberofhiddenneurons[i],
hiddenAcFnc[i],hiddenLayer.get(i-1)
.getNumberOfNeuronsInLayer()
));
}
hiddenLayer.get(i-1).setNextLayer(hiddenLayer.get(i));
}
}
if(numberOfHiddenLayers>0){
outputLayer=new OutputLayer(numberofoutputs,outputAcFnc,
hiddenLayer.get(numberOfHiddenLayers-1)
.getNumberOfNeuronsInLayer()
);
hiddenLayer.get(numberOfHiddenLayers-1).setNextLayer(outputLayer);
}
else{
outputLayer=new OutputLayer(numberofinputs, outputAcFnc,
numberofoutputs);
inputLayer.setNextLayer(outputLayer);
}
}
}5. Run the program
code show as below:
package neuralnet;
import neuralnet.math.IActivationFunction;
import neuralnet.math.Linear;
import neuralnet.math.RandomNumberGenerator;
import neuralnet.math.Sigmoid;
public class NeuralNetConsoleTest {
public static void main(String[] args){
RandomNumberGenerator.seed=0;
int numberOfInputs=2;
int numberOfOutputs=1;
int[] numberOfHiddenNeurons= { 3 };
IActivationFunction[] hiddenAcFnc = { new Sigmoid(1.0) } ;
Linear outputAcFnc = new Linear(1.0);
System.out.println("Creating Neural Netword...");
NeuralNet nn = new NeuralNet(numberOfInputs,numberOfOutputs,
numberOfHiddenNeurons,hiddenAcFnc,outputAcFnc);
System.out.println("Neural Network Network...");
double [] neuralInput = { 1.5 , 0.5 };
System.out.println("Feeding the values {1.5;0.5} to the neural network");
double [] neuralOutput;
nn.setInputs(neuralInput);
nn.calc();
neuralOutput=nn.getOutputs();
System.out.println("OutPut 1:" + neuralOutput[0]);
neuralInput[0] = 1.0;
neuralInput[1] = 2.1;
System.out.println("Feeding the values {1.0;2.1} to the neural network");
nn.setInputs(neuralInput);
nn.calc();
neuralOutput=nn.getOutputs();
System.out.println("OutPut 2:" + neuralOutput[0]);
}
}This completes all the code we neural network: The following is the source code archive. Needy students can download and run.