Artifical Neural Networks, an example: Hopfield (by AVL)

What is an Artificial Neural Network?

Author: André Vinícius Lopes


An Artificial Neural Network is an attempt to imitate the way that our brain works in a mathematical-logical function. A way of copying the way that humans think. Using a real Neuron as a main reference, Scientists have tried to create a proper code model of it.

This is a basic Article of what an ANN (Artificial Neural Network) is and its use with LibGDX and games, for more information I recommend a book from Jeff Heaton:

"Introduction to Neural Networks for Java" by Jeff Heaton
ISBN: 1-60439-008-5

ANN's make heavy use of mathematic matrices, so you might want to review before coding or reading the code.

What is the use of an Artificial Neural Network?

Well, ANN's are awesome to solve problems that are either NP or that aren't suitable for our tradional code. With NP problems, I mean problems that are, quoting Wikipedia:

"nondeterministic polynomial time"

Problems that can't be solved in:

  • Polynomial time: f(x) = x^z + ... With z >= 2
  • or linear time: f(x) = x¹ + ...
  • or even f(x) = 2

Obviously ANNs aren't always suitable, but when they are, they performs well. ANNs can solve pattern recognition problems, precognition, data mining and classification. Next I will present a Hopfield neural network, the "Hello World" of neural networks.

What is a Hopfield Neural Network?

A Hopfield Neural network only accepts Bipolar binary values. What does that mean? Normally we have booleans that are 1 == true and 0 == false. In Hopfield, we have 1 == true and -1 == false.

The equation to transform boolean to bipolar is as follows:

Y = 2x – 1

F(X) = 2x - 1 Y
0 -1
1 1

But what exactly is a Hopfield neural network? Well, if you see the images contained in this magnificient article By Jeff Heaton:
http://www.heatonresearch.com/node/687
You can have an idea of what it is graphically. He also explains the theory and code in the book, but let's focus on the code now.

Artificial Neural Networks have some important methods you should understand:
train() and pattern(Pattern)

The train() method trains the network with the values in the input and the pattern asks the ANN to recognize a pattern.

The output will be either:

  • The Desired Pattern Recognized
  • No Convergence whatsoever which means the Pattern wasn't recognized.
  • The Inverted pattern*

* Suppose you train the Hopfield neural network to recognize 10100 . The Hopfield neural network will also recognize 01011. Why? Well, because of the way the Hopfield works and is made, it will also recognize the opposite of the pattern it was trained with.

App made with LibGDX that uses a Hopfield neural network

app

How does it work?

When you click a square, it becomes green (on), that means the bipolar value == 1 and when its off, it's 0.

So what? So you can get the whole matrix status, on(1) or off(­1 , make it as a vector of bipolar values and train the ANN with it. Then you can do the same to present the pattern.

Don't understand it? Well, maybe you need see a bit of the code! Let us see it!!!

* Obs, in this case we are using a Matrix API made by me. In future we will use EJML because it is way faster.

The link below will show the implementation based on the code provided by Jeff Heaton in his book.
https://bitbucket.org/andrelopes1705/annimagerecognizer/src/61e8ff3136bb0227dce324e4d8d1c3e2b795720d/src/com/heatonresearch/book/introneuralnet/neural/hopfield/HopfieldNetwork.java?at=default

The ShortMatrix class is as follows: http://pastebin.com/8AxhZNkf

Why did I use short instead of int/double/float?

Well, basically since we are just using ­1 or -1 as values, we have no use for 64 bit precision. We don't need to waste memory, short will do just fine and will be faster. In fact, I could have also done with byte, which I will do after I finish this article, but it works just fine.

If we have a look at Java Docs:
http://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html

"short: The short data type is a 16­bit signed two's complement integer. It has a minimum value of ­32,768 and a maximum value of 32,767 (inclusive). As with byte, the same guidelines apply: you can use a short to save memory in large arrays, in situations where the memory savings actually matters."

Now the most important, the .jar so you can test it and check the source code:
https://bitbucket.org/andrelopes1705/annimagerecognizer/downloads/AnnImageRecognizer_0.3.zip

Conclusion

I hope this article explained at least a bit of the use of a Hopfield ANN with LibGDX. This is my first article and i would also like to thank Robin Stu (dermetfan) for making part of the view logic and publishing this article.

Feedback and Knowledge Base