Turing equivalence of neural networks with second order connection weights

It is widely acknowledged that in principle a potentially infinitely large neural network (either in number of neurons or in the precision of a single neural activity) could possess an equivalent computational power as a Turing machine. In the present work, the authors show such an equivalence of Turing machines to several explicitly constructed neural networks. It is proven that for any given Turing machine there exists a recurrent neural network with local, second-order, and uniformly connected weights (i.e., the weights connecting the second-order product of local 'input neurons' with their corresponding 'output neurons') which can simulate it. The numerical implementation and learning of such a neural Turing machine are also discussed.