A new method to build an associative memory model
Keywords:
Artificial neural networks, Associative memory, Binary operation, Morphological associative memory, Perfect recallAbstract
An associative memory is an artificial neural network model designed to store and recall input-output patterns pairs by association. A new method for obtaining two associative memories +M and -M is presented in this article, which uses a framework focused on two new binary operations boxplus and boxminus, and a unary operation called projection. The conditions to obtaining perfect recovery and the noise boundaries that the memories can tolerate are studied. Memories +M and -M are robust to additive and subtractive noise, respectively, both types converge in one-step and operate in heteroassociative and autoassociative modes. The performance of the proposed memories is tested against other memory models under identical conditions using the gamma binary distance for measuring similarity between binary patterns. The computer simulation results based on the average of 500 trials with the binary set of 26 lowercase letters of 7x7 pixel size, showed that the proposed memories in autoassociative mode exhibited a gamma binary distance of above .8, .75 and .7 by distorting up to 10 pixels by additive, subtractive, and mixed noise, respectively, which implied that the recovered images had a high similarity. In absence of noise the performance was excellent, i.e., the 100% of the recovered images were identical.
Downloads
References
M. H. Hassoun, Associative neural memories: theory and implementation. New York, United States: Oxford University Press, Inc., 1993.
W. K. Taylor, “Electrical simulation of some nervous system functional activities,” in Proc. Information Theory: Third London Symposium, (Butterworths, London), p. 314–328, Sept. 1955.
K. Steinbuch, “Die lernmatrix,” Kybernetik, vol. 1, pp. 36–45, Jan. 1961.
T. Kohonen, “Correlation matrix memories,” IEEE Trans. Comput., vol. C-21, pp. 353–359, Apr. 1972.
K. Nakano, “Associatron-a model of associative memory,” IEEE Trans. Syst., Man, Cybern., vol. SMC-2, pp. 380–388, July 1972.
J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proceedings of the National Academy of Sciences of the United States of America, vol. 79, pp. 2554–2558, Apr. 1982.
R. J. McEliece, E. C. Posner, E. R. Rodemich, and S. S. Venkatesh, “The capacity of the hopfield associative memory,” IEEE Trans. Inf. Theory, vol. 33, pp. 461–482, July 1987.
B. Kosko, “Fuzzy associative memories,” in Proc. of the 2nd Joint Technology Workshop on Neural Networks and Fuzzy Logic, (California, United States), pp. 3–58, Feb. 1990.
F.-L. Chung and T. Lee, “Towards a high capacity fuzzy associative memory model,” in Proc. of 1994 IEEE International Conference on Neural Networks (ICNN’94), (Florida, United States), pp. 1595–1599, July 1994.
P. Xiao, F. Yang, and Y. Yu, “Max-min encoding learning algorithm for fuzzy max-multiplication associative memory networks,” in Proc. 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation, (Orlando, United States), pp. 3674–3679, Oct. 1997.
G. X. Ritter, P. Sussner, and J. L. D´íaz de Le´ón, “Morphological associative memories,” IEEE Trans. Neural Netw., vol. 2, pp. 281–293, Mar. 1998.
N. Feng, X. Cao, S. Li, L. Ao, and S. Wang, “A new method of morphological associative memories,” in Proc. Emerging Intelligent Computing Technology and Applications. With Aspects of Artificial Intelligence. ICIC 2009, (Ulsan, South Korea), pp. 407–416, Sept. 2009.
T. D. Bui, T. H. Nong, and T. K. Dang, “Improving learning rule for fuzzy associative memory with combination of content and association,” Neurocomputing, vol. 149, pp. 59–64, Feb. 2015.
N. Feng and Y. Yao, “No rounding reverse fuzzy morphological associative memories,” Neural Network World, vol. 26, no. 6, pp. 571–587, 2016.
P. Sussner and M. E. Valle, “Implicative fuzzy associative memories,” IEEE Trans. Fuzzy Syst., vol. 14, pp. 793–807, Nov. 2006.
A. Ferreyra-Ramírez, E. Rodríguez-Martínez, C. Avilés-Cruz, and F. López-Saca, “Image retrieval system based on a binary auto-encoder and a convolutional neural network,” IEEE Latin America Transactions, vol. 100, pp. 1–8, Sept. 2020.
N. Feng, Y. Qiu, F. Wang, and Y. Sun, A Unified Framework of Morphological Associative Memories, vol. 344 of Lecture Notes in Control and Information Sciences, ch. Intelligent Control and Automation: International Conference on Intelligent Computing, ICIC 2006 Kunming, China, August 16–19, 2006, pp. 1–11. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006.
B. Sun and N. Feng, “Exploring the new application of morphological neural networks,” in Intelligent Computing Theories and Application, (Liverpool, United Kingdom), pp. 113–120, Aug. 2017.
C. Yáñez-Márquez, I. López-Yáñez, M. Aldape-Pérez, O. Camacho-Nieto, A. J. Argüelles-Cruz, and Y. Villuendas-Rey, “Theoretical foundations for the alpha-beta associative memories: 10 years of derived extensions, models, and applications,” Neural Processing Letters, vol. 48, p. 811–847, Oct. 2018.
N. Feng and B. Sun, “On simulating one-trial learning using morphological neural networks,” Cognitive Systems Research, vol. 53, pp. 61–70, 2019.
G. Urcid and G. X. Ritter, Noise Masking for Pattern Recall Using a Single Lattice Matrix Associative Memory, pp. 81–100. Berlin, Heidelberg: Springer Berlin Heidelberg, 2007.
A. A. Mustafa, “Probabilistic binary similarity distance for quick binary image matching,” IET Image Processing, vol. 12, no. 10, pp. 1844–1856, 2018.
M. Hattori, A. Fukui, and H. Ito, “A fast method of constructing kernel patterns for morphological associative memory,” in Proc. 9th International Conference on Neural Information Processing, 2002. ICONIP’02, (Singapore, Singapore), pp. 1058–1063, Nov. 2002.
P. Sussner, “Observations on morphological associative memories and the kernel method,” Neurocomputing, vol. 31, pp. 167–183, Mar. 2000.
J. Y. Montiel, R. Romero, and J. L. L´opez, “Generation of trip routes on the basis of a geographic information system,” IEEE Latin America Transactions, vol. 2, pp. 196–200, Sept. 2004.