Correlation Matrix Memories: Improving Performance for Capacity and Generalisation

Abstract

The human brain is an extremely powerful pattern recogniser, as well as being capable of displaying amazing feats of memory. It is clear that human memory is associative; we recall information by associating items together so that one may be used to recall another. This model of memory, where items are associated as pairs rather than stored at a particular location, can be used to implement computer memories which display powerful properties such as robustness to noise, a high storage capacity and the ability to generalise. One example of such a memory is the Binary Correlation Matrix Memory (CMM), which in addition to the previously listed properties is capable of operating extremely quickly in both learning and recall, as well as being well suited for hardware implementation. These memories have been used as elements of larger pattern recognition architectures, solving problems such as object recognition, text recognition and rule chaining, with the memories being used to store rules. Clearly, the performance of the memories is a large factor in the performance of such architectures.

This thesis presents a discussion of the issues involved with optimising the performance of CMMs in the context of larger architectures. Two architectures are examined in some detail, which motivates a desire to improve the storage capacity and generalisation capability of the memories. The issues surrounding the optimisation of storage capacity of CMMs are discussed, and a method for improving the capacity is presented. Additionally, while CMMs are able to generalise, this capability is often ignored. A method for producing codes suitable for storage in a CMM is presented, which provides the ability to react to previously unseen inputs. This potentially adds a powerful new capability to existing architectures.