What's catastrophic forgetting?

I should describe what catastrophic forgetting means in the context of cognitive science. Well, you'd think there would be a wikipedia entry for this but there's not -should probably go and write it down myself, meanwhile here's a nice introduction paper.

Anyway, catastrophic forgetting is the phenomenon by which some neural networks completely forget past memories when exposed to a set of new ones.

Naturally, this has been of some concerns for proponents of such systems which after all aim at simulating human memory functions. Humans do not, under normal circumstances, show this behavior. However it has been suggested and pretty convincingly argued that interleaved learning could circumvent the issue, at least in feed-forward networks trained with the backpropagation rule.

There is also a more catastrophic sense in which the notion has been used, and it is to describe the complete loss of any memories, past or recent, that occurs in e.g. the Hopfield network when exposed to undue numbers of patterns (interested in reading a recent thesis on the subject?). This is a subject I find utterly fascinating, having such ramifications as the function of dream sleep and palimpsest learning. More on that later.