IBM Research and academic collaborators propose replacing the log-sum-exponential energy in Dense Associative Memory with a log-sum-ReLU energy derived from the Epanechnikov kernel — the kernel that's been provably optimal for density estimation since 1969. The practical upshot: exact retrieval of all stored patterns (exponentially many, M ~ exp(d)) while the energy landscape simultaneously grows novel local minima at the centroids of overlapping basins. LSE can only do one or the other; Proposition 1 shows it never satisfies their global emergence condition at any β.
The emergent memories are mechanically boring — centroids of subsets of stored patterns — but when the patterns are VAE latents of MNIST or TinyImagenet, the decodes look like plausible generations. The authors themselves frame these as the memory-system analog of hallucinations: interference minima that look meaningful without being stored facts. The honest limitation is that the gradient vanishes exactly outside the basin supports, so far-field queries never retrieve anything without a query-dependent temperature schedule or an LSE hybrid.
Worth noting the field's arc here: Hopfield networks were declared dead circa 1995, resurrected in 2020 by noticing LSE is softmax attention, and are now gaining generative properties via kernel choice. The discipline is the same one that brought you CONFIG.SYS; we just keep relaunching it under new energy functions.
It's the oldest loop in computing: Hopfield nets ran on energy functions in 1982, got obsoleted by backprop, came back when someone noticed the energy gradient is literally softmax attention, and now the renaissance continues by swapping the kernel. Epanechnikov's result is from 1969 — older than the network it's being applied to.