Perineuronal nets (PNNs), a specialized form of extra cellular matrix (ECM), surround numerous neurons in the CNS and allow synaptic connectivity through holes in its structure. We hypothesize that PNNs serve as gatekeepers that guard and protect synaptic territory and thus may stabilize an engram circuit. We present high-resolution and 3D EM images of PNN-engulfed neurons in mice brains, showing that synapses occupy the PNN holes and that invasion of other cellular components is rare. PNN constituents in mice brains are long-lived and can be eroded faster in an enriched environment, while synaptic proteins have a high turnover rate. Preventing PNN erosion by using pharmacological inhibition of PNN-modifying proteases or matrix metalloproteases 9 (MMP9) knockout mice allowed normal fear memory acquisition but diminished long-term memory stabilization, supporting the above hypothesis.