Neurodevelopmental axonal pathfinding plays a central role in correct brain wiring and subsequent cognitive abilities. Within the growth cone, various intracellular effectors transduce axonal guidance signals by remodeling the cytoskeleton. Semaphorin-3E (Sema3E) is a guidance cue implicated in development of the fornix, a neuronal tract connecting the hippocampus to the hypothalamus. Microtubule-associated protein 6 (MAP6) has been shown to be involved in the Sema3E growth-promoting signaling pathway. In this study, we identified the collapsin response mediator protein 4 (CRMP4) as a MAP6 partner and a crucial effector in Sema3E growth-promoting activity. CRMP4-KO mice displayed abnormal fornix development reminiscent of that observed in Sema3E-KO mice. CRMP4 was shown to interact with the Sema3E tripartite receptor complex within detergent-resistant membrane (DRM) domains, and DRM domain integrity was required to transduce Sema3E signaling through the Akt/GSK3 pathway. Finally, we showed that the cytoskeleton-binding domain of CRMP4 is required for Sema3E’s growth-promoting activity, suggesting that CRMP4 plays a role at the interface between Sema3E receptors, located in DRM domains, and the cytoskeleton network. As the fornix is affected in many psychiatric diseases, such as schizophrenia, our results provide new insights to better understand the neurodevelopmental components of these diseases.