Quantum spin liquid and cluster Mott insulator phases in the Mo3O8 magnets

Abstract

Abstract We unveil the microscopic origin of largely debated magnetism in the Mo3O8 quantum systems. Upon considering an extended Hubbard model at 1/6 filling on the anisotropic kagomé lattice formed by the Mo atoms, we argue that its ground state is determined by the competition between kinetic energy and intersite Coulomb interactions, which is controlled by the trimerisation of the kagomé lattice into the Mo3O13 clusters, and the sign of hopping parameters, specifying the electron localisation at such clusters. Based on first-principles calculations, we show that the strong interaction limit reveals a plaquette charge order with unpaired spins at the resonating hexagons that can be realised in LiZn2Mo3O8, and whose origin is solely related to the opposite signs of intracluster and intercluster hoppings, in contrast to all previous scenarios. On the other hand, both Li2InMo3O8 and Li2ScMo3O8 are demonstrated to fall into the weak interaction limit where the electrons are well localised at the Mo3O13 clusters. While the former is found to exhibit long-range antiferromagnetic order, the latter is more likely to reveal short-range order with quantum spin liquid-like excitations. Our results not only reproduce most of the experimentally observed features of the Mo3O8 systems, but will also help to describe various properties in other quantum cluster magnets.