Analytical renormalization group approach to competing orders at charge neutrality in twisted bilayer graphene

Abstract

We investigate twisted bilayer graphene near charge neutrality using a generalized Bistritzer-MacDonald continuum model accounting for corrugation effects. The Fermi velocity vanishes for particular twist angles properly reproducing the physics of the celebrated magic angles. Using group representation theory, we identify all contact interaction potentials compatible with the symmetries of the model. This enables us to identify two classes of quartic interactions leading to either the opening of a gap or to nematic ordering. We then implement a renormalization group analysis to study the effect of quantum fluctuations on the appearance of these states for twist angles approaching the first magic value. This combined group theory-renormalization study reveals that the proximity to the first magic angle favors the occurrence of a layer-polarized, gapped state with a spatial modulation of interlayer correlations, which we call a nematic insulator.