Evidence of the Berezinskii-Kosterlitz-Thouless Phase in a Frustrated Magnet

The Berezinskii-Kosterlitz-Thouless (BKT) mechanism, building upon proliferation of topological defects in 2D systems, is the first example of phase transition beyond the Landau-Ginzburg paradigm of symmetry breaking. Such a topological phase transition has long been sought yet undiscovered directly in magnetic materials. Here, we pin down two transitions that bound a BKT phase in an ideal 2D frustrated magnet TmMgGaO₄, via nuclear magnetic resonance under in-plane magnetic fields, which do not disturb the low-energy electronic states and allow BKT fluctuations to be detected sensitively. Moreover, by applying out-of-plane fields, we find a critical scaling behaviour of the magnetic susceptibility expected for the BKT transition. The experimental findings can be explained by quantum Monte Carlo simulations applied on an accurate triangular-lattice Ising model of the compound which hosts a BKT phase. These results provide a concrete example for the BKT phase and offer an ideal platform for future investigations on the BKT physics in magnetic materials.