Site Mixing for Engineering Magnetic Topological Insulators

The van der Waals compound, MnBi₂Te₄, is the first intrinsic magnetic topological insulator, providing a materials platform for exploring exotic quantum phenomena such as the axion insulator state and the quantum anomalous Hall effect. However, intrinsic structural imperfections lead to bulk conductivity, and the roles of magnetic defects are still unknown. With higher concentrations of same types of magnetic defects, the isostructural compound MnSb₂Te₄ is a better model system for a systematic investigation of the connections among magnetic, topology and lattice defects. In this work, the impact of antisite defects on the magnetism and electronic structure is studied in MnSb₂Te₄. Mn-Sb site mixing leads to complex magnetic structures and tunes the interlayer magnetic coupling between antiferromagnetic and ferromagnetic. The detailed nonstoichiometry and site-mixing of MnSb₂Te₄ crystals depend on the growth parameters, which can lead to approx40% of Mn sites occupied by Sb and approx15% of Sb sites by Mn in as-grown crystals. Single crystal neutron diffraction and electron microscopy studies show nearly random distribution of the antisite defects. Band structure calculations suggest that the Mn-Sb site-mixing favors a FM interlayer coupling, consistent with experimental observation, but is detrimental to the band inversion required for a nontrivial topology. Our results suggest a long range magnetic order of Mn ions sitting on Bi sites in MnBi₂Te₄. The effects of site mixing should be considered in all layered heterostructures that consist of alternating magnetic and topological layers, including the entire family of MnTeBi$₂$Te$₃$_n, its Sb analogs and their solid solution.