Unconventional planar Hall effect in exchange-coupled topological insulator-ferromagnetic insulator heterostructures

The Dirac electrons occupying the surface states (SSs) of topological insulators (TIs) have been predicted to exhibit many exciting magnetotransport phenomena. Here we report the experimental observation of an unconventional planar Hall effect (PHE) and a gate-tunable hysteretic planar magnetoresistance in EuS/TI heterostructures, in which EuS is a ferromagnetic insulator (FMI) with an in-plane magnetization. In such exchange-coupled FMI/TI heterostructures, we find a significant (suppressed) PHE when the in-plane magnetic field is parallel (perpendicular) to the electric current. This behavior differs from previous observations of the PHE in ferromagnets and semiconductors. Furthermore, as the thickness of the 3D TI films is reduced into the 2D limit, in which the Dirac SSs develop a hybridization gap, we find a suppression of the PHE around the charge-neutral point indicating the vital role of Dirac SSs in this phenomenon. To explain our findings, we outline a symmetry argument that excludes linear Hall mechanisms and suggest two possible nonlinear Hall mechanisms that can account for all the essential qualitative features in our observations.