The square lattices formed by main-group elements such as Bi, Sb, Sn, and Si in layered materials have attracted a lot of interest, since they can create rich topological phases. In this paper, we report the slightly distorted square lattice of Sn in a noncentrosymmetric compound YSn2 generates multiple topologically nontrivial bands, one of which likely hosts a nodal line and tunable Weyl semimetal state induced by the Rashba spin-orbit coupling and proper external magnetic field. The quasiparticles described as relativistic fermions from these bands are manifested by nearly zero mass and nontrivial Berry phases probed in de Haas-van Alphen (dHvA) oscillations. The dHvA study also reveals YSn2 has a complicated Fermi surface, consisting of several three-dimensional (3D) and one 2D pocket. Our first-principles calculations show the pointlike 3D pocket at Y point on the Brillouin zone boundary hosts the possible Weyl state. Our findings establish YSn2 as a new interesting platform for observing novel topological phases and studying their underlying physics. \ 2018 American Physical Society.