Resource allocation plays a critical role in implementing D2D communications underlaying a cellular network. Game-based approaches are recently proposed to address the resource allocation issue. Most existing approaches employ deterministic game models while implicitly assuming that each player in the game is completely willing to exchange transmission parameters with other players. Thus each player knows the complete information of all others. However, this assumption may not be satisfied in practice. For example, users may be reluctant to disclose all their parameters to peers. In this paper, we fully consider this scenario, i.e., players have incomplete information of others, and investigate the resource allocation problem for multicell D2D communications where a D2D link utilizes common resources of multiple cells. To attack this problem, a game-theoretic approach under the incomplete information condition is proposed. Specifically, we characterize the Base Stations (BSs) as players competing for resource allocation quota from the D2D demand, formulate the utility of each player as payoff from both cellular and D2D communications leasing the resources, and design the strategy for each player that is determined based on prior probabilistic payoff information of other players. We conduct extensive simulations to examine the proposed approach and the results demonstrate that the utility, sum rate, and sum rate gain of each player under the incomplete information condition are surprisingly higher than the counterparts under the complete information condition.