We study how to communicate semantic information in the presence of an agent that can influence the decoder by providing side information. The agent's true intentions, which may be adversarial or helpful, is unknown to the communicating parties. Actions taken by the agent are governed by its intentions, and they may improve or deteriorate the communication performance. We characterize the optimal transmission policies to minimize the end-to-end average semantic error, i.e., difference between the meanings of intended and recovered messages, under the uncertainty in the agent's true intentions. We formulate the semantic communication problem as a Bayesian game, and investigate the conditions under which a pure strategy Bayesian Nash equilibrium exists. We then explore the structure of the encoding and decoding functions under the mixed strategy Bayesian Nash equilibrium, which for the semantic communication problem at hand always exists. Our results show that the optimal policies are strongly influenced by the belief the parties hold about the agent's true intention.