Minimum MSE design for multiuser MIMO relay

We study a joint optimal design of transceiver and relaying architecture in terms of mean square error (MSE) performance for multi-user multiple-input multiple-output (MIMO) relaying system. We show a duality relationship in the design of MSE-optimal transceiver and relaying architecture. Based on that, we devise an alternating algorithm, which converges and provides a suboptimal joint design of transceiver and relaying. By simulation, we show that the proposed joint design improves on the average MSE performance of the existing transceiver and relaying architectures.