This paper considers heterogeneous coded caching where the users have unequal distortion requirements. The server is connected to the users via an error-free multicast link and designs the users' cache sizes subject to a total memory budget. In particular, in the placement phase, the server jointly designs the users' cache sizes and the cache contents. To serve the users' requests, in the delivery phase, the server transmits signals that satisfy the users' distortion requirements. An optimization problem with the objective of minimizing the worst-case delivery load subject to the total cache memory budget and users' distortion requirements is formulated. The optimal solution for uncoded placement and linear delivery is characterized explicitly and is shown to exhibit a threshold policy with respect to the total cache memory budget. As a byproduct of the study, a caching scheme for systems with fixed cache sizes that outperforms the state-of-art is presented.