With a myriad of edge cameras deployed in urban areas, many people are seriously concerned about the invasion of their privacy. The edge computing paradigm allows enforcing privacy-preserving measures at the point where the video frames are created. However, the resource constraints at the network edge make existing compute-intensive privacy-preserving solutions unaffordable. In this paper, we propose slenderized and efficient methods for Privacy-preserving Surveillance as an Edge service (PriSE) after investigating a spectrum of image-processing, image scrambling, and deep learning (DL) based mechanisms. At the edge cameras, the PriSE introduces an efficient and lightweight Reversible Chaotic Masking (ReCAM) scheme preceded by a simple foreground object detector. The scrambling scheme prevents an interception attack by ensuring end-to-end privacy. The simplified motion detector helps save bandwidth, processing time, and storage by discarding those frames that contain no foreground objects. On a fog/cloud server, the scrambling scheme is coupled with a robust window-detector to prevent peeping via windows and a multi-tasked convolutional neural network (MTCNN) based face-detector for the purpose of de-identification. The extensive experimental studies and comparative analysis show that the PriSE is able to efficiently detect foreground objects and scramble frames at the edge cameras, and detect and denature window and face objects at a fog/cloud server to ensure end-to-end communication privacy and anonymity, respectively. This is done just before the frames are sent to the viewing stations.