Video surveillance systems became one of the most essential tools people rely on for keeping buildings, businesses, urban areas, and borders secure. As the number of deployed cameras increase rapidly, the ability of human observers to analyze the video in real-time is becoming almost impossible. Video analytics engines embedded in networked cameras and video analytics servers provide tools for automatically analyzing the scenes. However, the video compression hardware used in cameras is decoupled from and not aware of the embedded analytics engines, servers and other cameras possibly capturing the same scene. The objective of the proposed project is to design smart embedded video encoders that maximize the content and reliability of information extracted by the video analytics while minimizing the cost of deployment. Video compression algorithms that utilize the video analysis metadata to improve the prediction efficiency, computational resource and rate budget allocation will be designed. Objective video quality metrics specifically tailored for surveillance applications will be implemented to improve and judge the proposed techniques. Application specific quality measurement will be a new concept since previous studies in the literature focused on determining the perceptual distortion in images and videos. We also propose collaborative design of the video encoders located at cameras capturing overlapping scenes. Video analytics metadata will be utilized for joint multi-camera rate control and prediction algorithms. Finally, the project will tackle the robust data transmission challenges of the wireless surveillance systems. Wireless ad-hoc network state measurement and adaptive video streaming rate control techniques will be developed. A novel and scalable surveillance system will be devised upon the successful completion of the project.