The focus of this project is the development of algorithms that allow one to capture and analyse dynamic events taking place in the real world. For this, we intend to develop smart camera networks that can perform a multitude of observation tasks, ranging from surveillance and tracking to high-fidelity, immersive reconstructions of important dynamic events (i.e. 4D videos). There are many fundamental questions in computer vision associated with these problems. Can the geometric, topologic and photometric properties of the camera network be obtained from live images? What is changing about the environment in which the network is embedded? How much information can be obtained from dynamic events that are observed by the network? What if the camera network consists of a random collection of sensors that happened to observe a particular event (think hand-held cell phone cameras)? Do we need synchronization? Those questions become even more challenging if one considers active camera networks that can adapt to the vision task at hand. How should resources be prioritized for different tasks? Can we derive optimal strategies to control camera parameters such as pan, tilt and zoom, trade-off resolution, frame-rate and bandwidth? More fundamentally, seeing cameras as points that sample incoming light rays and camera networks as a distributed sensor, how does one decide which rays should be sampled? Many of those issues are particularly interesting when we consider time-varying events. Both spatial and temporal resolution are important and heterogeneous frame-rates and resolution can offer advantages. Prior knowledge or information obtained from earlier samples can be used to restrict the possible range of solutions (e.g. smoothness assumption and motion prediction). My goal is to obtain fundamental answers to many of those question based on thorough theoretical analysis combined with practical algorithms that are proven on real applications.