Terrestial implementation of UNB Super Camera and improvements to UNB-PanSharp

Abstract

Camera sensitivity is a significant challenge for many imaging applications, especially in low light conditions. Image recognition and presentation in low light conditions is highly dependent on camera sensitivity. Issues acquiring colour images in low light conditions are amplified because of the fact that the colour images are acquired in a narrow spectral band. To address this issue it is possible to collect images in black and white (monochrome). The wide spectral coverage of such monochrome cameras can improve the sensitivity of the resulting images with the same sensors, but the colour will be sacrificed in this strategy. Another solution would be to use lower resolution colour cameras to increase the signal-to-noise ratio. This solution will result in less spatial detail. In satellite systems, to improve the sensitivity of the images, a pair of high resolution monochrome and low resolution colour cameras is used. Fusion of the images from those cameras will result in a high sensitivity and high resolution colour image. This thesis investigates the potential to implement this technology in a terrestrial configuration, using a security camera application as an example. UNB Super Camera is a high resolution monochrome camera coupled with a lower resolution colour camera which, when processed using UNB PanSharp technique, produces high resolution colour video. In order to implement UNB Super Camera for a terrestrial application, a system with four components was designed (data collection, processing, display / storage and framework software). All of the components were researched and studied in this thesis with the results of this work being used as inputs into the design and development of a terrestrial based UNB Super Camera system. The data collection review included issues associated with the camera, and its associated hardware requirements. Data processing included frame-to-frame co-registration, photogrammetric calibration and orientation that facilitated image fusion, motion detection and tracking and 3D positioning. Data display / storage was facilitated with a standard monitor and computer storage facilities. The key component of the system design and implementation is the framework software which is .NET based and has been designed and developed to facilitate the real-time operation of the UNB Super Camera system. The system was been successfully implemented and the results obtained were assessed as to their quality using the criteria of sensitivity, resolution and colour rendering. It should be noted that while a complete UNB Super Camera system has been designed and implemented, ONLY the imagery components are addressed in detail in this thesis. The motion object detection / tracking / 3D positioning components, as required by a security camera application are not analyzed in detail. These subjects are the focus of other researchers. The results of the assessment proved that the UNB Super Camera had measurably higher sensitivity and resolution and colour rendering in comparison with the same generation of available high resolution colour cameras, especially in lower lighting conditions. Despite this improvement, the fused images / videos had colour distortions and stain in very low lighting indoor cases and sunshine condition in outdoor cases. Investigation into these issues showed that the different spectral coverage of the high resolution monochrome camera and low resolution colour camera was the source of the problems. To address the contaminations, four methods -- including Fixed Coefficient, Adaptive Component, Monochrome Correction and Differential Filtering -- were proposed and investigated. Implementation of these strategies showed that the differential filtering method provided the best results. However, all of the methods were successful in recovering the distortions and stains in different lighting conditions, to varying degrees. In addition, the sensitivity, resolution and colour rendering of the results were further improved. Beside the spectral coverage effects, a debayering issue has also appeared in this project. Debayering effects of the low resolution colour were inherited by the high resolution fused videos. To address this issue, a combined Gaussian debayering and binning strategy was proposed. Although the resulting debayered and binned video was slightly blurred in comparison with the original debayered and binned low resolution colour, the resulting fused video frames using this method led to measurably higher-quality frames. Moreover, this method was computationally faster in comparison with the other methods, which is important in real-time applications.