Thesis Examination Committee

Prof Baoling HUANG, MAE/HKUST (Chairperson)
Prof Amine BERMAK, ECE/HKUST (Thesis Supervisor)
Prof Jing KONG, Electrical Engineering and Computer Science, Massachusetts Institute of Technology (External Examiner)
Prof Chi Ying TSUI, ECE/HKUST
Prof Wing Hung KI, ECE/HKUST
Prof Yi-Kuen LEE, MAE/HKUST

Abstract

Smart and low-power CMOS image sensor (CIS) has seen an increased attention over the last decade specially targeting wireless sensor network applications. To further push power limit, mixed-signal image processing has been embedded within CIS to reduce efforts on signal quantization, communications and processing. In this thesis, four types of CISs with embedded mixed-signal image processing are presented, with focus on image compression, spatial and temporal feature extraction, and a combination of these features, respectively. The aim is to build a smart image sensor featuring low power, high energy efficiency, improved processing capability and accuracy as well as lower area overhead.

The first CIS employs a planned-sensor-distortion (PSD) algorithm to compress image data before storage or transmission. The 3-bit PSD imaging is enabled by a column-parallel microshift-guided SAR ADC based on a 3×3 predefined pattern. The data bandwidth is further compressed by a customized lossless encoder using predictive coding and run length coding. High reconstruction image quality with low data rate and low power consumption have been successfully demonstrated through our first prototype chip.

The second CIS focuses on local binary pattern (LBP) extraction and edge detection for object detection/recognition applications. A mixed-signal 4-pixel simultaneous group computation (GC) scheme is developed to extract 8-direction LBP and edge information. High-speed and energy-efficient column-parallel GC is realized through an innovative group-switchable multi-input-multi-output (MIMO) comparator. The proposed reconfigurable mixed-signal processing circuits have been successfully demonstrated featuring multi-mode operations with minimum area overhead and low power consumption.

The third proposed CIS embeds motion sensing for event-triggered applications. The sensor is reconfigurable for motion sensing, object segmentation or snapshot. High-frame-rate frame differencing (FD) and low-frame-rate background subtraction (BS) are combined as a cooperative motion sensing scheme to improve motion detection robustness with low power overhead. Object segmentation is realized through BS-based region-of-interest imaging so as to reduce both imaging and object localization overhead. The full imaging mode provides the snapshot function. The mixed-signal processing architecture is reconfigurable at both pixel and column levels to support multi-mode operation.

Last but not least a fourth CIS is demonstrated combining motion sensing, feature extraction and image compression as a camera System-on-Chip (SoC). This versatile CIS design provides scene-adaptive sensing with optimized power and bandwidth utilization. The proposed design combines CIS versatile smart functionalities with high energy-efficiency and compact area making the prospect of deploying such a sensor for wireless camera network applications a reality.