1. System Architecture

The overall system architecture of the proposed self-powered wake-up detector is shown in Fig. 2. It consists of four major functional blocks, i.e. energy-harvesting CIS, frame buffer SRAM, hand detection engine, and switched capacitor (SC) DC-DC converter. The energy-harvesting CIS works in dual modes of imaging and harvesting. In the imaging mode, the CIS captures incoming images and stores them in the frame buffer. The hand detection engine then finds hand shapes from the images and produces wake-up signal to the gesture UI system on hand detection. In the energy-harvesting mode, pixels in the CIS are used as photovoltaic cells to harvest energy from incident light. The SC DC-DC converter then produces 0.3 V supply voltage for the digital blocks such as frame buffer and hand detection engine from the energy harvested from the CIS and 0.6V for the analog blocks including the CIS and SC DC-DC converter itself. The chip initially runs from the external battery, but it switches to the on-die supply voltage for self-powered operation when the harvested energy reaches a sufficient voltage level of 0.6 V.

2. Near-threshold Dual-mode CIS with 3T Pixels

The dual-mode CIS with integrated imaging and harvesting operations has been studied for area- and energy-efficient system designs ⁽⁵⁾. The dual-mode CIS operates in two phases i.e. harvesting phase and imaging phase to serve as the energy source as well as image sensor for a system. The photodiode (PD) in conventional CIS pixels ⁽⁶⁾ cannot work as a photovoltaic cell because it needs forward bias for photovoltaic operation but its anode is fixed in reverse bias as shown in Fig. 3(a). Therefore, another diode (PD2) is added to the dual-mode pixel ⁽⁵⁾ that can be switched to forward bias by connecting its cathode to the ground in energy harvesting mode (EHM) as described in Fig. 3(b). This additional diode does not incur any area overhead to the pixel as it’s built upon the existing diode (PD1) and improves the fill-factors as well in the imaging mode.

The dual-mode pixels incorporate a source follower (SF) to transfer the pixel value to the comparator in the analog-to-digital converter (ADC) as illustrated in Fig. 4(a), but this SF imposes limitations on the dynamic range under low voltage operations. In the meantime, digital pixel sensor (DPS) was studied for low-voltage operation of the CIS ⁽⁷⁾. It takes out the comparator from the ADC and makes each pixel compare its own value within the pixel to directly output the result to the following ADC as described in Fig. 4(b). This approach eliminates the SF from the pixel and thus improves the dynamic range under low voltage operation. However, the DPS structure incurs a large area overhead due to the comparator incorporated in each pixel and therefore, shared-comparator structure ⁽⁸⁾ has been proposed for the DPS as shown in Fig. 4(c). It mitigates the area issue by leaving only the input device of the comparator in each pixel and taking out the other part of the comparator to make shared among the pixels in the same column. In this structure, a row-select device is added in each pixel for selective activation of the input device.

We basically adopt this shared-comparator DPS structure to our dual-mode pixel as shown in Fig. 4(d) for low-voltage operations. However, in our structure, the row-select device in Fig. 4(c) is taken out from the input stack and shunted with the diodes to reduce the number of stacked devices for more reduction in supply voltage down to near-threshold level. This structure also reduces the mismatches across pixels because process variation impacts only the single input device rather than both the stacked devices together. Thanks to this proposed dual-mode DPS structure, we achieve 0.6 V of near-threshold operation of the dual-mode CIS.

3. Subthreshold SRAM with Disturb-free 10T Bitcells

Conventional 6T SRAM bitcell achieves high memory-density with its simple structure but its low-voltage operation is limited around 0.7 V owing to the several disturbance issues associated with the 6T structure such as read disturbance, write disturbance, half selection and hold stability ⁽⁹⁾. Recently, a 12T bitcell was proposed to eliminate all the disturbances arising in a bitcell for robust subthreshold operations by adopting read buffer, cross-point write wordline, and power cutoff structures ⁽¹⁰⁾. However, the large transistor count in this structure impacts memory density so we propose a novel disturbance-free 10T bitcell to address the density issue without sacrificing the robustness under subthreshold region.

The proposed 10T SRAM bitcell is described in Fig. 5. It consists of data storage (M1-M6) and read/write port (M7-M10) as shown in Fig. 5(a). It exploits the single bitline structure for its area efficiency by unifying the dual read/write ports exploited in the 12T bitcell. The cell layout in Fig. 5(b) shows the layers from the diffusion up to metal-3. We place the PMOS transistors (M1-M4) on the left side of the cell and the NMOS transistors (M5-M10) on the right side to secure the space for preventing the well proximity effect (WPE). We put two write wordlines (i.e. WWLA and WWLB) between the PMOS and NMOS transistors to exploit the area secured for the WPE prevention. The layout takes 1.915 μm in width and 1.535 μm in height, thereby resulting in 2.94 μm2, which demonstrates 25.4% less area than that of 12T bitcell ⁽¹⁰⁾.

In this 10T structure, PMOS device M3 or M4 is turned off during write operations to eliminate the write disturbance arising from the contention between pull-up device (M1 or M2) and write driver. The read buffer composed of M9 and M10 isolates the data storage from the bitline to avoid any read disturbance caused by low-impedance path between storage and bitline. Half-select is also resolved as the connection between the storage node and the bitline can only be made for the fully selected bitcells, i.e. both the row-select wordline (RWL) and column-select write wordline (WWLA or WWLB) should be active together for a selection to be made. In addition, the source voltage of the read buffer is held high during the hold state to minimize bitline leakage, which increases the hold margin of the bitcell.

Fig. 6 shows 50k Monte Carlo simulations for the 0.3 V operations of the conventional 12T and proposed 10T bitcells, respectively. Fig. 6(a)-(c) present the graphs for the read noise margin, write noise margin, and hold margin of the 12T and the 10T bitcells, respectively, and show the proposed 10T bitcell has comparable noise margin with the 12T bitcell despite its smaller transistor count.

4. Hand Detection Engine with Skin-color Invariant Haar-like Filters

Adaboost is a classification algorithm to detect target objects through the votes from simple classifiers like Haar-like filters ⁽¹¹⁾. Adaboost is widely used for its low complexity but is weak at skin color variations since the Haar-like filter basically works on the pixel intensity. Therefore, we propose a modified Haar-like filter that is robust to skin color variations by utilizing the number of corners in the image rather than the pixel intensity. The modified Haar-like filter compares the attributes of gray and white rectangular regions in a similar way to the traditional Haar-like filter, as shown in Fig. 7, but it compares the number of corners rather than the pixel intensity of the gray and white regions. During the filtering, gray region is examined to see if it has enough number of corners to become a salient part of the image by comparing the number with its neighboring white region. The salient regions from this modified Haar-like filtering give more robust hand detection results since the variation in the number of corners remains small even when there is a large variation in the skin color. Each pixel in this modified Haar-like filter stores integral number of corners within a region starting from the origin to the pixel as illustrated in Fig. 8. In this way, the total number of corners inside a certain region can be simply calculated from the values of four pixels bounding the region as illustrated in the Fig. 8.

This algorithm is evaluated with the Cambridge hand dataset ⁽¹²⁾ shown in Fig. 9. The dataset consists of five sets of hand gestures with different lighting conditions, and each set includes five hand postures. As shown in Fig. 10, skin color variations according to the lighting conditions lowers the accuracy of the conventional Haar-like filter, but rarely affects the accuracy of the proposed modified Haar-like filter, maintaining its accuracy above 90%.

Fig. 11(a) shows the architecture of our hand detection engine accommodating buffer reuse capability in storing the integral numbers. The engine consists of corner detection, integral number generation, and Adaboost classifier blocks. As described in Fig. 11(b), the wake-up detector operates in harvesting and imaging phases alternatively, and the CIS does not use the frame buffer in the harvesting phase. Thus, the hand detection engine can reuse the idle frame buffer in the harvesting phase to store the computed integral number of corners. This approach saves the extra storage required for the integral numbers. In addition, the Adaboost weights are stored in a small lookup table, saving the accesses to the external memory.

5. Switched -Capacitor DC-DC Converter

The proposed CIS harvests 0.3 V of supply voltage that needs load regulation to reduce the ripples from non-uniform occurrence of load current over time. Moreover, this 0.3 V supply needs boosting up to 0.6 V for proper operation of analog domain. Therefore, we exploit the switched-capacitor (SC) DC-DC converter for cost-effective on-die regulation and conversion of the energy harvested from the CIS. It consists of loop controller, maximum power point tracking (MPPT) circuit and switched-capacitor power stage, as described in Fig. 12(a). The controller uses inverting amplifier and adopts pulse frequency modulation (PFM) for closed loop control of the power stage. The MPPT circuit monitors harvested voltage level across the photodiode and maintains the 0.3 V of MPP level by controlling the charge transfer to the storage capacitor (Coff) for the maximum efficiency in the harvesting. Fig. 12(b) illustrates how the 0.6 V supply is generated from the power stage. In phase Φ1, the 0.3 V of Coff is transferred to the flying capacitor (Cfly) of the power stage through the charge sharing between the Coff and Cfly with 100:1 capacitance ratio to minimize the voltage level degradation from charge sharing. The power stage then boosts the 0.3 V up to 0.6 V in phase Φ2 through the capacitive coupling of the Cfly. This conversion mechanism achieves 82% of power efficiency.