Biography
Enrollment Date: 2010
Graduation Date：2013
Degree：M.S.
Defense Date：2013.05.29
Advisors：Xiang Xie
Department：Institute of Microelectronics，Tsinghua University
Title of Dissertation/Thesis：Design of A Real-time Stereo Vision hardware System Based on FPGA
Abstract:
In recent years, with rapid development of Stereo vision technology, it has been wildly used. This technology gets depth information of object surface by calculating disparity between multi-view images captured from different viewpoints at the same time in same scene, and then produces stereo vision by some means. Although stereo vision have come to people’s lives, for example 3D movies, 3D effects are still not good enough for every scene. In order to Capture and get stereoscopic video with great human experience, we propose a project to implement a special hardware system that can acquire images using 2 or more cameras with high-speed interface and support different ways of 3D acquisition. This platform can be used for SoC design. The function of SoC are improving 3D effects and assisting 3D camera to capture. The SoC implementation is our final purpose.
There are many fields we need to do research in stereo vision technology. From capture to display, the whole processing chain include acquisition, sender side processing, 3D scene representation, Coding, Transmission, rendering and 3D video(3DV) or free viewpoint video(FVV) display. All these technologies are broad research areas on their own. After we summarized the other reseachers’ previous work, we found there are many questions on acquisition and sender side processing. Therefore, our research focus on below:
(1). Analyzing and calculating the parameters which mainly affect quality of 3D display in multi-view video acquisition. The way of acquisition directly affects the result of 3D effects. This Part is also complicated and very important in stereo vision technology. After research on production rules for Stereo Acquisition, we presented some math formulas to calculate these parameters in different situations, we also discussed their rationality.
(2). Hardware implementation of a real-time stereo vision system based on FPGA. We implemented capturing at same time using 3 image sensors and storing these real-time multi-view images using Flash Memory. This platform can be used to prove correction algorithm right or wrong and also get them hardware implementation. Hardware implementation of correction algorithm is a part of work in SoC design. FPGA, COMS image sensors, flash Memories and other devices constitute the system. FPGA is the core, CMOS image sensor is the image capture chip, flash Memory is the memory devices. The system could capture and store real-time 720p,25fps or 1080P,20fps multi-view images. That means the system could meet the requirements of standard definition video. Meanwhile We carefully analysed and tested the time between two image sensors captured multi-view images. After real-time stereo vision system is implemented, there are some prospect of our SoC design , and we also described its rationality.
(3). Combining the original image data into 3D video. We used the original multi-view images to produce 3D video that could be watched on 3DTV. By this step, we can directly prove the rationality of the system design and the theory about stereo acquisition.