Biography
Enrollment Date: 2015
Graduation Date：2018
Degree：M.S.
Defense Date：2018.05.23
Advisors：Xiang Xie
Department：Institute of Microelectronics，Tsinghua University
Title of Dissertation/Thesis：Research on Key Technology of The Projector-Camera Touchpad Human-Computer Interaction System
Abstract:
With the portability and miniaturization of intelligent devices, many traditional human-computer interaction devices, such as displays, keyboard and mouse, are not suitable for use on intelligent devices. In order to solve the portability and convenience of human-computer interaction, many new human-computer interaction methods have been applied to intelligent devices. The interactive mode based on the projector and the camera brings more flexible and wide application space to the future interaction.The aim of this project is to establish a human-computer interactive system based on single projection and single camera system, which can meet the user's needs for convenient human-computer interaction while adapting to the application scene of smaller portable intelligent devices. In the human-computer interaction system of the projection touchpad, it is hoped that the precision of the touchpad can reach more than 98% touch accuracy below 2mm, and the corresponding cursor movement error is 1 pixels in the movement of the finger after touch, and the whole system function is the same as the real touch board. The basic principle of the man-machine interaction system of the projection touchpad is to obtain the three-dimensional information of the surface of the object based on the three-dimensional measurement and structural light of the binocular system, and the finger model is used to judge the movement of the finger touch and the track after the finger touch. The main difficulties are as follows: 1) high precision calibration to obtain accurate projector camera model parameters, the accuracy of 3D information acquisition, and 2) how to obtain high precision information on the undercovered finger surface, so as to achieve high precision touch judgment; 3) the relationship between the finger and cursor and high precision trajectory tracking of the hand; 4) how to achieve real-time and hardware implementation of the algorithm to meet the needs of mobile devices.The main innovation points are as follows: 1) an improved projector camera system joint calibration method, which considers the projection angle perspective distortion, is adopted to obtain high precision system calibration parameters and achieve high precision three-dimensional measurement for the pointed position of the interested point; 2) use the color coded structured light to obtain the three-dimensional information on the upper surface of the finger in motion; 3) a precise finger model is used to solve the problem that the information of the finger cannot be obtained at the time of the finger touch. At the same time, it makes use of the three dimensional information of the key points on the finger model to judge the finger touch, simplifies the algorithm and reduces the storage space. It provides strong support for the real-time and hardware implementation of the system. 4) combining the finger model, finding the correspondence between the finger and the cursor, and using the key points on the finger to put forward and realize a high precision finger tracking method after finger touch based on Kalman filter method.The experiment platform is built on the basis of the personal computer and the projector camera system, and the experiment is carried out for the three parts of the calibration method, the touch judgment and the track tracking. After the calibration of the projector camera system is improved, the projection error of the projector is 0.1401. The accuracy of touch is up to 99% below the height of 2mm. The accuracy of tracking accuracy of finger movement is within 1 pixel. Under the illumination environment of 1000lux and the projection distance of 35cm, the overall performance index can meet the needs of projection touchpad interaction.