Vision Based Human Decoy System for Spot Cooling

This project aims to reduce the energy consumption of air conditioner usage while maintaining occupant comfort. Cooling down the unoccupied space can be considered as waste of energy. Therefore, a human decoy system is proposed to track any human in the detection area. Image contains depth data in each pixel which can be used to detect the presence of target subject as well as their position. The acquired position data is processed by using MATLAB and subsequently is transmitted to Arduino Mega using serial communication to control stepper motors. The experimental results show that the air conditioner airflow is successfully can be directed to the target human subject with average response of 0.860 seconds per movement within detection area.


Introduction
Air conditioner was a great invention of mankind. This can be proved by the first Prime Minister of Singapore, Mr Lee Kuan Yew who has highlighted air conditioner (AC) as one of the greatest invention of mankind [1]. People are trying to reduce the energy consumption as air conditioner is one of the most contributor to the electric bill. Nowadays, motion sensor and intelligent system are implemented to reduce the AC energy consumption. Motion sensor is able to detect the presence of human whereas the intelligent system is integrated to control the switching time on the air conditioner. Recently, an intelligent approach called Intelligent Eye System and 2 Area Intelligent Eye system are developed [2]. However, these systems have limitation in terms of detection area. Thus, this project proposes a new solution to directly track the human subject movement and subsequently increase the efficiency of the air conditioner system.

Research Method
The objective of the project is to develop an intelligent decoy tracking system based on vision application to detect human presence for spot cooling. The system will be implemented into the wall-mounted air conditioner. Figure 1 shows the block diagram of the developed Vision Based Human Decoy System for Spot Cooling. The Kinect sensor data is used to track human position in which the information is acquired and processed using MATLAB. MATLAB would process the position data before transmitted the signal to microcontroller. In this project, the microcontroller was used as a medium to control the movement of stepper motor. Stepper motor was attached on the air conditioner to control the movement of AC blades. Hence, the air will be directed according to the command given by microcontroller.  Figure 1. Block Diagram of the whole system

Acquisition of Target Human Position
This system involves Kinect sensor which is used to detect human presence. Figure 2 shows the layout of the sensor in the room and the required range of detection. Kinect sensor provides the x, y, and z coordinate of target subject. Nine specific points were taken from the detection area and their coordinates were recorded. In order to get precise reading, the average value of ten set of readings were taken. Table 1 shows the average reading for all nine points.

Human Tracking System Algorithm
Prior to data processing, Kinect sensor would detect and send the position data once there was someone stepped into the detection area. Then, the position data was used to check all the conditions so the system will know which area the location of target subject. Every area was assigned with certain angle of movement which is known as desired angle. After assignment of angle, the difference between new desired angle and old desired angle was calculated. Using Equation 1, the difference was then used to calculate the number of step that each stepper motor needed to take. Not forgetting the direction of motion, it was determined by the polarity of difference. Stepper motor would rotate in clockwise direction if the difference is negative value and vice versa. Important information such as number of step to be taken and required AC direction were send to Arduino via serial communication. Serial communication is used for Arduino to communicate with MATLAB. The data received from MATLAB could not be used directly without any conversion as it was in string format. Number of step needed to be converted into integer so it could have the numerical value. By using the product of conversion, the stepper motors could take number of step accordingly using for loop.

Results and Analysis
A series of experiments had been done to test the capability of the developed system. Testing aspect includes area of detection, position of human, and effect of human carrying item, covering some of the body part as well as effect of lighting.

Range of Detection
According to Microsoft website, the Field of View (FOV) of Kinect for Windows is 43° vertical and 57° horizontal [3]. The Kinect sensor location is discussed in previous section. A test has been conducted to determine the detection area of Kinect sensor based on Innovate Malaysia Design Competition (IMDC) requirement. Table 2 shows the results of the sensor detection range. The detected region is represented by green color whereas the red color denotes the undetected region.

Subjects Position
The capability of the Kinect sensor to detect human presence of human is investigated, in which the subject is required to stand up and sitting on the chair at coordinate of x = 150cm and y = 150cm. In addition, the test is also involved if there is other human (up to 2 subjects) presence in the detection area. Figure 4 shows that the Kinect sensor is able to detect human in standing condition. However, it was found that the sensor is not able to detect the subject in facing left condition as shown in Figure 5. This result may be explained by the fact that the sensor is unable to detect subject's head and shoulder in this condition. In contrast to standing position in which provides 20 joints reading, only 10 joints can be acquired in sitting position. In addition, Kinect sensor did not perform well on sitting facing both left hand side and right hand side. Figure 6 shows the results of subject facing left hand side. The ideal direction for sitting position is facing the sensor. Figure 7 shows the subject facing the sensor. Kinect sensor would be able to detect all 10 joints of the human body as they were exposed to the sensor, therefore it was able to detect the subject. The tests were repeated to determine the effect of an additional human presence as shown in Figure 8 and 9. On the other hands, facing sides gave the worth results. As can be seen the sensor provides negative result in detecting someone facing right hand side. Another interesting result came from detection of target in sitting position. It showed unstable results. For example, none of the subjects were detected in Figure 10 and Figure 11. Figure 10. Detection of two subjects who were standing facing the right hand side Figure 11. Detection of two subjects who were sitting facing right hand side

Effect on Carrying Items
Tests were carried out to find out the effect of carrying item to output of Kinect sensor as well. Results show items did not affect much on the detection as long as the items carried are not blocking big body part of the subject. In fact, the item carried will be detected as part of the body. For example, holding a laptop bag, the sensor may take the bag as the hand of subject as the length of the right hand and left hand were not same. The number of subjects does not give much effect as the maximum capability of the sensor is six persons. Figure 12 shows the detection of one subject was holding bag standing in front of the Kinect sensor.

Covered Body Parts
Tests were run to see whether the sensor was able to detect the subject when part of the body of the subject was covered. Result showed subject need to explore the head and shoulders in order to be detected. Kinect failed to detect the subject when the head and shoulders were covered but Kinect was able to detect the subject when the lower body of the subject was covered. Figure 13 and Figure 14 are showing the examples of detection for part of body covered. In addition, covering half of the body will make the sensor unable to detect the subject. Kinect sensor was not able to detect both the shoulders therefore it was not able to detect the subject. Figure 15 is showing the result of the detection when half of the body of the subject was covered. Figure 15 is one subject with half of body covered. Therefore, in order to be detected by the sensor, one must show the shoulders as well as the head. With these, Kinect sensor will detect the target and start tracking.

Different Light Conditions
It can be observed that the different light condition would not give much effect on the sensor output because Kinect sensor is using depth sensor as well as RGB sensor. Even it is under dark condition, the still able to function as if it is working under normal light condition. Figure 16 shows the detection of the sensor under low light condition. The detection is good as it is not affected by the light condition. The sensor is still able to detect the subject. Same result goes to two subjects' detection. Figure 17 is showing the detection of two subjects under low light condition.

The System Performance
Position data from Kinect sensor not only consists of the XY coordinate, it consists depth data as well. Position data was tested before it was used for further process. The area of detection was divided into 9 different points. Target was standing at different point in the detection area and the position data for each point were collection. The data was then analysis to get the mean for each point. Mean value was necessary to get more precise data. The area of detection was divided into nine areas as shown in Figure 18. Experiments were done based on the distribution of areas. Area A, B, C, D, E, F, G, H, and I were assigned "bottom left", "middle left", "top left", bottom middle", "center", "top middle", "bottom right", "middle right", and "top right" respectively in MATLAB. The respond of the system was good as it could detect all the area within the area of detection. In addition, the respond of the system was correct. The direction of the motion of stepper motors were correct. They followed the direction of motion of target.

Respond Time of the System
Next, the testing was carried out to investigate the respond time of the system. Respond time was taken when the Kinect began to detect target until first stepper motor started to move. The respond time for each area was recorded in Table 3. The average respond time of the system is less than 1 second. Therefore, the system can be considered to have fast response which is sufficient for this application.

Precision of the System
Desired angle was calculated using normal tangent rule as shown in Equation 2. The calculated results were shown in Table 4. Figure 19 and Figure   The angle of deflection of blades were measured and recorded. Further analysis was done to calculate the percentage of error for readings and tabulated in Table 5. From the tables, the mean of average error percentage for Blade A, Blade B, and Blade Y are 14.266%, 25.673%, and 4.796%. The mean of average error for Blade A and Blade Y are less than 15% which are considered accurate. The mean of average error percentage for Blade Y is relatively higher due to the high percentage on Area C detection. Angle of movement for Blade Y was limited by it mechanical structure. Hence, it could only go up to half of its desire angle. Besides the result of Blade Y in Area C, the others are considered good as the difference between desired angle and moved angle was small.

Conclusion
Vision based human decoy system is a system that able to track user once the target steps into the detection area and spot the target human. Kinect sensor is the only sensor that have been used for this project while the output which is the air flow of the air conditioner is controlled by the stepper motors. The system is an autonomous system, without any intervention of the user to control the air flow. The system has been tested to work in different light conditions. Interaction between MATLAB and Arduino Mega 2560 was initially not satisfying as the respond was a lot slower than expectation. However, this was overcome by using serial communication between MATLAB and Arduino instead of directly control Arduino. In addition, the system was able to detect both static and dynamic motion of human. Overall, the system was successfully developed and implemented onto the wall-mounted air conditioner after some modification.