Detection Hand Motion on Virtual Reality Mathematics Game with Accelerometer and Flex Sensors

Montessori method is a learning method using props. One of the developments props is to use the game as a medium of learning. The examples Game media as learning is the use of Virtual Reality or VR Technology. By using the VR, players will be brought into the virtual world as if the player is in the real world. The weakness of the VR game is the limited interaction with the outside world. Interaction uses only buttons and joysticks. In this paper we use Flex sensor and accelerometer sensor to detect hand movements for VR mathematic game. The result is VR games are more interactive and interesting with hand motion.


Introduction
Montessori Method is a learning method using props in teaching [1].This method continues to development, one of them is using the game.Examples of games as a medium of learning is game 'berhitung', 'sinau boso jowo' and so on.The game is effective as a learning medium because the game involves players in the game [2].
Game technology has many developments, one of them is virtual reality technology.Virtual reality is a technology that describes a three-dimensional environment in which the 3D environment is simulated by computer [3].In Virtual Reality, players can interact with the environment.
By using VR technology, the Montessori method will be more easily implemented because VR becomes a real learning tool.Players will be brought into the virtual world like in the real world.The player will be more easily accepted the value.This accordance with the level of the learning model on the Edgar Dal's pyramid.With doing the real thing, the lesson will be easier to remember.There is showing in the Figure 1 conical media learning Edgar dale.This figure provides an overview of the learning method according to Edgar Dal's pyramid [4].However.VR games are still a lot of weakness, which is this VR games has limited movement.This VR games depending on the head or the environment around the eye.If a player wants to go forward or backward, the player must use certain movements on his head like a bump to go forward or look up to retreat.This will reduce the appeal of play.Though according to Edgar Dale learning will be more remembered where player using all body movement.
For this reason, hand motion sensor is implemented on the VR games interface.This hand motion sensor will detect hand movements.By using hand gestures into VR games, the VR games will be more interactive and learning will be easier to remember.Hand motion sensor use a flex sensor and accelerometer sensor ADXL 335.

Related on Work
Brenda, Frade uses VR games for Math learning.They use of VR games as a medium of learning mathematics is still limited to the use of VR head sheets and have not used another interface [5], [6].Barmpoutis uses Kinect sensors in learning.This method produces effective learning [7].While Yulia, Conn, and Lee use glove hand sensors in virtual 3D [8][9][10].they use accelerometer and gyroscope sensors to detect hand gestures.Tageldeen use it also, Tageldeen using accelerometer and gyroscope sensors on VR to detect hand movements.Tangeldeen uses it for body arm rehabilitation [11].However, applications they made do not use finger detection or flex sensors.Flex sensors are usually used to detect finger movements.Vibol uses finger movements to be brought into the virtual world.A sensor used Vibol very sensitive because based on pressure and frequency vibration [12].Motion sensor using the flex sensor and accelerometer sensor is widely used to moving the robot's hand.Flex sensor and accelerometer are connected with Arduino.Then, Arduino sends signals to robot hand through xbee, direct cable and RF signal [13][14][15].

Literature Review 3.1. Virtual Reality
The term Virtual Reality (VR) was introduced by Jaron Lanier.At that time Lanier founded a company for VR business.Virtual Reality (VR) is a computer technology linked to the surrounding environment to connect with a simulation in a virtual world where it is a 3D image.VR is connected in real time and can connect with other users.In addition, this technology is pseudo-natural immersion or the user can move naturally, so the movement in the virtual world through the sensor components is just like the movement in the real world.Virtual Reality Elements [3]: a. Virtual world, i.e. the content of a given medium that can be either screen play or script.In this element, the user will experience the virtual world.b.Immersion, the sensation of existence in an environment.Immersion is divided into several types' i.e. mental immersion, physical immersion, and mentally immersion.c.Sensory feedback, which is information displayed to the senses of the user in the virtual world, sensory feedback can be visual, audio or touch.d.Interactivity, the virtual world that responds to user actions in real time.

Flex Sensor
The usual analog phenomena measured in the internal robot system relate to position, velocity, acceleration and inclination.From outside the robot system, the measurement is closely related to the determination of the robot coordinating position on the reference workspace [16].Figure 2 shows is the dimension of Flex Sensor.The following Figure 3 is the basic circuit of the Flex sensor.
Flex Sensor is a flexible sensor that has a length of 4.5 inches.This bending sensor is patented by Spectra Symbol.This flexible sensor resistance changes when the metal pads are out of bending.Specification: 1. Temperature coverage: -35o C to + 80o C.

ADXL 335
ADXL335 is a 3-dimensional motion/acceleration sensor with 13-bit resolution (213=8194 precision level) that can detect movement at range up to 16g (16 x 9.81 m/s 2 ~±157 m/s 2 ).This sensor is able to detect the tilt by monitoring changes in static pull and dynamic acceleration arising from movement and impact [17].Measurement method of ADXL 335 using rotation is illustrated in Figure 4. What is detected is the rotation of the sensor.It is not a shift of x, y, z but the rotation of: Φ-Roll-X rotation θ-Pitch-Y rotation Ψ-Yaw-Z rotation

Research Method
Figure 5 shows that there are 2 sensors applied i.e. flex sensor to detect the movement of fingers, and ADXL335 to detect the movement of the wrist.All sensors will be controlled by Arduino.Arduino will send it to the VR game application on android based mobile phone using Bluetooth.The Bluetooth application used is MH-04.Serial data will be captured by VR games.Engine Game applications utilize game engine unity.

Results and Analysis
The variation of flex sensor is compared with the output of digital data with flex sensor deflection position.Figure 6 shows Graph of voltage measurement comparison with flex sensor deflection.The result of accelerometer sensor testing can be shown in the following Figure 7, where the accelerometer output in digital data is compared with changes in 3 dimensions (x, y and z).The resulting measurement will be converted into digital data by Arduino, where the all sensors will be inputting on analog input in Arduino.Analog data will be converted into digital data and then sent to android smartphone via Bluetooth signal using component mh-04.The data are transmitted in serial form and the sampling data every 0.5 second.By android, the serial input data from Bluetooth is used as a hand drive in the VR game.The serial data is then converted to the existing layout position in the VR game.The weakness of accelerometer sensor only detects hand movement and does not detect the position of the hand.As a result, the hand sensor cannot follow the movement of the head or the eyes.To resolve this problem, the hand drawing on 3D will be made state following the head movement.The weakness of this app is hand gestures not really like the real state.Or in other words, the movement of the hand that can be detected under whereas the VR is on top.The settings for hand motion detection are as follows in Table 1.VR games are designed for math lessons in grade 1 elementary school children.The design of this math game is selecting the shape of the specified image, for example choosing the shape of the cube.The player will choose the shape of the displayed pictures.The player has 3 chances then the game ends.The player will get points for each correct answer.The appeared spatial forms are selected by hand.Figure 8 shows the VR game.

Conclusion
Real hand motion will control hand motion in virtual reality games.Hand movement is detected by using a sensor where the output is a different voltage.Variable voltage will be converted to digital by Arduino and then sent to VR game to control arm in 3D of VR world.Flex sensor to detect the movement of the finger while the accelerometer to detect hand motion.The use of the accelerometer only detects hand motion and does not detect the position of the hand, to resolve this problem; the hand drawing on virtual reality is made fixed

Figure 5 .
Figure 5. Design of motion sensor

Figure 6 .Figure 7 .
Figure 6.Graph of voltage measurement comparison with flex sensor deflection

Figure 8 .
Figure 8. Display on VR game

Table 1 .
Setup Motion Detection