Haptic Mouse Magnetic Field Based Near Surface Haptic and Pointing Interface

—In this paper, we are presenting a new type of pointing interface for computers which provides mouse functionalities with near surface haptic feedback. Further, it can be configured as a haptic display where users may feel the basic geometrical shapes in the GUI by moving the finger on top of the device surface. By mapping the motion and identifying the polarity of a neodymium magnet which attached to the finger tip, mouse pointer motion and control functions were achieved. Haptic feedback for user interactions was implemented by generating different like polarity haptic feedbacks using an electromagnet array which placed underneath of the device surface. This interface brings the haptic sensations above the device surface where previously it is felt only on top of the buttons of the haptic mouse implementations.


INTRODUCTION
OINTING devices used along with computers as an input device for more than four decades [1].They are used to control the mouse pointer or cursor in the graphical user interfaces (GUI).Movements and commands send by those pointing devices are echoed on the screen by movements of the mouse pointer (or cursor) and other visual changes [1].By far mouse is the most popular input device used.Track pad, stylus, Track ball and joystick are other kinds of pointing device implementations.
Pointing interfaces were continuously improved by adding new features like dragging, scrolling, and multitouch.Recently, there were some attempts to add haptic feedback sensations to the mouse.It can be understood that the addition of haptic sensations could enhance the attachment between the user and the computer.
Haptic Mouse is a new type of pointing interface which provides mouse interactions, haptic feedback and other enhanced features.The key advantage of this system over the other haptic pointing interfaces is that users do not required to touch the surface of the device.Instead the users could move the neodymium magnet worn on the fingertip near to the device surface and controls the cursor movement.This enables the haptic sensations in 3D space which will be a novel experience.
Different haptic sensations provided by this system can be felt like attraction, repulsion and various patterns of vibrations.Those sensations can be easily configurable as different feedbacks for different mouse commands using the interface driver we have developed.
This system provides attraction and repulsion sensations by changing the polarity of the electromagnets.Polarity is changed by swapping the positive and negative voltage supply to electromagnets using a controller circuit.When the neodymium magnet worn on the finger tips and the electromagnet array positioned in the opposite polarity (N -S or S -N) users feel an attraction towards the device surface.Users feel the repulsion sensation when those magnets are in like polarity (S -S or N-N) positions.
- --------------  Vibration sensations are provided by setting up neodymium magnet and magnetic array in a like polarity position and then rapidly switching on and off the electromagnetic array in certain frequencies.This rapid switching on and off dynamically changes the magnetic field it produces and affects the static magnetic flux developed by the neodymium magnet worn on the finger tips.While electromagnet is switched off neodymium magnet comes down but when the electromagnet is switched on it rises and this is felt by the user as a vibration.
Haptic Mouse allows users to both control and interact with the graphical user interfaces as same as the other pointing interfaces."Fig.2", shows the related gestures of 2D cursor movements and Commands of the system.Users can easily move their fingers on top of the device surface and control the GUIs.This can be visualized as a moving, invisible mouse on the mouse pad.
According to the default device configurations cursor movements are handled once the north pole of the neodymium magnet face downwards to the device surface.In order to execute mouse commands, this interface user has to rotate the finger 180 degrees which would face the south pole of the neodymium magnet downwards and perform appropriate gestures (Fig 2b).
Furthermore, this interface can be configured as a full Haptic display.It is possible for a user to move his/her finger on top of the surface and sense the basic shapes of the objects on the screen.This is achieved by providing a unique vibration pattern once the user moves the cursor on top of the interested object and then change to a different vibration patterns once the cursor crosses the border of that object.Simple geometrical shapes which are bigger than 200 pixels can be sensed and identified.
As further developments, if there is an application which is restricting the user to a particular window, this device can use the haptic feedback and let the user know about the virtual boundary.Further, the sensing of simple gestures will be helpful for users to increase their interaction with computers.Moreover, this system could be developed in assisting visually disabled in their interactions with computers.

RELATED WORK
This section will discuss prior research related/similar to this work which will present the authors with the advantage of highlighting the novelty value of the Haptic Mouse.
Liquid Interface [2] is a previous work of the author which has provided the base technologies for the current pointing interface.It is an organic user interface that utilizes ferrofluid as an output display and input buttons embodied with musical notes.Using a matrix of Hall Effect sensors, magnetic fields generated by neodymium magnets worn on the fingertips are measured and then converted into signals that provide input capability.This input actuates an array of electromagnets.Both Hall Effect sensors and electromagnets are contained beneath the surface of the ferrofluid.By matching like polarities between the electromagnets and the neodymium magnet, haptic force feedback can be achieved.Haptic feedback of this system is limited to the number of buttons in the display.It only detects switch on and switch off type of interactions and used to develop a ferrofluid based piano.
FingerFlux [4] is an output technique to generate nearsurface haptic feedback on interactive tabletops.It combines electromagnetic actuation with a permanent magnet attached to the user's hand.FingerFlux lets users feel the interface before touching, and can create both attraction and repulsion.This enables the development of applications such as reducing drifting, adding physical constraints to virtual controls, and guiding the user without visual output.They have achieved the vibration sensations up to 35mm above the table.As limitations, Fingerflux could only works with table top computers.It does not add magnetic based sensing and the maximum vibration feeling height is comparatively lower than our system.
Tactile Explorer [5] is a device which provides access to computer information for the visually disabled based on a tactile mouse.The tactile mouse resembles a regular computer mouse, but differs in having two tactile pads on top that have pins that move up and down.These translate the data on the screen to tactile sensation.Tactile Explorer provides possibilities to find and select desirable on-screen information and study it with different options.Kumazawa [19] introduces a mouse with a tactile display that generates tactile feedback for multi-modal user interface.The control system is applied to the multimodal user interface that uses tactile information in addition to the auditory information to assist visually handicapped people to operate a web browser that uses the tactile information to guide cursor operation.
Kyung [15] describes a haptic mouse implementation which conveys kinesthetic and tactile information simultaneously in virtual environments.This system provides 2-DOF translational force feedback, vibration, normal pressure, skin stretch and thermal feedback.This allows users to sense object's shape, stiffness and surface feedback properties.The system uses a 6x8 pin array to generate texture and thermal feedback feelings.Park's [17] haptic mouse is actuated by an electromagnet which works like a normal optical mouse but attraction force is generated between the electromagnet and a ferromagnetic mouse pad.It provides functionalities like generating magnetic attraction to allow the user to find the target easily, produce tactile feedback when the mouse cursor passes lines.
Kyung [18] propose Ubi-Pen, a pen-like haptic interface providing texture and vibration stimuli.
Preliminary evaluations indicate it can satisfactorily represent tactile patterns.He also evaluated its supportivity for GUI operations by producing a simple click-like feedback when buttons are pressed.In addition, this system provides texture sensation when a user rubs an image displayed on a touch screen.Choi [20] presents a novel haptic mouse system as a new human computer interface, which has a force feedback capability.A proposed haptic mouse can reflect 1 dof grabbing force as well as 2 dof translation force which can simulate more realistic virtual experience.Udea [16] proposed a mouseshaped haptic device which allows multiple finger inputs in order to carry out complicated tasks and in order to highly adapt to the environment.To develop the multifingered haptic device, we focused on anatomical knowledge and neurophysiology.
Microsoft tactile mouse [6] will be a commercially available mouse implementation which combines haptic sensation and will be developed to support rich features of their latest operating system.This mouse has a touch sensitive strip which contains two buttons, one on each end.Haptic-feedback, in the form of vibration through the touch-sensitive strip, indicates which one of the three scrolling speeds has been selected.Most of the haptic mouse implementations discussed above provides tactile sensation for users and they supports enhanced haptic interactions.however; operations and sensations are limited to the device surface.Further, the haptic actuation is limited to a small area of the device surface.Haptic Mouse implementation discussed in this paper provides a novel way for implementing haptic sensations in pointing interfaces.In addition it enables users to do mouse interactions and feel haptic sensation above the surface as well.

SYSTEM DESCRIPTION
Haptic Mouse contains three modules:

Neodymium Magnet attached to the finger and Hall Effect sensors Grid
The neodymium magnet attached to the finger allows users to actuate the Hall Effect sensors grid which is placed bellow the acrylic surface.Polarity of the neodymium magnet and various gestures made by the user is identified and measured by the Hall Effect grid.Neodymium magnet could generate higher density of magnetic flux compared to other permanent magnets.Therefore, it is easier to fix on the fingertip and especially size and weight of it became comparatively lower.
When the users interact with the interface, they can sense a subtle haptic feedback; this enables feedback for input and feels the objects in the computer screen in variable vibrations.Attraction and repulsion forces can also configured according to the application needs like guiding user automatically towards the default button when a dialog box appears.
We have used an Arduino [3] based microcontroller for processing the Hall Effect sensor readings.Analogue voltage readings of the sensors then converted to digital values using the built-in analog to digital converters and fed in to interface driver software to identify the gestures and commands.
It is observed that the sensor readings are stands approximately the same between 0cm to 2cm.When the power supply was set to 10V, the drop in the Hall Effect sensor reading over the distance appears to be greater.As a result we have decided to place the Hall Effect sensors grid 2cm bellow the device surface.With the placement of the Hall Effect sensors on the device surface the change of sensor values became lesser.Since there is not much difference between the values it became impossible to track the position of the neodymium magnet.However, by placing the sensor array 2cm below the surface we were able to track the position accurately.The Effective tracking area was then limited to 4cm above the surface.Also sensor grid has failed to detect magnetic field of the neodymium magnets once it is more than 6cm away from the grid.

Software Interface Driver
For the precise operation of the pointing device, there has to be a device driver which can integrate with the operating system.Therefore, using the Windows API we have developed a software driver for this device.
This driver accepts the row sensor values converted to digital from the microcontroller of the Hall Effects sensors grid as the input.When the North Pole of the neodymium magnet is positioned downward the ADC values are in between 512 -1024 range and when the South Pole is downward sensor values remain from 0 to 512 ranges.These sensor values are sorted in the descending order and if the magnet is North Pole downwards, software searches for the positions of the sensors in the grid where it received the maximum readings.Sensors which are nearest to the neodymium magnet, output the maximum values.Based on those intensity values relative distance to the neodymium magnet from the nearest three sensors are calculated.By finding the position of the neodymium magnet and comparing it with the next position, relative X,Y displacement can be calculated.Then these relative displacements are mapped to the last coordinates of the mouse curser position and moves the cursor to a new X,Y location.
In the case of identified mouse commands, firstly, driver identifies the neodymium magnet which is placed South Pole downwards by reading the digitally converted values.If the magnet is South Pole downwards, software driver searches for the three minimum sensor reading values and determines the coordinates of those sensors.Then, the distance to the neodymium magnet from each sensor is calculated and its position is determined.The movement path of the neodymium magnet is tracked and if the path follows the gestures defined for the mouse commands, the driver activates the appropriate commands.As the final step, it updates Electromagnet controller circuit about the necessary vibration pattern which would eventually provide the user with the vibration feeling.
In the case of sensing the shapes driver software keeps a selected vibration pattern until the user move the mouse cursor on top of the interested object in screen.Once the cursor is moved away from the object boundary, driver sends commands to the microcontroller of the electromagnet controller circuit to change the output frequency.

Electromagnetic Array and Controller Circuit
This part of the system is made with six electromagnets, Magnet controller circuit and Arduino based microcontroller.As the total power required by the electromagnets array is high at 6V and 13A [7], it becomes necessary to control the power supplied to the electromagnets via a relay circuit.This is because the voltage and current from the microcontroller pins amounts is only 5V, and 40mA respectively [3], which is insufficient to drive the electromagnet.To address this, the relay circuit acts as a mechanism that is able to switch on a much larger power to drive the electromagnets.For this power up electromagnets, six N-Type MOSFET [9] were used, one for each electromagnet.
The connections to the MOSFET are configured such that the MOSFET will enter linear region and produce a drain current ID, of approximately 1.9A when the Arduino outputs a 5V signal to turn on the electromagnet.When the Arduino outputs 0V signal and the MOSFET turns off, the drain current drops to 0A which turns off the electromagnet as illustrated in the "Fig.5".
When a PWM pin goes to high, the voltage at gate VG is at 5V, and voltage between gate and source VGS = 5V, causing the MOSFET to enter linear operating region.Since VDS is slightly more than 0V, a drain current, ID of approximately 1.9A is produced that is used to drive the electromagnet.When the PWM pin goes to low, VG = 0V, and VGS = 0V and there is no drain current (ID = 0) to power the electromagnet, causing it to turn off.
By programming the Arduino to switch continuously from high to low and vice versa in rapid succession, a PWM output wave is produced which in turn causes the MOSFET to continuously turn on and off the main power supply like a relay, generating another PWM output signal with enough current to drive the electromagnet.A diode connected in parallel to the electromagnet to prevent the damage to MOSFET by the backflow of current.A resistor is connected in parallel to the Arduino Pin and acts as a safety turn off mechanism.This design was replicated 6 times to drive the 6 electromagnets.
The electromagnets require PWM to run.The purpose of PWM is to simulate an analog voltage by rapidly toggling a digital pin between on and off.The percentage of time the digital pin is "ON" over the total time period is known as duty cycle [8].To output the PWM values to the MOSFET Arduino's hardware PWM pins were used.Software Interface Driver sends a 20 character length data frame for every 10ms via the serial connection to the microcontroller to activate the required electromagnets.These data frames are interpreted as commands to turn on the electromagnets that correspond to the Haptic feedback sensations felt by the user.Due to the limitations of the electromagnet, the maximum frequency that can be achieved is 100 Hz.Therefore, different frequencies between 5 Hz to 100Hz were used to provide different Haptic sensations to the users.
The relationship between the PWM and the maximum height that haptic sensation can be sensed follows an increasing linear trend, suggesting that the system is linearly controllable.However, it is noted that the haptic sensations were started to feel from PWM running on 11%.Further, when the PWM values are between 90% and 100%, it is hard to notice the maximum difference of the actuation.With these results we were able to provide the maximum height of haptic sensations from above the device surface to 6cm.

RESULTS
Two technical experiments were carried out to measure the capabilities and limitations of the system.
The purpose of this experiment was to measure the accuracy of sensor readings and algorithms written in the interface driver software.This experiment was conducted by moving the neodymium magnet on top of the device surface towards for direction as four straight lines in 3 different heights respectively; 0cm, 2cm and 4cm.The results are illustrated in the "Fig.6" , "Fig.7", and "Fig.8".
Hall Effect sensor grid used in this device is a 4*3 array (4 sensors along the X axis and 3 sensors along the Y axis).The space between two Hall Effect sensors were allocated 100 pixels.Therefore, all the sensor values recorded are represented as X,Y coordinates (0-300 in X axis and 0 to 200 in Y axis).
According to figure 6, the sensors were capable to detect the motion of the neodymium magnet in near liner fashion on the surface.Further, sensors managed to detect the position more than 90% of points.This line does not reflect the movement of the mouse curser.Mouse curser position is calculated by adding the difference of the X,Y displacement between two neodymium magnet position readings.Therefore, the accuracy of the movement of the mouse cursor was improved.

Tracking Accuracy measurements and improvements
Figure 7 shows the position detection readings of the neodymium magnet from 2 cm above the device surface.Sensors were able to track the position in near linear fashion more than 60% in this height.Large white spaces between the line segments show the places where sensors failed to detect the position of the neodymium magnet.By cancelling out those position changes, interface driver software still able to move the mouse cursor in the correct direction but slower than on the device surface.
In 4cm of height the sensor array only able to track the position of the neodymium magnet in near liner fashion less than 40% of the positions.In this height, the curser movement became fairly difficult.

DISCUSSION
Current physical computing interfaces emphasis on two things, mainly efficiency and speed.However, technical achievements alone may not be enough [13].In the design of user experience in HCI methodology, the emotional needs and desires of the human psyche has been largely ignored.Haptic Mouse seeks to address this by incorporating features that fulfils the emotional needs of humans can be achieved.Fels [14] postulates that an intimacy relationship must be established between the person and the interface to allow for an engaging experience.She described four types of relationships that users can have with an object, namely, the user communicating with the object, the user embodying the object, the object communicating with the user and finally, the object embodying the user.
The relationship of users communicating with the object can be seen in the way the user interacts with the interface.Moving the fingers above the interface initiates a communication relationship when the interface responds to the gesture.Khaslavsky states that in order to draw out the interest of a person, the interface must appear to be inviting and enticing enough for them to use it [12].Therefore, to investigate the users connectedness towards this kind of interface, a user study needs to be conducted, the results of which can be used to identify the ideal variables that will make for an optimum user experience.Several possible problems to look at include whether the loss of haptic feedback will affect the user experience, is resting hand on the device surface once using the device rather than keep some distance above from the surface become more easy and productive, successfulness of mapping clicks and scrolling for gestures are needed to be answered.

CONCLUSION
The haptic feedback resolution of the interface can be improved.These involve the use of smaller electromagnets in larger numbers.An increased resolution offers a better accuracy and representation of the virtual objects in the computer screen when the device used as a haptic display.This would improve upon the relationship of the user communicating with the object and user embodying the object.This device can be improved as an interface for visually handicapped who rely mostly on touch sensation.In order, to improve to this level of proficiency, this system is required to minimize the size of the electromagnets and increase the density of electromagnets packed in the electromagnets array which will provide a better resolution.This device could also be improved as an easy learning tool for children, which can be used to draw some basic shapes or characters that will enhance the interactive enjoyment.The neodymium magnet could be replaced with other forms of magnetized materials in future.
To conclude, in this paper, we have presented a new type of computer interface which provides basic pointing interface functionalities with near surface haptic feedback up to 6 cm of height.The advantages and limitations of the system were also discussed with the related works.Using three technical studies we were able to show that system can perform in an adequate manner that has strong potentials to be improved.Haptic Mouse provides the base tools to combine magnetic field based devices with computers.

PFig. 1 .
Fig. 1.User could freely move the neodymium magnet attached to the finger above the device surface and interact with the computer Fig. 2. Cursor movement and Commands gestures of the system.

Fig. 5 .
Fig. 5. Cursor movement and Commands gestures of the system.

Fig. 6 .
Fig. 6.Position of the neodymium magnet detected by the Hall Effect sensors array once the magnet is moving on the surface.

Fig. 8 .
Fig. 8. Position of the neodymium magnet detected by the Hall Effect sensors array once the magnet is moving 4 cm above the surface.

Fig. 7 .
Fig. 7. Position of the neodymium magnet detected by the Hall Effect sensors array once the magnet is moving 2 cm above the surface.