E-GLOVE Part-1 -- James Bond

E-GLOVE -- AN ARDUINO BASED AIR-MOUSE

I recently got my hands on an accelerometer, the MMA7361 on a nice breakout board for use with the Arduino and that got me thinking about its practical applications. Inspired by that idea and some Googling, I got around to making myself an e-glove, something like an air-mouse for the computer with which I could control my mouse with hand movements. Firstly, I had to figure out how to use the accelerometer by hooking it up to the Arduino and how to get understandable readings from it.

Fig.1.The final glove with wrist-strap

Initial Googling got me to the Sparkfun website, which distributes the said accelerometer and that got me some useful information regarding hooking it up to the Arduino (the page was in Chinese, I think) and a sweet little library that did all the maths for me. Sparkfun MMA7361, MMA7361 Library MMA7361 Connections. For those of you interested in further reading and details on how to do the maths, read it here.

Once that was hooked up, the next step was to go ahead and get the mouse to move in response to the accelerometer readings. So, I went ahead and got myself an Arduino Leonardo, which supports on-the-fly mouse control using the ATmega32u4 that has built-in USB communication, eliminating the need for a secondary processor.  This allows the Leonardo to appear to a connected computer as a mouse and keyboard, in addition to a virtual (CDC) serial / COM port. The official details for the board are given here.

Now, came the tough part. The readings of X and Y axis coming from the accelerometer had to be translated into real movement of the mouse on the screen. Taking a look at the Mouse control function of the Leonardo, this was the syntax :

Mouse.move(<x-axis>, <y-axis>, <z-axis>);

But, the readings from the accelerometer were slightly different than expected. The X-Axis readings from the accelerometer were the Y-Axis movement on the monitor and vice-versa. Since we wanted hand-up motion to move mouse up and hand-down motion to move mouse down, we had to take –X readings for the up movement and +X readings for the down movement. Once that was cleared out, the next hurdle was that the mouse needed to be stopped from moving if hand was at a central position, i.e. at rest. So, to facilitate that, we figured out a simple graph system which would allow us to set thresholds based on readings taken in from the accelerometer and move the mouse only if that threshold was crossed. The graph below is basically what we used as the basis for all mouse movements. Initially, the mouse movement was fixed to 4 directions, but we added separate code to facilitate movement in all 8 directions. The blue areas in the graph below are essentially the threshold values we used on the basis of readings from the Arduino. The mouse was now working quite nicely with relative smoothness in all directions.

Fig.2.The Graph used to figure out threshold values

The next step of the project was to actually try and make this wireless, since the main aim was to use it for something like controlling a robotic arm from a distance or shoot some bad guys in Call of Duty. So, I pulled out my trusty RF transmitter and receiver and got started with trying to integrate that into this glove. This meant that I had to get hold of an Arduino Uno to start acting as the transmitter, which would sit on the glove and the Leonardo as the receiver, which would remain connected to the computer. The RF uses the Virtual Wire Library to send and receive data on a 433Mhz frequency. After connecting the RX and TX to the Leonardo and the Uno respectively as given here, I proceeded to code both ends. The Uno would take care of receiving all the accelerometer input and transmitting them to the Leonardo receiver which would then move the mouse on the system.The RF works at 4800bps (bauds per second) which gave me pretty good resolution on the mouse.To improve on the efficiency of transmission, the coding was done to optimize the data sent to a minimum amount. So, we went for low level encoding and brought down the data sent to around 10 or 15 bits as opposed to sending the X readings and Y readings separately as integers. We also added an extra bit to the setup()portion of the code on the transmitter so that each time the user would start his glove, the receiver would calibrate the glove based on initial readings of the accelerometer.

The basis of calibration was that it would receive around 5 to 10 initial readings after which it would add to it a threshold value based on which the device would respond. This gave us a dynamic range within which the mouse would move, irrespective of how the user was wearing his glove. Also, we added a sensitivity option on the basis of said calibration that would allow the user to control the speed of mouse movement based on the degree of movement of his hand. This was done using a float value based on accelerometer readings that gave a nice smooth transition from one speed to another, like an exponential curve.

The next step was to add control for the mouse clicks for which we employed two Flex Sensors. This is a cool piece of hardware that changes its resistance based on how much it is bent. Using information available here, I set up the flex sensors on the glove’s fingers and added some calibration bits for this too. The Leonardo syntax for mouse clicks is :

Mouse.press(MOUSE_LEFT | MOUSE_RIGHT | MOUSE_MIDDLE);

Mouse.release(MOUSE_LEFT | MOUSE_RIGHT | MOUSE_MIDDLE);

Mouse.isPressed(MOUSE_LEFT | MOUSE_RIGHT | MOUSE_MIDDLE);

Using these three lines of code, we figured out a way to send a bit for each click, left and right, and then, on the receiver, check if they were pressed or released separately or simultaneously. When pressed simultaneously, the Mouse.scroll() function was called and that allowed for scrolling of screens based on X axis movement.

This, in my opinion, was the hardest part of the coding, since we also had to take care of drag-and-drop functionality. Basically, that was it. After around a week of coding and frustrating amounts of testing along with around 6 hours of PCB soldering, we had a working prototype of the wireless e-glove. We added a wrist-strap that held the Uno and the battery (9V) on the glove and taped things up a bit to keep things steady on the glove.

The members of the team are;

Kanishka Ganguly – Hardware and software

Nimesh Ghelani  - Coding

Vishesh Dokania – Coding and general visualization of concepts

Neel Lahiri – Coding and general visualization of concepts

Mainak Basu – PCB soldering and guidance

Fig.3.The PCB after fabrication

Our main aim with this project was a proof-of-concept which we hope to achieve by purchasing a robotic arm and connecting the receiver to it to control the arm remotely. The main agenda is to prove that such systems can be used in rovers to remotely defuse mines and rescue people from wreckage, if produced on a larger scale. Also, such systems may be extrapolated to be put to use in remote surgery and similar such applications.The next step on the agenda is to try and use gyroscopes and/or Inertial Measurement Units (IMUs) in order to achieve much more fluid gestures and movement in 3D space.

The code for this is currently in a private repo on Github, which we cannot release due to intellectual property rights at the moment.

This is a work in progress and we hope to put up videos and other progress soon , and may be some day James Bond maybe using it .... 'shaken, not stirred' !!!