The watch-drone project

Controlling a drone from your wrist

Bond checking the time

One of my favourite parts from James Bond movies has always been Q introducing the latest gadgets from MI6's research laboratory. This project began over 6 months ago. I had a smartwatch that ran Android (5.1 for the curious), but I could not think of a use for it other than its intended purpose -- one that I found was largely irrelevant to my lifestyle after I wore it for a couple of weeks. I finally decided I wanted to try to control one of my drones from the watch, similar to what you might see in a Bond or Mission Impossible film. I ended up getting caught up in trying to install Linux on my laptop and so I never properly began the project. I already had everything I needed -- the drone, laptop and smartwatch. It was only until recently that I decided to finish what I had started.

The smartwatch, a Finow X3 plus

The AR Drone 2.0 was released in 2012, and represented the low-mid end of consumer drones at the time. It was designed to be used without any previous knowledge of piloting multirotors.

My AR Drone 2.0, with an XT60 adaptor

Before I get into the AR drones' technical specifications, it should be noted I have been working with drones and model aircraft since around the time the AR Drone 2.0 came to market. I started out with a Super Cub from Horizon Hobby, as well as a couple toy drones, such as my trusty Syma X1. I then built David Windestal’s tricopter 2.6HV, before taking it to the US a week later and flying it in various places around California and Death Valley.

Since then, I’ve added some racing drones to the fleet, which I’ll talk more about in a future post. Needless to say, I’ve been in the hobby for a while, and I haven’t been much of a fan of pre-built quadcopters in the past. Now, the drones coming from companies such as DJI are exponentially better than what could be scratch-built for a similar price, and I feel that due to the growth of the hobby, new pilots are more mindful about where they fly (unlike in the past). Let's get back to the Parrot.

My latest build, a ZMR 210r

The AR Drone 2.0 runs Linux and interfaces with your mobile device via WiFi. Since it’s essentially a flying access point, you can connect to it directly from any WiFi-enabled device, which translates to every smartphone, tablet and laptop from the last decade. This convenience comes at a cost though -- atrocious range. Whereas DJI’s OccuSync technology in their latest quadcopters allows for up to full HD video sync and a <7km range, the Parrot relies on the sensitivity and power of the WiFi radio inside your device, which results in abysmal performance, as they aren’t designed to work over longer ranges. Many people have modded their AR Drones to use higher-gain antennas, increasing range, but that ends up becoming a whole other project. Besides, the stock range scales well with the restricted flight capabilities the watch screen’s size imposes. Compared to my other drones, its flight performance is pretty pathetic, but because of its lack of power and heavy-handed stabilisation it’s relatively easy to fly without any experience. The FreeFlight app made by Parrot is pretty good, and there are also a number of 3rd party apps that add extra features such as Bluetooth gamepad support.

I decided the best way to approach the problem would be to connect the watch to a PC, and use a third-party application to actually do the communicating with the drone, as I didn't want to rely on the watch in case I needed to quickly land. There is (or was) a community of developers that began programming for the AR Drone after Parrot released their API. One website that compiles these scripts is nodecopter.com. I was originally planning on using  a couple of their node.js modules, but after many, many hours of messing around with errors when installing them on my laptop’s Ubuntu installation, I decided that it was not worth the hassle to try to use code that I couldn’t be sure would talk to each other in the first place. Luckily, there is an alternative -- AutoFlight by Electronics Kitchen. It’s a desktop client that allows you to control the AR Drone from Windows or Linux machines.

The AutoFlight main screen

This only allows communication between the drone and the laptop, however. We still need to bring the watch into the equation. There are a few services I tried out in order to get input from the watch onto the laptop, but one stood out in particular. DroidPad has been around for a while now, but it has a very useful feature that is relevant to the project: the ability to make custom joystick configurations. Since the watch’s display has a very unusual physical resolution (320x320 cropped into a circle), other apps would not scale properly as they had fixed locations for each joystick on the screen, and as a result were rendered unusable. I had originally planned on using this functionality of DroidPad's to turn the watch into a joystick which would directly communicate with AutoFlight. However, I also had major difficulties installing DroidPad on Ubuntu, to the point where I downgraded to Ubuntu 14.04.1 from the latest version, 17.04, as it was the last build to be officially confirmed to work with DroidPad. Still, I could not get it to completely install. Instead, I decided to try DroidPad on my Windows desktop, where it worked (almost) flawlessly, leading me to make a Windows 10 virtual machine on my laptop and call it quits.

In order to communicate between the phone and PC, DroidPad requires both machines to be on the same network. I was hoping to use a USB WiFi adapter that I had bought for network security training to create an access point that both the VM and my watch would connect to. Again, though, I was not able to get it working in the intended way, and so I settled on creating a hotspot through my phone. Yes, it adds another device to the equation, but at the same time, it simplifies things and makes the process of setting up more straightforward. This frees up the adaptor, which has the benefit of increasing the range of the system when used to connect to the drone as it is more powerful than the laptop's internal WiFi card.

The TL-WN722N

Problems solved, right? Not so fast. After getting DroidPad and Autoflight set up on my VM, (along with disabling all Windows telemetry and uninstalling Candy Crush after it downloaded itself via my mobile hotspot), I opened up the vJoy configurator and -- nothing. Turns out the error message that pops up when installing Droidpad does actually affect the abilities of the program. Even after manually installing the included driver, it wouldn't work properly.

At this point, I was ready to call off the project. I had spent enough time messing around with compiling programs from their GitHub source and Googling error messages. But then I realised -- why not use the mouse functionality of DroidPad? It was the only part of it that actually worked on my setup. That’s when I found MouseToJoystick.

The program, according to its GitHub page, was written so the author could play the 1998 Tie Fighter game without a joystick, instead using his mouse. For me, what the program meant is that I could use mouse inputs from DroidPad to control my drone. The program uses the cursor’s position on the screen and checks it for movement, then converts that movement along either the X or Y axis into a vJoy output. Autoflight directly supports vJoy input devices, and so I finally had a way of getting proportional input from the watch to the drone.

Here is a final diagram of the setup:

It’s time to fly.

After turning on the watch and connecting it my phone’s hotspot, booting up the Windows VM and opening DroidPad, MouseToJoystick and Autoflight, I was finally ready to go. However, I still faced an issue -- I could connect the Macbook’s internal WiFi card to the AR Drone’s network, but not the higher-power USB adaptor I had planned on using. After trying to connect to other networks, it was clear the problem was with the adaptor. I finally found the root of the issue on askubuntu.com: the adaptor’s MAC address was being randomised automatically, meaning it would constantly connect to the network, change its MAC address and attempt to reconnect to apply its changes. A few lines added to the NetworkManager.config file and a reboot later, and all systems were go.

Everything set up in the Windows VM

Unfortunately it was a blustery Autumn afternoon when I maidened the WatchDrone, so further flight tests will need to be carried out in order to sort out the controls (at least one of the two vJoy axis need inverting). All the systems did work together as I had hoped, and as such I can officially call the project a success. Look out for a future post where I’ll take the WatchDrone to a real flying field and tune the AutoFlight setup so it is actually controllable.

To keep up-to-date on this and future projects, join the mailing list by pressing the “Subscribe” button at the top of the page.