The Misadventures of Quinxy truths, lies, and everything in between!


Cell Phones as R/C Quadcopter FPV TX/RX and Homing Beacon

Flying multi-rotor (particularly quadcopter) radio-controlled vehicles is a lot of fun and you can do amazing things with them, in particular some beautiful aerial photography.  While most r/c pilots do this by looking up at their craft from whatever their distance happens to be at the moment, a growing number of r/c enthusiasts are using FPV (first person view) to remotely control their vehicles.  By using a tiny video camera with an attached transmitter the pilot can virtually fly their quadcopter as though they were a miniature pilot located inside it.  Aside from just being a lot of fun, this perspective makes it possible to fly over far longer distances than one could by merely looking at the craft from the ground.  The equipment to do FPV is not cheap, however, with decent entry-level setups of camera, transmitter, and receiver costing $1,000.  And so I couldn't help but wonder why no one talks about using the ubiquitous smart cell phone with its included cameras as an alternative solution.  The cell phone has a number of advantages over a typical FPV setup, namely that with the right apps running it can record its own video, it can operate over almost infinite distance with its use of cellular networks for data transmission, it can operate with very high bandwidth over 4G or wifi (though very limited distance with wifi), it can log its own flight path by recording its GPS positions, and if it crashes it can transmit its location to make recovery easy.  With all these advantages in a package that can cost just $100 it's hard to imagine it not used by everyone!

From what I understand in talking to a few people, the issue comes down to video quality, latency, and the possibility that the connection just drops out.  Few people want to risk their $1,500 and up r/c darling on Sprint's or AT&T's potentially spotty and variably efficient cell coverage.  And the current video streaming software available as apps for the key FPV feature are not intended for mission critical, near real time transmission.  Imagine trying to drive a car or pilot a plane with Skype.  You might do just fine for a while, as long as sender and receiver have good signals but if either gets into trouble the video suddenly becomes erratic, delayed, and pictured objects become indistinct and it would be impossible to make critical operating decisions based on that.

Given the enormous complexities being overcome daily by dedicated enthusiasts of r/c flying this seems like a challenge that can fairly easily be overcome.  One of the key pieces of software this community has developed and continually refined is the "flight controller", or the software which takes signals from the pilot's transmitter (the device with the joysticks that he uses to control the vehicle) and turns those into adjustments to motors and control surfaces.  Many flight controllers now even come with amazingly sophisticated auto pilot features, like the ability to hover motionless in one spot (despite winds, etc.), to fly home in the event that signal is lost or a fault develops, to automatically land if batteries are low, and even to navigate on its own, flying between coordinates previously supplied to it.  If all that can be achieved surely the FPV via cell phone problem can be overcome!

There are three problems that need to be addressed: loss of signal, degradation of signal, and overall quality of video.

Loss of video signal is clearly a very real problem and no amount of clever software can make up for a lack of inputs from the pilot, but such situations can be appropriately handled to minimize negative impact.  As I mentioned above, many flight controllers now include safety modes such as hover,  automatic return home, and automatic landing.  The pilot with such an onboard autopilot can at the flick of a switch tell his craft to do the appropriate thing, presumably either hovering to wait and see if signal is restored or begin to return home at least enough to regain the signal.  Also, the FPV app in the cell phone can be optimized to recover quickly from any network failure (once the underlying data connection is restored).  And if this was to all develop and become more sophisticated clearly an integration of the FPV app and the onboard flight controller would be ideal, allowing the FPV app to command auto pilot features directly in the event of network loss as well as be the means of transmitting flight controller telemetry to the ground during routine flight (and perhaps controlling other features of the flight controller as well).

Degradation of signal and overall quality of video are related problems.  The key here is, I believe, to develop a video codec or perhaps just an application of existing codecs that focuses on the critical visual data FPV pilots need.  While users of video streaming applications like Skype want overall picture quality to be good, an FPV pilot is primarily focused on visual information related to the orientation of their craft relative to ground, potential physical obstacles in their path, and anything necessary to continue whatever flight motion they were executing.  If the signal degrades and there is less bandwidth over which to send video it's most important that the critical parts of the pilot's picture continue to appear in near real time!  How exactly one extracts or prioritizes those features I am not sure at this point, but I am fairly certain it's an achievable goal.  One need only think of situations they have been in when their own vision has degraded due to environmental factors like darkness, fog, smoke, etc. to realize that our brains can seize upon very small and sometimes indistinct cues to maintain orientation.  An algorithm could be developed to prioritize the sending of lesser quality video data related to horizon and to nearby obstacles (elements of the frame which the algorithm have noted move more relative to the overall background).  And with the ongoing development in the areas of computers interpreting images to extract features like faces, eye positions, smiles, body positions, etc. it should present little challenge to have the FPV app be able to maintain an awareness of very crude items like the horizon and those objects most likely to represent near obstacles.  This specific data could even be transmitted using ultra-low bandwidth as mere vector data rather than actual color images, in other words allowing the second receiving cell phone app to reconstruct the approximate figures overlapping whatever video may or may not be coming in.

Hopefully these things will see development in the near future because it can hardly be argued that the potential here is huge.

^ Quinxy

Comments (5) Trackbacks (0)
  1. The Storm Drone has a optional iphone holder,.

  2. I have been trying this for a while now using 2 mobile phones, 1 attached to my Blade quadcopter and 1 attached to my controller/transmitter. I have no problem at all with lag or poor signal (maybe my area has good 4G coverage). I have been using skype on both phones. The only problem I have is that I cannot record the flight video. Both my phones are android and I would like to record the flight. There does not seem to be a program for android that can record skype video. The optimal situation would be to have the phone that is attached to the Blade be recording video while skype also accesses the camera for the FPV side. So the best solution would be a camera (video camera) app that can use the phones camera in the background while still granting skype access to it, or an android app that can record a video skype call. Seems easy enough, I don’t understand why no one has made such an app.

  3. I’ve been thinking about this too. Why not use the ustream app, I think it will try to reconnect after failure plus you can stream to the world. I was looking into dtmf encoding as a way to send commands over cell. Possibly also over a clandestine numbers stations style burst broadcast over short wave too 😀 solar cell that bad boy up waterproof it, fly it low, hide it and charge, move on etc etc explore the country by proxy 😀

  4. If you’ve never soldered or tinkered with programming, don’t even think about it.
    If you aren’t familiar with them, I suggest you look into these very
    durable and multifaceted camera systems. The FAA is currently accepting public comment on proposed regulations, with final law expected sometime in 2015.

  5. I have a question. I’m always tinkering with soldering this and that and well I know that quadcopters are now using a combination of buttons for commanding the quadcopter to do a 360 flip in 4 directions. I’m trying to figure out which wires do I solder to the board to figure out the language so I can either add commands to make it do more tricks or add extra special features. or even make it compatible with whatever I want basically.

Leave a comment

No trackbacks yet.