The Misadventures of Quinxy truths, lies, and everything in between!


Lytro Alternative: Automatic, Intelligent Focus Bracketing

Now in my early post-Lytro days I've been wondering how I could achieve the same effect with better results, not wanting to wait the years it might take for them to come up with a suitable next generation model.  Lytro's only real selling point at this moment is it's ability to take "living pictures" (their parlance), which really just means a photo which is interactive in as much as you can focus on different items in the picture by tapping those items.  The technology may be capable of quite a bit more, but that's all it currently delivers, and it delivers that with poor resolution, graininess, and restrictive requirements on lighting/action.

Living Pictures without Lytro

Why couldn't I achieve the exact same effect with far better results using my existing digital camera? I could, and did! Here's my "living picture" proof, using just an ordinary digital camera and a bit of human assistance.

No Lytro was required for this "living picture", just an ordinary digital camera (in this case a Sony NEX 5).  Click on different objects in this scene to change focus depth!

...and now for Lytro's version...

Lytro's "living picture"!  Click on different objects in this scene to change focus depth!

It doesn't take an expert in photo analysis to see that the non-Lytro picture looks much better: sharper, higher resolution, less grainy, and more realistic colors.

Faking Lytro Manually

The simple fact is, if you can capture a succession of photos each at a different focal setting and then view these photos with a JavaScript or Flash program that responds to clicks by selecting the photos whose clicked region was in focus, you can create the pleasing "living picture" effect without any fancy light-field camera required. And that is just what I did for the above demo. I took a series of pictures focusing on the different major elements in the scene and then made the regions in the photos clickable by way of a simple image map which specifies which image in my sequence corresponds to which area. While the approach I took for the demo was manual, it's not hard to see how easily any camera in combination with some very simple software could be made to do this, producing final results better than is possible with the Lytro camera.

Making Cameras Support Lytro-Like Effects

Many cameras these days have a feature called "exposure bracketing" which takes reacts to a shutter button press by taking a series of pictures at different exposure settings.  You then review the photos later and determine which photo looked the best.  Why then could you not have a "focus bracketing" feature which does the same thing but with focus?  The simplest approach would be to take multiple photos as the camera automatically walks the focus back from infinity to macro taking as many photos as necessary to achieve a desirable effect, perhaps as few as 5 or 10 would be needed to achieve a reasonable effect; with the aperture appropriately set any given picture's depth of field is wide enough to allow significant ranges to be sharply focused.  You would then need some mechanism for assigning clickable regions to the photo frames which happened to be in focus in that region.  This would likely be a fairly trivial software problem to solve.  All of this could be done with minimal camera intelligence, since it would just be varying focus distance in a fixed manner.  A far better but slightly more complicated solution would be to do automatic, intelligent focus bracketing using the camera's built-in autofocus system. Many cameras (particularly in phones) allow you to select a region of the scene which should be in focus. It would thus be easy for the camera to break down the scene into a search grid it would scan looking for objects upon which to focus, taking a single picture at each focus depth (one photo per range, according to the depth of field). The camera could record which grid location contained an object at a certain focal distance away, this being usable later to relate clicks on an image to a particular focused frame. The advantage of this approach is that it might be far quicker and more efficient, needing only as many frames as a scene objects' depths require. A scene which had two people in the foreground hugging and a church in the background would probably require just two photos to make a "living picture", people around a table at a birthday table with a cake in the center and a bounce house in the background may require 5 or 6 photos to make a Lytro-like image.

Working with Motion

These approaches share one significant weakness which is the fact that using multiple sequentially taken images negates the ability to capture any action-oriented scenes.  While modern digital cameras take rapid-fire photos, and those with exposure bracketing take three or so shots in a half second, that's certainly slow enough to make any significant movement within the scene noticeable when switching between frames.  Still, as action is easily blurred with the first generation Lytro, this hardly seems any sort of argument against this alternate approach.  An interesting solution to this problem and that of the inability to easily alter most existing camera's firmware, would be to use a replacement lens that split a single digital frame into multiple differently focused reproductions of the scene.  Just as I use a Loreo 3-D lens to merge the images captured by two lenses onto one digital frame, so to could one produce a system that would use four or perhaps nine lenses to capture one instant onto one digital frame through small lenses focused at slightly different depths.  Software could then easily split apart the single digital image into its component frames and do an easy focus analysis to determine what regions in each were in focus, with viewer software showing those as appropriate in response to touch.  The limitations of this approach would be related to the increased lighting requirements (or decreased action) as a result of the smaller lens, the expected poorer quality of each lens (related to cost and it being more a novelty manufacture than embraced by lens giants), and the reduced resolution (as your effective megapixel image would be the original value divided by the number of lenses within the lens).  Many stereo photographer setups coordinate two cameras to take their photos in concert, getting around all these issues, which you could also do to solve this problem, though I can imagine nothing more cumbersome.

The Lomo Oktomat as seen on Lomography has eight lenses which it uses to 2.5 seconds of motion across a single analog film frame it has divided into 8 regions. The same multi-lens approach but used simultaneously, with each lens focused slightly differently, could capture motion with Lytro-like aesthetics.

Focus Bracketing leads to Focus Stacking (Hyperfocus)

As I began to look into the practicality of these approaches I was pleased to discover that "focus bracketing" was being done manually, though with an intriguingly different goal.  Rather than produce a living picture where you can focus on different elements in a scene, a process called focus stacking is used to take and then (using software) merge photos taken at different focus settings to produce a single image in which everything in the scene is in focus.  The software involved analyses each photograph in the stack, each of the identical scene where only the focus is varied, and uses the regions of each photo which are in focus to produce the combined image in which everything is in focus.   This approach produces very impressive results.  The only limitations to this system is the requirement for a still scene, and the strong recommendation (if not requirement) that you use a tripod when taking your shots so as little varies as possible.  

Series of images demonstrating a 6 image focus bracket of a Tachinid fly. First two images illustrate typical DOF of a single image at f/10 while the third image is the composite of 6 images. From Focus Stacking entry on Wikipedia.

The aesthetic of a photo in which most things are in focus is quiet different from one in which only those things you select are in focus, but from a technical standpoint they are quite similar, since both situations require one possess the data pertaining to every element in a scene being in focus.  And a viewer could (and likely would) be given the option of viewing such a photo as he/she wished.  Do they want to see the photo traditionally (one, non-interactive focus point), as a "living picture" where they can choose the object in focus, or as a photo in which everything is in focus?

Focus Bracketing Available Today on your Canon PowerShot

A little further research led me to find a rather intriguing ability to add automatic focus bracketing to an entire range of camera models, via the Canon Hack Development Kit (CHDK).  CHDK allows you to safely, temporarily use a highly configurable and extensible alternative  firmware in your Canon PowerShot.  And users have used this to add focus bracketing for the purposes of focus stacking, and included detailed instructions on just how you can do it, too.

Coming Soon as an iPhone & Android Camera App

This integration of camera and software is a natural fit for an iPhone and Android app, where the app can control the capturing of the image and intelligent variation of focus and then do the simple post-processing to make the image click-focus-able. While I haven't seen such an app, I'm sure it'll come soon. I'd write it myself if I had the time.

Until the Future Comes

The point is that until Lytro demonstrates just what can be done with a light field camera, beyond merely creating a low-resolution "living picture", there's really no technical justification for placing the technology in people's hands when the same problem could be solved as effectively with traditional digital cameras.  If demand existed (and perhaps it will come) for this image experience, no light fields need apply.  Hopefully traditional digital camera companies will see the aesthetic value and include support in their firmware (for intelligent focus bracketing) and co-ordinated desktop software, app developers will launch good living picture capturing app cameras, and hopefully Lytro will demonstrate the additional merits of capturing and reproducing images from light fields.

^ Quinxy


Lytro Experience, Day Three: Not For Me

I pre-ordered the Lytro back in October, excited by the stunning live demos on their site.  A camera that captures a focus-less light field and allows you to do the focal interpretation later was just too amazing not to buy.  The potential and real advantages were immediately obvious: stunning "live" photos, potential for effective single lens 3-D capture, potential for faster picture taking with no need to wait for auto focus, no out of focus pictures, ability to take better pictures in worse light conditions, ability to capture subtler image detail across the objects photographed.

My Lytro was one of the first to be shipped and it arrived just three days ago.  Sadly Lytro's absolute requirement that you have a Mac with version 10.6.6 or higher to view your photos, it wasn't until today that I could actually try it out.  My initial excitement that had become frustration rebounded at the chance to see just what this camera could do!  Sadly upon viewing the photos I'd taken it beat a hasty retreat.

The Lytro is cool, but I cannot imagine myself actually using this thing in my daily life.  It can take amazing pictures, as proven by the stunning live demos on the Lytro site.  But I now appreciate just how many pictures must have been sifted through to pick out those hypnotically good ones.  If you've got the time and the artistic inclination I have no doubt you can and will do amazing things, but the vast majority of my shots look awful.

Here's what I discovered:

  • The camera's effective resolution is low!  Your pictures are 1024 x 1024.
  • The lens requires a lot of light*!  Unless conditions are right your images will be extremely grainy.
  • Everything must be still*!  Motion, both your own and other people's, must be minimized otherwise your photo will likely be blurry.
  • Mac and only Mac!  Unless you normally use a Mac daily (which I don't) you're just going to be annoyed by the absolute necessity of the Mac software.  You cannot view images or export images without using the Mac software.

* Obviously capturing a scene sharply involves a trade-off between light and motion; less light is fine if all is still, and more motion is fine if there is enough light.  I'm just saying that in "ordinary" life situations where people move, where light can be low, and where your hand isn't stabilized, this camera can be trouble.

My experience of my Lytro has, therefore, been pretty disappointing.  I imagined myself taking this camera with me everywhere, eager to capture "living pictures" to use their lingo, freezing moments in a manipulatable form.  But now I imagine carrying this thing around would only breed frustration as I could never rely on images I took coming out  right.  Some would stun but all too many moments would be unenjoyably grainy and blurry.  As it stands I'm better off with the clearer dimensional realities captured by my ever-present Evo 3-D and where appropriate my Sony NEX 5.

And so my Lytro is now up for sale on eBay.  If it were more amazing or much cheaper I'd keep it for those special moments where I could afford to experiment, but at $499 I don't want to be a guinea pig.

^ Quinxy


Lytro Experience, Day One: Unending Frustration! And a Mac is Required!

My bleeding edge Lytro light-field camera arrived today and thus far it's been an awful experience!  And I haven't even used the camera yet!  So what's gone so wrong?

  • The packaging is scary bad!  It's designed to be like a watch box or something where you lift off the bulk of the  box to reveal the beauty of the Lytro camera.  But unlike a watch box the Lytro isn't secured to its base, so if you happen to pull off the top while the box is at an angle the Lytro will come tumbling out, as mine did!  By some miracle I happened to be  sitting on a soft pile carpeted floor at the time, and my Lytro fell unharmed a mere 2 inches.  I have to believe others aren't finding themselves as lucky.  I pity those poor folks who've waited for months to get this thing and then see it damaged before they even got a chance to turn it on.
  • A huge shock was to discover upon connecting it to my PC that you you cannot even view or export any images without Mac! The official notice merely says that Windows support will be added in 2012.  I did not realize what this meant when I ordered the camera, nor can I conceive of how this is even possible.  Every camera I've ever owned can be used on any computer I've ever owned.  Sure, some cameras include custom OS-specific software for managing photos, uploading firmware, etc. but the software was always optional.  You could always just plug in a USB cable and view/copy the photos, whether you were running Windows, Mac OS X or Linux.  But not so with Lytro!  Unless you have a Mac running OS X 10.6.6 or higher your photos will never leave your camera!  There's not even an HDMI out.  Surely the entire point of having a camera is to take pictures you can show others.  If they have some means to share Lytro photos, which includes the image processing necessary, why on Earth do they not allow you to upload the raw files?  I really think they should have done a better explaining this limitation as it defies all logic and historical precedent.
  • I happened to have an old Apple Mini from my brief foray into iPhone app development a few years ago.  I dug it out, hooked it up, and discovered that the Lytro software requires 10.6.6 (Snow Leopard or Lion), and my Mini was 10.5 (Leopard)...  I figured I'd be able to easily upgrade online, but nope!  Snow Leopard (10.6) can only be ordered as a physical DVD.  And, to make things even more frustrating, Lytro software requires 2 GB memory, whereas my stock Mini has 512 MB.  Ah well, I figured I'd just run to Best Buy or the Apple Store buy what I needed and be able to use my Lytro by nightfall.  Nope!  No shops sell OS X 10.6, not even the Apple store!  So now I've got to wait at least two days for my Amazon orders to arrive.

Ah well...  My Lytro day one experience was a rotten one!

^ Quinxy