Making Google Street View into 3D

I was very excited about Google Street View in Australia. I think in a computer vision point of view, it provides such a huge amount of usable multiple views data to play with.

One idea I have is to:
1. Run the SIFT descriptor (by David Lowe) over the photos along one street to grab the interest points,
2. Ran RANSAC with rather loose tolerance to find homography match, and classify regions into planar surfaces. Maybe some super pixel algorithm will be useful to grow small region into bigger one based on colour information.
3. Transform the Street View into a 3D cardboard style walk through like the Photo Popup I mentioned before.

I think it would work. And because we know it's on a street, there are many constraints that can be added like we know the ground under is flat and it's pretty much sky on top kind of things.

If you're reading this and implement the algorithm and got a paper published or something, just give me an honourary mention will do. ;)

On a non-technical note, here is one place that it didn't quite work out on Street View. There's clearly a road there, but it didn't show it.



Or on Street View live:

View Larger Map