Animators, illustrators, design engineers and others creating 3D images have been deprived of one of the wonders of the modern age: copy and paste. But a new system promises to bring the power and convenience of copy/paste to the world of 3D image editing.
Wan-Yen Lo, of UC San Diego and University of Bern, along with Jeroen van Baar, of Disney Research Zürich, and their colleagues, has developed a software system for stereoscopic copy and paste that will greatly simplify editing of 3D images. This development is just in time for what could be a proliferation of such images.
The boy and other elements were copied and pasted from a different image using new stereoscopic editing technology (source: ACM Siggraph and the researchers).
Composing a realistic-looking 3D image from different source materials currently requires meticulous, tedious work. Typical tasks include essentially painting shadows, textures and other artifacts by hand. Spatial and lighting differences between the copied graphic and the image it’s pasted into have to be resolved. To avoid looking distorted, an image needs to include all the visual cues our eyes and brain know from reality.
As the research paper explains, “3D copy & paste has to take stereopsis into account and avoid stereopsis rivalry: conﬂicting cues to the human visual system in the left and right eye images which could severely strain the visual system, or even destroy the 3D illusion altogether.”
The biggest challenge is preserving a sense of depth—and this is one of the major things the research team has accomplished. To preserve the volume of the element being pasted, the team developed what it calls a stereo billboard method for rendering the copied object. (Thisvideo provides more detail.)
Let’s say you’d taken some photos with one of the 3D cameras Fuji now sells. Each scene would consist of two shots: one red, one cyan (hence the red/blue glasses), from slightly different perspectives. The researchers’ software essentially computes the difference between the two versions to determine the depth between objects. The resulting depth map is used to guarantee that when the copied object is pasted into the composite image, it maintains the visual cues needed to make it appear realistic.
Which fruit was copied and pasted from different source images? Only the image editor knows (source: ACM Siggraph and the researchers).
The researchers have developed a complete editing system, including a precision tool for selecting an element of an image, similar to what you’d have with a program like Photoshop. They describe their software, which they introduced at the recent ACM Siggraph conference, as “intuitive” and responsive. Limitations to their current solution include vertical disparities and dealing with stereo baseline changes. They also point out that illumination differences between the copied image and the destination image are a “larger problem” that’s beyond the scope of this project.
The team hopes that someday these techniques can be applied to editing of stereoscopic 3D video.
Asked about any product plans, Wan-Yen Lo told Smarter Technology that this is “a research project, and commercialization is not planned yet.” But with an expected increase in 3D imagery—especially if the science fiction movies are right—there would be big demand for the capability she and her colleagues have developed. Once upon a time, the techniques embodied in Photoshop were research projects that no one planned to turn into a commercial product.