Real-time Edge-Aware Image Processing with the Bilateral Grid

Jiawen Chen, Sylvain Paris, Frédo Durand

In ACM Transactions on Graphics (Proceedings of the ACM SIGGRAPH 2007 conference)

teaser

Abstract

We present a new data structure---the bilateral grid, that enables fast edge-aware image processing. By working in the bilateral grid, algorithms such as bilateral filtering, edge-aware painting, and local histogram equalization become simple manipulations that are both local and independent. We parallelize our algorithms on modern GPUs to achieve real-time frame rates on high-definition video. We demonstrate our method on a variety of applications such as image editing, transfer of photographic look, and contrast enhancement of medical images.

Update (12/21/2007)

I'm sorry, but due to patent issues, we've had to remove version 0.2 of our code, which included Real-time Video Abstraction.

Update (10/22/2007)

Code version 0.1 and data are now available! We apologize for the long delay.

Preprint of paper (PDF)
Video (MPEG-4 with H.264, plays in Quicktime 7, 60 MB)
Video (WMV, plays in Windows Media Player, 100 MB)

Slides from SIGGRAPH (Powerpoint 2003 + WMV videos inside ZIP, 86 MB)
Individual Files:
Slides (PPT)
Slides (PDF, no animations)
Interactive Local Tone Mapping (WMV)
Bilateral Grid Painting 3D Visualization (WMV)
Bilateral Grid Painting Screen Capture (WMV)
Bilateral Filtering Screen Capture (WMV)
Multiscale Video Abstraction Screen Capture (WMV)
Photographic Style Transfer (WMV)
Photographic Style Manipulation Live Demo Screen Capture (WMV)

Clarifications

For the results and applications, the numbers we report are for the NVIDIA GeForce 8800 GTX. Only the Bilateral Filter is benchmarked using the other GPUs.
In the places where we report memory usage, it is how much overhead the data structure takes. It does not include the amount used by the input and output images.

Acknowledgements

We thank the MIT Computer Graphics Group and the anonymous reviewers for their comments. We are especially grateful to Jonathan Ragan-Kelley for fruitful discussions on GPU programming and Tom Buehler for his assistance in making the video. This work was supported by a National Science Foundation CAREER award 0447561 "Transient Signal Processing for Realistic Imagery," an NSF Grant No. 0429739 "Parametric Analysis and Transfer of Pictorial Style," and a grant from Royal Dutch/Shell Group. Jiawen Chen is partially supported by an NSF Graduate Research Fellowship and an NVIDIA Fellowship. Frédo Durand acknowledges a Microsoft Research New Faculty Fellowship and a Sloan Fellowship.

Last updated: 10/22/2007