InstructIR ✏️🖼️ 🤗
High-Quality Image Restoration Following Human Instructions
Marcos V. Conde, Gregor Geigle, Radu Timofte
Computer Vision Lab, University of Wuerzburg | Sony PlayStation, FTG
TL;DR: quickstart
InstructIR takes as input an image and a human-written instruction for how to improve that image. The (single) neural model performs all-in-one image restoration. InstructIR achieves state-of-the-art results on several restoration tasks including image denoising, deraining, deblurring, dehazing, and (low-light) image enhancement. 🚀 You can start with the demo tutorial. Check our github for more information
Abstract (click me to read)Image restoration is a fundamental problem that involves recovering a high-quality clean image from its degraded observation. All-In-One image restoration models can effectively restore images from various types and levels of degradation using degradation-specific information as prompts to guide the restoration model. In this work, we present the first approach that uses human-written instructions to guide the image restoration model. Given natural language prompts, our model can recover high-quality images from their degraded counterparts, considering multiple degradation types. Our method, InstructIR, achieves state-of-the-art results on several restoration tasks including image denoising, deraining, deblurring, dehazing, and (low-light) image enhancement. InstructIR improves +1dB over previous all-in-one restoration methods. Moreover, our dataset and results represent a novel benchmark for new research on text-guided image restoration and enhancement.
Disclaimer: please remember this is not a product, thus, you will notice some limitations. This demo expects an image with some degradations (blur, noise, rain, low-light, haze) and a prompt requesting what should be done. Due to the GPU memory limitations, the app might crash if you feed a high-resolution image (2K, 4K).
The model was trained using mostly synthetic data, thus it might not work great on real-world complex images. However, it works surprisingly well on real-world foggy and low-light images. You can also try general image enhancement prompts (e.g., "retouch this image", "enhance the colors") and see how it improves the colors.