Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report

Marcos V. Conde; Manuel Kolmet; Tim Seizinger; Tom E. Bishop; Radu Timofte; Xiangyu Kong; Dafeng Zhang; Jinlong Wu; Fan Wang; Juewen Peng; Zhiyu Pan; Chengxin Liu; Xianrui Luo; Huiqiang Sun; Liao Shen; Zhiguo Cao; Ke Xian; Chaowei Liu; Zigeng Chen; Xingyi Yang; Songhua Liu; Yongcheng Jing; Michael Bi Mi; Xinchao Wang; Zhihao Yang; Wenyi Lian; Siyuan Lai; Haichuan Zhang; Trung Hoang; Amirsaeed Yazdani; Vishal Monga; Ziwei Luo; Fredrik K. Gustafsson; Zheng Zhao; Jens Sjölund; Thomas B. Schön; Yuxuan Zhao; Baoliang Chen; Yiqing Xu; Jixiang Niu

doi:10.1109/CVPRW59228.2023.00166

Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report

Marcos V. Conde, Manuel Kolmet, Tim Seizinger, Tom E. Bishop, Radu Timofte, Xiangyu Kong, Dafeng Zhang, Jinlong Wu, Fan Wang, Juewen Peng, Zhiyu Pan, Chengxin Liu, Xianrui Luo, Huiqiang Sun, Liao Shen, Zhiguo Cao, Ke Xian, Chaowei Liu, Zigeng Chen, Xingyi YangSonghua Liu, Yongcheng Jing, Michael Bi Mi, Xinchao Wang, Zhihao Yang, Wenyi Lian, Siyuan Lai, Haichuan Zhang, Trung Hoang, Amirsaeed Yazdani, Vishal Monga, Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön, Yuxuan Zhao, Baoliang Chen, Yiqing Xu, Jixiang Niu

Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

20 Scopus citations

Abstract

We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge. Recent advancements of mobile photography aim to reach the visual quality of full-frame cameras. Now, a goal in computational photography is to optimize the Bokeh effect itself, which is the aesthetic quality of the blur in out-of-focus areas of an image. Photographers create this aesthetic effect by benefiting from the lens optical properties.The aim of this work is to design a neural network capable of converting the the Bokeh effect of one lens to the effect of another lens without harming the sharp foreground regions in the image. For a given input image, knowing the target lens type, we render or transform the Bokeh effect accordingly to the lens properties. We build the BETD using two full-frame Sony cameras, and diverse lens setups.To the best of our knowledge, we are the first attempt to solve this novel task, and we provide the first BETD dataset and benchmark for it. The challenge had 99 registered participants. The submitted methods gauge the state-of-the-art in Bokeh effect rendering and transformation.

Original language	English (US)
Title of host publication	Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023
Publisher	IEEE Computer Society
Pages	1643-1659
Number of pages	17
ISBN (Electronic)	9798350302493
DOIs	https://doi.org/10.1109/CVPRW59228.2023.00166
State	Published - 2023
Event	2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023 - Vancouver, Canada Duration: Jun 18 2023 → Jun 22 2023

Publication series

Name	IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
Volume	2023-June
ISSN (Print)	2160-7508
ISSN (Electronic)	2160-7516

Conference

Conference	2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023
Country/Territory	Canada
City	Vancouver
Period	6/18/23 → 6/22/23

All Science Journal Classification (ASJC) codes

Computer Vision and Pattern Recognition
Electrical and Electronic Engineering

Access to Document

10.1109/CVPRW59228.2023.00166

Cite this

Conde, M. V., Kolmet, M., Seizinger, T., Bishop, T. E., Timofte, R., Kong, X., Zhang, D., Wu, J., Wang, F., Peng, J., Pan, Z., Liu, C., Luo, X., Sun, H., Shen, L., Cao, Z., Xian, K., Liu, C., Chen, Z., ... Niu, J. (2023). Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report. In Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023 (pp. 1643-1659). (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops; Vol. 2023-June). IEEE Computer Society. https://doi.org/10.1109/CVPRW59228.2023.00166

@inproceedings{34934c3dcd664c1daa25ad93e5d36a21,

title = "Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report",

abstract = "We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge. Recent advancements of mobile photography aim to reach the visual quality of full-frame cameras. Now, a goal in computational photography is to optimize the Bokeh effect itself, which is the aesthetic quality of the blur in out-of-focus areas of an image. Photographers create this aesthetic effect by benefiting from the lens optical properties.The aim of this work is to design a neural network capable of converting the the Bokeh effect of one lens to the effect of another lens without harming the sharp foreground regions in the image. For a given input image, knowing the target lens type, we render or transform the Bokeh effect accordingly to the lens properties. We build the BETD using two full-frame Sony cameras, and diverse lens setups.To the best of our knowledge, we are the first attempt to solve this novel task, and we provide the first BETD dataset and benchmark for it. The challenge had 99 registered participants. The submitted methods gauge the state-of-the-art in Bokeh effect rendering and transformation.",

author = "Conde, {Marcos V.} and Manuel Kolmet and Tim Seizinger and Bishop, {Tom E.} and Radu Timofte and Xiangyu Kong and Dafeng Zhang and Jinlong Wu and Fan Wang and Juewen Peng and Zhiyu Pan and Chengxin Liu and Xianrui Luo and Huiqiang Sun and Liao Shen and Zhiguo Cao and Ke Xian and Chaowei Liu and Zigeng Chen and Xingyi Yang and Songhua Liu and Yongcheng Jing and Mi, {Michael Bi} and Xinchao Wang and Zhihao Yang and Wenyi Lian and Siyuan Lai and Haichuan Zhang and Trung Hoang and Amirsaeed Yazdani and Vishal Monga and Ziwei Luo and Gustafsson, {Fredrik K.} and Zheng Zhao and Jens Sj{\"o}lund and Sch{\"o}n, {Thomas B.} and Yuxuan Zhao and Baoliang Chen and Yiqing Xu and Jixiang Niu",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023 ; Conference date: 18-06-2023 Through 22-06-2023",

year = "2023",

doi = "10.1109/CVPRW59228.2023.00166",

language = "English (US)",

series = "IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops",

publisher = "IEEE Computer Society",

pages = "1643--1659",

booktitle = "Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023",

address = "United States",

}

Conde, MV, Kolmet, M, Seizinger, T, Bishop, TE, Timofte, R, Kong, X, Zhang, D, Wu, J, Wang, F, Peng, J, Pan, Z, Liu, C, Luo, X, Sun, H, Shen, L, Cao, Z, Xian, K, Liu, C, Chen, Z, Yang, X, Liu, S, Jing, Y, Mi, MB, Wang, X, Yang, Z, Lian, W, Lai, S, Zhang, H, Hoang, T, Yazdani, A, Monga, V, Luo, Z, Gustafsson, FK, Zhao, Z, Sjölund, J, Schön, TB, Zhao, Y, Chen, B, Xu, Y & Niu, J 2023, Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report. in Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, vol. 2023-June, IEEE Computer Society, pp. 1643-1659, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023, Vancouver, Canada, 6/18/23. https://doi.org/10.1109/CVPRW59228.2023.00166

Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report. / Conde, Marcos V.; Kolmet, Manuel; Seizinger, Tim et al.
Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023. IEEE Computer Society, 2023. p. 1643-1659 (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops; Vol. 2023-June).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report

AU - Conde, Marcos V.

AU - Kolmet, Manuel

AU - Seizinger, Tim

AU - Bishop, Tom E.

AU - Timofte, Radu

AU - Kong, Xiangyu

AU - Zhang, Dafeng

AU - Wu, Jinlong

AU - Wang, Fan

AU - Peng, Juewen

AU - Pan, Zhiyu

AU - Liu, Chengxin

AU - Luo, Xianrui

AU - Sun, Huiqiang

AU - Shen, Liao

AU - Cao, Zhiguo

AU - Xian, Ke

AU - Liu, Chaowei

AU - Chen, Zigeng

AU - Yang, Xingyi

AU - Liu, Songhua

AU - Jing, Yongcheng

AU - Mi, Michael Bi

AU - Wang, Xinchao

AU - Yang, Zhihao

AU - Lian, Wenyi

AU - Lai, Siyuan

AU - Zhang, Haichuan

AU - Hoang, Trung

AU - Yazdani, Amirsaeed

AU - Monga, Vishal

AU - Luo, Ziwei

AU - Gustafsson, Fredrik K.

AU - Zhao, Zheng

AU - Sjölund, Jens

AU - Schön, Thomas B.

AU - Zhao, Yuxuan

AU - Chen, Baoliang

AU - Xu, Yiqing

AU - Niu, Jixiang

PY - 2023

Y1 - 2023

N2 - We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge. Recent advancements of mobile photography aim to reach the visual quality of full-frame cameras. Now, a goal in computational photography is to optimize the Bokeh effect itself, which is the aesthetic quality of the blur in out-of-focus areas of an image. Photographers create this aesthetic effect by benefiting from the lens optical properties.The aim of this work is to design a neural network capable of converting the the Bokeh effect of one lens to the effect of another lens without harming the sharp foreground regions in the image. For a given input image, knowing the target lens type, we render or transform the Bokeh effect accordingly to the lens properties. We build the BETD using two full-frame Sony cameras, and diverse lens setups.To the best of our knowledge, we are the first attempt to solve this novel task, and we provide the first BETD dataset and benchmark for it. The challenge had 99 registered participants. The submitted methods gauge the state-of-the-art in Bokeh effect rendering and transformation.

AB - We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge. Recent advancements of mobile photography aim to reach the visual quality of full-frame cameras. Now, a goal in computational photography is to optimize the Bokeh effect itself, which is the aesthetic quality of the blur in out-of-focus areas of an image. Photographers create this aesthetic effect by benefiting from the lens optical properties.The aim of this work is to design a neural network capable of converting the the Bokeh effect of one lens to the effect of another lens without harming the sharp foreground regions in the image. For a given input image, knowing the target lens type, we render or transform the Bokeh effect accordingly to the lens properties. We build the BETD using two full-frame Sony cameras, and diverse lens setups.To the best of our knowledge, we are the first attempt to solve this novel task, and we provide the first BETD dataset and benchmark for it. The challenge had 99 registered participants. The submitted methods gauge the state-of-the-art in Bokeh effect rendering and transformation.

UR - http://www.scopus.com/inward/record.url?scp=85159615398&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85159615398&partnerID=8YFLogxK

U2 - 10.1109/CVPRW59228.2023.00166

DO - 10.1109/CVPRW59228.2023.00166

M3 - Conference contribution

AN - SCOPUS:85159615398

T3 - IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

SP - 1643

EP - 1659

BT - Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023

PB - IEEE Computer Society

T2 - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023

Y2 - 18 June 2023 through 22 June 2023

ER -

Conde MV, Kolmet M, Seizinger T, Bishop TE, Timofte R, Kong X et al. Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report. In Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023. IEEE Computer Society. 2023. p. 1643-1659. (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops). doi: 10.1109/CVPRW59228.2023.00166

Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this