中文版 | English
CSpace
Dual-stream spatio-temporal decoupling network for video deblurring
Ning, Taigong1; Li, Weihong1; Li, Zhenghao2; Zhang, Yanfang1
2022-02-01
摘要It is very important to obtain spatio-temporal information in video deblurring based on deep learning. The existing methods usually jointly learn the spatio-temporal information of blurred videos through single-stream networks, which inevitably limit spatio-temporal information learning and video deblurring performance of networks. Therefore, we propose a dual-stream spatio-temporal decoupling network (STDN), which can learn the spatio-temporal information of blurred videos more flexibly and efficiently with the decoupled temporal stream and spatial stream, for solving this problem. Firstly, in the temporal stream of STDN, we propose a video deblurring pipeline, that is motion compensation plus 3D CNNs, for solving the drawback of 3D CNNs that its receptive field cannot effectively cover the same but misplaced contents of different frames. Thus, the temporal stream can aggregate temporal information of frame sequences and handle inter-frame misalignments more effectively. Specifically, we design a novel deformable convolution compensation module (DCCM) to achieve motion compensation of this pipeline more accurately. Then, we develop a 3DConv module optimized by the designed temporal, spatial, and channel decoupling attention block, named the CTS, to achieve 3D CNNs of this pipeline. Secondly, we design a spatial stream in which two types of wide-activation residual modules are stacked, for learning more spatial features of the central frame to supplement the temporal stream. Finally, extensive experiments on the baseline datasets demonstrate that the proposed STDN has better performance than the latest methods. Remarkably, using the proposed temporal stream alone already can achieve competitive video deblurring performance than the existing methods. (C) 2021 Elsevier B.V. All rights reserved.
关键词Video deblurring Decoupling learning Dual-stream network Motion compensation 3D CNNs
DOI10.1016/j.asoc.2021.108342
发表期刊APPLIED SOFT COMPUTING
ISSN1568-4946
卷号116页码:16
通讯作者Li, Weihong(weihongli@cqu.edu.cn)
收录类别SCI
WOS记录号WOS:000768204300007
语种英语