SA-RFR: self-attention based recurrent feature reasoning for image inpainting with large missing area

Wang, J., Wang, L., He, S., Alfarraj, O., Tolba, A. and Sherratt, R. S. ORCID: https://orcid.org/0000-0001-7899-4445 (2022) SA-RFR: self-attention based recurrent feature reasoning for image inpainting with large missing area. Human-centric Computing and Information Sciences, 12. 31. ISSN 2192-1962 doi: 10.22967/HCIS.2022.12.031

Abstract/Summary

With the recent emergence of artificial intelligence, deep learning image inpainting methods have achieved fruitful results. These methods generated plausible structures and textures in repairing images with small missing areas. When inpainting an image with an excessively large missing area (the mask ratio is more than 50%), however, it usually produces a distorted structure or a fuzzy texture that is inconsistent with the surrounding area. Therefore, we propose a self-attention based recurrent feature reasoning (SA-RFR) network. First, SA-RFR uses self-attention (SA) to enhance the correlation between known pixels and unknown pixels and the constraints on the hole center, so that the repaired content details are clearer and the edges are smoother. In addition, because ordinary convolution has feature redundancy for the generated feature map, some unnecessary information is generated, and some models are difficult to train. Therefore, we also propose an adaptive ghost convolution (AGC) to replace part of the ordinary convolution. Using the PReLu activation function instead of the ReLu activation function in the ghost module, AGC can effectively improve the overfitting problem of the model and the quality of the repaired image without increasing the computational cost. The proposed model has undergone extensive experiments on several public datasets, and the results show that our method is superior to the state-of-the-art methods.

Item Type	Article
URI	https://reading-clone.eprints-hosting.org/id/eprint/107542
Identification Number/DOI	10.22967/HCIS.2022.12.031
Refereed	Yes
Divisions	Life Sciences > School of Biological Sciences > Biomedical Sciences Life Sciences > School of Biological Sciences > Department of Bio-Engineering
Publisher	Springer Berlin Heidelberg
Download/View statistics	View download statistics for this item