This article is the University of Kentucky: doctoral thesis (author Vijay Venkatesh Mahalingam), a total of 128.
Digital inpainting is a technique that uses information from surrounding areas to fill in missing areas in an image or video. This technology has a wide range of applications in fields such as error recovery, multimedia editing, and video privacy protection.
This paper mainly discusses three major challenges related to existing and emerging patching algorithms and applications. The three key areas of influence are 1) the structure of image restoration algorithms, 2) a fast and effective object-based video restoration framework, and 3) the perceptual evaluation of large-area image restoration algorithms.
One of the main methods of existing image repair algorithms to repair missing information is to follow a two-stage process. Through the structure completion step, outline the boundary of the hole area, and then use the advanced texture synthesis method in the texture completion process. Although the texture synthesis stage is very important, it can be said that the structure completion step is an important part of improving the quality of perceptual image restoration. To this end, we propose a global structure completion algorithm with symmetry as the main feature. Although the existing symmetry completion method requires prior information, our method uses a non-parametric method and uses the invariance of curvature to complete the missing boundary.
Turning our attention from images to video restoration, we can easily observe that the existing video restoration technology has developed into an extension of the image restoration technology. As a result, they suffer from a variety of shortcomings, including the inability to handle large areas of lost space-time, significantly slow execution time making them unsuitable for interactive use and the presence of space-time artifacts. In order to solve these main problems, we propose a method based on the object framework to improve the performance of the video repair algorithm. This paper proposes a modular restoration scheme. First, the obtained background model is used to segment the video into multiple objects, and then the static background area and dynamic foreground area are restored. For the restoration of the static background area, we use simple background replacement and occasional image restoration. In order to repair the dynamically moving foreground area, we introduce a new dissimilarity measurement method based on sliding window under the dynamic programming framework. This technology can effectively repair large-area occluded areas, repair completely missing objects in multiple frames, change the size and posture of objects, and have minimal blur and motion artifacts.
Finally, we will focus on the experimental research on the evaluation of the perceptual quality of large-area image restoration algorithms. The perception of large-area restoration technology is essentially a subjective process, but previous studies have not yet started from the subjective nature of the human visual system. We used 24 subjects’ eye-tracking devices to conduct subjective experiments to analyze the effects of repair on human gaze. Experiments show that the existence of repair artifacts directly affects the gaze of unbiased observers, which in fact directly affects the observer's subjective evaluation. Specifically, when the repaired artifact is obvious, the gaze energy of the repaired image cavity area shows a clear deviation from the normal behavior.
Digital inpainting is the technique of filling in the missing regions of an image or a video using information from surrounding area. This technique has found widespread use in applications such as restoration, error recovery, multimedia editing, and video privacy protection. This dissertation addresses three significant challenges associated with the existing and emerging inpainting algorithms and applications. The three key areas of impact are 1) Structure completion for image inpainting algorithms, 2) Fast and efficient object based video inpainting framework and 3) Perceptual evaluation of large area image inpainting algorithms. One of the main approach of existing image inpainting algorithms in completing the missing information is to follow a two stage process. A structure completion step, to complete the boundaries of regions in the hole area, followed by texture completion process using advanced texture synthesis methods. While the texture synthesis stage is important, it can be argued that structure completion aspect is a vital component in improving the perceptual image inpainting quality. To this end, we introduce a global structure completion algorithm for completion of missing boundaries using symmetry as the key feature. While existing methods for symmetry completion require a-priori information, our method takes a non-parametric approach by utilizing the invariant nature of curvature to complete missing boundaries. Turning our attention from image to video inpainting, we readily observe that existing video inpainting techniques have evolved as an extension of image inpainting techniques. As a result, they suffer from various shortcoming including, among others, inability to handle large missing spatio-temporal regions, significantly slow execution time making it impractical for interactive use and presence of temporaland spatial artifacts. To address these major challenges, we propose a fundamentally different method based on object based framework for improving the performance of video inpainting algorithms. We introduce a modular inpainting scheme in which we first segment the video into constituent objects by using acquired background models followed by inpainting of static background regions and dynamic foreground regions. For static background region inpainting, we use a simple background replacement and occasional image inpainting. To inpaint dynamic moving foreground regions, we introduce a novel sliding-window based dissimilarity measure in a dynamic programming framework. This technique can effectively inpaint large regions of occlusions, inpaint objects that are completely missing for several frames, change in size and pose and has minimal blurring and motion artifacts. Finally we direct our focus on experimental studies related to perceptual quality evaluation of large area image inpainting algorithms. The perceptual quality of large area inpainting technique is inherently a subjective process and yet no previous research has been carried out by taking the subjective nature of the Human Visual System (HVS). We perform subjective experiments using eye-tracking device involving 24 subjects to analyze the effect of inpainting on human gaze. We experimentally show that the presence of inpainting artifacts directly impacts the gaze of an unbiased observer and this in effect has a direct bearing on the subjective rating of the observer. Specifically, we show that the gaze energy in the hole regions of an inpainted image show marked deviations from normal behavior when the inpainting artifacts are readily apparent.
-
引言 - Literature Review
- Symmetrical completion and global image inpainting
- Efficient object-based video repair method
- Perceptual evaluation of image restoration
- Conclusion and Outlook
For more exciting articles, please pay attention to the public account: