Nvidia has presented its own research into a way to perform image inpainting which is the elimination of holes or damaged parts in an image. It uses deep learning for this with its own technique that would provide better results.
According to Nvidia the method can not be used alone to fill up missing parts of an image, but also to remove elements from the image. The company has published a video in which it shows how its method works. This could be applied in image processing software, says Nvidia. The company claims that its model can handle holes of any shape, size and distance to the edge of the image. The details of the research are described in a paper .
In it, the researchers write that their research differs from earlier work, which would mainly focus on rectangular image parts in the center of the image. However, the new method would also work with irregular shapes. Moreover, the methods proposed so far often lead to visual artifacts in the final image. Nvidia wants to counteract this by not making output for missing pixels dependent on the input for those pixels, according to the company.
The researchers used a dataset of about 55,000 masks for training purposes. It concerns missing parts in images. The images used came from other datasets, namely ImageNet, Places2 and CelebA-HQ. The training took place on a Nvidia V100 accelerator with 16GB of memory.