Image Inpainting — Object Removal and Loss Recovery from Images using Artificial Intelligence and Deep Learning.
How many times have you clicked a picture and have had something unwanted appear in the frame? How you wish you could’ve timed the shot a little better or have the ability to remove it seamlessly? Well, here we have proposed a solution that solves this problem.
The proposed solution also addresses the problem of loss recovery of images after transmission, which is a pressing problem in this digital world.
With my team members and I as interns at LeadingIndia AI under the mentorship and guidance of Dr. Suneet Gupta, we propose a solution to the above mentioned problem.
LeadingIndia AI is an initiative that is aimed at promoting Artificial Intelligence and Machine Learning skills amongst the engineering students of India to reduce the skill gap of the students and make these skills mainstream, as the future is Artificial Intelligence. I am proud to have been an intern in this initiative.
Overview About Image Inpainting And Our Proposed Solution
Image inpainting is the task of filling “patches” in an image, which can be used in various aspects of image processing. One can remove unwanted parts of the image while keeping the integrity of the image intact. Image inpainting presents a lot of implications in real life, such as recovery of a lossy image after transmission, different varieties of image/painting restorations.
However, most of these traditional methods rely heavily on rectangle shaped regions near the center of the image and expensive post-processing. Traditional methods use Conv2D layers built in Keras for this task, which often leads to colour discrepancy and blurriness. The task at hand is to remove the need of expensive post-processing and blending operations, while dealing with irregular holes/masks robustly and produce a meaningful and semantically correct image. For this purpose, we thought of removing the convoluted layers by replacing them with partially convoluted layers and mask-updates.
The solution that we proposed includes an autoencoder decoder approach that has a series of Conv2D layers along with Transpose layers to achieve the desired result, but we introduced partially convoluted layers instead of the traditional Conv2D layer along with updating of the mask in each layer, thus creating a PConv2D layer. A partial convolution layer comprises of a masked convolution operation which has been re-normalized and then is followed by a mask-update setup. Doing so makes the model not consider all the pixels that are part of the patch/mask. The U-Net structure was used to design the architecture of the model. The activation function used for triggering the layers is ‘relu’ and for the last layer we have used the ‘sigmoid’ function.
The prediction model gave better results than traditional Autoencoder-decoder models. The use of partially convoluted layers improved the colour-correction, kept the edges intact, and saved us the expensive blending methods to create a semantically correct image composition. The result so obtained is shown below.
My Personal Experience
My experience at this internship was really fulfilling and I thank the team at LeadingIndia AI for providing me this wonderful opportunity.
I was tasked with developing the front-end and integrating the modules during this internship. I learnt a lot from this and I realized the importance of UX/UI design and I got an opportunity to grasp an in-depth knowledge of front-end design, development and integration of the front-end with the back-end.
I also learnt about various modules and technologies while working in the project which helped me a lot. On the whole, the learning outcome that I found to be of utmost importance was the ability to search for information and use it effectively, about communicating effectively and teamwork skills.
You can view the Webapp here: https://github.com/69690/Flask-Webapp