Generalisation of sketch to photo-realistic inversion techniques to other drawing techniques and styles using Convolutional Neural Networks

Abstract

Photo-to-sketch inversion techniques can be useful for artistic and forensic purposes. However, there are no existing algorithms that are applicable on realworld problems yet. The Convolutional Sketch Inversion model is one model that showed that state-of-the-art inversions can be achieved on one particular sketch technique. In this model, an adapted model is proposed that overcomes the limitation that it is not able to generalize to an arbitrary drawing or painting technique. The key idea of the adaptations are face feature extractions to train the network on high level featural information and an adversarial training paradigm between two networks that counteract with each other to make the model in general more robust. Although the output was not of the quality as expected, the model shows that it is able to invert images of di erent styles. This can be seen as an improvement on the former CSI-model.