The most important research articles are listed here. It's a hand-curated list of the most recent AI and data science breakthroughs, organized chronologically with a link to a more in-depth article.

GAN-based image composition

The complex relationships between lighting, geometry, and partial occlusion, which cause different parts of an image to interact, make it very hard to blend features from multiple images seamlessly. For example, even though recent work on GANs has made it possible to create realistic hair or faces, it is still hard to combine them into a single, believable image instead of a bunch of disjointed image patches. Based on GANinversion, the researchers show a new way to blend images, especially for the problem of transferring hairstyles. In addition, the researchers develop a new latent space for combining images better at keeping details and encoding spatial information. They also create a new GAN-embedding algorithm that can slightly change images to fit a typical segmentation mask.

Their new way of representing things lets them transfer the visual properties of multiple reference images, including specific details like moles and wrinkles. Because we blend images in a latent space, we can make images that make sense. Their method avoids the problems that other forms have with mixing and finds an image that is the same everywhere. In a user study, their results show a significant improvement over the current state of the art. More than 95 per cent of the time, users preferred their blending solution.

Paper: Barbershop: GAN-based Image Compositing using Segmentation Masks

Click here for the code

Text aesthetics transfer

This new AI model from Facebook can translate or change the text in an image in your language and style.

The researchers come up with a new way to separate the content of a text image from how it looks overall. The appearance representation that we come up with can then be used on new content to copy the style of the source content all at once. The researchers learn how to separate things on their own. Their method processes whole word boxes without having to separate text from the background, process each character separately, or make assumptions about the lengths of strings. The researchers show results in different text types, such as scene text and handwritten text, which used to be handled by specialized methods.

The researchers show a way to separate real text photos by getting an opaque representation of the text and letting this appear in new content strings. Their method is said to have several new features. The most important thing is that our TSB only needs a single source style example: Style transfer can only happen once. Their method for untangling knots is trained more under self-supervision, which lets them use real photos without style labels for training. Lastly, the researchers show results that were made artificially for both scene text and handwritten text. 

Paper: TextStyleBrush: Transfer of text aesthetics from a single example

Click here for the code

Picture animation

In this paper, the researchers show how we can turn a still image into a looping video that looks like it is animated. The researchers look for scenes with constant fluid motion, like water moving or smoke billowing. The researchers use an image-to-image translation network to encode the motion priors of natural scenes from online videos. This method lets them make a motion field that matches a new photo. The motion animates the image using a deep warping technique: pixels are as in-depth features, those features using Eulerian motion, and the resulting warped feature maps as images.

Furthermore, the researchers have developed a new way to loop video that moves parts forward and backwards in time and then mixes the results. This way, they can make video textures that loop continuously and smoothly. Researchers show that our method works and is reliable by using it on many examples, such as beaches, waterfalls, and rivers that flow.

Paper: Animating Pictures with Eulerian Motion Fields

Click here for the code