This New Tool By Nvidia Will Allow You To Turn Doodles Into Landscapes

Nvidia is no newcomer when it comes to creating groundbreaking technology, like fixing grainy photos or even creating portraits of people using AI. And now they surprised us once again, this time by coming up with a tool that turns your doodles into stunning works of art.

“GauGAN allows users to draw their own segmentation maps and manipulate the scene, labeling each segment with labels like sand, sky, sea or snow,” writes Nvidia.

The company says the AI has been trained on a million images: “Draw in a pond, and nearby elements like trees and rocks will appear as reflections in the water. Swap a segment label from “grass” to “snow” and the entire image changes to a winter scene, with a formerly leafy tree turning barren.”

Since the technology was trained using real images, it understands that real bodies of water have reflections and is able to generate them.

“It’s like a coloring book picture that describes where a tree is, where the sun is, where the sky is,” says Bryan Catanzaro, the vice president of applied deep learning research. “And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows and colors, based on what it has learned about real images.”

GauGAN allows the user to change the generated images to adapt the style of a painter of their choice and change the daytime of the scene.

“This technology is not just stitching together pieces of other images, or cutting and pasting textures,” says Bryan Catanzaro. “It’s actually synthesizing new images, very similar to how an artist would draw something.”

The research paper describing this new technology will be presented in the Computer Vision and Pattern Recognition (CVPR) conference in June. And those attending the GPU Technology Conference this week can even try out GauGAN themselves in the Nvidia booth!

Check out the video explaining how this technology works below!