Movie Posters Based on Dreams

Exploiting the flaws of AI art to create movie posters for my recurring dreams

The Idea

I have always found the text I see in my dreams interesting - it often looks like English but isn’t. When I first started playing with Stable Diffusion, I noticed that the text it generates is very similar:

A local art gallery was accepting pieces for an exhibition of visual art based on dream imagery. My dreams often play out like movies. This gave me an idea: movie posters for three of my favorite types of dreams:

• being chased
• the world coming to an end
• being able to fly

The Tools

• Stable Diffusion v1.5 (AUTOMATIC1111 Stable Diffusion WebUI)
• AI Upscaling (R-ESRGAN 4x+, ESRGAN 4x+)
• Adobe Photoshop

The Process

I wanted to create three movie posters, with three unique styles. The movie posters would use a portrait orientation, with credits and a title overlaid on the imagery.

This presented a challenging problem for Stable Diffusion 1.5 for two reasons:

• I had specific demands for the content of the images
• I had specific demands for the text which should be overlaid on the images

It is extremely challenging for Stable Diffusion 1.5 to generate accurate text and imagery in a single image. If it were required to generate the movie poster text and imagery at the same time (i.e. with a single image generation prompt), the odds of obtaining a successful image would be quite low. To avoid this problem I took a compositing approach:

1. Generate poster images in a portrait orientation
2. Generate title and credit text
3. Create a movie poster by combining imagery and text

1. Poster Imagery

I began by deciding what style of movie posters I wanted to create. Based on the differing themes, each dream was best represented by a different movie poster genre:

• being chased: muscle car grindhouse movie

• the world coming to an end: alien invasion summer blockbuster

• being able to fly: children’s movie

With these styles in mind, the compositions came naturally:

• muscle car grindhouse movie: front view of a black muscle car driving away from an explosion in the desert
• alien invasion summer blockbuster: aerial image of a menacing object hovering over the Earth
• children’s movie: back view of a child floating above the clouds

Next came the prompts to produce the desired imagery. I treated each prompt as the combination of two distinct prompts: one portion for the content, the other portion for the visual style.

After considerable experimentation, I arrived at the following prompts:

black muscle car leaving a trail of fire, driving into the camera, front view, in an empty desert, dark black smoke in the sky, surrounded by flames, watercolor painting, action movie poster

asteroid hitting earth, explosion, debris, apocalypse, destruction, aerial view from above, oil painting, disaster movie poster

a boy basejumping without a parachute with arms stretched wide, wearing sneakers, flying above the surface of the earth, top down view of the back of his head and body, white altocumulus clouds above the surface of the earth visible underneath, bright day, oil painting, fantasy movie poster

From the thousands of generated images I chose my favorites:

I was now ready to generate the text for the movie posters.

2. Movie Titles and Credits

With the poster imagery generated, I shifted focus to generating images purely for their text content. This meant I could tailor my prompts to increase the odds that appropriate text (in style and content) was generated.

After some experimentation, I arrived at the following prompts:

grindhouse muscle car movie poster text credits written by directed by

classic asteroid destruction disaster movie poster with text credits at bottom

fly movie poster, cartoon, zany, wacky, large font

From the thousands of generated images I chose the most appropriate title and credit text for each poster:

Since I was printing these movie posters for display in an art gallery, I targeted the common 11 × 17-inch mini-poster format. Using the standard 300dpi printing resolution, this required the movie posters to have pixel dimensions of 3300px x 5100px. Stable Diffusion 1.5 was not designed to generate images directly at this print resolution, so I needed to upscale the imagery before compositing.

After some experimentation, I found that using R-ESRGAN 4x+ to upscale images, and ESRGAN 4x+ to upscale text worked best.

I could now combine these pieces into the final movie posters.

3. Compositing

In Photoshop, I isolated the text from its background, refined the text outlines and colors, and played with various compositions. Once happy, I applied Gaussian noise to the entire image to compensate for any smoothing which occurred during upscaling. This also made the text and background imagery blend more naturally.

The final composites:

The Result

The posters were printed and framed:

Then mounted for display at the art gallery:

Those who liked it, liked it a lot:

I later used custom LoRAs and ControlNet for Japanese Citypop Landscapes, The Back Side of the Moon, and SEM Landscapes.