Paint, Paper, Panoramas, and Python

I’m an artist and a python developer, two things that rarely occupy the same worlds, let alone the same sentence. However, I have recently found a way to combine these two passions: Panoramas.

My current smartphone takes excellent pictures. It does a great job at figuring out colors, lighting, and focus, even in low lighting. As an artist, this is important to me because I often use my phone to snap quick pictures of a scene as a reference to take back to my studio. It’s a huge improvement in the technology I had in my hands even five years ago. There is one thing about my old phone that I miss though – its ‘panorama’ photo mode, but not because it was better.

I miss how amazingly awful it used to be, and more importantly, the freedom to make awful pictures it allowed. I’d point the lens out the window of the car as it sped along (as a passenger of course) to make jagged and confusing images of tiny bits of the landscape that the phone struggled to hodgepodge together. I’d tilt and move the phone in random directions to make weird swirls of the horizon. Even when being used ‘as directed’, it would usually struggle with focus and lighting coming up with spontaneously and wonderfully terrible photos with abstract light glare or menacing dark patches. It’s hard to explain, but sometimes as an artist, a terrible photo can be just as inspiring as those picture perfect reference pics I take with me back to the studio.

My current phone is too smart for that though, and it snatches away any joy of bad photography by making consistently beautiful and seamless panoramas. Not only that, but it accomplishes this mostly by yelling at you (“You’re going too fast!”) or by using angry arrows to make sure you can only move the phone in one direction, and then abruptly ending the photograph when you don’t cooperate. So, I did what anyone does when they get nostalgic for awful photography – I made a python script to make my own terrible panoramas.

My plan was simple. First, I would shoot short videos where my phone wouldn’t yell at me for moving, tilting, and spinning the image as much as I wanted. Next, use Python to convert each frame of the video clip to an image, crop the image into a tiny sliver out of the center of the image and then glue them all together. The results are imperfect. And gloriously so.

Side note: Although I used my smartphone to shoot some video, this script could be applied to any video. Think of the wild panoramas you could create from some Russian dash cam footage, or a GoPro strapped to a fish, or a tiny clip from the Lord of the Rings. However, this script works best on videos that are less than 10 seconds long or else it produces mile long panoramic images. Currently, I don’t bother limiting the image size at all, but theoretically I could by using one out of every five frames for instance, or by cutting down the image slice size based on video length.

The Python

I used ffmpeg for turning each video frame into an image. It was simple to install, just download and unzip. Here’s a handy installation guide -> https://github.com/adaptlearning/adapt_authoring/wiki/Installing-FFmpeg

The Python Image Library is the only other requirement, installed with pip.

pip3 install pillow

The script works by pulling all videos out of a source directory based on file suffix and creating a panorama for each. This could easily be modified to convert just one video at a time by removing the loops and passing the path to the desired video directly to ffmpeg.

directory = Path('my\\videos\\dir')

vids = []
for vid in directory.iterdir():
    if vid.suffix.lower() in ('.mp4', '.mkv'):
        vids.append(vid)

Every frame pulled out by ffmpeg is stored in a file. I delete the directory and recreate it before ffmpeg runs to delete the old frames from the last run.

for vid in vids:
    shutil.rmtree("pics", ignore_errors=True)
    os.makedirs("pics", exist_ok=True)

    print(f'Creating panoramic {vid.stem}')
    result = run(
        f'ffmpeg -i {vid.absolute()} '
        f'-y pics\\thumb%04d.jpg -hide_banner', 
        shell=True, stderr=PIPE)
    result.check_returncode()
    print(result.stderr.decode('utf-8'))

After it finishes pulling out all the frames, I start the panorama by creating an empty image. I need to have the dimensions of the finished image to create it. To get the final width, I multiply the number of frames ffmpeg pulled out by the width of my image slice (40 pixels). For the height, I open up one of the frames and use it size as a reference. I also use the sample image’s dimensions to figure out the center of the image for cropping everything down later.

Then, I loop through all the frame images in reverse order (because … long story short, it usually looks better that way) and then work on slicing each image down to 40 pixels wide to glue into the panorama.

    
    sample = Image.open("pics/{}".format(os.listdir("pics")[0]))
    width, height = sample.size
    center = width / 2

    panoramic = Image.new('RGB', (len(os.listdir("pics")*40), height))
    
    # This offset is so PIL knows where to start adding 
    # each image slice to the panorama
    x_offset = 0

    for i in reversed(os.listdir("pics")):
        img = Image.open("pics/{}".format(i))
        area = (center - 20, 0, center + 20, height)
        cropped_img = img.crop(area)
        panoramic.paste(cropped_img, (x_offset, 0))
        x_offset += 40

    panoramic.save(f'{vid.stem}.jpeg')
    panoramic.close()

The Painting

So far, I am quite happy with the results of this adorable little script.
It has definitely given me the creative inspiration I was missing. In the past two weeks, I have done three series of paintings based on panoramas I have created using it, with plans for many more. Here’s an example of how I used it to create some artwork!

I took this video:

turned it into this panorama:

played with some paint and smudged around some charcoal and pastels:

and came up with this:

Final Thoughts

It feels really amazing to apply Python to unusual problems, even if that challenge is finding a unique way of creating original art. Plus, if the inspiration ever dries up, I have some ideas for making this script even more fun:

• grab each slice from a random spot rather than dead center of the image for something much more jumbled and abstract
• options to not use every frame for longer videos
• PIL ‘effects’, like a black and white mode, over saturation, or extra blurry images
• an ‘up and down’ mode for tall panoramas

I hope you enjoyed! Feel free to check out my website or my instagram for more artwork if you are interested.