Django ImageField backed by DigitalOcean Spaces

Posted by on 10 March 2020

Django has a wonderful tool for handling image uploads that has been carefully thought out and works very well. Consider all the things you would have to manage if you were to roll your own.

  1. Where to store it
  2. How to handle the same image being uploaded twice but in different records
  3. On change, remove the old image and replace with the new one
  4. Deleting it when the record gets deleted

Django handles some of these things for you right out of the box. However it begins to break down if you have more than one node because it keeps track of where they are stored on disk and if you have more than one node, some images could be found on one node but not the other so you would need a way to synchronize all the nodes or use a shared store like S3 or DigitalOcean Spaces. If you do this then the ImageField will need a little help. You can help it by either downloading a library to take care of this for you, or you can roll your own Storage. I opted for rolling my own storage and that is what I'm going to talk about today.

My first attempt was not to create my own custom storage however, I thought just hooking into the model might be "good enough"...and it did kind of work, until I was making some edits and the load balancer switched nodes on me for whatever reason and Django started throwing 500 errors. It turned out that it was because it was looking for the image on disk but it was on another node, not the current one. That is when I discovered custom storage. With a custom storage you can tell the ImagField to use your new storage and you don't have to manage all of that stuff in the model. Making a custom storage is not that hard, you just need to subclass "Storage" and override the methods you need. You can read more about the process here: https://docs.djangoproject.com/en/3.0/howto/custom-file-storage/. Here is the storage I came up with for DigitalOcean Spaces.

import boto3
import logging
import os

from botocore.exceptions import ClientError
from django.core.files.storage import Storage
from django.utils.deconstruct import deconstructible


def get_client():
    """ Initialize a session using DigitalOcean Spaces. """
    session = boto3.session.Session()
    spaces_region = os.environ.get('SPACES_REGION', '')
    spaces_endpoint = os.environ.get('SPACES_HOST', '')
    spaces_access_key = os.environ.get('SPACES_USER', '')
    spaces_password = os.environ.get('SPACES_PASSWD', '')

    return session.client('s3',
                          region_name=spaces_region,
                          endpoint_url=spaces_endpoint,
                          aws_access_key_id=spaces_access_key,
                          aws_secret_access_key=spaces_password)


@deconstructible
class SpacesStorage(Storage):
    bucket = "zol-images"

    def _save(self, name, content):
        """ Save the image to a DigitalOcean Space """
        client = get_client()
        try:
            client.upload_fileobj(Fileobj=content,
                                  Bucket=self.bucket,
                                  Key=name,
                                  ExtraArgs={
                                      'ACL': 'public-read',
                                      'ContentType': content.content_type
                                  })
        except ClientError as e:
            logging.error(e)

        return name

    def delete(self, name):
        """ Deletes image files on `post_delete` """
        print("spaces-delete: %s" % name)
        client = get_client()

        try:
            client.delete_object(Bucket=self.bucket, Key=name)
        except ClientError as e:
            logging.error(e)

    def exists(self, name):
        """ Check if the image name already exists in the Space """
        client = get_client()

        try:
            client.get_object(Bucket=self.bucket,
                              Key=name)
        except ClientError as e:
            logging.error(e)
            return False

        return True

    def url(self, name):
        """ Return the URL to access the image on the CDN """
        return "https://zol-images.zolmok.org/%s" % name

So now your new storage knows how to read, write and delete images we just need to tell the model about it.

fs = SpacesStorage()

class Post(models.Model):
    ...
    hero_image = ImageField(
        blank=True,
        default='',
        storage=fs,
        upload_to="gallery"
    )

That should be enough to upload your image but if you use the built-in Django admin you want to replace the image with a new one you still have a problem because the admin doesn't do that automatically. I solved this by overriding "ImageField".

from django.db import models


class ImageField(models.ImageField):
    def save_form_data(self, instance, data):
        if data is not None:
            file = getattr(instance, self.attname)

            # delete the image from the DigitalOcean space
            # if the "clear" checkbox was checked
            if file.name != '' and data is False:
                file.delete(save=False)

        super(models.ImageField, self).save_form_data(instance, data)

You can see in the model code above that I'm already using the custom "ImageField" and that should be all you need. Now you can add, edit, and remove images backed by DigitalOcean Spaces. There may be a better way and if you have any suggestions for improvement I would love to hear about it.

Tags: #django #python

Categories: #technology

Tags

Categories