In this guide, we'll host an imaginary viggor Django project on Heroku.

Don't mind the v* commands, they are aliases I use to manage Python virtual envs.

Setup virtual environment and install low-level dependencies:

export APP_NAME=viggor
vmk $APP_NAME 3.8
v $APP_NAME
pip install -U pip wheel setuptools pip-tools
sudo apt-get install -y postgresql postgresql-contrib libpq-dev # for psycopg2

Define and install Python dependencies that I frequently use:

{ cat <<EOF > requirements.in
django
django-enumfields
django-filter
django-guardian
django-heroku
django-impersonate
django-jinja
djangorestframework
gunicorn
markdown
psycopg2
requests
whitenoise
EOF
} && { cat <<EOF > requirements-dev.in
-c requirements.txt
freezegun
pip-tools
pytest
pytest-cov
pytest-django
pytest-mock
requests-mock
EOF
} && pip-compile requirements.in \
  && pip-compile requirements-dev.in \
  && pip install -r requirements.txt -r requirements-dev.txt

Create and configure a new Djago project:

django-admin startproject $APP_NAME . \
  && cat <<EOF > .env
DATABASE_URL=postgres:///$APP_NAME
DEBUG=True
PYTHONUNBUFFERED=1
SECRET_KEY=`python -c "import secrets; print(secrets.token_urlsafe())"`
EOF

Mangle the settings file a bit:

# your milage may vary with new Django versions but this worked in Django 3.1
sed -i \
    -e "1s/^/import os\nimport django_heroku\nfrom django_jinja.builtins import DEFAULT_EXTENSIONS\n/" \
    -e "/# .*/d" \
    -e "s/SECRET_KEY = .*//" \
    -e "s/STATIC_URL = .*//" \
    -e "s/DATABASES = {/DELETE_ME = {/" \
    -e "s/DEBUG = True/DEBUG = os.getenv('DEBUG', False)/" \
    $APP_NAME/settings.py \
  && { cat <<EOF > $APP_NAME/session_serializer.py
import json
import uuid
from django.contrib.sessions.serializers import JSONSerializer

class SessionSerializer(JSONSerializer):

    def dumps(self, obj):
        return json.dumps(obj, separators=(',', ':'), cls=ExtendedJSONEncoder).encode('latin-1')

class ExtendedJSONEncoder(json.JSONEncoder):

    def default(self, obj):
        if isinstance(obj, uuid.UUID):
            return obj.hex
        return super().default(self, obj)
EOF
} && { cat <<EOF > $APP_NAME/permissions.py
import copy
from rest_framework import permissions

class ExtendedModelPermissions(permissions.DjangoModelPermissions):

    def __init__(self):
        self.perms_map = copy.deepcopy(self.perms_map)
        self.perms_map['GET'] = ['%(app_label)s.view_%(model_name)s']
        self.perms_map['OPTIONS'] = ['%(app_label)s.view_%(model_name)s']
        self.perms_map['HEAD'] = ['%(app_label)s.view_%(model_name)s']
        super().__init__()

class ExtendedObjectPermissions(permissions.DjangoObjectPermissions):

    def __init__(self):
        self.perms_map = copy.deepcopy(self.perms_map)
        self.perms_map['GET'] = ['%(app_label)s.view_%(model_name)s']
        self.perms_map['OPTIONS'] = ['%(app_label)s.view_%(model_name)s']
        self.perms_map['HEAD'] = ['%(app_label)s.view_%(model_name)s']
        super().__init__()

EOF
} && { cat <<EOF >> $APP_NAME/settings.py
AUTH_USER_MODEL = '$APP_NAME.User'
LOGIN_REDIRECT_URL = '/'
SESSION_SERIALIZER = '$APP_NAME.session_serializer.SessionSerializer'

AUTHENTICATION_BACKENDS = (
    'django.contrib.auth.backends.ModelBackend',
    'guardian.backends.ObjectPermissionBackend',
)

# https://django-guardian.readthedocs.io/en/stable/userguide/custom-user-model.html#custom-user-model
GUARDIAN_MONKEY_PATCH = False

INSTALLED_APPS.extend([
    '$APP_NAME',
    'django_jinja',
    'guardian',
    'impersonate',
    'rest_framework',
])

MIDDLEWARE.extend([
    'impersonate.middleware.ImpersonateMiddleware',
])

TEMPLATES.insert(0, {
    'BACKEND': 'django_jinja.backend.Jinja2',
    'APP_DIRS': True,
    'OPTIONS': {
        'extensions': DEFAULT_EXTENSIONS,
        'context_processors': TEMPLATES[0]['OPTIONS']['context_processors'],
        'match_extension': '.jinja',
        'debug': True,
    }
})

# https://www.django-rest-framework.org/api-guide/settings/
# https://www.django-rest-framework.org/api-guide/permissions/
REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework.authentication.SessionAuthentication',
    ),
    'DEFAULT_PERMISSION_CLASSES': [
        '$APP_NAME.permissions.ExtendedObjectPermissions',
    ],
    # 'DEFAULT_FILTER_BACKENDS': [
    #     'rest_framework_guardian.filters.ObjectPermissionsFilter',
    # ],
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination',
    'PAGE_SIZE': 100,
}

# django-heroku will handle most of the default configuration
# https://github.com/heroku/django-heroku/blob/master/django_heroku/core.py#L49
django_heroku.settings(locals())
EOF
}

Add an index view and its template:

mkdir -p $APP_NAME/views \
  && echo 'from .index_view import IndexView' >> $APP_NAME/views/__init__.py \
  && { cat <<EOF > $APP_NAME/views/index_view.py
from django.views.generic import TemplateView

class IndexView(TemplateView):
    template_name = 'index.jinja'
EOF
} && mkdir -p $APP_NAME/templates \
  && { cat <<EOF > $APP_NAME/templates/index.jinja
<html>
    Hello {{ user }}!
</html>
EOF
}

Overwrite URL configuration:

cat <<EOF > $APP_NAME/urls.py
from django.contrib import admin
from django.urls import include, path

from $APP_NAME.api.router import router
from $APP_NAME.views import IndexView

urlpatterns = [
    path('admin/', admin.site.urls),
    path('api/', include(router.urls)),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework')),
    path('impersonate/', include('impersonate.urls')),
    path('', IndexView.as_view(), name='index'),
]
EOF

Customize the core user model:

mkdir -p $APP_NAME/mixins \
  && echo 'from .times_mixin import TimesMixin' >> $APP_NAME/mixins/__init__.py \
  && { cat <<EOF > $APP_NAME/mixins/times_mixin.py
from django.db import models

class TimesMixin(models.Model):
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

    class Meta:
        abstract = True
EOF
} && echo 'from .uuid_mixin import UUIDMixin' >> $APP_NAME/mixins/__init__.py \
  && { cat <<EOF > $APP_NAME/mixins/uuid_mixin.py
import uuid
from django.db import models

class UUIDMixin(models.Model):
    id = models.UUIDField(
        primary_key=True,
        default=uuid.uuid4,
        editable=False,
        verbose_name='ID'
    )

    class Meta:
        abstract = True
EOF
} && mkdir -p $APP_NAME/models \
  && echo 'from .user import User' >> $APP_NAME/models/__init__.py \
  && { cat <<EOF > $APP_NAME/models/user.py
from django.contrib.auth.models import AbstractUser
from guardian.mixins import GuardianUserMixin
from $APP_NAME.mixins import UUIDMixin
from $APP_NAME.mixins import TimesMixin

class User(UUIDMixin, TimesMixin, AbstractUser, GuardianUserMixin):
    pass
EOF
} && mkdir -p $APP_NAME/admin \
  && echo 'from .user_admin import UserAdmin' >> $APP_NAME/admin/__init__.py \
  && { cat <<EOF > $APP_NAME/admin/user_admin.py
from django.contrib import admin, auth
from django.utils.translation import gettext_lazy as _
from $APP_NAME.models import User

@admin.register(User)
class UserAdmin(auth.admin.UserAdmin):

    list_display = ('username', 'email', 'is_active', 'is_staff', 'is_superuser')
    readonly_fields = ('created_at', 'updated_at')

    def get_fieldsets(self, request, obj=None):
        fieldsets = super().get_fieldsets(request, obj)
        if obj:
            fieldsets += ((_('Custom'), {'fields': ('created_at', 'updated_at')}), )
        return fieldsets
EOF
}

Add a simple API for the user models:

mkdir -p $APP_NAME/api \
    && { cat <<EOF > $APP_NAME/api/user_serializer.py
from rest_framework import serializers
from $APP_NAME.models import User

class UserSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = User
        fields = ['url', 'username', 'email']
EOF
} && { cat <<EOF > $APP_NAME/api/user_view_set.py
from rest_framework import viewsets
from $APP_NAME.api.user_serializer import UserSerializer
from $APP_NAME.models import User

class UserViewSet(viewsets.ModelViewSet):
    queryset = User.objects.all()
    serializer_class = UserSerializer
EOF
} && { cat <<EOF > $APP_NAME/api/router.py
from rest_framework import routers
from $APP_NAME.api.user_view_set import UserViewSet

router = routers.DefaultRouter()
router.register('users', UserViewSet)
EOF
}

Create development database:

# sudo -u postgres createuser -s $USER  # here, $USER is the sudoing user
createdb $APP_NAME
# psql $APP_NAME -c 'SELECT 1;'  # just a test
heroku local:run python manage.py makemigrations $APP_NAME

Create Procfile for Heroku:

cat <<EOF > Procfile
release: python manage.py migrate --noinput
web: gunicorn $APP_NAME.wsgi --log-file=-
EOF

Setup testing for development:

{ cat <<EOF > pytest.ini
[pytest]
DJANGO_SETTINGS_MODULE=$APP_NAME.settings
python_files = *_tests.py
EOF
} && { cat <<EOF > conftest.py
import pytest
from django.contrib.auth.models import Group
from rest_framework.test import APIClient

@pytest.fixture
def random_user(django_user_model):
    User = django_user_model
    return User.objects.create_user('john', 'john@example.com', 'johndoe123')

@pytest.fixture
def random_group():
    return Group.objects.create(name='some-people')

@pytest.fixture
def admin_user(admin_user):
    return admin_user

@pytest.fixture
def anonymous_client(client):
    return client

@pytest.fixture
def random_client(client, random_user):
    client.force_login(random_user)
    return client

@pytest.fixture
def admin_client(admin_client):
    return admin_client

@pytest.fixture
def anonymous_api_client():
    return APIClient()

@pytest.fixture
def admin_api_client(anonymous_api_client, admin_user):
    anonymous_api_client.force_authenticate(user=admin_user)
    return anonymous_api_client
EOF
} && { cat <<EOF > $APP_NAME/api/user_api_tests.py
import pytest
from guardian.shortcuts import get_perms, assign_perm, remove_perm
from rest_framework.reverse import reverse

# https://www.django-rest-framework.org/api-guide/routers/#usage
# https://developer.mozilla.org/en-US/docs/Learn/Server-side/Django/Authentication

@pytest.mark.django_db
def test_user_api_as_anonymous(anonymous_api_client):
    user_list_url = reverse('user-list')
    response = anonymous_api_client.get(user_list_url)
    assert response.status_code == 403  # Forbidden

# TODO: # user_detail_url = reverse('user-detail', args=[response.wsgi_request.user.pk])
# TODO: https://github.com/rpkilby/django-rest-framework-guardian/blob/master/src/rest_framework_guardian/filters.py

@pytest.mark.django_db
def test_user_api_as_random(random_client, random_group):
    user_list_url = reverse('user-list')
    response = random_client.get(user_list_url)
    user = response.wsgi_request.user
    assert response.status_code == 403  # Forbidden

    assign_perm('$APP_NAME.view_user', response.wsgi_request.user)
    response = random_client.get(user_list_url)
    assert response.status_code == 200  # OK

    remove_perm('$APP_NAME.view_user', response.wsgi_request.user)
    response = random_client.get(user_list_url)
    assert response.status_code == 403  # Forbidden

@pytest.mark.django_db
def test_user_api_as_admin(admin_api_client):
    user_list_url = reverse('user-list')
    response = admin_api_client.get(user_list_url)
    assert response.status_code == 200  # OK

@pytest.mark.django_db
def test_permissions_configuration(random_user, random_group):
    common_permissions = ['add_group', 'change_group', 'delete_group', 'view_group']
    for perm in common_permissions:
        assert not random_user.has_perm(perm, random_group)
    for perm in common_permissions:
        assign_perm(perm, random_user, random_group)
        assert random_user.has_perm(perm, random_group)
EOF
} && { cat <<EOF > $APP_NAME/views/index_view_tests.py
def test_index_view_as_anonymous(anonymous_client):
    response = anonymous_client.get('/')
    assert 'Hello AnonymousUser!' in str(response.content)

def test_index_view_as_admin(admin_client):
    response = admin_client.get('/')
    assert 'Hello admin!' in str(response.content)
EOF
} && { cat <<EOF > $APP_NAME/models/user_tests.py
import uuid
import pytest
from $APP_NAME.models import User

@pytest.mark.django_db
def test_user_creation():
    User.objects.create_user('john', 'john@example.com', 'johndoe123')
    user = User.objects.get(username='john')
    assert type(user.id) == uuid.UUID
    assert user.email == 'john@example.com'
    assert user.password != 'johndoe123'  # it wil be properly hashed
    assert not user.is_staff
    assert not user.is_superuser
EOF
} && heroku local:run pytest

TODO: from django.core.exceptions import PermissionDenied

def users_list_view(request): if not request.user.has_perm('auth.view_user'): raise PermissionDenied()

See that it works:

heroku local:run python manage.py migrate
heroku local:run python manage.py createsuperuser
heroku local:start web -p 4000
# http://127.0.0.1:4000/
# http://127.0.0.1:4000/admin and try to login

Run on Heroku

Tell Heroku which Python version to use:

# https://devcenter.heroku.com/articles/python-support#supported-python-runtimes
echo python-3.8.8 > runtime.txt

cat <<EOF > .gitignore
.idea/
.pytest_cache/
__pycache__/
staticfiles/
.env
EOF
git init
git add -A
git commit -m 'Initial commit'
heroku auth:whoami  # heroku login

export HEROKU_APP=$APP_NAME
heroku apps:create --region us --remote $HEROKU_APP
heroku config:set SECRET_KEY=`python -c "import secrets;print(secrets.token_urlsafe())"`
heroku addons:create heroku-postgresql:hobby-dev
heroku config  # you should see DATABASE_URL in there
git push heroku master
heroku run python manage.py createsuperuser
heroku open

Debugging Heroku:

# see service logs...
heroku logs --tail

# open interactive debug shell...
heroku run python manage.py shell

Add error reporting

export APP_NAME=viggor
export HEROKU_APP=$APP_NAME && echo $HEROKU_APP
echo 'sentry-sdk' >> requirements.in
pip-compile requirements.in && pip install -r requirements.txt

# https://docs.sentry.io/platforms/python/guides/django/
# https://devcenter.heroku.com/articles/sentry#integrating-with-python-or-django
heroku addons:create sentry
# you will recieve a message from Sentry, recover password to set it

# settings.py...
import os

if os.getenv('SENTRY_DSN'):
    import sentry_sdk
    from sentry_sdk.integrations.django import DjangoIntegration

    sentry_sdk.init(
        dsn=os.environ['SENTRY_DSN'],
        integrations=[DjangoIntegration()],
        traces_sample_rate=1.0,
        send_default_pii=True,
    )


import sentry_sdk
try:
    a_potentially_failing_function()
except Exception as e:
    # Alternatively the argument can be omitted
    sentry_sdk.capture_exception(e)


# reporting about a non-exception issue like a 404 or a warning:
import sentry_sdk
sentry_sdk.capture_message('Something went wrong')
sentry_sdk.capture_message('Something went wrong', level='error')

Add caching

export APP_NAME=viggor
export HEROKU_APP=$APP_NAME && echo $HEROKU_APP
echo 'django-redis' >> requirements.in
pip-compile requirements.in && pip install -r requirements.txt

cat <<EOF >> $APP_NAME/settings.py
# https://devcenter.heroku.com/articles/heroku-redis#connecting-in-django
CACHES = {
    'default': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': os.getenv('REDIS_TLS_URL', os.getenv('REDIS_URL', 'redis://127.0.0.1:6379/1')),
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
            'CONNECTION_POOL_KWARGS': {'ssl_cert_reqs': False}
        }
    }
}
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"
EOF

heroku local:start web -p 4000  # and try to login

# now you can use the cache backend in your code:
# from django.core.cache import cache
# cache.ttl('foo')
# cache.set('foo', 'value', timeout=5)
# cache.get('foo')
# with cache.lock('somekey'):
#     do_some_thing()
# and all libraries that support Django cache backend will use it

heroku addons:create heroku-redis:hobby-dev
heroku addons:wait heroku-redis
heroku config  # there should be REDIS_TLS_URL and/or REDIS_URL
git add -A
git commit -m 'Add Redis for cache and sessions'
git push heroku master
heroku open  # and try to login

Adding scheduled jobs

If you need something ran every 10 mininutes, every hour or once per day, Heroku Scheduler is an easy free option.

export APP_NAME=viggor
export HEROKU_APP=$APP_NAME && echo $HEROKU_APP
heroku addons:create scheduler:standard
heroku addons:open scheduler
# and configure it to run a command e.g. echo hello or python manage.py something

For more complex jobs: https://devcenter.heroku.com/articles/clock-processes-python

Better Auth

echo 'django-allauth' >> requirements.in
pip-compile requirements.in && pip install -r requirements.txt

Frontend

https://www.valentinog.com/blog/drf/#setting-up-react-and-webpack

Pipelines

https://devcenter.heroku.com/articles/pipelines https://devcenter.heroku.com/articles/github-integration-review-apps

Reverting all changes; local and remote

export APP_NAME=viggor
export HEROKU_APP=$APP_NAME && echo $HEROKU_APP
heroku apps:destroy --confirm $HEROKU_APP
cd ..
dropdb $APP_NAME
vrm $APP_NAME
rm -rf $APP_NAME