Challenges Learning Django Rest Framework

I had only a small amount of experience with Django when I started my current project. Naively, I thought that learning this new tech stack would be a breeze since I had worked with many different frameworks before. However, as I dug deeper into Django and Django REST Framework, I discovered surprising differences that left me feeling blindsided.

For those who aren’t aware, Django is a Python web framework that follows the model-template-view pattern. Django REST Framework is a toolkit that sits on top of Django and provides serialization, permission verification, and abstractions centered around views and endpoints.

Before Django, I mostly worked with the Spring framework and had some experience with the Phoenix framework and various JavaScript backends. Despite this, Django introduced me to techniques and methods I had never experienced before.

The most challenging aspects of working with Django and Django REST Framework were the ORM pattern used by Django, the implicit endpoints created by Django REST Framework’s viewset abstractions, and some of the logic placement patterns found in Django.

ORM Layer Pattern

My previous experience had me working with frameworks following the data mapper ORM pattern. In this pattern there exists a data access layer which acts as a liaison between the in-memory representation of a record and the actual database implementation. Therefore, the class representation of a record is typically more detached from the underlying data and a separate repository class is used to interact with the database instead of methods within the record class. A record was always represented as an immutable data structure, retrieved using a get function like UserRepo.get(userId), copied with desired changes, and saved using a save function such as UserRepo.save(newUser).

On the other hand, the ActiveRecord pattern which is used by Django and many other frameworks use the record class to represent both the data of a specific row in the table, and the actions that can be performed on that row.  Many have argued that this closer coupling leads to more intuitive and straightforward code, but since this concept was so new to me I struggled to get used to it.

This approach was particularly challenging for me when it came to relationships to other records. In Django, relationships are represented as lazy-loaded properties. This means they are not loaded at first when the record is retrieved but are loaded as soon as the property is accessed for the first time. Although this may improve performance for experienced users, I found that it led to many situations where the actual queries being executed were obfuscated. This often resulted in me having to optimize code that I didn’t realize was running hundreds of queries per request. Because the number and type of queries being run is determined by which model properties are accessed, it can be much more difficult to analyze query performance since it is spread throughout the codebase.  Detecting issues such as N+1 problems requires checking serializers, services, signals, and other models to determine which properties are accessed during every endpoint or task execution. These situations are so prevalent that we added a library called nplusone specifically to identify instances of inappropriate lazy loading.

The following will fetch all users then print their list of posts. The for loop will execute an SQL select statement to fetch all the posts for each user resulting in an n+1 amount of queries being executed where n is the amount of users.

users = User.objects.all()
for user in users:
  print(user.posts)

Using the select_related or prefetch_related methods will fix the N+1 problem, but requires you to know which properties will be used later in the code.

users = User.objects.all().prefetch_related("posts")
for user in users:
  print(user.posts)

Although I eventually got better at understanding Django query performance through practice, it was one of my biggest hurdles during the learning process.

Implicit Endpoint Functions

During my brief experience with Django REST Framework, it became apparent to me that it is all about abstracting away boilerplate code and enabling developers to create full CRUD endpoints quickly. This is extremely valuable, but it requires a different approach to code searching and navigation than I was accustomed to with other frameworks.

In other frameworks, each endpoint typically had an associated controller function that contained the logic for that endpoint. While there may be other functions or utilities that the controller calls, you could always start at the controller function and dig down until you found what you were looking for.

In contrast, the Django REST Framework viewset class system implicitly defines a majority of endpoints, which means they don’t map to a singular controller function but are automatically created using overridden methods of the viewset class. While this is helpful for quickly creating CRUD-style endpoints, it can make it more challenging for unfamiliar users to locate specific logic.

The following shows an example of viewset class where the logic defining the endpoints is in three separate places.

class PermissionClass(BasePermission):
  def has_permission(self, request, view):
    # Logic

# Implicitly defines CRUD endpoints
class UserViewSet(ReadOnlyModelViewSet):
  permission_classes = [PermissionClass]

  def get_serializer_class(self):
  # Logic

  def get_queryset(self):
  # Logic

Compared with standard Django view endpoints which define all of the logic in the view functions.

def index(request)
  # Logic
  return HttpResponse(result)

I often found myself in situations where an endpoint seemed to be performing unexpected effects, and I had to search for quite a while to determine if the issue was caused by the get_queryset method, a method field in the serializer, or another part of the code entirely. Initially, I wanted to avoid the implicit endpoints by manually defining endpoints using the @action decorator, and became frustrated when working with existing endpoints written in the viewset style. However, after learning all the various method overrides and pockets of logic that contribute to the endpoint’s behavior, I was able to become comfortable using the Django REST Framework viewsets properly.

Hidden Logic

In addition, I found that, similar to Django REST Framework’s implicit endpoints, models in Django placed a lot of logic in method overrides such as save or clean. While I believe this object-oriented approach can be advantageous for abstracting away much of the logic concerning model validation and saving, I also think it can be easily misused.

I fell into many traps where a model’s save override performed side effects like sending an email or saving other records, resulting in hours of debugging before I understood what was happening. Although I eventually learned to check for overridden methods in the model, it can still be challenging, particularly in code that references multiple models.

This creates a user model which, upon save, will send an email about a new user being created.

class User(Model):
  name = CharField()
  def save(self, *args, **kwargs):
    super().save(*args, **kwargs)
    send_new_user_email()

It isn’t immediately clear that the following code will send an email, especially when writing test setups or working in the interactive shell.

User().save()

Conclusion

Despite encountering many issues while learning Django and Django REST Framework, I have grown to respect their design and discovered many incredible benefits. For instance, I found the Django migration system to be extremely valuable, as it supports complicated migrations when required and allows for basic migrations to be generated with little to no manual effort. Additionally, I found the testing library included with Django to be very easy to use and a significant asset to development, with excellent support for API integration testing.

Overall, my experience learning Django and Django REST Framework has demonstrated their ability to create robust, low boilerplate model classes with minimal endpoint redundancy. However, the cost of these abstractions is a more opinionated framework that can make problems requiring unique implementations difficult and navigation more challenging for someone without a background in the paradigms Django employs.