How MVC Frameworks Taught Us Bad Habits — Part 1 ActiveRecord

Let’s look back in the past and try to figure out how frameworks taught us not to notice the wrong things.

Bad habits

Intro

Let’s go back in 200X. It was a boom of MVC frameworks. Ruby on Rails, Django, ZendFramework, Spring, etc. Various frameworks in different programming languages created, copied, and improved various features.

Frameworks have taken a really huge step in web development. And they brought a lot of good things into the industry and forbade doing bad things for developers. Using frameworks made it easy for beginners to start. My career as a web developer began in 2007. For several years I used the frameworks without thinking about the architecture and tools that these frameworks provided.

We are so used to using frameworks that we can hardly imagine that there may be any alternative options. They inadvertently taught us to rely on the approaches that they use. The problem is that approaches that work great for small to medium-sized apps don’t work so well for large applications. The converse is also true. So if you are dealing with small and medium-sized applications, you have nothing to worry about. But if you feel that your applications already lack the former flexibility — you are welcome.

I want to share some ideas about the shortcomings that I see for large applications that frameworks cannot handle effectively. I want you to look at familiar things from a different angle

And we start with the ActiveRecord pattern. Not all frameworks use it. So if your favorite framework uses the repository pattern, then everything is fine. You can skip this article.

Why do we like ActiveRecord

Remember how cool it was to create a blog in 15 minutes. Back at this time, many frameworks provided the CLI or even a Web interface for creating models, controllers, and views in one action. You got a skeleton similar to this one.

def create
  @user = User.new(user_params)
  if @user.save
    redirect_to @user
  else
    render 'new'
  end
end

Convention Over Configuration and Keep It Simple were the main rules. And the cornerstone in building such a simple code was ActiveRecord. Not all, but very many frameworks used and use Active Record, not offering any alternative

Nowadays you can find pretty same code in various framework guides. The code is short and elegant. So what’s wrong with ActiveRecord?

What’s wrong with ActiveRecord

At first glance, it is very convenient and straightforward. And indeed it works great while you are dealing with simple applications.

But in real life, creating a new user or an order is not just writing to a database. If we are talking about user creation — it is a complicated registration process. Let’s estimate what additional actions may appear during registration:

• save avatars
• pre-fill data from 3rd party sources
• non-trivial validation — calling 3rd parties
• different scenarios for registration — registration with FB, with the invite, with form, etc
• additional verifications — phone number verification, assigning and verification of a credit card
• blacklists checking
• additional post logic - notify other services about the new user
• etc

Order creation, appointment scheduling, even post creation, etc — in real life, business logic is much more complicated than it might seem at first glance. How ActiveRecord offer to cope with all these business rules?

Extend your validation rules

The frameworks offer extensive validation capabilities. Your validation rules grow like weeds, trying to keep up with business rules. (PHP and Yii1)

<?php 

class Yii1Model 
{
    public function rules()
    {
        return array(
            // Required fields
            array('customer', 'required', 'message' => Yii::t('rezdy', 'Please create or select a customer'), 'on' => 'create, update'),

            // Safe on booking form
            array('customer, customer[mailingAddress], additionalFields, paymentOption, payFullAmount', 'safe', 'on' => 'bookingForm, partners'),
            array('customerAgreedToTermsAndConditions', 'safe', 'on' => 'bookingForm'),

            // Safe on create and update
            array('status, surcharge, sourceChannel, agentUserName', 'safe', 'on' => 'create, update'),
            array('internalNotes, resellerComments, resellerReference, agentUserName', 'safe', 'on' => 'partners'),

            // Safe on commissions report
            array('startTime, endTime, dateRange, resellerId, reconciled, lastInvoice', 'safe', 'on' => 'commissionDetails'),


            // Top level fields with children
            array('customer, items, paymentDetails, creditCard', 'safe'),
            array('status, searchString, startTime, endTime, dateRange, productId, paymentStatus, entityState', 'safe', 'on' => 'search'),

            // Special fields
            array('date', 'date'),
            array('customer[phone], customer[mobile]', 'match', 'pattern' => '/^([0-9+() -])+$/'),

            // Trim
            array('comments, internalNotes, voucherCode, resellerComments, resellerReference', 'filter', 'filter' => [$this, 'sanitize']),

            // Email validation
            array('customer[email]', 'ext.validators.EmailValidator', 'allowEmpty' => true),

            //Check for TC agreement
            array('customerAgreedToTC', 'ext.validators.EConditionalValidator', 'conditionalRules' => array('companySettings[explicitAgreementToTermsAndPolicy]', 'compare', 'compareValue' => true), 'rule' => array('required'), 'on' => 'bookingForm'),
            // Max length
            array('email, customer[email]', 'length', 'max' => 254),
            array('customer[phone]', 'length', 'max' => 25),
            array('customer[firstName], customer[lastName], customer[mailingAddress][addressLine], customer[mailingAddress][addressLine2], customer[mailingAddress][city], customer[mailingAddress][state]', 'length', 'max' => 200),
            array('customer[mailingAddress][countryCode]', 'length', 'max' => 4),
            array('customer[mailingAddress][postCode]', 'length', 'max' => 10),
        );
    }
}

If your model deals with nested or relative models we still can add a few more options. Ruby on Rails

# https://api.rubyonrails.org/classes/ActiveRecord/NestedAttributes/ClassMethods.html
class Member < ActiveRecord::Base
  has_many :posts
  accepts_nested_attributes_for :posts, allow_destroy: true,  update_only: true, reject_if: proc { |attributes| attributes['title'].blank? }
end

If you need to have some post-update logic you can do that with

class PictureFile < ApplicationRecord
  after_commit :delete_picture_file_from_disk, on: :destroy
  def delete_picture_file_from_disk
    if File.exist?(filepath)
      File.delete(filepath)
    end
  end 
end

Your model knows how to

• validate user input for different scenarios
• save updates to your datastore
• find your models for different scenarios
• update and even validate related models
• make some post-update logic (often not directly related to the current domain)

Correct me if I am wrong, but a very common pattern looks like this

• you need a new feature
• you create a new table in your database
• create a model in your codebase that allows doing CRUD operations
• you keep adding functionality related to that domain model into the single class

If your project continues to evolve and new requirements appear regularly, after a few months you discover that your models folder has 200+ classes. And all your models know about each other.

After all, you will get monsters like this one. It’s just too big to put it here but take a look into it. Now it’s a little embarrassing to admit, but the author of this work was me 🤫. It was written around 2014. But I have seen much larger models.

Such huge models break the Single Responsibility Principle. The number of methods no longer allows our brain to work effectively with it (find out why is it hard for a brain to understand spaghetti-code). In the end, it’s no longer important does your class have 1K or 10K lines? And even considering that you’re OK to deal with Ctrl+F all the time the main problem of that approach - everything is tightly coupled. Now you are starting to realize that your code is desperately lacking in flexibility (although you can still resist it). You need to check a lot more places to be sure that you won’t break anything. The problem is that this is a fairly typical code, which is common now in 2020. When a team and code grows, you begin to understand that changes now take more time. And what was very convenient at the beginning now lacks flexibility.

ActiveRecord is just a tool

Have you ever had to port your application from one framework to another? Usually, it’s so laborious that it’s easier to do everything from scratch. And the culprit of these problems is tightly coupled models.

Patterns like AcriveRecordhave become the de-facto standard in some frameworks. Instead of building a domain layer around business rules, we built business rules around the provided tools like ActiveRecord, following provided guides. Your business rules don’t care whether you use ActiveRecord or Repository pattern do you use Mysql or DynamoDB. Business rules remain an integral part of your system, while Active Record is just a tool. As a developer, we can change the way how we save the data, we can change the storage but we cannot change business rules. Business dictates the rules. We build applications for these business rules.

Business rules should not be dependent on tools. You must first design the high-level components that implement your business rules. And only then pick up low-level tools. Not the other way around. A database is just a detail that you can change.

How to do it right

Let’s imagine that you need to implement the part of the application that processes the order. The order was created in the previous step and now you need to make a payment. You have the following requirements:

• you need to validate user input
• you need to recalculate the total amount and make sure that it matches the one we showed in the previous step
• process payment via X payment gateway
• update order state in the DB
• notify the warehouse module about the new order
• send a receipt for the customer

Instead of creating a new table in the database and related ActiveModel, let’s make a draft with these high-level requirements

class class OrderService {
    public Order processOrder(Order order) {
        // validate input
        // re-calculate total amount with TAX and make sure that the match previous price
        // process payment
        // update order state in DB
        // notify warehouse
        // send receipt
    }
    public createOrder(...)  {}
}

As you can see, saving data is just one step among many others. High-level operations and the interaction between them is more important now. And I want to postpone the implementation of component that saves data. How can I do this?

public class OrderService {
    priivate final OrderRepository repository;
    public OrderService(OrderRepository repository) {
        this.repository = repository;
    }

    public Order processOrder(Order order) {
        this.orderValidator.validate(order);
        Long total = this.orderPrice.calculateTotal(order);
        if (!total.equal(order.getTotal())) {
            throw new OrderPriceException("Price has been changed")
        }
        try {
            this.stripePaymentGateway.processOrder(order);
        } catch (StripeException exception) {
            // 
        }
        order = this.repository.confirm(order);
        this.applicationEventPublisher.publishEvent(new OrderConfirmed(order));
        this.customerNotifier.sendOrderReceipt(order);
    }
    public createOrder(...)  {}
}

Where OrderRepository is an interface, not even real implementation. I used the Repository pattern. Ok, now I’m ready to test my OrderService.

@ExtendWith(MockitoExtension.class)
public class ChargeServiceTest {

    @Mock
    private OrderRepository orderRepository;

    @InjectMocks
    private OrderService orderService;

    @Test
    public void testProcessOrderSuccess() {
        // test validation
        // test order price recalculation
        // etc
        when(orderRepository.confirm(order)).thenReturn(order);
    }
}

We were able to build and test our high-level component that implements our business rules.

Note:

• I do not need a database or integration with DynamoDB or any other storage to do this
• The code does not depend on the framework or on a specific library to deal with the database.
• It is easy to test. We consider OrderRepository as another unit. It will be tested with another test. The test is unit and fast since we don’t need to write data into a real database.
• the structure of the component operate ubiquitous language that is clear for the business and developers

If we want to migrate our application to the new framework we will just copy that code as it is and it will continue to work. It doesn’t have any hardcoded dependencies.

Wait but we actually need to save data to storage. We need a real implementation. Seriously? Oh yeah right. Ok, let’s add it. What do you prefer these days? Mysql or DynamoDb

@Repository
public class MysqlOrderRepository implements OrderRepository {
 
    @PersistenceContext
    private EntityManager entityManager;
    @Transactional
    public void confirm(Order order) {
        order.setConfirmed(true);         
        this.entityManager.persist(person);
    } 
}

Conclusion

• Although ActiveRecord violates SRP and tends to the couple components, still it could be good option for small and medium-size applications.
• ActiveRecord is just a tool. Always look at alternatives and choose what you need according to business needs.
• Repository&Entity is better though 😛
• Design your application based on business rules. High-level is important, low-level details can be postponed.