Mocks are a powerful tool for testing Ruby code, but they're frequently misused.
Test mocks in Ruby are sort of strange. They’re not usually discussed when developers first are introduced to Ruby, usually via the Rails Way of building web applications. Nonetheless, testing is an ingrained part of Rails and Ruby culture in general, so eventually developers learn how to use them. The problem with this is that developers often learn how to use mocks but not why to use them. This post seeks to bridge that gap by discussing why and when to use mocks in a Ruby application.
In her influential book, Practical Objected-Oriented Design, An Agile Primer Using Ruby, Sandi Metz discusses the subject of mock usage in the final chapter of her book. Sandi states that outgoing messages, method invocations on other objects, can be understood as either commands or queries. She believes that outgoing messages which trigger no side-effects should not be tested. These messages are referred to as “queries”. On the other hand, she asserts that method invocations with side-effects should be verified by using mocks. These sorts of messages are called “commands”.
So what does this mean in practice? Imagine a situation in which we’re designing some code for assigning a user account to a league on a fasntasy sports site.
class League
def assign(user:)
assignment_audit = AssignmentAudit.new(user:, league: self)
if assignment_audit.allows_assignment?
user.invite_to(league: self)
true
else
false
end
end
end
class AssignmentAudit
RULES = [FreemiumLimiter, DuplicationDetector, SuspensionDetector]
def initialize(user:, league:)
@user = user
@league = league
end
def allows_assignment?
# Calls `evaluate` on elements of the RULES constant, returning `true` if all
# evaluations return `true`.
end
end
class User
def invite_to(league:)
# Logic for inviting a user to a league
end
end
When calling League#assign, the method sends two outgoing messages. First it sends allows_assignment? to an instance of AssignmentAudit, and then it potentially sends invite_to to an instance of User. In this case, AssignmentAudit#allows_assignment? exclusively returns data to the method being called, making it a query. On the other hand, User#invite_to triggers side-effects relevant to the user receiving the message. It triggers invitations to be sent via whatever channels the application uses. When we test League, the tests for assign could look like this.
RSpec.describe League do
describe '#assign' do
subject { league.assign(user:) }
let(:league) { build(:league) }
context 'when the assignment audit passes' do
let(:user) { build(:user) }
before { allow(user).to receive(:invite_to) }
it { is_expected.to be true }
it 'sends invitations to the user' do
subject
expect(user).to have_received(:invite_to).with(league:)
end
end
context 'when the assignment audit fails' do
let(:user) { build(:user, :invalid_for_assignment) }
it { is_expected.to be false }
end
end
end
Here we’ve followed Sandi Metz’ suggestion for testing outbound messages. The invocation of AssignmentAudit#allows_assignment? is not mocked, meaning it is neither verified nor is its response stubbed. We do however mock invite_to sent to the instance of User. This is a great example of how mocks can make practical improvements to test suites by isolating the object under test. User#invite_to could send invitations via email or push notification, but the consequences of that outbound message aren’t important to the implementation of League#assign. Consequently, if stakeholders ask us to add text message invitations in the future, we won’t have to modify our test for League#assign.
So if mocking the outgoing message sent to the instance of User improved the isolation of the object under test, why wouldn’t we want to also mock the message sent to the instance of AssignmentAudit? Wouldn’t that make our test setup less onerous? Sandi Metz suggests we shouldn’t mock the outgoing message since it’s a query, but let’s look at what the practical implications would be.
Let’s say we did mock the message sent to the instance of AssignmentAudit. We might get specs that look like this.
RSpec.describe League do
describe '#assign' do
subject { league.assign(user:) }
let(:user) { build(:user) }
let(:league) { build(:league) }
let(:audit_double) { instance_double(AssignmentAudit) }
before { allow(AssignmentAudit).to receive(:new).and_return(audit_double) }
context 'when the assignment audit passes' do
before do
allow(audit_double).to receive(:allows_assignment?).and_return(true)
allow(user).to receive(:invite_to)
end
it { is_expected.to be true }
# Omitting the mock expectation on `user` for brevity
end
context 'when the assignment audit fails' do
before { allow(audit_double).to receive(:allows_assignment?).and_return(false) }
it { is_expected.to be false }
end
end
end
With this mock, if the behavior of AssignmentAudit#allows_assignment? changes, we no longer need to modify our test data. On its face, that seems like a good thing. Unfortunately, it is not. In fact, the ability to use these regression tests to assert the behavior of activities like refactoring has now declined.
Consider the situation in which the rules that are used to audit league assignment become more complex. For instance, imagine that the site starts allowing public leagues, and there are new spam filter results which we want to persist to the database. Now AssignmentAudit#allows_assignment? returns an instance of a class with multiple boolean fields. This is a huge problem. AssignmentAudit#allows_assignment? will never return a falsey value now!
Unfortunately since we’ve stubbed AssignmentAudit#allows_assignment? to always return boolean values, our regression suite isn’t doing it’s job! We made a change to AssignmentAudit#allows_assignment? which breaks League#assign, but specs won’t fail like they should. We’ve reduced our ability to confidently refactor code by preventing our test suite from indicating when behavior has changed.
Now, let’s consider another situation. Perhaps we’ve identified a circumstance in which we can create a nice duck type in our application, but we need to change the interface of AssignmentAudit to make it work. So, we change allows_assignment? to the method name we need to satisfy the shared interface. Then we change all references to allows_assignment? in our app folder, and finally we run our regression suite. Uh oh. Every single class that depends on AssignmentAudit#allows_assignment? now has failing tests. This is counter-intuitive. We made an implementation-only refactoring of assign, but somehow the specs are failing. That’s not the behavior we want to see from our test suite, and it’s going to be time consuming and frustrating to go change all of the mocks. That frustration will be multiplied if you aren’t the person who wrote the tests in the first place, or if you wrote them a long time ago.
Keen readers might have noticed that the previous argument neglects an important fact. Command messages, which this article suggests should be mocked, would have the same problem. For instance, if we changed the interface of User so as to rename invite_to, we’d have to go change all of those mocks as well. That’s correct, although I still believe that these situations are different in nature. The behavior of League#assign is completely dependent on the value returned by AssignmentAudit#allows_assignment?. In order to accurately assess the behavior of League#assign, the value of AssignmentAudit#allows_assignemnt? must be accurate. The response of sending AssignmentAudit#allows_assignment? directly impacts the behavior of League#assign. In the case of User#invite_to, sending invite_to to an instance of User is the behavior of League#assign. Other than sending the outbound message, the results of User#invite_to have no further impact on League#assign.
Although it is a misuse of mocks to verify and stub queries, that doesn’t mean tests inherently have to be bound to the data setup required by their dependencies. In fact, there’s a way of inverting this relationship so that dependencies have to abide by an interface decided by their caller. This technique is called Dependency Injection, and it’s a complex and nuanced topic of its own which I’d like to address in a separate article.
Test mocks are a powerful tool which can help prevent a test file from including unrelated behavior. Mocks are a sharp knife though, and using them to stub query messages can lead to tests suites which don’t serve their purpose. It also adds a maintenance burden to the test suite which is cumbersome and unnecessary.