Inicio > English > TDD in Clojure

TDD in Clojure

16 febrero, 2016

(This is an article I published a few weeks ago in Funding Circle’s engineering blog, here)

It’s been a few months since I first approached Clojure, and approaching a functional language for the first time is quite an experience that makes you revisit and reevaluate many of your past experience and know-how with object-oriented (OO) languages.

Testing and TDD/BDD is one of the main foundations of high quality software. But TDD/BDD doesn’t feel as natural in functional languages as it does in OO languages; this has been one of my main issues working with Clojure.

Do I really need to test as much as before? How do I test code composed of functions only? How do I mock? Should I mock? Are functional languages hard to test or am I doing something wrong?

This post is aimed at giving some advice to how to TDD in Clojure, given that many OO techniques both for testing and testability, are not directly applicable to Clojure. This is a summary of the little information I found about adapting TDD to programming in Clojure, together with my own experience. Looking back, I can summarize three main concerns:

  • Differences in TDD flow
  • What testable code means in Clojure
  • How to implement common testing techniques (i.e. mocking)

Warning: These advices come from just few months of experience in Clojure, so they might be incomplete, or just plainly wrong!

Differences in TDD flow

The main difference I’ve found is moving from the classical top-down design, to a bottom-up strategy.

A bottom-up approach feels more natural, helps keeping your impure functions under control, and removes a lot of overhead by avoiding mocking in many situations.

Uncle Bob Martin recommends this approach in the article TDD in Clojure but mainly because of a lack of mocking tools in Clojure by the time he wrote it (2010). That’s no longer the case, but I would still recommend giving a try to a bottom-up approach. For example, try to build your DB layer first, then start going up till you reach the UI. This makes easier to keep all I/O isolated in a single namespace.

In any case, I recommend ignoring the quite spread opinion that functional languages do not need unit testing or only a small amount. I think this belief is based on a misunderstanding of certain properties of some functional languages.

Some functional languages do need much less testing, and people mistakenly confer this to its functional nature. In the case I’m familiar with, Haskell, the reduced amount of testing is due to the combination of its functional paradigm, its side-effect encapsulation (monads), and its static type system with type inference. It’s not that it doesn’t need tests; it’s that it has a built-in testing system called type system. And it only helps with some low-level testing: you still need your usual amount of business logic tests.

But Clojure is a dynamic language. The compiler does not reason about whether a certain function can receive an invalid type, using type inference. Not only that, but Clojure does not help with keeping side effects under control. Because of this, I recommend to keep your test coverage as high as you do with any other OO language. Even if a reduced-test approach is feasible, I don’t think it’s a good approach for a Clojure newbie.

Testability in Clojure

I would say that most of the problems I’ve had with TDD, happened because of low testability of my code.

What does testability mean in Clojure? Most people agree on pure functions being the main testability enhancer in a functional language. Also, dependency injection helps both testability and function pureness. Avoiding untestable code in namespaces and changing your OO design mindset is also important.

Pure functions

A pure function can be defined as a stateless function, or a referencially transparent function, or plainly as a function that has no side effects and, for a given input, always returns the same output. Usually, this mean no state, no I/O, no side effects.

For example, random() is not a pure function, because it is not deterministic, you may get different output when calling it with the same input. A function that writes to a database is not pure, because it has a side effect. A function that calls three impure functions and “branches” the execution is not pure, because it has side effects (well, it depends). A function that makes a decision by reading from the database, is not pure either because it can return different results for the same inputs depending on the content of the database.

So, you should always try to have a clear boundary between pure and impure functions in your code. Usually, you’ll want to keep your impure functions free of business logic and locked up in the lower (DB) and the upper (UI) layers, keeping the rest of the code as a chain of pure functions. Just as a comment, I haven’t managed to do that yet, but the closer to that approach I am, the easier it is to test my code.

Dependency injection

Dependency injection can help in two different ways. The more obvious is, it makes mocking easier: just pass the mock as an injected dependency, and you are good to go. The second way is, it allows you to turn an impure function into a pure one.

Dependency injection in Clojure has its own difficulties in contrast with OO languages. Where in OO languages you typically inject an object or a class as a dependency and store it in your state, in functional languages you only deal with functions being injected into other functions, usually without a state for storing the injected dependencies.

This adds quite an overhead. Given that you have no internal state like an object does, you cannot store your injected dependency in that state. So, you need some other technique to pass the injected function to the receiver function.

Five faces of dependency injection contains a compendium of five different techniques to deal with this. I would recommend either using function arguments, or try a reader monad.

In any case, injection adds an overhead. So, we should try using it only when facing a hard to mock function, or for increasing pureness.

Or try to replace dependency injection with function composition. For example:

(defn should-i-print-this?
  [arg]
  (= arg :yes))

(defn print-it
  [arg decider]
  (when (decider arg)
    (prn arg)))

(print-it :yes should-i-print-this)

but with function composition:

(defn filter-to-print
  [arg]
  (when (= arg :yes)
    arg))

(defn print-it
  [arg]
  (when arg
    (prn arg)))

(def print (comp print-it filter-to-print))

(print :yes)

Deciding whether the composition or the injection version is better depends on the situation. It’s just another tool that might be useful!

Increasing pureness with dependency injection

Dependency injection can help to keep impureness under control, by extracting impure behavior to functions with as little business logic as possible, and then injecting them where needed. This way, we can keep an otherwise impure function pure:

(defn transform-if-exists
  "This function is pure."
  [entity checker]
  (when (checker (:id entity))
    (transform entity)))

(transform-if-exists entity db/checker)

instead of

(defn transform-if-exists
  "This function is not pure"
  [entity]
  (when (db/checker (:id entity))
    (transform entity)))

(transform-if-exists entity)

The first function is pure. You can test it without worrying about fixtures, factories or similar. The second one is not, it needs you to redefine db/checker or use some db fixtures. It’s harder to test. It’s slower to test.

Assembly line vs Object interactions

As OO developers we are used to have code whose execution path branches instead of keeping a single line of execution. We make our objects to speak to a lot of other objects and our programs end up being a composition of objects interacting with each other.

For example, if we need to save a record, then turn some flag on it, and then transform it, we tend to do this:

(defn- save
  [thing]
  (db/save thing))

(defn- mark
  [thing]
  (db/update (assoc thing :mark true)))

(defn- transform
  [thing]
  (assoc thing :name "I'm transformed"))

(defn save-mark-and-transform
  [awesome-thing]
  (db/save awesome-thing)
  (db/mark awesome-thing)
  (transform awesome-thing))

The problem is, this approach is not suited for functional languages because of the amount of side effects.

Instead, we should try to think about our programs as assembly lines, where there is one single flow of execution, and the output of one element is the input of the next one. The previous code example would look like this:

(defn- save
  [thing]
  (db/save thing)
  thing)

(defn- mark
  [thing]
  (let [marked-thing (assoc thing :mark true)]
    (db/update marked-thing)
    marked-thing))

(defn- transform
  [thing]
  (assoc thing :name "I'm transformed"))

(defn save-mark-and-transform
  [awesome-thing]
  (-> awesome-thing
      save
      mark
      transform))

The difference is not only syntactic, but structural. Each function takes previous function’s output as its input. This means, we are enforcing each function to return something, instead of triggering a side effect, and we are keeping a single flow of execution. Also, this mean we can just use composition for building new functions:

(defn- save
  [thing]
  (db/save thing)
  thing)

(defn- mark
  [thing]
  (let [marked-thing (assoc thing :mark true)]
    (db/update marked-thing)
    marked-thing))

(defn- transform
  [thing]
  (assoc thing :name "I'm transformed"))

(def save-mark-and-transform (comp save mark transform))

The more I code this way in Clojure, the easier to maintain my code is (at least for now: ask me in six months, maybe I’m wrong!).

Untestable code in namespaces

Avoid complex code to execute at namespace load time. Nothing more than that.

Testing techniques

I really recommend reading chapter four of Test-Driven Development in Clojure (Niclas Nilsson, 2015). It’s a good compendium of basic testing techniques for Clojure.

Mocking

With dependency injection

If your function accepts its dependencies as arguments:

(defn awesome-function
  "I'm so awesome I have my dependencies injected"
  [some-function stuff1 stuff2]
  (let [stuff {:a-key stuff1
               :another-key stuff2}])
    (some-function stuff))

testing it can be as simple as:

(deftest awesome-function
  (let [mock-function (constantly "whatever")]
    (is (= "whatever" (awesome-function mock-function 3 5)))))

Without dependency injection

If your function does not allow to inject a certain dependency, you can still test it. Assuming we have:

(defn crappy-function
  "I'm so crappy I have my dependencies hardcoded"
  [stuff1 stuff2]
  (let [stuff {:a-key stuff1
               :another-key stuff2}])
    (some-namespace/some-function stuff))

testing it would be:

(deftest crappy-function
  (with-redefs [some-namespace/some-function (constantly "I am really crappy")]
    (is (= "I am really crappy" (crappy-function mock-function 3 5)))))

If we use Midje, a test framework for Clojure, it looks slightly better:

(facts "about `crappy-function`"
  (fact "returns whatever some-namespace/some-function returns"
    (crappy-function 3 5) => "I am really crappy"
    (provided (some-namespace/some-function anything) => "I am really crappy")))

But in any case, it's better to inject the dependency when possible. Using redefinitions is not always feasible —for example, with Clojure protocols— and using dependency injection tends to produce more pure functions, which are easier to test.

Spies

What if you want to check whether a function has been called, and with which
parameters?

Well, in that case you should first realise you are testing a side effect, and decide whether you really need it or you can somehow avoid it.

Using the previous example, and using midje:

(facts "about `crappy-function`"
  (fact "returns whatever some-namespace/some-function returns"
    (crappy-function 3 5) => "I am really crappy"
    (provided (some-namespace/some-function {:a-key 3 :another-key 5}) => "I am really crappy" :times 1)))

Note the different provided statement, together with the number of calls we expect.

You can also spy using clojure.test, using atoms to store the calls to a mock function:

(deftest crappy-function
  (let [calls (atom [])]
    (with-redefs [some-namespace/some-function (fn [arg] (swap! calls conj arg) "I am really crappy")]
      (is (= "I am really crappy" (crappy-function mock-function 3 5)))
      (is (= @calls [{:a-key 3 :another-key 5}])))))

So, this is more or less my compendium of lessons learned during my first six months with Clojure. Many things are probably wrong, or I will change my mind in a few more months. But in the meantime, I hope these ideas are useful to someone else!

A %d blogueros les gusta esto: