Sometimes things break

I was very surprised to find the lispy build broken after I pushed some minor update, like a change to README.md. I mean, how in the world would a few words in README.md break the Elisp tests? Upon investigation, it turned out that only one test was broken ¹. This one:

(ert-deftest clojure-thread-macro ()
  (require 'cider)
  (should
   (string=
    (lispy-with
     "|(map sqr (filter odd? [1 2 3 4 5]))" "2(->>]<]<]wwlM")
    "(->> [1 2 3 4 5]\n  (map sqr)\n  (filter odd?))|")))

The culprit was an update in clojure-mode's indentation. The previous behavior:

(->> [1 2 3 4 5]
   (map sqr)
   (filter odd?))

is now replaced with:

(->> [1 2 3 4 5]
     (map sqr)
     (filter odd?))

Thankfully, the Travis CI in combination with cask is keeping me up to date. Apparently, there were some heated discussions accompanying the change, and there was some reverting going on. Anyway, it looks to me that both approaches have merit: the first one is more logical, since ->> is an operation akin to Elisp's with-current-buffer, where the first argument is different from the others, while the second one is more aesthetically pleasing. Fine with me either way, I'm not complaining:)

Also, the key sequence in the test is pretty ancient. These days I'd probably use: 2(->>C-fd<j<skwAM. I've recently done a more complex Elisp refactoring screencast, check it out here. Later on, I plan to do more Emacs-related screencasts (not just lispy-related) on my channel.

If you haven't tried lispy yet, you're missing out - doing this refactor operation feels like you're doing the 15-number puzzle:

And that's fun in my book. But let me get back to the short overview of the Emacs testing tools that lead me to this post, mainly cask.

cask: what does it do?

According to its own documentation:

Cask is a project management tool for Emacs Lisp to automate the package development cycle; development, dependencies, testing, building, packaging and more.

Yes, please, I'd like to do that! But after the exciting intro sentence, there's very little followup documentation-wise. It took me ages to figure out how cask can actually give me some tangible benefits, since I thought that package.el is enough to maintain my own config (it still is).

tangible benefits of cask

I'd like to be sure that my packages work across recent Emacs versions. I'm using the bleeding edge myself, but people who download my packages from MELPA might be using something older, like emacs-24.3.

So I want to run my tests on both versions. Also, even for just one version, the tests need to be run in a minimum environment, i.e. with only the dependencies loaded, so that my personal configuration does not interfere with the tests.

This is where cask actually shines: it can bootstrap a whole new .emacs.d, separate from your own, just for running tests. It can do it on your machine as well as on Travis CI.

Here's my Cask file for lispy:

(source gnu)
(source melpa)

(package-file "lispy.el")

(files "*.el" (:exclude "init.el" "lispy-test.el"))

(development
 (depends-on "helm")
 (depends-on "ace-jump-mode")
 (depends-on "noflet")
 (depends-on "iedit")
 (depends-on "multiple-cursors")
 (depends-on "cider")
 (depends-on "slime")
 (depends-on "geiser")
 (depends-on "projectile")
 (depends-on "s")
 (depends-on "highlight"))

And here's the Makefile:

EMACS = emacs
# EMACS = emacs-24.3

CASK = ~/.cask/bin/cask
CASKEMACS = $(CASK) exec $(EMACS)
LOAD = -l lispy-inline.el -l lispy.el -l lispy-test.el

all: test

cask:
    $(shell EMACS=$(EMACS) $(CASK))

compile:
    $(CASKEMACS) -q  $(LOAD) lispy.el \
    --eval "(progn (mapc #'byte-compile-file '(\"lispy.el\" \"lispy-inline.el\" \"le-clojure.el\" \"le-scheme.el\" \"le-lisp.el\")) (switch-to-buffer \"*Compile-Log*\") (ert t))"

test:
    $(CASKEMACS) -batch $(LOAD) -f ert-run-tests-batch-and-exit

clean:
    rm -f *.elc

As you can see, the Makefile has two separate testing targets: an interactive one (compile) and a non-interactive one (test). There's actually some validity to this, since it happened once that the same tests we failing in non-interactive mode, but passing in interactive mode. Also, compile obviously compiles, testing for compilation warnings/errors. I can change the Emacs version at the top, although I don't have to do it too often.

Finally, here's .travis.yml:

language: emacs-lisp
env:
  matrix:
    - EMACS=emacs24

before_install:
  - sudo add-apt-repository -y ppa:cassou/emacs
  - sudo apt-get update -qq
  - sudo apt-get install -qq $EMACS
  - curl -fsSkL --max-time 10 --retry 10 --retry-delay 10 https://raw.github.com/cask/cask/master/go | python

script:
  - make cask
  - make test

So each time I push a change to github, Travis CI will

• install emacs24
• install cask
• install the packages from MELPA:
  • helm
  • ace-jump-mode
  • noflet
  • iedit
  • multiple-cursors
  • cider
  • slime
  • geiser
  • projectile
  • s
  • highlight
• load Emacs with these packages
• load lispy-test.el and run it
• show up green if make test returned 0

Seems a bit wasteful, but it's the Cloud - what can you do?