Emacs package management

Lately, I've been spending some time to automate and publish my Emacs config. Being able to quickly reproduce your config has many advantages, the main one being that you no longer have to spend time to make your config reproducible.

Happily, most of my config is already published in many packages, I only have to figure out a nice layer to glue them together. Below, I'll show some automation for the packages managed by package.el.

Step 1: get the main directory

This is an important step that many other peoples' configs miss, even the ones that are designed to be distributed. You can't just assume that the config will be located in ~/.emacs.d and rely on Emacs defaults. Instead, it's nice to be able to clone the config into a random directory and launch an Emacs from there, without messing with the currently installed Emacs.

It's also useful for having multiple repositories for different versions of Emacs. ELPA packages are byte-compiled, and the byte code can be incompatible between versions (for instance, 24.3 and 24.4). Having two independent checkouts with ELPA directory auto-generated really helps in that case.

So here is the code to get the main directory and define an ELPA directory with respect to that:

(defconst emacs-d
  (file-name-directory
   (file-chase-links load-file-name))
  "The giant turtle on which the world rests.")

(setq package-user-dir
      (expand-file-name "elpa" emacs-d))

Step 2: decide what you like

Next, I initialize the package and define some of the packages that I like, omitting the dependencies that they bring. Note that the code of the whole post is stored in a separate file packages.el that is not intended to be loaded on start up, so it's fine to call package-refresh-contents here:

(package-initialize)
(setq package-archives
      '(("melpa" . "http://melpa.milkbox.net/packages/")
        ("gnu" . "http://elpa.gnu.org/packages/")))
(package-refresh-contents)

(defconst ora-packages
  '(auto-compile auto-yasnippet ace-link ace-window
    company eclipse-theme flx-ido function-args
    headlong ido-occasional ido-vertical-mode lispy
    magit smex swiper use-package guide-key
    powerline projectile slime cider worf
    org-download make-it-so ukrainian-holidays
    netherlands-holidays j-mode)
  "List of packages that I like.")

Step 3: install and upgrade

The install step is pretty straightforward: install a package unless it's already installed. I tried to do something fancier for the upgrade, but in the end it was much more simple to just call the interactive interface. The last two lines are basically equivalent to pressing Uxy interactively:

;; install required
(dolist (package ora-packages)
  (unless (package-installed-p package)
    (package-install package)))

;; upgrade installed
(save-window-excursion
  (package-list-packages t)
  (package-menu-mark-upgrades)
  (package-menu-execute t))

Step 4: make it callable

Finally, I just create a Makefile with the following contents:

emacs ?= emacs
upgrade:
    $(emacs) -batch -l packages.el

run:
    $(emacs) -Q -l init.el

up: upgrade run

Thanks to the first line, I can issue stuff like this on the shell:

emacs=emacs24 make up

This will use the emacs24 executable, instead of whatever emacs points to. Since the up target depends on upgrade and run targets, they will be executed in that order:

• the upgrade will install / upgrade all packages in a non-interactive Emacs
• the run target will start an interactive Emacs with already updated packages

I really like putting stuff in Makefiles, since they are very flexible, yet so easy to call. In the very same Makefile, I have a profile target from the post on profiling Emacs start up. I also wrote two packages related to Makefiles: helm-make and make-it-so. The latter one is actually very interesting and deserves its own post, I should maybe just clean it up a bit.

Outro

I'll just cite Gall's law here:

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.

I think it applies from both sides w.r.t. my Emacs config: it kind of works, but I really wish it was reproducible from the start, before making it complex.

A new Swiper demo on Youtube

Youtube video

Today, I've fixed a few bugs in both swiper and ivy. Finally, the number of candidates display has also been added. You can see the whole thing in the one minute video demo.

Here are the bindings that I'm using:

(global-set-key "\C-r" 'swiper)
(global-set-key "\C-s" 'swiper)

Integration tests

I've also added some integration tests, if you're interested. I didn't know how to do exactly this type of testing before (when there's input from minibuffer). Turns out, it's pretty easy to do using execute-kbd-macro:

(require 'ert)

(defvar ivy-expr nil
  "Holds a test expression to evaluate with `ivy-eval'.")

(defvar ivy-result nil
  "Holds the eval result of `ivy-expr' by `ivy-eval'.")

(defun ivy-eval ()
  "Evaluate `ivy-expr'."
  (interactive)
  (setq ivy-result (eval ivy-expr)))

(global-set-key (kbd "C-c e") 'ivy-eval)

(defun ivy-with (expr keys)
  "Evaluate EXPR followed by KEYS."
  (let ((ivy-expr expr))
    (execute-kbd-macro
     (vconcat (kbd "C-c e")
              (kbd keys)))
    ivy-result))

(ert-deftest ivy-read ()
  (should (equal
           (ivy-read "pattern: " nil)
           nil))
  (should (equal
           (ivy-read "pattern: " '("42"))
           "42"))
  (should (equal
           (ivy-with '(ivy-read "pattern: " '("blue" "yellow"))
                     "C-m")
           "blue"))
  (should (equal
           (ivy-with '(ivy-read "pattern: " '("blue" "yellow"))
                     "y C-m")
           "yellow"))
  (should (equal
           (ivy-with '(ivy-read "pattern: " '("blue" "yellow"))
                     "y DEL b C-m")
           "blue"))
  (should (equal
           (ivy-with '(ivy-read "pattern: " '("blue" "yellow"))
                     "z C-m")
           nil)))

Outro

Give the package a try, if you haven't yet. You can get it from MELPA.

Find file in a Git repo with ivy

I'm really enjoying using ivy for matching stuff.

Here is today's addition:

(defun couns-git ()
  "Find file in the current Git repository."
  (interactive)
  (let* ((default-directory (locate-dominating-file
                             default-directory ".git"))
         (cands (split-string
                 (shell-command-to-string
                  "git ls-files --full-name --")
                 "\n"))
         (file (ivy-read "Find file: " cands)))
    (when file
      (find-file file))))

This one will allow you to find a file in your Git repository. I've bound it like this:

(global-set-key (kbd "C-c f") 'couns-git)

Here's how it looks like for selecting a file in the Emacs repo:

The speed isn't an issue for 3500 candidates, although I should try to add the number of candidates display pretty soon. It's just that there isn't a good spot in the minibuffer to show that.

I've also updated ivy-next-line and ivy-previous-line to switch to the previous history element in case ivy-text is empty. This is the exact behavior of isearch, so if you bind swiper to C-s and C-r like I do, you'll find that C-s C-s and C-r C-r work as expected. Thanks to @johnmastro for the suggestion.

Here's the current state of the keymap:

(defvar ivy-minibuffer-map
  (let ((map (make-sparse-keymap)))
    (define-key map (kbd "C-m") 'ivy-done)
    (define-key map (kbd "C-n") 'ivy-next-line)
    (define-key map (kbd "C-p") 'ivy-previous-line)
    (define-key map (kbd "C-s") 'ivy-next-line)
    (define-key map (kbd "C-r") 'ivy-previous-line)
    (define-key map (kbd "SPC") 'self-insert-command)
    (define-key map (kbd "DEL") 'ivy-backward-delete-char)
    (define-key map (kbd "M-<") 'ivy-beginning-of-buffer)
    (define-key map (kbd "M->") 'ivy-end-of-buffer)
    (define-key map (kbd "M-n") 'ivy-next-history-element)
    (define-key map (kbd "M-p") 'ivy-previous-history-element)
    (define-key map (kbd "C-g") 'minibuffer-keyboard-quit)
    map)
  "Keymap used in the minibuffer.")

You can also try counsel for completing Elisp and couns-clj for completing Clojure. As you can see, the implementation is very simple: you just get a list of strings, and you're done.

If you want to implement some ivy completion for your favorite mode, I recommend to find the corresponding ac-source and see where it gets its list of strings.

More Info

I really liked Marcin Borkowski's post on Info dispatch, where he describes how to open several *info* buffers for the most commonly used manuals. However, as I tried to call one of these functions today, I forgot the key binding.

Hydra to the rescue!

Here's what I've come up with:

(defun ora-open-info (topic bname)
  "Open info on TOPIC in BNAME."
  (if (get-buffer bname)
      (progn
        (switch-to-buffer bname)
        (unless (string-match topic Info-current-file)
          (Info-goto-node (format "(%s)" topic))))
    (info topic bname)))

(defhydra hydra-info-to (:hint nil :color teal)
  "
_o_rg e_l_isp _e_macs _h_yperspec"
  ("o" (ora-open-info "org" "*org info*"))
  ("l" (ora-open-info "elisp" "*elisp info*"))
  ("e" (ora-open-info "emacs" "*emacs info*"))
  ("h" (ora-open-info "gcl" "*hyperspec*")))

(define-key Info-mode-map "t" 'hydra-info-to/body)

The Plan

I already have the <f1> i key binding hard wired to my fingers. So after I'm in any Info buffer, I plan to press t and get this dispatch:

Getting the Hyperspec

I've heard people lauding HTML while dumping on Info. I'd suggest them to compare the Common Lisp Hyperspec web site to this info file extracted from there. Their contents are identical, but it's easier and more pleasant to use Info.

After I downloaded the file, I extracted it to ./etc/info/gcl.info, where . is my emacs-d. Then I just added this directory to the Info path:

(setq Info-additional-directory-list
      (list (expand-file-name "etc/info/" emacs-d)))

Outro

Reading Info is pleasant and educational. In case you're new to Info, there's Info for Info. In Info format! Just press <f1> i h.

Try J

Intro

I imagine that if you're reading this blog, you like to tinker with Emacs. And people who like to tinker with stuff probably also like to learn new programming languages, just for fun. In that case, if you ever want to learn a non-mainstream language, I highly recommend J.

From its homepage:

J is a modern, high-level, general-purpose, high-performance programming language. J is portable and runs on Windows, Unix, Mac, both as a GUI and in a console. J systems can be installed and distributed for free.

The Appetizer

Things J has going for it that are high on my list:

• GPL licensed source (although it's gibberish to read)
• a REPL
• short code
• high performance

What attracted me to J in the first place is that it consistently has the shortest solutions (and fast-running) on Project Euler. What kept me going after the initial wow-effect, was the extremely elegant standard functions implementation dividing things into verbs, adverbs, and conjunctions.

Something impressive: a Sudoku solver

This code is taken from the ob-J page that I wrote some time ago (you can find a lot of additional info there):

#+begin_src J :exports both
i =: ,((,|:)i.9 9),,./,./i.4$3
c =: (#=[:#~.)@-.&0
t =: [:(([:*/_9:c\])"1#])i&{+"1 1(>:i.9)*/[:i&=i.&0
r =: [:,`$:@.(0:e.,)[:;(<@t)"1
s =: 9 9&$@r@,
]m =: 9 9 $"."0'200370009009200007001004002050000800008000900006000040900100500800007600400089001'
s m
#+end_src

#+RESULTS:
#+begin_example
2 0 0 3 7 0 0 0 9
0 0 9 2 0 0 0 0 7
0 0 1 0 0 4 0 0 2
0 5 0 0 0 0 8 0 0
0 0 8 0 0 0 9 0 0
0 0 6 0 0 0 0 4 0
9 0 0 1 0 0 5 0 0
8 0 0 0 0 7 6 0 0
4 0 0 0 8 9 0 0 1

2 8 4 3 7 5 1 6 9
6 3 9 2 1 8 4 5 7
5 7 1 9 6 4 3 8 2
1 5 2 4 9 6 8 7 3
3 4 8 7 5 2 9 1 6
7 9 6 8 3 1 2 4 5
9 6 7 1 4 3 5 2 8
8 1 3 5 2 7 6 9 4
4 2 5 6 8 9 7 3 1
#+end_example

It's pretty amazing that the whole implementation, not counting the example input matrix, takes only 169 characters. You can also see how functional the language is.

Something simpler: a factorial

To have a more simple example, here's how to write down incrementally the factorial of 20:

#+begin_src J
i.20
#+end_src

#+RESULTS:
: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

#+begin_src J
1 + i.20
#+end_src

#+RESULTS:
: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

#+begin_src J
*/ 1 + i.20
#+end_src

#+RESULTS:
: 2432902008176640000

The spaces are optional, I've only included them to make the code more clear.

Something visual: a sine plot

#+begin_src J
load 'plot'
plot 1 o. 0.1 * i.200
#+end_src

By simply pressing C-c C-c on this source block you get this image generated and opened in your browser:

The Emacs Tooling

As mentioned above, there's org-mode babel support for J, including session interaction. More importantly, there is j-mode. The same package also provides a REPL via jconsole.

Additionally, I wrote down a learning/assisting tool helm-j-cheatsheet. Here's one of the screenshots:

It allows to:

• insert a function by English name
• look up the English name of the function by symbol
• open the J documentation for a function

Outro

J is a very cool language to try and I hope you give it a go. I haven't yet managed to find a nice use for it, but you could say that, just as learning LISP, learning J can make you better at other languages.