elf-mode - view the symbol list in a binary

Recently, I've been looking at libigl. I didn't manage to fully figure out their CMake build system for tutorials: although each tutorial has a CMakeLists.txt, it's only possible to build them all at once.

So I decided to replace CMakeLists.txt with a good-old Makefile; how hard can it be? Concerning includes, not at all hard: the missing files are found with counsel-locate and added to the include path.

But I had some trouble matching a missing ld dependency to a library file. Fixed it with a bunch of googling and guesswork; I still wonder if there's a better way. But in the process, I've found this useful command:

readelf --syms libGL.so

which produces e.g.:

Symbol table '.dynsym' contains 2732 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 000000000004faf0     0 SECTION LOCAL  DEFAULT    8
     2: 00000000000e8f20     0 FUNC    GLOBAL DEFAULT   11 glGetIntegerui64i_vNV
     3: 00000000000e13e0     0 FUNC    GLOBAL DEFAULT   11 glGetMultiTexEnvfvEXT
     4: 00000000000d7440     0 FUNC    GLOBAL DEFAULT   11 glProgramUniform2uiv
     5: 00000000000cfdc0     0 FUNC    GLOBAL DEFAULT   11 glMultiTexCoord3sv

This is a pretty good representation of a binary file: in this example, instead of one megabyte of gibberish I see a bit more than 2732 lines describing the functions this file uses and provides.

Viewing the symbol list automatically

I liked the above representation so much that I want to see it by default. In Emacs, it's pretty easy to do with auto-mode-alist:

(add-to-list 'auto-mode-alist '("\\.\\(?:a\\|so\\)\\'" . elf-mode))

The above code instructs Emacs to call elf-mode function whenever the file name ends in *.a or *.so.

And here's the body of elf-mode:

(defvar-local elf-mode nil)

;;;###autoload
(defun elf-mode ()
  (interactive)
  (let ((inhibit-read-only t))
    (if elf-mode
        (progn
          (delete-region (point-min) (point-max))
          (insert-file-contents (buffer-file-name))
          (setq elf-mode nil))
      (setq elf-mode t)
      (delete-region (point-min) (point-max))
      (insert (shell-command-to-string
               (format "readelf --syms %s" (buffer-file-name)))))
    (set-buffer-modified-p nil)
    (read-only-mode 1)))

The idea is very simple: elf-mode is a toggle function that replaces the buffer contents with the shell command output. It carefully uses read-only-mode and set-buffer-modified-p so that the file will not be overwritten by accident with the symbol names.

Using autoload to avoid overhead

As you might imagine, looking at binaries isn't really a common task. Is it worth to be dragging this code around from now on, loading it on each start? The answer is yes, of course. Since the actual cost is negligible until the feature is used.

If you look above, elf-mode has an ;;;###autoload cookie before it. The cookie results in this line in my loaddefs.el:

(autoload 'elf-mode "modes/ora-elf" "" t nil)

My init.el always loads loaddefs.el, but never loads ora-elf.el where the function is defined. That file is only loaded when the function elf-mode is called for the first time. The above autoload statement simply instructs Emacs to load a particular file when elf-mode needs to be called.

When you use the package manager, the autoloads file is generated and loaded for you automatically:

$ tree elpa/ace-link-20160811.112/

elpa/ace-link-20160811.112/
├── ace-link-autoloads.el
├── ace-link.el
├── ace-link.elc
└── ace-link-pkg.el

0 directories, 4 files

Here, the package manager will always load ace-link-autoloads.el, which instructs Emacs to load ace-link.el when one of the ;;;###autoload functions is called and ace-link.el isn't yet loaded.

As an example of how useful delayed loading is: my 6000 line config starts in 1.8 seconds. About 40% of that time is spent on (package-initialize), which I assume is the package manager loading all those *-autoloads.el files that I have in my elpa/.

Outro

Let me know if there's interest to have elf-mode on MELPA. Also, if anyone knows how to set mode automatically based on the first few chars of the file (all binaries seem to start with ^?ELF), I'd like to know that as well. Happy hacking!

Swipe all the files!

If you've ever tried the swiper-all from swiper, forget everything about it. The command was super-awkward, since it had to parse all your open files before giving you a chance enter anything, resulting in dozens of seconds before the prompt.

Recently, I've had some time to examine and improve it and the result looks very promising. The new command is now async, which means there's no delay before the prompt comes up. Here's a result I got with no delay while having around 50 buffers open:

The shortcut I'm using:

(global-set-key (kbd "C-c u") 'swiper-all)

For efficiency's sake a small trade off had to be made: the line numbers are no longer displayed. This actually results in an advantage that you can select different candidates on the same line.

There are still a few things I plan to try for the new command, like adding file-less buffers, caching for incremental regexes and maybe even newlines in wild cards, but even now it seems very usable. So give it a try, enjoy and happy hacking!

Bookmark the current window layout with Ivy

Today's post is about the newest feature related to the ivy-switch-buffer command. If you use ivy-mode, you're probably already using ivy-switch-buffer since it overwrites the built-in switch-to-buffer.

The cool thing about ivy-switch-buffer is that it's not only buffers that are offered for completion. Other buffer-like entities can be there as well: bookmarks, recently opened files (recentf), and finally, window layouts. Since all of those are relatively the same concept, it's very convenient to have them all in one place available for completion.

Here are the relevant settings:

;; Enable bookmarks and recentf
(setq ivy-use-virtual-buffers t)

;; Example setting for ivy-views
(setq ivy-views
      `(("dutch + notes {}"
         (vert
          (file "dutch.org")
          (buffer "notes")))
        ("ivy.el {}"
         (horz
          (file ,(find-library-name "ivy"))
          (buffer "*scratch*")))))

I did mention ivy-views before in the ivy-0.8.0 release post. But now, instead of setting ivy-views by hand, you can also bind ivy-push-view to a key and store as many window configurations as you like, really fast.

What gets stored:

• The window list - all windows open on the current frame.
• The window splits relative to each other as a tree. Currently, the size of the split isn't saved, all windows are split equally.
• The point positions in each window. If you use just one window, you've got something similar to bookmark-set.

Recommended key bindings

Here's what I use currently:

(global-set-key (kbd "C-c v") 'ivy-push-view)
(global-set-key (kbd "C-c V") 'ivy-pop-view)

Typical workflow

Suppose I have two files open: the file 2016-06-23-ivy-push-view.md and the _posts directory. By pressing C-c v I am prompted for a view name with the default being e.g. {} 2016-06-23-ivy-push-view.md _posts 2.

I can still name the view however I want, but I typically just press RET. The parts of the automatic view name are:

• {} - this is a simple string marker to distinguish the views in the buffer view. If I enter only {} into ivy-switch-buffer prompt, the candidates will normally filter to only views, since very rarely will a file or a buffer name match {}.
• 2016-06-23-ivy-push-view.md _posts is the list of buffers stored in the view. This view has only two buffers, but ivy-push-view can handle as many windows as you can cram into a single frame.
• 2 means that I already have two views with the same buffers, each new view with the same buffers gets an increased number for the suggested name. And it's not useless to have many views for the same buffers, since the views also store point positions, not just the window list.

Here's the beauty of it for me: when I type _posts into ivy-switch-buffer I can chose to open the _posts directory in a variety of ways:

• If the buffer is currently open, I can just switch there.
• If the buffer is currently closed, I can re-open it, thanks to recentf.
• I can open the buffer as part of a stored view(s) in ivy-views.

Finally, if I decide that I don't need a particular view any more, I can delete it with C-c V (ivy-pop-view). It's possible to delete many views at once by pressing C-M-m (ivy-call), as usual with most ivy completion functions.

Breaking API change

While implementing ivy-set-view I decided that the current way alist collections are handled together with actions is sub-optimal. Here's the new way of working:

(let (res)
  (ivy-with
   '(ivy-read "test: "
     '(("one" . 1) ("three" . 3))
     :action (lambda (x) (setq res x)))
   "t C-m")
  res)
;; =>
;; ("three" . 3)

Previously, the return result would be 3, i.e. the cdr of the selected candidate. Any code using ivy-read with an alist-type collection will break. I fixed all instances in counsel.el, and there actually aren't too many uses in the published third party packages.

A simple fix to the problem is to use cdr in the action function. Additionally, having more information available in the action function will serve to improve the code.

Set an Emacs variable with double completion

I'd like to show off a certain Elisp productivity booster that I've had in an unfinished state for a while and finished just today.

A large part of tweaking Elisp is simply setting variables. The new command, counsel-set-variable, allows to set them quite a bit faster.

Completion stage 1:

First of all, you get completion for all variables that you have defined:

Completion stage 2:

Once a symbol is selected, the code checks whether the symbol is a defcustom with type 'boolean or 'radio. Since then it is possible to offer all values that the symbol is allowed to become for completion.

For example, here's a typical 'radio-type definition:

(defcustom avy-style 'at-full
  "The default method of displaying the overlays.
Use `avy-styles-alist' to customize this per-command."
  :type '(choice
          (const :tag "Pre" pre)
          (const :tag "At" at)
          (const :tag "At Full" at-full)
          (const :tag "Post" post)
          (const :tag "De Bruijn" de-bruijn)))

And here's a completion screen offered for this variable:

It is worth noting that the current value of the variable is pre-selected, to give a nice reference point for the new setting.

In case the symbol isn't a boolean or a radio

Then you get a completion session similar to M-x read-expression, but with the initial contents already filled in. For example:

The read-expression part combines well with this setting in my config:

(defun conditionally-enable-lispy ()
  (when (eq this-command 'eval-expression)
    (lispy-mode 1)))

(add-hook
 'minibuffer-setup-hook
 'conditionally-enable-lispy)

Here's a series of commands using lispy-mode that I would typically use for the screenshot above, to set ivy-re-builders-alist to a new value:

1. C-f (forward-char) to get into special.
2. -e (lispy-ace-subword) to mark the plus part of the code.
3. C-d (lispy-delete) to delete the active region.

After the C-f -e C-d chain of bindings, the minibuffer contents become:

(setq ivy-re-builders-alist '((t . ivy--regex-|)))

I press C-M-i (completion-at-point) to get completion for all symbols that start with ivy--regex-. Since I have ivy-mode on, C-M-i starts a recursive completion session. I highly recommend adding these settings to your config:

;; Allow to read from minibuffer while in minibuffer.
(setq enable-recursive-minibuffers t)

;; Show the minibuffer depth (when larger than 1)
(minibuffer-depth-indicate-mode 1)

Finally, I would e.g. select e.g. ivy--regex-fuzzy and RET RET to finalize the eval. The first RET exits from completion-at-point and the second RET exits from counsel-set-variable.

Outro

I think this command, especially the newly added read-expression part, is quite a bit faster than what I did before. That is switching to *scratch* and typing in the setq manually and evaluating with C-j. Here's my binding for the new command:

(global-set-key (kbd "<f2> j") 'counsel-set-variable)

It's not very mnemonic, but it's really fast. Just a suggestion, in case you don't know where to bind it. Happy hacking!

Ivy 0.8.0 is out

Intro

Ivy is a completion method that's similar to Ido, but with emphasis on simplicity and customizability.

New package names

Changes on MELPA

Due to multiple requests, in an attempt to simplify things a new package ivy has been released on MELPA. The old package swiper, which used to provide most of the ivy features, now only provides swiper.el and depends on ivy. The third package counsel, which provides most of the goodies using ivy hasn't been changed and is still on MELPA. All three packages have the version 0.8.0 currently.

To reiterate the dependencies:

• ivy depends on Emacs version larger than 24.1, preferably at least 24.3 (the most common one bundled currently with Linux distributions).
• swiper depends on ivy and provides basically 3 commands: swiper, swiper-multi and swiper-all.
• counsel depends on swiper and provides around 50 commands for all kinds of stuff. My favorites are counsel-M-x, counsel-git-grep, counsel-rhythmbox and counsel-grep-or-swiper.

Changes on GNU ELPA

On GNU ELPA, a single package ivy-0.8.0 has replaced the previous stable version swiper-0.7.0. This package provides all the files combined of the three separate MELPA packages.

Release summary

The release consists of 282 commits over 5 months by 15 authors. The detailed Changelog is available here, thanks to the ever useful Org mode export. The raw Org file is in doc/Changelog.org in the main repository.

The detailed documentation is available as an (ivy) Info node and also in HTML form here. If anyone wants to document something that's missing there, I'd appreciate the help: simply edit doc/ivy.org and send me a PR.

Release highlights

Below, I'll highlight some of the new features.

Allow to compose collections

For example, to stack the top 10 elements of recentf on top of counsel-locate, use this code:

(defun small-test ()
  (cl-subseq recentf-list 0 10))

(ivy-set-sources
 'counsel-locate
 '((small-test)
   (original-source)))

Here, (original-source) represents the async candidates of counsel-locate. All extra sources are static - each function is called once to generate a list of strings, which will be filtered later.

See #373 for more info.

Improved documentation

If you're not yet familiar with Ivy, you can get a quick reference card by pressing C-h m during any completion session. It will pop up an Org-mode buffer that describes most of the minibuffer key bindings.

Additionally, C-o (hydra-ivy/body), which serves a quick reference as well, received a small restructuring and a new binding. Press D to go to the definition of this hydra. This is useful to see what each key does, you might even want to customize some of it.

Completion in region

From now on, ivy-mode will also set completion-in-region-function. This means that functions like:

• C-M-i complete-symbol in many major modes,
• TAB while in the M-: (eval-expression) minibuffer,
• TAB in a shell buffer,

will use ivy for completion.

Many improvements to ivy-occur-mode

You can "permanently" save any completion session by pressing C-c C-o (ivy-occur). This will generate a new buffer in ivy-occur-mode with all your current candidates inserted there. Clicking or pressing f on any of the candidates in that buffer will result in the appropriate action being called with that candidate.

The *ivy-occur ...* buffers can actually be customized per collection type. Specifically for swiper, counsel-git-grep, counsel-grep and counsel-ag, the customizations are already in place that allow you to:

• Edit the buffer with wgrep by pressing C-x C-q (ivy-wgrep-change-to-wgrep-mode).
• Refresh the buffer due to the original files being changed by pressing g (ivy-occur-revert-buffer).

The second feature is often useful to me when I want to somehow change a symbol throughout the project. First I make a list of all occurrences via e.g. swiper and ivy-occur. After I went through some of the occurrences, I can press g to refresh the search for the same symbol and see how many I still have left.

Yet another cool feature is to press c (ivy-occur-toggle-calling) to toggle calling the action after each line movement and cycle through candidates by holding either j (ivy-occur-next-line) or k (ivy-occur-previous-line).

ivy-set-action can work on all commands

Here's the code I'm using currently in my config:

(defun ivy-insert-action (x)
  (with-ivy-window
    (insert x)))

(ivy-set-actions
 t
 '(("I" ivy-insert-action "insert")))

This allows me to press M-o I to insert the current candidate into the buffer. For instance, if I want to quote an Emacs command, I can M-x (counsel-M-x), select the command I want and press M-o I to insert it instead of calling it.

Virtual views in ivy-switch-buffer

Here, "virtual" buffer means something that's close to a buffer but not an actual buffer. If you were using the setting ivy-use-virtual-buffers, you'd have your bookmarks and recentf items available to you as virtual buffers.

The new feature allows to select a whole window configuration with many buffers inside nested horizontal and vertical splits from ivy-switch-buffer.

To use it, set ivy-views, since it's nil by default. For instance, here's what I have in my config:

(setq ivy-views
      '(("dutch + notes {}"
         (vert
          (file "dutch.org")
          (buffer "notes")))
        ("ivy {}"
         (horz
          (file "ivy.el")
          (buffer "*scratch*")))))

For a more elaborate and detailed use, see this post by Manuel Uberti.

Magic slash in file name completion

From now on, if you want to enter a directory, simply select a candidate which is a directory and press /. You can still use the old binding C-j (ivy-alt-done), but / is shorter and easier to get used to if you're switching from Ido.

Note that this does not prevent the use of old functionality like:

• // to enter the root directory,
• /ssh: RET to connect via TRAMP.

A better way to search with counsel-grep-or-swiper

If you've ever been annoyed with the long start-up time of swiper in huge buffers, switch to this setting:

(global-set-key "\C-s" 'counsel-grep-or-swiper)

This command will use swiper for small buffers, and counsel-grep for large buffers.

Something very similar to this command was highlighted in this post by Karl Voit.

Just to give you an idea of how fast counsel-grep is:

• It has 0s start-up time, since it's async.
• For a two million line file weighing 50MB produced by copying org.el a few times, it takes 0.2s to find the 17,664 occurrences of the word hook. It still takes 7.5s to search for org in that file, simply because there are 500,000 candidates and it takes time for Emacs to simply receive that input.

A list of all new commands

The new commands since 0.7.0 are: counsel-tmm, counsel-imenu, counsel-decbinds, counsel-list-processes, ivy-switch-buffer-other-window, counsel-git-stash, counsel-git-log, counsel-pt, counsel-linux-app, counsel-ace-link, counsel-esh-history, counsel-shell-history, counsel-grep-or-swiper.

If any of those sound interesting, go ahead and try them out.

Outro

Thanks to all the contributors. Happy hacking!