My name is Charlie Groves and I have a stunning confession to make: I wrote a terminal pager in 2022.
It’s not 1984, when the best known pager was created. It’s 2022. Why now?
Just one word: hyperlinks.
HTML popularized hyperlinks in the early 1990s. For the next 25 years, terminal authors slept on linking technology. Then, in 2017, the authors of Gnome Terminal and iTerm2 added terminal codes for hyperlinks. In the intervening years many other excellent terminals added support.
That critical mass of terminals meant more applications can emit links. Your compiler or test runner or recursive grep could link to a character in a line in a file they thought you might find interesting. I wanted to press one key in my editor to compile and test my code in a separate terminal pane and then navigate to the first error if there was one.
I wrote ate to do that. Before we get into what ate does, let’s talk about how terminal links work and make it possible.
Terminal Hyperlinks #
Like setting a color and many other terminal commands, links are started by printing an escape sequence to the terminal stream.
\e]8;;file://feh/home/groves\e\\ starts a link to
/home/groves on a host named
The terminal will know that text printed after that links there.
\eis the C escape sequence for the escape character in ASCII. The escape character starts terminal escape sequences.
\eis a common way to see it in when writing them out. Escape is the 27th character in ASCII, so you may also see it as 0x1b or 033, which are 27 in hex or octal respectively.
\\is the C escape for
\e\\is called “string terminator” in terminals and it ends many escape sequences.
The other characters are literal ASCII characters.
\e]8;;file://feh/home/groves\e\\ to the equivalent HTML,
<, it tells the parser that a tag is coming
a, it indicates what the tag is
href=file://feh/home/groves", it’s the data for that tag
>, it says we’re out of the tag and back to stuff to show the user
To close a hyperlink in the terminal print
It’s equivalent to
</a> in HTML.
Text printed after that won’t be linked, at least not until another link is started.
To put it all together, run
printf '\e]8;;https://sevorg.org/posts/why_ate/\e\\Why Ate\e]8;;\e\\' in your terminal.
If your terminal supports links, it will print the text “Why Ate” linking to this page.
Since you’re already on this page, I hope that’s the most useless terminal hyperlink you encounter.
Hyperlink Params #
You may have been wondering about the
; before the
file:// URI in the escape sequence.
Terminal hyperlinks may also include another chunk of data before that
Params are key-value pairs separated by
line=12:column=5 would create a key
line with the value
12 and a key
column with the value
5 in a hyperlink.
That would look like this in a full hyperlink escape sequence:
An application reading that link could open the file in your editor at the specific line and column given in the params.
Put a Link On It #
That’s all there is to creating links.
As someone authoring a program that runs in the terminal, if you’re printing something that relates to a local file, you put a
file:// link on it.
If you’re printing something that has a home on the web, you put an
http:// link on it.
Some terminal applications are already doing that:
- delta links to files in git diffs and revisions in git log.
- ls 8.28 links to files if given a
- gcc 10 links to the description of a problem it finds in your code.
There’s a bit of a bootstrapping problem in getting widespread application support for linking though. Until enough users want hyperlinks, application authors won’t add them. Until enough applications add hyperlinks, users won’t have a reason to use them. Application authors will likely still add them since it’s a cool and useful feature, but it will take a while for it to spread.
Luckily, since terminal applications are directly manipulable text, you don’t need to wait for an application to emit links itself. Kovid Goyal, the author of the kitty terminal, wanted links to the matches in ripgrep. He created a wrapper around ripgrep that inserts those links.
The link insertion code is straightforward:
- take every line of ripgrep’s output
- extract which file we’re matching
- wrap search results in a link to the matched line in the file
- print ripgrep’s original output with those links
With that, you can run
hyperlinked_grep anywhere you’d run ripgrep,
pass the same flags you’d pass to ripgrep,
get the same displayed output you’d get from ripgrep,
but also get links embedded in that output.
We don’t need to track the current file in cargo output, so the link insertion code is straightforward enough to walk through here:
# Use regular expressions to match certain output from cargo # Match assertion failures eg # right: `0`', src/main.rs:1012:9 assert_pat = re.compile(br' +(?:left|right):.+ (.+):(\d+):(\d+)') # Match backtrace lines eg # at /build/rustc-1.63.0-src/library/core/src/panicking.rs:181:5 btrace_pat = re.compile(br' +at (.+):(\d+):(\d+)') # Match compile errors eg # --> src/main.rs:55:1 num_pat = re.compile(br' +--> (.+):(\d+):(\d+)') # This gets called for every line cargo prints # write - a function that writes a line to our output # raw_line - the input line from cargo with styling included # clean_line - the text from raw line without styling def line_handler(write, raw_line, clean_line): for pat in [assert_pat, btrace_pat, num_pat]: if m := pat.match(clean_line): # One of our patterns matched! # Surround raw_line with a link and print it write_hyperlink(write, line=raw_line, # Link to the file in the first parentheses in the pattern path=m.group(1), # Make the URI fragment the line from the second parentheses frag=m.group(2)) return # None of the patterns matched. Original line, please drive through write(raw_line)
All of cargo’s output is fed to that line_handler function and you get linkified cargo out the other side.
If you’re using kitty, you already have
Follow the setup instructions to start using it.
If you’re not using kitty and you want
hyperlinked_grep, or you want my cargo linkifier, you can install hyperer.
It installs the ripgrep wrapper as
hyperer-rg and the cargo wrapper as
If you want links in the output of another command, hopefully it seems straightforward to write a wrapper now. hyperer can be a good starting point for writing a wrapper. Please send PRs to hyperer with any generally useful wrappers.
So, why ate? #
With an understanding of terminal hyperlinks and how to get them, we can now talk about the motivation for ate. Like many developers, I love a tight edit-compile-test loop. An integrated development environment(IDE) “integrates” tools to make that loop tight. I don’t want to give up the control that an IDE requires, so I need a way to integrate arbitrary tools. ate and terminal hyperlinks do that integration.
In that video, I:
- make an edit
- hit F4 to rerun my last shell command,
,cargo test, which is this script
- fix the compile error that running
,cargo testbrings me to in my editor
- hit F4 to rerun
,cargo testand see the compile error fixed
That’s a tight edit-compile-test loop.
What is ate doing to make that possible?
It looks through the output that hyperer-cargo produces.
If there’s a link in the output, ate runs the command in the
ATE_OPENER environment variable passing it the URI because
ATE_OPEN_FIRST is set and telling it to open the first link it finds when it runs.
ate also breaks long output into pages like less and other pagers.
It lets you move back and forth between links with
It searches in links if you type
All that’s to say that ate doesn’t do much. It’s the bridge between commands that produce links and whatever you want to do with those links. That it’s simple is a feature: you can make any terminal program produce links and you can send them to any other program. ate lets you make that connection however you like, and that’s why ate.