Tips and Tricks for Hacking Docutils (and Sphinx)

Return to Home

Docutils is incredibly powerful. And one of it's greatest features is how easy it is to get it to do something it was not made to do. Or, in other words, hacking docutils.

I'm keeping this page both as a personal reference so that I can go back and revisit some of the hacks I did, and for others if they need it.

Contents

Warning

This is a document about hacks, not elegant and pretty solutions that appeal to software engineers. If you wanted those, go somewhere else.

Quick & Dirty Custom Roles

You may be tempted to use the style directive on a substitution:

blah blah |blah| blah

.. |blah| style:: foo

As it turns out, not only is it a little bit ugly, but as of this time of writing, it is not actually implemented in docutils. So it won't even work anyway.

I propose you just use a custom role, which will not only be a bit better looking, but it actually works too. First, make yourself a custom role real fast:

.. role:: foo

Then, use your role:

blah blah blah blah :foo:`blah` blah

Finally, add the foo class to your CSS and do what you want with it, or if you're using LaTeX related stuff, see XeLaTeX stuff.

As a good example of what to do with this, I made a thvert role for typsetting vertical table headers.

Better Custom Roles

If you are using Sphinx, you can add a bunch of custom code in your conf.py. Here's an example I use for quickly inserting links to slides I have in Sphinx (as an example):

import docutils.parsers.rst as rst
import docutils.nodes as nodes

def slides_role(role, rawtext, text, lineno, inliner, options={}, content=[]):
    with open('source/slides/{}.rst'.format(text)) as f:
        title = f.readline().strip()
    return [
        nodes.reference(rawtext, title, refuri='/slides/{}.html'.format(text), internal=True),
        nodes.Text(', ('),
        nodes.reference(rawtext, '4:3 PDF', refuri='/_static/slides/{}-43.pdf'.format(text), internal=True, newtab=True),
        nodes.Text(', '),
        nodes.reference(rawtext, '16:9 PDF', refuri='/_static/slides/{}-169.pdf'.format(text), internal=True, newtab=True),
        nodes.Text(', '),
        nodes.reference(rawtext, '16:10 PDF', refuri='/_static/slides/{}-1610.pdf'.format(text), internal=True, newtab=True),
        nodes.Text(')'),
    ], []
rst.roles.register_local_role('slides', slides_role)

Yeah. It's a wee bit ugly. But it works. This page was about hacks, not pretty solutions to problems.

As a note here, newtab was not something provided by docutils or Sphinx, I created this for my own use. More on this in Sphinx Hacks.

You can do this with things that are not Sphinx as well. Should be exactly the same process (minus the conf.py).

XeLaTeX stuff

rst2xetex works great out of the box. It makes really nice looking output.

Templating, Custom Roles, and Custom Classes

If you want to customize some of the output, you can make a custom template file. You can make custom formatting for roles and classes real fast by overriding \DUroleNAME and DUCLASSNAME (as an environment) for a role or class with name NAME respectively. A minimal example of this might be something like this:

$head_prefix% generated by Docutils <http://docutils.sourceforge.net/>
% rubber: set program xelatex
\usepackage[margin=20mm]{geometry}

\usepackage{multicol}
\newenvironment{DUCLASScontact}{\begin{multicols}{3}}{\end{multicols}}

% \defaultfontfeatures{Scale=MatchLowercase}
% straight double quotes (defined T1 but missing in TU):
\ifdefined \UnicodeEncodingName
  \DeclareTextCommand{\textquotedbl}{\UnicodeEncodingName}{%
    {\addfontfeatures{RawFeature=-tlig,Mapping=}\char34}}%
\fi
$requirements
%%% Custom LaTeX preamble
$latex_preamble

%%% User specified packages and stylesheets

$stylesheet
%%% Fallback definitions for Docutils-specific commands
$fallbacks$pdfsetup
$titledata

%%% Body
\begin{document}
$body_pre_docinfo$docinfo$dedication$abstract$body
\end{document}

Put this in a file, and add --template=xelatex.tex (or whatever you named the file) to your rst2xetex command.

Making Admonitions Look Better (and work with verbatim!)

Admonitions are implemented as LaTeX commands, not environments. As a side effect of this, verbatim environments will not work within admonitions [1]. Of course, there exists a pretty way to solve this problem (subclassing the XeLaTeX writer to write enviornments instead). But this is not a page about pretty solutions, but rather hacks. So we will implement it using a pretty ugly hack.

First, we're going to need something that supports libpcre. Python's re won't do, we're going to need recursive regular expressions. And Perl's implementation of recursive expressions is kinda borked. As it turns out, PHP uses libpcre and was installed on my system, so that's going to be my weapon of choice [2]:

<?php
$re = '/\\\\DUadmonition\[([^\]-]*)(?:-[^\]]*)?\]\{\n\\\\DUtitle\[\1(?:-[^\]]*)?\]\{([^}]*)\}\n((?:[^{}]++|\{(?3)\})*)\}/smx';
$data = stream_get_contents(STDIN);
while (preg_match($re, $data)) {
    $data = preg_replace($re, '\\\\begin{\1box}{\2}' . "\n" . '\3' . "\n" . '\\\\end{\1box}', $data);
}
echo $data;
?>

Next, implement environments for each kind of NAMEbox you are using. I used tcolorbox, as it looks pretty great and also makes this pretty easy.

\usepackage{tcolorbox}
\newtcolorbox{systembox}[1]{colback=red,colframe=red!50!black,coltitle=white,coltext=white,fonttitle=\bfseries,title=#1}
\newtcolorbox{importantbox}[1]{colback=orange!20!white,colframe=orange,fonttitle=\bfseries,title=#1}
\newtcolorbox{notebox}[1]{colback=yellow!50!white,colframe=yellow!50!black,fonttitle=\bfseries,title=#1}
\newtcolorbox{warningbox}[1]{colback=red!15!white,colframe=red,coltitle=white,fonttitle=\bfseries,title=#1}
\newtcolorbox{admonitionbox}[1]{colback=blue!10!white,colframe=blue,fonttitle=\bfseries,title=#1}

You may have to implement a few more than that, depending on your needs.

That's all. Just pipe the output of rst2xetex into your PHP script and you should be good to go. Don't let any software engineers see your code.

[1]Try it yourself. XeLaTeX will blow up all over the place.
[2]I already told you this was a hack. Please stop freaking out.

Emoji Support

Chances are that you are not using a font with builtin emoji support in your document. I implemented the font-change using quick & dirty custom roles:

.. role:: emoji

Next, add some formatting stuff (XeTeX and CSS shown below):

\newfontfamily\emojifont[]{Noto Emoji} % or some other emoji font
\newcommand\DUroleemoji[1]{{\emojifont #1}}
.emoji {
    font-family: "Noto Color Emoji", "Segoe UI Emoji", sans-serif;
}

That's all. Surround your emojis with the emoji role.

Beamer Hacks

First off, you should be using the rst2beamer3k package rather than the original rst2beamer, which is way out of date.

No Support for Custom Document Classes

rst2beamer hardcodes the document class to beamer, even though you may want to use your own document class that derives from beamer. If this is the case, implement everything you would have in your document class in a file called beamerthemeNAME.sty and set --theme=NAME when you call rst2beamer. This should give you near equivalent functionality.

Compiling Beamer Things as HTML, LaTeX, etc

To a certain extent, rst2beamer provides you with equivalent container directives to its own beamer-* directives. Use these where you can.

But, you may need to make use of the beamer-columnset and beamer-column directives, which have no container equivalent. In this case, simply patch these up to be containers in things that are not rst2beamer (e.g., Sphinx):

import docutils.parsers.rst as rst
import docutils.parsers.rst.directives.body as body

class LooseContainer(body.Container):
    option_spec = {'name': str, 'width': str}
    def __init__(self, *args, width=None, **kwargs):
        super().__init__(*args, **kwargs)
rst.directives.register_directive('beamer-column', LooseContainer)
rst.directives.register_directive('beamer-columnset', LooseContainer)

Pause, and the like

You could just use a raw directive:

.. raw:: latex

    \pause

But I might reccomend making this a substitution to make it a bit easier:

.. |pause| raw:: latex

    \pause

Sphinx Hacks

Sphinx is a pretty great tool for documentation. If you are using it for it's intended purpose, this is probably not the section for you. I use Sphinx for my course website for CSCI-400, which is certainly not its intended usage.

Hiding the relbar

If you are not using Sphinx for documentation, the relbar might not be of any use for you. For the classic theme, you can disable it by overriding it in _templates/layout.html:

{% extends "!layout.html" %}
{% block relbar1 %}{% endblock %}
{% block relbar2 %}{% endblock %}

Using Docutils for your CV

I use docutils to generate my CV. It's not a terribly involved task: look at the code on my GitHub if you want an example.