I use org-roam to maintain my wiki locally, and I use org-publish to render my wiki as a site of static html files. This page is specifically to describe how I use org-publish on my org-roam files to get them up on my public site.
1. Why org-publish?
org-publish feels a bit slow and clunky to me sometimes, but it's also very hackable and has gotten me pretty far so far. I'm not an expert on it by any means. One might prefer to use something like ox-hugo, firn, or similar (see options for publishing an org-roam-based digital garden). I do like the idea of just using the built-in org-publish capabilities, but I have started to get to the point where I've customised it so much, I wonder if it would be better to use something else. But, it works for me for now.
Just to note that I've built this all up incrementally - I'm an advocate for starting simple, and you don't need half the stuff I'm describing here just to get started. See the history section at the end to see the progression, and how to start simple.
2. My org-publish setup
I have the various publish-related functions in a publish.el file. I have a Makefile that I can use to call these functions - this to avoid the annoying way in which org-publish completely blocks emacs when you call it interactively. I used to publish from my local machine via the Makefile, but now I have the publish functions called in a gitlab pipeline. (Publishing org-roam via GitLab CI).
As far as the org-publish configuration goes, it's generally a fairly standard org-publish setup I think, with a couple of extra bits for org-roam purposes (e.g. backlinks and graph), along with a bit of extra JS and CSS for Miller columns.
2.1. Using a makefile
I used to use a Makefile to publish from my local machine. Now I use a gitlab pipeline triggered on commit. But this section left for posterity (or until properly gardened and tidied up…)
I first read about how to publish with a Makefile here: How to blog with Emacs Org mode. All this does is call through to various publish functions in a new process - avoiding blocking Emacs while I'm publishing.
So for example, if I'm in a file in my org-roam project, I could do
SPC c c (which triggers
helm-make-projectile) and then the
publish option. This triggers the publish make target. (You could also just run
make publish from the command line of course).
The publish target is:
publish: publish.el @echo "Publishing... with current Emacs configuration." emacs --batch --load publish.el --funcall org-publish-all
This run emacs in batch mode, loading my publish.el as a library file, calling
org-publish-all, and then exiting once finished.
In the previous case I'm calling one of org-publish's built-in functions. I can also call a custom function, e.g. with my graph target
graph: publish.el @echo "Graphing..." emacs --batch --load publish.el --funcall ngm/build-graph @chmod 755 graph.svg
This calls my own
ngm/build-graph function in my publish.el.
I deploy the html files to my server with rsync via a make target, too.
rsync: publish.el @echo "rsyncing published site to hosting..." rsync -chavz /var/www/html/commonplace/ dloop:/var/www/commonplace/
You could have any kind of deployment step here, like a push to Netlify.
So, to re-iterate, the Makefile is just a non-blocking way to call various publishing functionality. With an extra benefit that I can have an 'all' target, that does a few steps in a row, e.g:
all: graph republish rsync
You can find my full Makefile here on Gitlab.
2.2. Configuring and extending org-publish in your publish.el
My publish.el file is where I put all of the bits of org-publish configuration, as well as custom publish functions I've written.
It's worth pointing out that this is a constant work-in-progress - you can always find the latest version here on Gitlab.
I have a few differents types of things in my publish.el.
2.2.1. standard org-publish config
This is the standard stuff you'll find on a million posts out there for publishing a blog with org-mode. I'm not really doing anything special here for org-roam/wiki purposes.
I've tweaked head-extra a little to include variou css/js files, and preamble and postamble a little to link to my graph and my sitemap.
2.2.2. org-roam backlinks
I've got a couple of functions for including org-roam's backlinks during the publish process.
2.2.3. org-roam graph
I've got some functions for publishing the graph that org-roam produces to my digital garden. I also tweaked the process a little bit so as to allow the links on the nodes of the graph to link to the HTML pages on my site, as opposed to the org-protocol links they link to out of the box.
2.2.4. Hacking the HTML output
This is a bit messy and I'd like to improve on it. I'm basically overriding
org-html-template, which is a bit of a no-no, but it's the only way I could see so far to tweak the HTML enough to let me do some of the custom page layout that I wanted.
2.3. gitlab pipeline for automatic publishing
In February 2021 I set up a publishing pipeline to allow me to simply commit the org files to my git content repo, and have it all published from there. See: Publishing org-roam via GitLab CI.
3. Custom CSS and JS
You can use your own custom CSS and JS as you like - just include the links in your
4. Extra bits and pieces
Some other things I've picked up along the way.
4.1. Doing some things interactively
One downside of putting stuff in a publish.el is that it isn't loaded by default as it would be if it's in your .emacs config. So you can't out-of-the-box then call stuff like
org-publish-current-file. What I'm doing is just quickly opening publish.el and then evaling the buffer with
, e b, then all the config and functions are available for use. That works for now, but I wonder if I can set it as part of dir-locals or in some kind of hook to get it eval'ed automatically whenever I'm in the project.
4.2. Publishing single files
I actually don't run
make all that often. It's too slow.
What I tend to do is
org-publish-current-file on any new files I'm working on, and
SPC c c and
rsync. Then occasionally I'll do
make all to rebuild and republish everything (which will include a new graph, sitemap, and update the backlinks on all the pages).
org-publish uses htmlize to do syntax highlighting of code blocks.
I found that this doesn't work immediately if you're publishing via –batch. htmlize is a package that needs to be included. It's included by default if you're publishing within the project when you're running Emacs, but not from –batch.
I learned from here https://gjhenrique.com/meta.html how to get htmlize to work from –batch.
4.4. Publishing unchanged files
org-publish-current-file with the universal argument (i.e. do
SPC u first if you're in spacemacs) - that will force a republish of the file even if its actual content hasn't changed. So for example you'd do this if you want to update the backlinks on a published page.
5. A little bit of history
I think It's useful to have the progression of how I got to where I currently am, because you might not need all the bits and pieces as I have them now.
I started with all my publishing options in my .spacemacs config file. It did the job, but with two issues. Firstly, I wanted to share the full files more easily for others to reference (I do have my .dotfiles in a public repo, but I wouldn't expect people to look there and find the relevant bits).
Secondly, I found the interactive org-publish to be really slow, and it blocks Emacs, so I couldn't do anything during the publish process.
So I then moved them in to the publish.el file in the same folder as all the files, and set up the makefile.
As far as the look'n'feel goes, I used to have just a single page at once on the screen, before I switched to the Miller columns style. I think I like the Miller columns navigation for a personal wiki, but not 100% convinced - I may just go back to single page again at some point.