Sendmentions and Commentions - webmention plugins for Kirby 3

With IndieWebCamp Berlin taking place in early May 2019, I finally had a deadline to assemble the bits and pieces I had been working on to create my personal webmentions solution for the CMS Kirby 3. This post describes the process and a fully functional prototype.

Over the past months, I fell in love with Kirby CMS, more specifically its third incarnation, Kirby 3. After a joy- and successful Kirby-based client project, there is simply no way I’d ever voluntarily go back to Wordpress. Yet, in consequence, this means to eventually migrate my own site to Kirby as well. A major obstacle is the desire to keep some beloved Indieweb functionalities - most importantly: webmentions.

So I need a way to make Kirby 3 send and receive webmentions for my content; and since we are at it, a system that processes user comments in general. All this under the premise that my motives are slightly different than in most webmention plugins: I have no interest in automatically displaying social media “likes” and “retweets” on my site (the so called backfeed, the most popular feature of webmentions for many), but am only interested in generic Webmentions sent directly from other websites …which I treat like comments, which always undergo a manual approval process.

Some benchmarks

Kirby is not the first CMS in need of webmention support, and I am not the first (nor likely the last) to attempt creating one, so it makes sense to assess existing work to build upon and learn from the experience of others.

An essential benchmark, Bastian Allgeier created a “first draft for a Kirby Webmentions plugin” during IndieWebCamp Düsseldorf in May 2015. It has barely seen any activity since, but appears to be used on websites in production and reliably deliver core interactions of a webmentions flow. For my work, this provides valuable insight at how the creator of Kirby himself would implement this.

Sebastiaan Andeweg subsequently forked the plugin to overcome some of its limitations - documented in Github issues (e.g. here and here) and on the Kirby forum - for his own use. The need for asynchronous processing of webmentions, as suggested by the spec for performance and security reasons, is the most challenging issue, since Kirby does not provide a task queue engine out of the box. Other challenges are posed by the fact that, while webmentions are a stunningly simple technique in principle, the devil is in the details; everybody who has worked with the Indieweb stack before has tales to tell. And the code that takes care of sending/receiving them always has to be complemented with according pieces in themes, markup and processes: a Webmention plugin will almost always be just one piece in a bigger puzzle.

As a long-time user of the powerful Wordpress Indieweb plugins by Matthias Pfefferle, David Shanske and others, I learned from their work as well as from stories about their challenges. In addition, there is a wide range of more or less mature implementations for various CMS, which can be peeked into for inspiration.

Lastly, after Sebastiaan had created one for Kirby 2, Bart Vandeputte provides the Kirby 3 community with a functional Kirby Queue plugin that could be used for asynchronous processing - an important building block, not only for webmentions.

Separating outgoing and incoming webmentions

In true IndieWeb fashion (“scratch your own itch” and other principles), I am first and foremost writing this Kirby 3 solution for my own, personal use: I want my Kirby site to deal with webmentions in exactly the way I see fit. Nonetheless, after creating plenty of spaghetti code in earlier endeavours, I decided that for once it would be nice if at least some of what I create could potentially ease others’ pain as well.

Since my plugin solution would inevitably be heavily opinionated (on the receiving side, in particular), I - after some deliberation - decided to create not one, but two plugins. While webmention plugins for CMS commonly come with the aim to be a “one-stop-shop”, I believe that a more modular approach can be of value. This particularly applies to a system like Kirby, where the solution does not have to be deeply integrated with default structures (as e.g. on Wordpress all received Webmentions have to be processed within the boundaries of its built-in comment system), hence the smaller the pieces the better.

As a first design decision, I outlined the creation of

• one plugin dealing with everything related to publishing my content and pinging all websites referenced, and

• a second plugin that first and foremost implements a simple comments system and, in addition, provides an endpoint to submit a “comment” to that system using the webmention protocol.

This way, site owners who already have a comments solution implemented, or who conceptually frame incoming webmentions in a different way than I do, could still get their site to send out webmentions by using the first plugin, while building an alternative solution for the incoming webmentions.

Sendmentions - a plugin that notifies other sites

For me, the web is a medium of dialogue. As I write my own content, hyperlinks put it in relation to others’ work and thoughts. This is why I am so fond of webmentions: after adding some recommended microformat markup to my HTML, I can not only notify referenced sources that they have been linked to (the original idea behind Pingback), but attach semantic meaning to these pings (e.g. imply that my intention to link back is that of a comment, a bookmark, or even the RSVP to an event).

So, ultimately, all the plugin has to do is to contact linked websites and tell them that they have been linked to. Luckily, there is a ready-made Indieweb PHP library (mention-client-php) providing that functionality. Since it has a built-in fallback to Pingback in case the target site does not have a webmention endpoint, I decided to keep it that way.

In addition, I decided to implement something I had hacked into my Wordpress site a long time ago: the plugin can at the same time also send out pings to archive.org, triggering their Wayback machine to archive the linked pages.

The flow of the Sendmentions plugin is simple:

• When a page is set to “published”, all URLs in the content are extracted and added to a job queue (currently still processed instantly, later to be dealt with asynchronously to avoid the long lag in the panel UI)

• As the queue is processed, every URL is pinged by first looking for a webmention endpoint and then sending either a webmention or a pingback

• If configured, the URL is also sent to archive.org as a “Save this page” request

• All notified URLs, with the response code returned by the receiving side are stored in a JSON file in the page’s content folder (and, for archive.org pings, the URL of the archived copy as well)

• When accessing a page in the admin panel, an optional widget displays a list of URLs that have been pinged

And that’s pretty much it. I added a few configuration options (such as limitation to certain template types) and started assembling the inevitable backlog of ideas/todos for the future. For now, webmentions are only sent once per URL as the code is yet lacking a smarter logic when to resend requests while avoiding to flood sites with webmention requests on subsequent minor edits - no doubt dealing with re-sending webmentions after edits or deletions is the most important item on the roadmap.

https://github.com/sebastiangreger/kirby3-sendmentions

Commentions - a webmention-enabled comments solution

As stated earlier, my personal interest when receiving webmentions is not about collecting reactions from syndicated copies on social media sites, but about enabling others to “submit a semantically enriched comment” from their own websites into my comments system. Since Kirby does not have a default comments solution, yet another round of benchmarks was called for - looking mostly into Kirby 2 solutions, as there do not seem to be any mature Kirby 3 comments plugins, yet.

In general, there seem to have been three main approaches to dealing with comments in Kirby. One is to make use of an external tool, such as Disqus - very easy to implement, but comes with loss of control and potential risks of relying on a third party. In the remaining two approaches comments are stored within Kirby: either as a (hidden) subpage structure below the original page, or as a data structure within the content’s markdown file. While I cannot recall seeing it in my benchmarks, it would also be possible to keep a separate database, centralised or as a JSON file in each page’s folder, but at least at the time I started out, that didn’t seem to sit quite as natural with the beauty of Kirby’s flat-file structure and my ultimate aim for a minimalist solution.

I also ruled out creating comment subpages for myself; while I see the great fit with Kirby’s overall logic and there is value in keeping comments apart from the original content, I just don’t want to deal with hundreds of subpages. The alternative somehow fits better with my logic: storing reader comments as “meta data” right within the markdown file sounds like a great light-weight solution.

My initial plan was to adopt the approach of writing comments, marked as unapproved, directly into the content file. However, one major shortcoming is that in this way my markdown files might change while I am editing the content or other comments in the panel - leading to a potential loss of comments that arrived in the meantime when saving. Since I always approve comments and webmentions before displaying them publicly, my personal solution is to establish an inbox of comments and only write them into the content files on approval: no more risk of overwriting).

Like I did for incoming webmentions, I decided to use another Indieweb library, php-comments, which does most of the heavy lifting, as they need to be verified and their semantics extracted. Using Aaron Parecki’s XRay would likely be the more powerful alternative, but that seemed like too big a dependency for my tiny plugin.

The Commentions plugin’s logic is:

• Comments submitted directly on the website are stored as a JSON file in an inbox folder

• Incoming webmentions are first queued for asynchronous processing by writing a JSON file into a queue folder; as the queue is processed, verified webmentions and their meta data are rewritten as JSON files into the same inbox folder; the JSON files from the queue folder are deleted

• In a comments UI, comments and webmentions can be approved or deleted one by one

• If approved, the data from the JSON file is written into a YAML data structure in the according content’s markdown file

• Processed JSON files are deleted from the inbox

• Stored comments can be reviewed, edited and deleted on the according page’s panel view (requires adding a field to the blueprint)

While I consider the Sendmentions plugin above to be already reasonably stable to be used (except for its lack of asynchronity), this latter plugin is still more in a stage of prototype/proof-of-concept/exploration. I decided to make it available for review regardless, but - as always - this should be treated with utmost caution. In addition I would like to highlight once more that the philosophy of the Commentions plugin’s concept may not be the best fit for everybody, as it significantly diverts from certain mainstream usage patterns of webmentions. This, after all, is my very personal take on a Kirby webmention solution.

https://github.com/sebastiangreger/kirby3-commentions

A great way to learn more about Kirby

While the work initially started out of the need for certain functionalities, creating these plugins has above all proven to be a great exercise to dive deeper into Kirby 3. And while the learning curve has been a challenge at times (in particular when it comes to the Vue.js-based panel code; an entirely new framework for me), this has been great fun and my appreciation for the ingenuity of this system has only grown over time.