Misc » ActivityPub

Sebastian Greger This is a living document
Last update:

Notes on interacting with the Fediverse using the ActivityPub protocol

There are essentially two approaches:

  • cross-posting content to Mastodon (creating posts on Mastodon that feature a link to content published on a separate blog, aka. POSSE)
  • or making blog content appear as “native” Fediverse content to be interacted with – i.e. turning a personal website into a Fediverse server.

My preference is with the latter, as it leads to single canonical content and allows full ownership of the domain-based identifier.

This website is now a Fediverse citizen

Some background about my motivation and approach.

Useful resources

Existing implementations

There are some functional and open implementations that may be used, assessed for reference, or build upon:

Documentation

  • Always a good starting point for anything Indieweb is the indieweb.org wiki; it also contains loads of links to people working on their own implementations
  • The official ActivityPub spec is an enlightening read, but is a little abstract to be of value for figuring out the more intricate details; it’s a great read to understand the principle, but not my go-to resource for troubleshooting
  • activitypub.rocks promises to be a one-stop shop for all things ActivityPub; unfortunately the most handy feature, a testing suite, seems to have been defunct for several years.
  • The official Mastodon documentation has pages about ActivityPub, Webfinger and Security that have some well-structured information for implementation details
  • Several years old already, but this visualization of different Mastodon timelines can be handy to grasp the mechanisms behind federation
  • This GitHub issue has some interesting discussions about the inner workings of the ActivityPub protocol.

Reports from other “self-hosters”

  • Lewis Dale shares a Netlify/Eleventy setup; this easy-to-read summary is actually the most concise write-up of elementary ActivityPub functionality I’ve seen (it does not go into the inbox and other more advanced features, though).
  • Justin Garrison describes the most minimal setup for a Mastodon-compatible ActivityPub server, essentially consisting of six static files.
  • Tom MacWright outlines a journey very similar to my own, and rightfully points out how this is actually more a Mastodon-compatible implementation than a standards-based ActivityPub implementation. I feel the same way, but Mastodon is currently so dominant that it feels impossible to not walk down that path.
  • Aral Balkan highlighting how the computational load involved with federation comes with a potentially massive server load; this is an important concern for anybody planning to self-host a Fediverse server.

Tools

In addition to using tool for making cURL/API requests manually (e.g. Insomnia or some REST-enabled browser extension), I can recommend this helper I recently discovered:

JSON Crack - Crack your data into pieces jsoncrack.com

A very handy tool when working with complex JSON files.

Making a blog available on the Fediverse

As I took some notes while experimenting, below are some of my learnings from my first steps with ActivityPub. Maybe they are of help to someone. They are likely not bug-free, and I cannot provide any support on the subject, but would of course be most happy to hear if you spot an obvious mistake.

I’d also be curious to read about your own experiences – feel free to submit a URL of your own documentation via the form at the bottom of this page and I’ll add it here for others’ benefit.

Discovery via Webfinger

A key precondition to be discoverable, in particular by Mastodon which relies heavily on this mechanism, is Webfinger. It is a file located at the pre-defined URL https://myserver.tld/.well-known/webfinger that provides JSON-formatted information about an ActivityPub profile’s location (among possibly other things).

In its most minimal form, the Webfinger file should be returned with an application/json header as:

{
  "subject": "acct:username@myserver.tld",
  "aliases": [
    "https://myserver.tld/username"
  ],
  "links": [
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://myserver.tld/username"
    }
  ]
}

In order to accommodate multiple users, clients usually request this file with a resource attribute: https://myserver.tld/.well-known/webfinger?resource=acct:username@myserver.tld should return the JSON data for the requested user with an HTTP code 200. On single-user instances, it would also be possible to return it when no attribute is present, otherwise it seems common practice to react to a missing or non-existent user name by returning HTTP error 422 with this JSON payload (though Mastodon simply responds with an empty text/html response and code 400):

{
  "detail": [
    {
      "loc": [
        "query",
        "resource"
      ],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}

The user profile (the “actor”)

Once the client has discovered the profile location for user username on server myserver.tld, it will request that URL with an Accept: application/json or Accept: application/activity+json header. Most implementations return a human-readable HTML profile when no such header is present (e.g. the Mastodon profile URL returns either HTML or the JSON-LD payload depending on that header’s presence); this ensures that the canonical user URL can be read by both humans and machines.

The profile is a JSON-LD file (lingo “the actor”) containing the information needed to interact with the Fediverse. A minimal version may look like this:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams"
  ],
  "id": "https://myserver.tld/username",
  "type": "Person",
  "name": "Full Name",
  "preferredUsername": "username",
  "summary": "<p>Cogito, ergo sum</p>",
  "published": "2022-11-11T11:11:11Z",
  "icon": {
    "type": "Image",
    "mediaType": "image/jpeg",
    "url": "https://myserver.tld/avatar-square.jpg"
  },
  "image": {
    "type": "Image",
    "mediaType": "image/png",
    "url": "https://myserver.tld/banner-1500x500.png"
  },
  "attachment": [
    {
      "type": "PropertyValue",
      "name": "Website",
      "value": "<a rel=\"me\" href=https://myserver.tld>myserver.tld</a>"
    }
  ],
  "manuallyApprovesFollowers": false,
  "inbox": "https://myserver.tld/username/inbox",
  "outbox": "https://myserver.tld/username/outbox",
  "followers": "https://myserver.tld/username/followers",
  "following": "https://myserver.tld/username/following",
  "featured": "https://myserver.tld/username/featured",
  "publicKey": {
    "id": "https://myserver.tld/username#main-key",
    "owner": "https://myserver.tld/username",
    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIThisIsOfCourseFakeFAAOCAg8AMIICCgKCAgEAuGLSHHAV2lAGVeLrKrmN\ndFcesBnWEJvFux2n2Gl3OGmnxaZimdPUvr058Ib3d4P6zVF7rHkdENX7lJZceMeH\noeDJlPapEgQegQ2D98tAfgntWKMjWGdrSkwAkffYDSKHqua+SfPd5hxqAvLtrWyh\nxHtgOh85tYZGTgUnmXRxMJZ+GdQHZTluJnpc4saiTu3RuusrFZHhFMZVIVGFOF4B\nhstsm9RipifdyDI5ItNxI8tw/lhWpL6pZFw/DoikvMXTttz76IiZg/OL+nqzKGia\nIQnZpKQbCUHErjgKawRzS69qoMtAnRaBqHIAfKrFne9UOJ1KwTeXmL0b/e2n1TTm\nTm1hrWIuJ4kvKAL7j8YUQ5UCAwEAAQ==\n-----END PUBLIC KEY-----\n"
  }
}

This intro by Eugen Rochko, the author of Mastodon, explains all the basics really well:

How to implement a basic ActivityPub server blog.joinmastodon.org

Apart from the rather obvious data like name, preferredUsername etc, the precondition for a functioning Fediverse server are the publicKey object for signing HTTP requests, and the URLs to various endpoints.

To generate the required key pair, I used the generateNewKeyPair function from Aaron’s Nautilus and replaced all line breaks with \n.

The actor profile of others needs to be fetched whenever we want to initiate an interaction ourselves – for example to find out the inbox URL of another person. This is essentially a simple GET request to the URL in question, but some Mastodon servers are configured to run in “secure mode”, which expects signed requests even for such basic interactions – the signing process is identical to the one described further below, except that the digest header is omitted.

The outbox

The outbox file (a list of existing federated content) is, among other uses, accessed by Mastodon when displaying a user’s profile. The spec supports some form of pagination (look at sample outbox files of Mastodon users), but in its most simple form (it may even be possible to simplify more?) can take this format:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams"
  ],
  "id": "https://myserver.tld/username",
  "type": "OrderedCollection",
  "totalItems": 1,
  "orderedItems": [
    {
      "id": "https://myserver.tld/url-for-this-create-activity",
      "type": "Create",
      "published": "2022-11-11T11:11:11+02:00",
      "actor": "https://myserver.tld/username",
      "to": [
        "https://www.w3.org/ns/activitystreams#Public"
      ],
      "cc": [
        "https://myserver.tld/username/followers"
      ],
      "object": {
        "id": "https://myserver.tld/url-for-this-create-activity",
        "type": "Note",
        "published": "2022-11-11T11:11:11+02:00",
        "url": "https://myserver.tld/public-post-url",
        "attributedTo": "https://myserver.tld/username",
        "summary": null,
        "inReplyTo": null,
        "content": "<p>Note text</p>",
        "to": [
          "https://www.w3.org/ns/activitystreams#Public"
        ],
        "cc": [
          "https://myserver.tld/username/followers"
        ]
      }
    }
  ]
}

The id of the activity as well as the actor and attributedTo fields has to be the URL of the “actor” (profile) file. In addition, it is important that the URL with the placeholder https://myserver.tld/url-for-this-create-activity points to a URL that actually returns the same content object as in this item list (otherwise the post will not show up on Mastodon); this can be done by setting up a separate path that returns this object or by enabling a JSON-LD representation of a blog post under its canonical URL when the Accept header request application/json or application/activity+json.

However, the outbox is probably not absolutely needed and could be either omitted from the profile or return an empty item list – the distribution of new content to followers is not a pull but a push flow, i.e. our server needs to actively ping the instances used by our followers to federate new content.

The to and cc properties are more relevant when pushing new activities, but since they show up here, good to reference Mastodon's ActivityPub docs again:

  • the “magic collection” https://www.w3.org/ns/activitystreams#Public should be in to for public posts, and in cc for unlisted posts (not showing up in local/federated timelines for non-subscribers)
  • To post to followers, it is not necessary to include them one by one, but instead our local /followers URL, in either to or cc
  • For private statuses or direct messages, special rules apply (see the linked document), but that is out of scope for my explorations

The inbox

In a minimal implementation, the inbox does not really return any relevant data to a request. It can simply return something like:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams"
  ],
  "id": "https://myserver.tld/username",
  "type": "OrderedCollection",
  "totalItems": 0,
  "orderedItems": []
}

Its more important role is to process incoming data, to which it commonly should reply with an HTTP status code 202. This will be any activities by the people we follow, hence relevant if we want to develop our site to not only be a publishing outlet but also a “Fediverse feed reader”, but most crucially all incoming “Follow” activities – requests by other users to subscribe.

For developing an individual implementation, it makes sense to set up some means to log incoming requests (including their payload and headers), use a Mastodon account to test the interactions and build the required processes from that.

Before processing the incoming request, it is important to validate the signature headers. Philipp Waldhauer’s implementation can serve as a boilerplate for that.

Following and being followed

The core mechanism is again explained really well in Eugen Rochko’s primer:

How to make friends and verify requests: Implementing an ActivityPub inbox blog.joinmastodon.org

Reacting to incoming “Follow” requests

The body of an incoming “Follow” activity (aka. a subscription request) will look like this:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams"
  ],
  "id": "https://remoteserver.tld/unique-uri",
  "type": "Follow",
  "actor": "https://remoteserver.tld/username",
  "object": "https://myserver.tld/username",
}

Here, the actor is the profile URL of the remote users expressing their wish to follow and object is the local profile URL of the user receiving the request (as they are the “object” of that request).

Unless we need a mechanism to manually approve followers, we can instantly reply to this request by sending a cURL request to the remote server in the background. For that, we first have to poll the requesting user’s profile and parse it to retrieve the URL of their inbox (see above for the spec of the profile JSON).

Now we send the following JSON object (an “Accept” activity that contains the original “Follow” activity as its object) to that inbox:

{
    "@context": [
        "https://www.w3.org/ns/activitystreams"
    ],
    "id": "https://myserver.tld/unique-uri",
    "type": "Accept",
    "actor": "https://myserver.tld/username",
    "to": "https://remoteserver.tld/username",
    "object": {
        "id": "https://remoteserver.tld/unique-uri",
        "type": "Follow",
        "actor": "https://remoteserver.tld/username",
        "object": "https://myserver.tld/username"
    }
}

The request to the remote inbox has to be a POST request and requires a cryptographic signature. By signing our request with our private key, the receiving side can validate it against our public key from our “actor” profile and therefore ensure it is coming from who claims to be its author.

This part is a little tricky. It is rather well explained in How to implement a basic ActivityPub server, but since there is no test suite to send test messages to, if things don’t go well Mastodon’s error messages can be rather obscure.

For my experiments with Kirby CMS, I created the following code – heavily inspired by both Aaron Parecki’s Nautilus and a snippet by Philipp Waldhauer:

// prepare the headers needed
$date = gmdate('D, d M Y H:i:s T', time());
$digest = sprintf('SHA-256=%s', base64_encode(hash('sha256', $body, true)));

// assemble the signature string
$plaintext = sprintf(
    "(request-target): post %s\nhost: %s\ndate: %s\ndigest: %s",
    parse_url($inboxurl)['path'],
    parse_url($inboxurl)['host'],
    $date,
    $digest
);

// sign the request; signature returned in $signature
openssl_sign(
    $plaintext,
    $signature,
    openssl_get_privatekey('/path/to/activitypubprivate.pem')),
    OPENSSL_ALGO_SHA256
);

// send post request to remote target
$request = [
    'headers' => [
        'Content-Type: application/json',
        'Date: ' . $date,
        'Signature: ' . sprintf(
            'keyId="%s",headers="(request-target) host date digest",signature="%s"',
            site()->url() . '/username#main-key',
            base64_encode($signature)
        ),
        'Digest: ' . $digest
    ],
    'method'  => 'POST',
    'data'    => $body, // this has to be json
];
$response = Remote::request($inboxurl, $request);

Make sure to store a list of subscribers somewhere; at a minimum the profile URL (“the actor”) of each subscriber.

A similar workflow is needed for incoming “Undo” activities, which in case of an active subscriber means an “unsubscribe”.

Publishing a list of followers

Once we have subscribers, we can populate the JSON object to be returned from the /followers URL. This again can be a paginated object or – with small subscriber numbers at least – all in one file:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams"
  ],
  "id": "https://myserver.tld/username",
  "type": "OrderedCollection",
  "totalItems": 0,
  "orderedItems": [
    "https://remoteserver.tld/username"
  ]
}

The items in the list of followers are simply the profile/actor URLs of each subscriber.

If, for privacy reasons or due to a general personal preference, you prefer to keep your followers/following lists private, simply omit the orderedItems element from the JSON file (at least that’s what Mastodon does when selecting the “Hide your social graph” option from the “Edit profile” settings page, and it does not to affect functionality otherwise). Others can still see who else on their instance is following your profile (that is instance-internal data, not under outside control), but the list won’t contain followers from other instances.

Publishing new content

Once we publish a new post, we have to send a “Create” object to all our subscribers. We therefore need to poll (or retrieve from a cache) all profiles for the inbox URLs. Many platforms include a "sharedInbox" URL, which allows to send only one activity to a server which then takes care of distributing it to all followers (massively reducing the amount of API calls in the case of big instances with mutiple followers, like mastodon.social or whatever is popular in your niche of the Fediverse).

Work in progress: I’ll try to add this in more detail later; essentially, a “Create” activity resembles the “Accept” activity (incl. the requirement for a signature) but the type is "Create" and the content object sent is the same as the object in the outbox. In addition, we need to make sure that it is properly addressed; based on my preliminary tests, it suffices to address it to https://www.w3.org/ns/activitystreams#Public and https://myserver.tld/username/followers as in the outbox, but this is not yet based on more thorough testing (it is probably not necessary to address the object to specific user actors as long as they are followers which the receiving server should be aware of).

“Remote following”

Subscribing to people on the Fediverse is not quite as simple as on a centralized platform, as “Follow” buttons have to trigger a process on the user’s own server, not on the profile of the person they want to follow (unless, of course, both are on the same Mastodon instance or similar).

The “remote follow” technique essentially allows to offer a “Follow” button that then connects to a user’s own instance to initiate the required server-level interactions.

How to implement remote following for your ActivityPub project hughrundle.net

NB. At least when dealing with Mastodon, the URL to be called is not, as stated in that article https://remote.social/authorize_interaction?uri=https%3A%2F%2Fexample.social%2Fuser%2Fmolly but https://remote.social/authorize_interaction?uri=molly%40example.social (i.e. despite the argument name uri, the endpoint actually expects an account ID).

Performance considerations

This may seem far fetched for a small independent website connecting to the Fediverse, but it is well worth spending some thoughts on a robust implementation. Due to the nature of federation, there may be peaks of high traffic from potentially thousands of Mastodon instances, so a proper server-side caching strategy is really important.

Just to name a few pitfalls:

  • Since Mastodon instances naturally do not have a shared cache for link previews (as Twitter does, rendering cards from a cached version; as everybody who ever fixed a typo in a headline of an already-tweeted article is aware of), a popular post being shared around the Fediverse may essentially result in a DDoS attack (the “Mastodon stampede”) unless served from a properly configured cache (another example here)
  • Depending on the amount of followers and the degree to which you want to distribute your activities, also Aral Balkan's article is worth a read
  • As you are processing incoming inbox requests, you will be surprised by the sheer amount of meta activities (like notifications about accounts deleted from instances that you never interacted with, etc.). Again, good caching and/or asynchronous processing are paramount. This scales with the amount of other servers yours is interacting with.
  • Think twice about sharing posts with photos or other media from your self-maintained Fediverse server; as these get federated, every time they are displayed in any Mastodon instance, the media files are being loaded from your server

There are some well documented experiments regarding the massive amount of traffic involved with federation in this article by Kris Nóva.

Moving between instances

I haven’t done this myself, but in case somebody already has an active Mastodon account and wants to move to a self-hosted implementation, this documentation by Manton Reece probably has all the details needed to set up an alias and move an account.

I'm Sebastian, Sociologist and Interaction Designer aiming to bring toge­ther social science and design for inclusive, privacy-focused, and sustainable "human-first" digital strategies. This is my "digital garden" with carefully curated resources. For a more stream-like outlet, see my journal.

My occasionally sent email newsletter has all of the above, and there is of course also an RSS feed or my Mastodon/Fediverse profile.