Brussels / 4 & 5 February 2023


Building a social app on top of Matrix

Fighting surveillance capitalism for fun and profit

I wanted a secure and convenient way to share baby photos with friends and family, so I used Matrix to create an app with a social network interface and end-to-end encryption underneath. In this talk, I describe how we use Matrix rooms to represent human social structures, from loose disorganized circles of friends, to well-defined organized groups. I will talk about what worked well, where we found some pain points, and what we did to work around them.

I wanted the security of Signal Private Messenger together with the convenience of Facebook. So I created an app called Circles that uses Matrix as its server. All posts are end-to-end encrypted, so even the server admins can't read what users post or look at their pictures.

In Matrix, everything happens in a room. So Circles uses Matrix rooms to store all of its posts. We use Matrix rooms in two different ways to let users connect and share in an organic way.

First, we can use a Matrix room to store one individual user's own posts, similar to their "wall" in Facebook. The user can invite other users to follow them by joining the room. Followers can also post emoji reactions and replies to the room owner's posts. In the app, a social "circle" consists of one such "wall" room belonging to the current user, plus similar "wall" rooms belonging to the user's friends. For example, Alice's "Friends" circle might consist of her own "Friends" room, her buddy Bob's "Friends" room, and their friend Carol's "Friends" room. The app hides most of the complexity of the multiple rooms and instead presents the user with a single unified timeline of posts collated from all the rooms in the circle.

Second, we can use a single Matrix room to represent a social group (like a Facebook group) where everyone in the group knows each other. All that is required here is to render Matrix messages as a timeline of "social" posts rather than as chat messages.

We estimate that building on Matrix saved us 18-24 months of development time for our first minimum viable prototype. Overall the experience has been very good.

However, we did run into a few pain points. These include:

  1. Decryption errors with the iOS SDK
  2. Needing two passphrases to enable secure server-side storage, or hoping that the user never loses their first device
  3. Problems with the 3rd party identity server that we initially used for email verification
  4. No ability to post both text and media in the same message
  5. No ability to give a new user access to the earliest encrypted posts in the room
  6. Questions about server performance and scalability

To address #2 and #3, we built a new authentication service using the Matrix user-interactive authentication API.

We are looking forward to upcoming improvements in the Matrix spec, including extensible events, rooms as profiles, and sliding sync.


Charles Wright