Private Groups – first milestone

Earlier this year, we were selected by NLnet under the NGI Assure fund to work on SSB Private Groups and bring them to Manyverse for real world usage. Today, we're proud to announce our first milestone has been achieved, after months of research and development by a team of five.

Our first milestone is titled "Space-efficient replication of group messages" and it has to do with organizing how group content is stored locally, and how it gets replicated remotely.

Metafeeds

It's important to store and replicate group content in such a way that doesn't waste device storage. If you're a member of 100 different groups, some of your friends should not necessarily replicate your (encrypted!) content in those groups, because they may not even belong to those groups. So we need a way to categorize your content and allow peers to replicate only some categories. Metafeeds allows us to do exactly that.

Metafeeds is an extension to the SSB protocol developed by the SSB NGI Pointer group in 2021. Two people in our current team were part of that team. In traditional SSB, all of one user's content is published on a single append-only chain we call a "feed". With metafeeds, a single user can have several feeds and they're linked hierarchically, as a tree (see diagram below). This allows friendly peers to replicate only portions of the tree.

So far, the assumption has been that all feeds in a metafeed tree are publicly knowable. In our NGI Assure project, we identified how the metafeeds specification could be improved to support encrypted portions of the tree, which in turn better supports private group use cases. We updated the metafeeds spec, in particular to add the v1 tree structure with a concept called "shards" to organize the feeds and support partial replication.

Metafeed group spec

An important deliverable in this milestone is the metafeeds group specification, a new document that prescribes exactly how group-dedicated feeds are managed and replicated. This is the primary document that informs our implementation efforts.

The key components of that specification are:

• A new "group/additions" feed

  • All invitations to new members will be published here
  • It is publicly knowable, i.e. its announcement is not encrypted
  • Messages published on this feed are, however, encrypted

• Group feeds

  • For each group, each group member has their own "group feed" where all their activity in that specific group is published on
  • It is secretly knowable only to other group members, i.e. its announcement is encrypted
  • Messages published on this feed are also encrypted

The document outlines how these two feeds are created, managed, discovered, and replicated.

ssb-tribes2

The other important deliverable is a new Node.js library, ssb-tribes2. The new library is a successor to ssb-tribes, and adapts SSB Private Groups for metafeeds, implementing the aforementioned spec. It contains APIs such as:

• start() to initialize your group/additions feed and track for invitations received from others
• create() to create a new group and its group-dedicated feed on your metafeeds tree
• addMembers() to publish an invitation to new members, revealing the secret key of the group
• get() to get more information about a group you're a member of
• list() to list all groups you're a member of
• listMembers() to list all members in a group
• publish() a helper function to easily publish messages encrypted to a group

Improvements in other modules

Our work wasn't limited to the specification and the new library. We had to adapt various other modules, and improving them in the process. The following is a list of pull requests we merged in the last few months:

• ssb-db2 (SSB database): #389, #393, #394, #397, #398, #401, #405
• ssb-ebt (SSB replication algorithm): #76, #73, #72, #71, #69
• ssb-replication-scheduler: #9, #10
• ssb-meta-feeds (CRUD for metafeeds): #69, #71, #72, #74, #75, #81, #82, #84, #85, #86, #87, #89, #90, #92, #93, #94, #95, #96, #98, #100, #102, #103, #104, #105, #106, #107, #108, #110, #111
• ssb-box2 (encryption format): #12, #13, #14, #15, #16, #17, #18, #20, #21, #22
• ssb-keyring (data store for encryption keys): #15, #16, #17, #18
• private-group-spec (schemas and spec for private group metadata): #18, #19, #21, #22
• ssb-private-group-keys (encryption key utilities for groups): #14, #15

What's next

It had to be this milestone first, because it sets the stage for Groups in the context of metafeeds. There are two other milestones that this milestone unlocks:

• "Private chats extended to unlimited members": this brings Groups to production in Manyverse, but in a way that preserves the current UX of private chats. Users will hardly notice a change, but the underlying technology (metafeeds and group replication) will be there.
• "Removal of members by moderators": this is about adding a permission layer to Groups, and the creation of new group feeds that reflect a new "epoch" for the group. This will require careful protocol design, and the deliverables will be specs and libraries.

We're excited too see this all come together, and we're once again grateful to NLnet (and particularly the European Commission for the NGI fund) for funding this work!