The meta-federated social network

Today’s social network are all introvert in a way or an other at varying degrees. Some are fully centralized where interaction is done only with users from the same network and server. Some are decentralized and interactions occur between users on the same network spread on different servers. Some others go even further and let interactions occur between users from different networks and servers. This is the federation. Networks falling in the last category are few and a real federation is not there yet.

The fediverse is not federated

Networks should settle on a common protocol in order to understand each other. This certainly won’t happen. Years have passed and no unified protocol have been defined yet. Who said OStatus? XMPP you say?

I might say that each network speaks its own language and doesn’t do any effort to learn a foreign one. It eventually learns a word or two as a second languages like « Hello », and « show me your picture pelase ». Please take note of the typo the the second sentence.

A huge hindrance to federation on the one hand is jailing networks like the Eff, Tee or Gee+ ones that are completely centralized and introverts by design. On the other hand federated networks provide the poorest and outdated documentation possible making it difficult for other developers to bypass the lack of a standard protocol.

The current state of things is that if a network talks to another, the communication is poor. For example, a user from GNU Social can subscribe to a user from diaspora* but not the inverse. More than that, the GNU Social user would stop receiving updates from that diaspora* user after a period of time.

You cannot subscribe to a user living in a jail network.

There exist bridges that allows a user post on a jail network and have his updates mirrored on an open network and vice versa. Even though, you still need to have two accounts to use a bridge.

My proposition

I propose to create a network that is polyglot. A network that thinks in its own way and expresses itself in the language of the one it communicates with. Lets name this social network The Polyglot.

A polyglot account would provide access to as many networks as network-languages it can speak. E.g. A polyglot account gives an account on GNU Social, diaspora*, Redmatrix, pump.io, friendica, and XMPP all in the same place under one user identifier. Speak to it in OStatus, it replies in OStatus. Ask it in pump.io, it answers in pump.io.

The polyglot, like humans, can understand (read-only) what jail networks say, but is too afraid to speak to them. It may resort to brokers or intermediaries to speak to them.

How to achieve this is rather simple. The polyglot needs to know the basics of all the networks-languages it wants to talk to. Those basics are:

1. Get a user’s profile including his avatar, header, bio, location, email and URL
2. Get a user’s publications. Also known as notices, dents, updates, statuses, queets, tweets, etc.
3. Post a publication

These basics can be extended by basics level 2:

1. Post a reply
2. Get replies to a notice.

A post has a textual content, a posting user, a created at date and can have a type or not. It has also many meta-data.

A reply is a post that replies to another post. Depending on the network, a reply may be considered a comment.

The polyglot is intelligent enough to understand if it dealing with posts, replies, comments, photos, videos, check-ins or a personalized post type.

When the polyglots learns how to talk to enough networks, it can then formulate complex sentences and take complex actions like one-click remote subscriptions, reply to any post anywhere or populate a user photo gallery from a user on a network with no gallery notion. It is real smart a polyglot, isn’t it?

On a more technical side this polyglot should know all the important API end-points and how to use them. The polyglot should be developed in Object Oriented Programming and makes extensive use of Inheritance and abstraction.

This network is better written in a framework like symfony2 so it benefits from a strong MVC architecture, routing and templating. By templating I mean it either replies in HTML5, JSON, plain XML, Atom, RSS, RSS2, ActivityStream or anything.

Lets say we have an abstract base class Polyglot defining all the vital and necessary operations. This is what I referred to as thinking in its own way. The common parts would take care of storing updates, profile information and all the data in a unified and structured way. The remaining methods will all be abstract, obliging any class inheriting from it to implement the basic operation we want.

What I want to come to is that if we need our polyglot to speak a new network-language, we just create a new class defining how those basic operations work on that new network.

When we have all the data we want, we do what ever we want with it like translating it to other networks in their own languages.

For jail networks, we only need an API key and secret that would power a broker. This broker will get a list of users, read what they say, and replicate their statements locally. In other words, the broker monitors the subscription process and look for all the remote that are in the jail it is in charge of, and dresses a list of them. It will then, depending on the configuration of the local server, retrieve all the updates and data of the selected users in real-time, near real-time or at intervals. Once the brokers does its work, the polyglot knows what the jailed users said, and inform its local users. Local users might also ask the broker to transmit messages to jailed users.

Practically, you might subscribe to @user@twitter.com, mention him and the broker will take care of informing the polyglot and thus the local user of all what @user@twitter.com says, and even transmit queets to him inform tweets. Obviously, those tweet will be posted by a bot, the broker, and contain the original user username and all the data needed by the jail network in the format it wants. Given enough permissions, the broker can do the same for any network of any kind as long as it has public API or is decentralized.

Eventual problem we might face

We might have a problem if two networks use the same API end point while using different data structure and format. API end points are the principal indicator of the protocol. If a social network doesn’t identify itself in the headers, which certainly no network does, then there is no other way than the API end point to identify it. This can be problematic. Really.

An eventual solution is to give a Schrodinger reply. We first determine all networks using this API endpoint and generate a response to all of them. Then if all those networks use the same data structure. e.g. all use JSON, try to put somehow all the response one after the other hoping the network would choose the right one for it. If they use different data structure, it is more problematic. We may give a response using some kind of separator, or multiple successive replies. If the network doesn’t understand the response will drop it but hopefully takes the next one in consideration.

A real solution is prefixing APIs with the name of the protocol. Instead of having example.com/api/ as the base, we would have example.com/ostatus and example.com/diaspora as bases. This is possible since networks have absolutely no clue on how that base should look like and just search for it in the <head> section of a DOM page.

Conclusion

Only a polyglot can decide what language (protocol) is the most practical and efficient. So while nobody knows what any protocol should look like, how efficient it should be federation wise, lets all feed our polyglot so it tells us. This proposition is practical and feasible. Clients like Andstatus can operate on many networks, but they don’t do so under one and same identity.

 

Update

Friendica apparently does most of what I describe and is the most federated. It can interact with, GNU Social, diaspora, redmatrix and itself. Why isn’t as wide spread as others?