If you've used anything built upon ActivityPub, think Mastodon, Misskey (and forks), Pixelfed and a plethora of others, you probably relate to this problem:
You're reading a post, it links an article and has a snippet, and you could see before clicking on it that it has a few replies. You click into it, expecting discourse, but you find yourself empty handed. You cannot see any replies, there is barely any likes/favourites and barely any boosts either.
You think to yourself, "surely this is not true?" and visit the origin instance. Truth be told, this post is a lot more popular than you initially thought, but none of the data about this post ever made it to your instance.
Why is that? Can't my instance just get the replies from the origin? Why did it not federate here to begin with? Is this the fault of the software I use?
Lots of questions...
So, you start digging into the problem. You find bug reports filed about something called "backfilling", and - reluctantly - you learn more about the platform's internals. You start to notice a pattern in these issues, and eventually you realise... "this really is a problem with the software I use"
Or well, at least that's what I - the person that already knows a lot more than the average person - believes this all to sound like in a normal person's head; Refer to XKCD 2501
The issue here is that, in ActivityPub, there is no consensus over the "truth" about an actor or any post that is made, as well as post relationships. There is no external source you can use, and you cannot rely on your instance to be correct because chances are, unless it's a local post, it isn't correct.
To alleviate this, extensions such as substitoot were made, clients for Android and iOS started implementing features to fetch remote likes and boosts from remote instances as well; All of these are bandaid fixes to the underlying problem.
So the question naturally becomes: how do we solve this problem?
Collecting ideas...
Of course, the most obvious solution is the one I've already mentioned: Simply fetching the data from a remote instance when needed.
This would be the easiest to implement, but - for the sake of argument - if we imagine all instances adopted this at once, it would probably increase the overall bandwidth required to run fedi as a network by an incredible amount (which I will not calculate I do not have the numbers anyways). So the need for caching or duplicating data is probably still there.
Another - quite frankly insane idea - would be...State Resolution.
Yes, like Matrix, except maybe don't goof up the algorithm this time (we would not want Split-Brained Fediverse posts)
Jokes aside, a state-resolution-esque solution would effectively have fediverse instances collaborate with eachother to determine the current state of any given post. This could be computationally expensive though, and may not work in practice (again, see matrix, which is incredible right up until it breaks)
One solution that I'm explicitly trying to avoid here is introducing any kind of central entity - even one that is distributed or can be changed - that defines the 'global state' of a post or actor. The reason for this is...Bluesky, ironically
Bluesky (NOT ATproto) uses a relay-based system, wherein data lives on users' personal data servers, which then gets digested by a relay over WebSockets into a 'firehose' of data. This firehose can then be used by any application that wishes to do so.
This however means that anyone that doesn't use, say, Bluesky's relay, is effectively culled from the network. They exist somewhere, but not on the same network that you do. This may be useful for private federations, but not much beyond that.
In other words....all my ideas kinda suck!? I guess things are like that sometimes, but that is why I'm publishing this after all!
I want you to ping me on Fedi at @cyrus@wetdry.world or on Bluesky at @cyrneko.eu and tell me about any ideas you might have based on these or entirely new ideas...because Fedi is kind of in a pickle with this and it affects just about every Fedi user.
If you wanna support me:
