167 lines
15 KiB
HTML
167 lines
15 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
|
|
<head>
|
|
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
|
|
<title>Improvements on GNU social's network systems - Tech Report - GNU social Summer of Code 2019</title>
|
|
<link rel="icon" href="../../favicon.png">
|
|
<meta charset="utf-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
<link rel="stylesheet" href="https://www.diogo.site/projects/excalibur_template/assets/css/main.css">
|
|
<style>
|
|
@page {
|
|
size: A4 portraint;
|
|
}
|
|
|
|
@page :blank {
|
|
@top-center { content: "This page is intentionally left blank." }
|
|
}
|
|
|
|
|
|
h1 {
|
|
page-break-before: always;
|
|
}
|
|
|
|
h1, h2, h3, h4, h5 {
|
|
page-break-after: avoid;
|
|
}
|
|
|
|
table, figure {
|
|
page-break-inside: avoid;
|
|
}
|
|
|
|
@page:right{
|
|
@bottom-right {
|
|
content: "Page " counter(page) " of " counter(pages);
|
|
}
|
|
}
|
|
</style>
|
|
</head>
|
|
|
|
<body>
|
|
<header id="header">
|
|
<nav id="side-menu">
|
|
<label for="show-menu" id="menu-button">Menu</label>
|
|
<input id="show-menu" role="button" type="checkbox">
|
|
<ul id="menu">
|
|
<li><a href="../../"><strong>← GS GSoC 2019</strong></a></li>
|
|
<li><a href="#about">About</a></li>
|
|
<li><a href="#openid">OpenID</a></li>
|
|
<li><a href="#routing">Routing</a></li>
|
|
<li><a href="#private-messaging">Private Messaging</a></li>
|
|
<li><a href="#activitypub">ActivityPub</a></li>
|
|
<li><a href="#remote-actions">Remote Actions</a></li>
|
|
<li><a href="#todo">TODO</a></li>
|
|
</ul>
|
|
</nav>
|
|
<h1>Improvements on GNU social's network systems</h1>
|
|
<p>Developed by <strong><a href="https://http://loadaverage.org/tenma">Bruno Casteleiro</a></strong> during <strong><a href="https://summerofcode.withgoogle.com/archive/2019/#6563945798696960">Google Summer of Code 2019</a></strong></p>
|
|
<p>Mentored by <a href="https://www.diogo.site/">Diogo Cordeiro</a></p>
|
|
</header>
|
|
<h2 id="about">About</h2>
|
|
<h3 id="what-is-this-document">What is this document?</h3>
|
|
<p>This is a technical Google Summer of Code Project Report describing briefly what I did from May 27 to August 19.</p>
|
|
<p>The whole code I wrote during this period is available <a href="https://notabug.org/diogo/gnu-social/src/c2a0f85274d01b1ea15caebe8d328d2fd290f228">here</a>.</p>
|
|
<p>This document helps understanding what was done in a simple and informal way, highlighting the most relevant parts, why some decisions were made and what I've learned with it. Unless explicitly said, the order in which I'll write about my tasks may
|
|
not exactly correspond to the real order by which I did the work.</p>
|
|
<h3 id="what-is-gnu-social">What is GNU social?</h3>
|
|
<p>GNU social is a social communication software used in federated social networks.</p>
|
|
<h3 id="abstract">Abstract</h3>
|
|
<p>My work for this summer involved, in a general way, working the different network systems of GNU social. Due to an unexpected increase of the workload in some of the tasks, my plan, although still related to network systems, turned to be a lot more
|
|
focused in the ActivityPub protocol, a communication standard for federated software. During my journey, other systems like the OpenID protocol or the internal url mapper were part of my work.</p>
|
|
<h4 id="benefits">Benefits</h4>
|
|
<ul>
|
|
<li>Ensuring federation of GNU social with other software that only implement the ActivityPub protocol.</li>
|
|
<li>Extending the ActivityPub implementation to correctly handle new interactions important in a social network.</li>
|
|
<li>Fix, extend and improve core code, ensuring better and greater usability.</li>
|
|
</ul>
|
|
<h2 id="openid">OpenID</h2>
|
|
<p>OpenID is an authentication protocol that allow users to use an unique identifier to login in multiple sites without needing to individual registration in each of them.</p>
|
|
<p>In GNU social, OpenID support is provided by a plugin and my task was to perform a revision of it.</p>
|
|
<h3 id="related-work">Related work</h3>
|
|
<p>During my revision it was necessary to update the external libraries used by the plugin. Since no code changes were required afterwards, this was a trivial update.</p>
|
|
<p>The OpenID user interface was also updated. Users had no option to choose whether to sync or not their GNU social accounts when adding an OpenID identifier in the settings. I've extended the interface with such sync option.</p>
|
|
<h3 id="learnings">Learnings</h3>
|
|
<p>Being the first task, reviewing OpenID helped me to better understand how plugins were structured and the back/front-end code interaction.</p>
|
|
<h2 id="routing">Routing</h2>
|
|
<p>Internally, the <code>URLMapper</code> class is responsible for the creation of the many routes used in GNU social. This class was known for having a few issues and, having affected some of my work during other tasks, I was responsible to fix some
|
|
of those issues in different times during the summer.</p>
|
|
<h3 id="related-work-1">Related work</h3>
|
|
<p>Fully reworking the Mapper was considered as an option at a certain time, however, taking into account the many routes spread all over the codebase and my work schedule, I decided that fixing the existent class in the best way possible was the safest
|
|
and more rentable option I could take.</p>
|
|
<h4 id="accept-headers">Accept-Headers</h4>
|
|
<p>One of the first changes to the Mapper were result of needing to have two or more routes with exactly the same path to coexist, differentiated only by the accept-headers that should be set in the request.</p>
|
|
<p>This necessity was solved by extending the route creation function with an option array of accept-headers. When populated, the function verifies the request accept-headers and in case of any match, it stores/substitutes the new route in the appropriate
|
|
variables.
|
|
</p>
|
|
<h4 id="wrong-matching">Wrong matching</h4>
|
|
<p>In some cases, some paths that would differ from one another only in their regex attributes were being routed. Although fundamentally equal, these paths generated different regex's during route creation. Because of this and because of being stored
|
|
by order of arrival, path matching was matching the wrong route in some cases.</p>
|
|
<p>This problem was solved by simply storing new routes at the head of the according array variable.</p>
|
|
<h4 id="dynamics-first">Dynamics first!</h4>
|
|
<p>The Mapper distinguishes what are dynamic and static routes. Static routes are the ones with regex attributes in their paths. Because of both being stored in the same place, grouped by the action to be executed, some static routes were incorrectly
|
|
being matched in the place of the dynamic ones, simply because of being found first in the storage array.</p>
|
|
<p>It makes sense to look all dynamic routes first, specially when these don't have the problem of wrongly match with a request for a static route. Therefore, this problem was solved by separating both types in different variables and explicitly enforcing
|
|
the lookup of the dynamic routes first.</p>
|
|
<h4 id="so-many-routes">So many routes!</h4>
|
|
<p>Spread all over the codebase, a great deal of the calls to the route creation function were making an improper use of the function parameters. All these calls were appropriately fixed.</p>
|
|
<h3 id="learnings-1">Learnings</h3>
|
|
<p>Specially because of some undo/redo that these changes required, this task was important for me to be more cautious with core code changes. Verification of the surrounding code and careful analysis of the changes to be done is very important.</p>
|
|
<h2 id="private-messaging">Private Messaging</h2>
|
|
<p>GNU social private messaging support is provided with the <code>DirectMessage</code> plugin . By requirement of the changes related to ActivityPub, this plugin end up being totally reworked by me.</p>
|
|
<h3 id="related-work-2">Related Work</h3>
|
|
<p>The plugin suffered many changes in practically all of its code, making it hard to write about every change. Still, because of their importance, three changes are worth of an highlight spot:</p>
|
|
<h4 id="porting-to-the-notice-db-table">Porting to the Notice DB table</h4>
|
|
<p>GNU social already had notices, so it didn't make much sense to have a separated table just for private messages. The most important part of this port was ensuring an upgrade function and the correct handling of a newly introduced Notice <code>MESSAGE_SCOPE</code> scope.</p>
|
|
<h4 id="supporting-multi-recipient">Supporting multi-recipient</h4>
|
|
<p>Porting to the Notice table automatically enabled text mentioning, so with just a few adjustments the plugin was handling multi-recipient messages.</p>
|
|
<h4 id="supporting-federation">Supporting federation</h4>
|
|
<p>Of course, the whole reason for this task. Events were added in key points of the private messaging flow for the federation plugins to subscribe and handle in their own way.</p>
|
|
<h3 id="learnings-2">Learnings</h3>
|
|
<p>Because of its big extent, this task was a test to everything I had learned so far and, therefore, completing it gave me great pleasure and some energy boost for what was to come. In particular, I've learned how to add upgrading logic to plugins.</p>
|
|
<h2 id="activitypub">ActivityPub</h2>
|
|
<p>Support for the ActivityPub protocol in GNU social is given by means of a plugin wrote by my own mentor, during last year's edition of GSoC. Since the last tests with other federated software some things changed and it was known that some interactions
|
|
weren't properly working, so part of my job was to review and ensure ActivityPub could federate again. Furthermore, some functionality was yet to be implemented and other came along the way, summing up the reasons as for why this task to consume
|
|
more time than the expected.</p>
|
|
<h3 id="related-work-3">Related Work</h3>
|
|
<p>Following, the most important changes this plugin seen this summer:</p>
|
|
<h4 id="follow-collections">Follow collections</h4>
|
|
<p>The followers/following endpoints for ActivityPub users weren't retrieving the most accurate information due to accounting invalid users. This was easily fixed by porting and correctly adapting the subscriptions functions from the core to the plugin.
|
|
Furthermore, I've handled the caching of these collections.</p>
|
|
<h4 id="ensuring-federation">Ensuring federation</h4>
|
|
<p>This sub-task involved ensuring the operation of all interactions supported by the plugin, both from the sender and receiver sides. These interactions are: Accept-Follow, Create-Note, Delete-Note, Follow-Person, Like, Undo-Follow, Undo-Like and Announce.
|
|
In some of the cases the changes were something trivial, others require small function rewrites and bug-fixing. The interactions were tested with remote instances of GNU social, Pleroma, Mastodon and Pixelfed.</p>
|
|
<p>Later I've also introduced and ensured the operation of the Delete-Person activity.</p>
|
|
<h4 id="queue-support">Queue support</h4>
|
|
<p>The GNU social queue system was added for ActivityPub notice distributions, these were proving to be quite a time consuming experience.</p>
|
|
<h4 id="audience-targeting">Audience targeting</h4>
|
|
<p>At this point me and my mentor, Diogo, agreed that, despite not being supported in GNU social, we should still properly handle the different non-public types of notes that the inbox could receive, like the unlisted, followers-only or private/direct
|
|
types.</p>
|
|
<p>Handling private notes was fairly easy since I had already reworked the <code>DirectMessage</code> plugin, so it was only needed to introduce a new Create-Direct-Note activity and subscribe the DirectMessage events. It is worth mentioning that this
|
|
type of note has (yet) no standard way to be differentiated. After some research me and Diogo decided that making GNU social conforming with the discussed <a href="https://github.com/w3c/activitypub/issues/196#issuecomment-304958984">directMessage</a> flag was the best way to handle the activity.</p>
|
|
<p>The unlisted and followers-only types of note were handled by storing such notices with a special <code>GATEWAY</code> type flag. This flag was originally introduced to represent notices with origin in services other than the OStatus protocol. Being
|
|
that way guarantees that the notice won't show in the public timelines. For the time being, I believe this is the best option for handling such note types: 1. easy to change (if required) in the future when proper support for these types is added
|
|
and 2. provides at least the minimum requirements of visibility scope.</p>
|
|
<h3 id="learnings-3">Learnings</h3>
|
|
<p>With this task I've learned how to use more systems of the program, like the caching and queue systems. Furthermore, I gained further knowledge in different aspects of the ActivityPub standard.</p>
|
|
<h2 id="remote-actions">Remote Actions</h2>
|
|
<h3>Remote-follow button</h3>
|
|
<p>Aiming to provide better user experience and to ensure federation in a general way, this task led to a new <code>RemoteFollow</code> plugin. This plugin is responsible to fully control the remote-follow button that shows on user profiles and to provide
|
|
the necessary events for all federation plugins to subscribe and make this button work.</p>
|
|
<h3 id="related-work-4">Related work</h3>
|
|
<p>As there isn't no standard way to treat a remote-follow button ActivityPub, the common way to do it is to extend the logic of OStatusSub, the OStatus way of handling such button. That is precisely what I did. I've ported the necessary actions related
|
|
to remote-following a user from the <code>OStatus</code> plugin and adapted it to be event-driven.</p>
|
|
<h3 id="learnings-4">Learnings</h3>
|
|
<p>This task was my first time working in the <code>OStatus</code> plugin, so I learned some details of its implementation.</p>
|
|
<h3 id="search-box">Search-box</h3>
|
|
<p>I've introduced the "ActivityPub way" of fetching remote users and notices, allowing remote-following users without needing to leave the local instance.</p>
|
|
<h2 id="todo">Anything left to do?</h2>
|
|
<p>Of course, there's always something more to do :)</p>
|
|
<ul>
|
|
<li>Further ActivityPub improvements (e.g Inbox Forwarding)</li>
|
|
<li>ActivityPub C2S (part of the original plan)</li>
|
|
<li>Some minor TODOs I left in the <code>DirectMessage</code> plugin (not crucial)</li>
|
|
</ul>
|
|
</body>
|
|
</html>
|