From 367cc5c5c71d32b693962779d9d622ad5e96382d Mon Sep 17 00:00:00 2001 From: Diogo Peralta Cordeiro Date: Tue, 3 Aug 2021 15:35:23 +0100 Subject: [PATCH] [DOCS][Dev] Add Queues --- docs/developer/src/queue.md | 110 +++++++++--------------------------- src/Core/Queue/Queue.php | 8 ++- 2 files changed, 33 insertions(+), 85 deletions(-) diff --git a/docs/developer/src/queue.md b/docs/developer/src/queue.md index 2b0cf43dfd..58268645a1 100644 --- a/docs/developer/src/queue.md +++ b/docs/developer/src/queue.md @@ -1,8 +1,10 @@ -## Queues and daemons +# Queue -Some activities that GNU social needs to do, like broadcasting with OStatus or -ActivityPub, SMS, XMPP messages and TwitterBridge operations, can be 'queued' -and done by off-line bots instead. +Some activities that GNU social can do, like broadcasting with OStatus or +ActivityPub, XMPP messages and SMS operations, can be 'queued' and done by +asynchronous daemons instead. + +## Running Queues Run the queue handler with: @@ -14,89 +16,33 @@ GNU social uses Symfony, therefore the [documentation on queues](https://symfony.com/doc/current/messenger.html#deploying-to-production) might be useful. -TODO queuing +## Definitions -#### OpportunisticQM plugin +* Message - A Message holds the data to be handled (a variable) and the queue name (a string). +* QueueHandler - A QueueHandler is an event listener that expects to receive data from the queue. +* Enqueuer - An Enqueuer is any arbitrary code that wishes to send data to a queue. +* Transporter - The Transporter is given a Message by an Enqueuer. The Transporter is responsible + for ensuring that the Message is passed to all relevant QueueHandlers. -This plugin is enabled by default. It tries its best to do background -jobs during regular HTTP requests, like API or HTML pages calls. +## Using Queues -Since queueing system is enabled by default, notices to be broadcasted -will be stored, by default, into DB (table queue_item). +Queues are akin to events. -Whenever it has time, OpportunisticQM will try to handle some of them. +In your plugin you can call `App\Core\Queue::enqueue` and send a message to +be handled by the queue: -This is a good solution whether you: +```php +Queue::enqueue($hello_world, 'MyFirstQueue'); +``` -* have no access to command line (shared hosting) -* do not want to deal with long-running PHP processes -* run a low traffic GNU social instance +and then receive with: -In other case, you really should consider enabling the queuedaemon for -performance reasons. Background daemons are necessary anyway if you wish -to use the Instant Messaging features such as communicating via XMPP. - -#### Queue deamon - -It's recommended you use the deamon, you must be able to run -long-running offline processes, either on your main Web server or on -another server you control. (Your other server will still need all the -above prerequisites, with the exception of Apache.) Installing on a -separate server is probably a good idea for high-volume sites. - -1. You'll need the "CLI" (command-line interface) version of PHP - installed on whatever server you use. - - Modern PHP versions in some operating systems have disabled functions - related to forking, which is required for daemons to operate. To make - this work, make sure that your php-cli config (/etc/php5/cli/php.ini) - does NOT have these functions listed under 'disable_functions': - - * pcntl_fork, pcntl_wait, pcntl_wifexited, pcntl_wexitstatus, - pcntl_wifsignaled, pcntl_wtermsig - - Other recommended settings for optimal performance are: - * mysqli.allow_persistent = On - * mysqli.reconnect = On - -2. If you're using a separate server for queues, install StatusNet - somewhere on the server. You don't need to worry about the - .htaccess file, but make sure that your config.php file is close - to, or identical to, your Web server's version. - -3. In your config.php files (on the server where you run the queue - daemon), set the following variable: - - $config['queue']['daemon'] = true; - - You may also want to look at the 'Queues and Daemons' section in - this file for more background processing options. - -4. On the queues server, run the command scripts/startdaemons.sh. - -This will run the queue handlers: - -* queuedaemon.php - polls for queued items for inbox processing and - pushing out to OStatus, SMS, XMPP, etc. -* imdaemon.php - if an IM plugin is enabled (like XMPP) -* other daemons, like TwitterBridge ones, that you may have enabled - -These daemons will automatically restart in most cases of failure -including memory leaks (if a memory_limit is set), but may still die -or behave oddly if they lose connections to the XMPP or queue servers. - -It may be a good idea to use a daemon-monitoring service, like 'monit', -to check their status and keep them running. - -All the daemons write their process IDs (pids) to /var/run/ by -default. This can be useful for starting, stopping, and monitoring the -daemons. If you are running multiple sites on the same machine, it will -be necessary to avoid collisions of these PID files by setting a site- -specific directory in config.php: - - $config['daemon']['piddir'] = __DIR__ . '/../run/'; - -It is also possible to use a STOMP server instead of our kind of hacky -home-grown DB-based queue solution. This is strongly recommended for -best response time, especially when using XMPP. +```php +public function onMyFirstQueue($data): bool +{ + // Do something with $data + return Event::next; +} +``` +GNU social comes with a set of core queues with often wanted data: TODO Elaborate. \ No newline at end of file diff --git a/src/Core/Queue/Queue.php b/src/Core/Queue/Queue.php index c8a07678a8..ded1955f68 100644 --- a/src/Core/Queue/Queue.php +++ b/src/Core/Queue/Queue.php @@ -42,10 +42,12 @@ abstract class Queue } /** - * Enqueue a $message in a configured trasnport, to be handled by the $queue handler + * Enqueue a $message in a configured transport, to be handled by the $queue handler * - * @param object|string - * @param mixed $message + * @param mixed $message + * @param string $queue + * @param bool $high + * @param array $stamps */ public static function enqueue($message, string $queue, bool $high = false, array $stamps = []) {