gnu-social/lib/util/iomanager.php

197 lines
5.6 KiB
PHP
Raw Permalink Normal View History

Major refactoring of queue handlers to support running multiple sites in one daemon. Key changes: * Initialization code moved from common.php to StatusNet class; can now switch configurations during runtime. * As a consequence, configuration files must now be idempotent... Be careful with constant, function or class definitions. * Control structure for daemons/QueueManager/QueueHandler has been refactored; the run loop is now managed by IoMaster run via scripts/queuedaemon.php IoManager subclasses are woken to handle socket input or polling, and may cover multiple sites. * Plugins can implement notice queue handlers more easily by registering a QueueHandler class; no more need to add a daemon. The new QueueDaemon runs from scripts/queuedaemon.php: * This replaces most of the old *handler.php scripts; they've been refactored to the bare handler classes. * Spawns multiple child processes to spread load; defaults to CPU count on Linux and Mac OS X systems, or override with --threads=N * When multithreaded, child processes are automatically respawned on failure. * Threads gracefully shut down and restart when passing a soft memory limit (defaults to 90% of memory_limit), limiting damage from memory leaks. * Support for UDP-based monitoring: http://www.gitorious.org/snqmon Rough control flow diagram: QueueDaemon -> IoMaster -> IoManager QueueManager [listen or poll] -> QueueHandler XmppManager [ping & keepalive] XmppConfirmManager [poll updates] Todo: * Respawning features not currently available running single-threaded. * When running single-site, configuration changes aren't picked up. * New sites or config changes affecting queue subscriptions are not yet handled without a daemon restart. * SNMP monitoring output to integrate with general tools (nagios, ganglia) * Convert XMPP confirmation message sends to use stomp queue instead of polling * Convert xmppdaemon.php to IoManager? * Convert Twitter status, friends import polling daemons to IoManager * Clean up some error reporting and failure modes * May need to adjust queue priorities for best perf in backlog/flood cases Detailed code history available in my daemon-work branch: http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-13 03:57:15 +00:00
<?php
/**
* StatusNet, the distributed open-source microblogging tool
*
* Abstract class for i/o managers
*
* PHP version 5
*
* LICENCE: This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* @category QueueManager
* @package StatusNet
* @author Evan Prodromou <evan@status.net>
* @author Sarven Capadisli <csarven@status.net>
* @author Brion Vibber <brion@status.net>
* @copyright 2009-2010 StatusNet, Inc.
* @license http://www.fsf.org/licensing/licenses/agpl-3.0.html GNU Affero General Public License version 3.0
* @link http://status.net/
*/
abstract class IoManager
{
const SINGLE_ONLY = 0;
const INSTANCE_PER_SITE = 1;
const INSTANCE_PER_PROCESS = 2;
/**
* Factory function to get an appropriate subclass.
*/
public static function get() {
throw new MethodNotImplementedException(__METHOD__);
}
Major refactoring of queue handlers to support running multiple sites in one daemon. Key changes: * Initialization code moved from common.php to StatusNet class; can now switch configurations during runtime. * As a consequence, configuration files must now be idempotent... Be careful with constant, function or class definitions. * Control structure for daemons/QueueManager/QueueHandler has been refactored; the run loop is now managed by IoMaster run via scripts/queuedaemon.php IoManager subclasses are woken to handle socket input or polling, and may cover multiple sites. * Plugins can implement notice queue handlers more easily by registering a QueueHandler class; no more need to add a daemon. The new QueueDaemon runs from scripts/queuedaemon.php: * This replaces most of the old *handler.php scripts; they've been refactored to the bare handler classes. * Spawns multiple child processes to spread load; defaults to CPU count on Linux and Mac OS X systems, or override with --threads=N * When multithreaded, child processes are automatically respawned on failure. * Threads gracefully shut down and restart when passing a soft memory limit (defaults to 90% of memory_limit), limiting damage from memory leaks. * Support for UDP-based monitoring: http://www.gitorious.org/snqmon Rough control flow diagram: QueueDaemon -> IoMaster -> IoManager QueueManager [listen or poll] -> QueueHandler XmppManager [ping & keepalive] XmppConfirmManager [poll updates] Todo: * Respawning features not currently available running single-threaded. * When running single-site, configuration changes aren't picked up. * New sites or config changes affecting queue subscriptions are not yet handled without a daemon restart. * SNMP monitoring output to integrate with general tools (nagios, ganglia) * Convert XMPP confirmation message sends to use stomp queue instead of polling * Convert xmppdaemon.php to IoManager? * Convert Twitter status, friends import polling daemons to IoManager * Clean up some error reporting and failure modes * May need to adjust queue priorities for best perf in backlog/flood cases Detailed code history available in my daemon-work branch: http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-13 03:57:15 +00:00
/**
* Tell the i/o queue master if and how we can handle multi-site
* processes.
*
* Return one of:
* IoManager::SINGLE_ONLY
* IoManager::INSTANCE_PER_SITE
* IoManager::INSTANCE_PER_PROCESS
*/
public static function multiSite()
{
return IoManager::SINGLE_ONLY;
}
/**
* If in a multisite configuration, the i/o master will tell
* your manager about each site you'll have to handle so you
* can do any necessary per-site setup.
*
* The new site will be the currently live configuration during
* this call.
Major refactoring of queue handlers to support running multiple sites in one daemon. Key changes: * Initialization code moved from common.php to StatusNet class; can now switch configurations during runtime. * As a consequence, configuration files must now be idempotent... Be careful with constant, function or class definitions. * Control structure for daemons/QueueManager/QueueHandler has been refactored; the run loop is now managed by IoMaster run via scripts/queuedaemon.php IoManager subclasses are woken to handle socket input or polling, and may cover multiple sites. * Plugins can implement notice queue handlers more easily by registering a QueueHandler class; no more need to add a daemon. The new QueueDaemon runs from scripts/queuedaemon.php: * This replaces most of the old *handler.php scripts; they've been refactored to the bare handler classes. * Spawns multiple child processes to spread load; defaults to CPU count on Linux and Mac OS X systems, or override with --threads=N * When multithreaded, child processes are automatically respawned on failure. * Threads gracefully shut down and restart when passing a soft memory limit (defaults to 90% of memory_limit), limiting damage from memory leaks. * Support for UDP-based monitoring: http://www.gitorious.org/snqmon Rough control flow diagram: QueueDaemon -> IoMaster -> IoManager QueueManager [listen or poll] -> QueueHandler XmppManager [ping & keepalive] XmppConfirmManager [poll updates] Todo: * Respawning features not currently available running single-threaded. * When running single-site, configuration changes aren't picked up. * New sites or config changes affecting queue subscriptions are not yet handled without a daemon restart. * SNMP monitoring output to integrate with general tools (nagios, ganglia) * Convert XMPP confirmation message sends to use stomp queue instead of polling * Convert xmppdaemon.php to IoManager? * Convert Twitter status, friends import polling daemons to IoManager * Clean up some error reporting and failure modes * May need to adjust queue priorities for best perf in backlog/flood cases Detailed code history available in my daemon-work branch: http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-13 03:57:15 +00:00
*/
public function addSite()
Major refactoring of queue handlers to support running multiple sites in one daemon. Key changes: * Initialization code moved from common.php to StatusNet class; can now switch configurations during runtime. * As a consequence, configuration files must now be idempotent... Be careful with constant, function or class definitions. * Control structure for daemons/QueueManager/QueueHandler has been refactored; the run loop is now managed by IoMaster run via scripts/queuedaemon.php IoManager subclasses are woken to handle socket input or polling, and may cover multiple sites. * Plugins can implement notice queue handlers more easily by registering a QueueHandler class; no more need to add a daemon. The new QueueDaemon runs from scripts/queuedaemon.php: * This replaces most of the old *handler.php scripts; they've been refactored to the bare handler classes. * Spawns multiple child processes to spread load; defaults to CPU count on Linux and Mac OS X systems, or override with --threads=N * When multithreaded, child processes are automatically respawned on failure. * Threads gracefully shut down and restart when passing a soft memory limit (defaults to 90% of memory_limit), limiting damage from memory leaks. * Support for UDP-based monitoring: http://www.gitorious.org/snqmon Rough control flow diagram: QueueDaemon -> IoMaster -> IoManager QueueManager [listen or poll] -> QueueHandler XmppManager [ping & keepalive] XmppConfirmManager [poll updates] Todo: * Respawning features not currently available running single-threaded. * When running single-site, configuration changes aren't picked up. * New sites or config changes affecting queue subscriptions are not yet handled without a daemon restart. * SNMP monitoring output to integrate with general tools (nagios, ganglia) * Convert XMPP confirmation message sends to use stomp queue instead of polling * Convert xmppdaemon.php to IoManager? * Convert Twitter status, friends import polling daemons to IoManager * Clean up some error reporting and failure modes * May need to adjust queue priorities for best perf in backlog/flood cases Detailed code history available in my daemon-work branch: http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-13 03:57:15 +00:00
{
/* no-op */
}
/**
* This method is called when data is available on one of your
* i/o manager's sockets. The socket with data is passed in,
* in case you have multiple sockets.
*
* If your i/o manager is based on polling during idle processing,
* you don't need to implement this.
*
* @param resource $socket
* @return boolean true on success, false on failure
*/
public function handleInput($socket)
{
return true;
}
/**
* Return any open sockets that the run loop should listen
* for input on. If input comes in on a listed socket,
* the matching manager's handleInput method will be called.
*
* @return array of resources
*/
function getSockets()
{
return array();
}
/**
* Maximum planned time between poll() calls when input isn't waiting.
* Actual time may vary!
*
* When we get a polling hit, the timeout will be cut down to 0 while
* input is coming in, then will back off to this amount if no further
* input shows up.
*
* By default polling is disabled; you must override this to enable
* polling for this manager.
*
* @return int max poll interval in seconds, or 0 to disable polling
*/
function pollInterval()
{
return 0;
}
/**
* Request a maximum timeout for listeners before the next idle period.
* Actual wait may be shorter, so don't go crazy in your idle()!
* Wait could be longer if other handlers performed some slow activity.
*
* Return 0 to request that listeners return immediately if there's no
* i/o and speed up the idle as much as possible; but don't do that all
* the time as this will burn CPU.
*
* @return int seconds
*/
function timeout()
{
return 60;
}
/**
* Called by IoManager after each handled item or empty polling cycle.
* This is a good time to e.g. service your XMPP connection.
*
* Doesn't need to be overridden if there's no maintenance to do.
*/
function idle()
{
return true;
}
/**
* The meat of a polling manager... check for something to do
* and do it! Note that you should not take too long, as other
* i/o managers may need to do some work too!
*
* On a successful hit, the next poll() call will come as soon
* as possible followed by exponential backoff up to pollInterval()
* if no more data is available.
*
* @return boolean true if events were hit
*/
public function poll()
{
return false;
}
/**
* Initialization, run when the queue manager starts.
* If this function indicates failure, the handler run will be aborted.
*
* @param IoMaster $master process/event controller
* @return boolean true on success, false on failure
*/
public function start($master)
{
$this->master = $master;
return true;
}
/**
* Cleanup, run when the queue manager ends.
* If this function indicates failure, a warning will be logged.
*
* @return boolean true on success, false on failure
*/
public function finish()
{
return true;
}
/**
* Ping iomaster's queue status monitor with a stats update.
* Only valid during input loop!
*
* @param string $counter keyword for counter to increment
*/
public function stats($counter, $owners=array())
{
$this->master->stats($counter, $owners);
}
}