Should fix issue #3027: twitter user avatars not getting imported.
Due to the change in URI, all twitter users that had been previously seen were getting new profile entries, which tried to save the same avatar. This would fail as Avatar.url has a unique index.
Note: now anything new seen in the last couple days in production will still potentially conflict.
There's a new menu layout in this version of the software. It was
implemented as a plugin in 0.9.x to avoid clashes with existing themes,
but we're going to break that compatibility in this version, so we're just going for it.
This change involved moving all the changes in NewMenuPlugin into the
default code that was calling it. In addition, since
accountsettingsaction and connectsettingsaction differed only by menu,
I removed them, changed all references to them to the settingsmenu, and moved
the combined nav to its own class.
Let's put that episode behind us.
The CSS shim that was loaded by NewMenuPlugin for certain themes and certain actions
was removed.
'admin' is a pretty common username that people try when installing;
it was blacklisted because all of our admin panels were at /admin/*,
which would conflict with the admin user's namespace.
Changed the location of all admin panels to /panel/*, blacklisted the
nickname 'panel', and allowed 'admin'. Tested with a fresh install;
seems to work great.
Original fixes in c169dcb5221cf3dd452c291bf97374bb459cc5b9; didn't get merged in 39cad55711 because the code had been broken out to another file, but manual merge went smooth.
These affect twitterstatusfetcher.php on all 32-bit installs and some 64-bit installs (depending on whether the version of the JSON library reads the large numbers as long or double internally). 64-bit bug is harder to see as it tends to manifest as off-by-one due to losing a bit of precision off the end.
Added StartRegistrationTry/EndRegistrationTry calls into those three, and moved the actual recording hook to EndUserRegister which is guaranteed to be called from User::register (so we don't need to worry about other auth methods forgetting to call the other UI-code hooks).
Fixes for Twitter bridge breakage on 32-bit servers. New "Snowflake" 64-bit IDs have become too big to fit in the integer portion of double-precision floats, so to reliably use these IDs we need to pull the new string form now.
Machines with 64-bit PHP installation should have had no problems (except on Windows, where integers are still 32 bits)
Conflicts:
plugins/TwitterBridge/twitterimport.php <- as this hasn't been broken out, the import code is NOT FULLY UPDATED HERE.
$config['twitter']['ignore_errors'] = true;
A longer-term solution is to patch up the indirect retry handling to count retries better, or delay for later retry sensibly.
Most annoying error case being where the notice was already faved or deleted on Twitter! :)
Such errors will now just fail out and log a note to the syslog -- the rest of what we were doing will continue on unhindered, so you can still delete, favorite, etc and it just won't sync the info over in that case.
This option may be useful for intranet sites that don't have direct access to the internet, as they may be unable to successfully fetch those resources.
When the retweet failed with a 403 error (say due to it being a private tweet, which can't be retweeted) we would end up mishandling the return value from our internal error handling.
Instead of correctly discarding the message and closing out the queue item, we ended up trying to save a bogus twitter<->local ID mapping, which threw another exception and lead the queue system to re-run it.
- Fixed the logic check and return values for the retweet case in broadcast_twitter().
- Added doc comments explaining the return values on some functions in twitter.php
- Added check on Notice_to_status::saveNew() for empty input -- throw an exception before we try to actually insert into db. :)
Introduce a table mapping notices to Twitter statuses. Initialize
this table at checkSchema() time. Save the mapping when we push
or pull statuses. Use the table to determine if a notice has a
Twitter equivalent.
addPlugin('TwitterBridge', array('adminImportControl' => true, ....));
Added a note on the label that it requires manual daemon setup. (Note that by default the admin panel won't be shown, so it's no biggie to be hiding this for now.)
Note that much of that form is duplicated several times for Twitter, Facebook, and OpenID registrations -- these need to be refactored to avoid having multiple out-of-sync copies of code and messages.
* added locale/en/LC_MESSAGES/statusnet.po to make it easier to start customizing English texts
* added notes to locale/README about customizing and how to disable languages you haven't customized
* renamed PO templates from *.po to *.pot to match general conventions and reduce confusion for people trying to find which file they're supposed to edit
Queue handlers for XMPP individual & firehose output now send their XML stanzas
to another output queue instead of connecting directly to the chat server. This
lets us have as many general processing threads as we need, while all actual
XMPP input and output go through a single daemon with a single connection open.
This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues
Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob.
This is based on Craig Andrews' work branch to generalize queues to take any
object, but conservatively leaving out the serialization for now.
Table updater (preserves any existing queued items) in db/rc3to09.sql
Code changes to watch out for:
* Queue handlers should now define a handle() method instead of handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is queued
directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage at
each run through the event loop to watch for memory leaks
To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the actual
notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice objects
to queues.
* Find a way to improve interactive performance of the database-backed queue
handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager. The
grouping used to split out the XMPP output queue is a bit awkward.
Key changes:
* Initialization code moved from common.php to StatusNet class;
can now switch configurations during runtime.
* As a consequence, configuration files must now be idempotent...
Be careful with constant, function or class definitions.
* Control structure for daemons/QueueManager/QueueHandler has been refactored;
the run loop is now managed by IoMaster run via scripts/queuedaemon.php
IoManager subclasses are woken to handle socket input or polling, and may
cover multiple sites.
* Plugins can implement notice queue handlers more easily by registering a
QueueHandler class; no more need to add a daemon.
The new QueueDaemon runs from scripts/queuedaemon.php:
* This replaces most of the old *handler.php scripts; they've been refactored
to the bare handler classes.
* Spawns multiple child processes to spread load; defaults to CPU count on
Linux and Mac OS X systems, or override with --threads=N
* When multithreaded, child processes are automatically respawned on failure.
* Threads gracefully shut down and restart when passing a soft memory limit
(defaults to 90% of memory_limit), limiting damage from memory leaks.
* Support for UDP-based monitoring: http://www.gitorious.org/snqmon
Rough control flow diagram:
QueueDaemon -> IoMaster -> IoManager
QueueManager [listen or poll] -> QueueHandler
XmppManager [ping & keepalive]
XmppConfirmManager [poll updates]
Todo:
* Respawning features not currently available running single-threaded.
* When running single-site, configuration changes aren't picked up.
* New sites or config changes affecting queue subscriptions are not yet
handled without a daemon restart.
* SNMP monitoring output to integrate with general tools (nagios, ganglia)
* Convert XMPP confirmation message sends to use stomp queue instead of polling
* Convert xmppdaemon.php to IoManager?
* Convert Twitter status, friends import polling daemons to IoManager
* Clean up some error reporting and failure modes
* May need to adjust queue priorities for best perf in backlog/flood cases
Detailed code history available in my daemon-work branch:
http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
Caching support will be added in future work after unit tests have been added.
* extlib: add PEAR HTTP_Request2 0.4.1 alpha
* extlib: update PEAR Net_URL2 to 0.3.0 beta for HTTP_Request2 compatibility
* moved direct usage of CURL and file_get_contents to HTTPClient class, excluding external-sourced libraries
* adapted GeonamesPlugin for new HTTPResponse interface
Note some plugins haven't been fully tested yet.
Caching support will be added in future work after unit tests have been added.
* extlib: add PEAR HTTP_Request2 0.4.1 alpha
* extlib: update PEAR Net_URL2 to 0.3.0 beta for HTTP_Request2 compatibility
* moved direct usage of CURL and file_get_contents to HTTPClient class, excluding external-sourced libraries
Note some plugins haven't been tested yet.
* 0.9.x: (247 commits)
Added in credits.
Use site's name for basic auth realm
Make apigroupcreate.php pass phpcs
Took out some unnecessary intializations
Implemented create group api
CamelCase all function names in the API code
These same params are used in most API actions; moved to base API class
Missed some of the references to the old TwitterApiAction - removed
Remove more redundant $formats
Remove dead code
Move all basic auth output and processing to base classes
$format is used by every API action. Set it in the base class.
Delete action/api.php and rename lib/twitterapi.php to lib/api.php
New actions for blocks via API
fix FBConnect so it doesn't muffle EndPrimaryNav
don't write session if it's unchanged
Fixed facebook connect primary nav to hide search option when site is private and user is not logged in
Fixed facebook connect primary nav to obey sms/twitter/openid settings
Fixed facebook connect login nav to obey openid settings
Fixed facebook connect nav to obey sms/twitter disabled
...
