UserActivityStream -- used to create a full activity stream including subscriptions, favorites, notices, etc -- normally buffers everything into memory at once. This is infeasible for accounts with long histories of serious usage; it can take tens of seconds just to pull all records from the database, and working with them all in memory is very likely to hit resource limits.
This commit adds an alternate mode for this class which avoids pulling notices until during the actual output. Instead of pre-sorting and buffering all the notices, empty spaces between the other activities are filled in with notices as we're making output. This means more smaller queries spread out during operations, and less stuff kept in memory.
Callers (backupaccount action, and backupuser.php) which can stream their output pass an $outputMode param of UserActivityStream::OUTPUT_RAW, and during getString() it'll send straight to output as well as slurping the notices in this extra funky fashion.
Other callers will let it default to the OUTPUT_STRING mode, which keeps the previous behavior.
There should be a better way to do this, swapping out the stringer output for raw output more consitently.
file_quota is adjusted from the defined value to take into account the maximum upload size limits in PHP, or cropped to 0 if uploads are disabled.
This can be used by client apps to determine maximum size for an attachment.
Search highlighting was being done with a regex on raw HTML text, followed by a second regex undoing replacements within double-quoted attribute values.
This broke on imported Twitter messages, as the way we generate the markup uses single quotes on the attributes, which didn't get matched by the second regex.
I've replaced this do-then-undo cycle by dividing up the import HTML into freetext spans and tags; the freetext gets replaced, while the tags are left untouched.
* 0.9.x:
* update translator documentation. * remove superfluous whitespace. * tab to spaces. * add FIXME for undocumented class.
* update translator documentation. * remove superfluous whitespace. * L10n updates. * small refactoring in publicrss.php. * remove PHP4-isms
Localisation updates from http://translatewiki.net.
Double quotes to single quotes.
* improve L10n consistency for English. For example proper punctuation for all button and label titles. * fix some i18n bugs (in-message variables). * update/add translator documentation. * remove superfluous whitespace.
add path separators for Plugin::path()
argument to send email summary to all users on all sites
fix indentation in siteemailsummaryhandler
fix indentation in sendemailsummary.php
fix indentation in Email_summary_status.php
fix indentation in EmailSummaryPlugin.php
fix indentation in usermailsummaryhandler.php
The ShowmessageAction was using the MailboxAction to do its display of
a single direct message. Since we redid the nickname management, this
was breaking (MailboxAction requires a nickname argument,
ShowmessageAction does not, and nickname validation that used to
quietly fail now throws an exception).
I've moved the message list processing to its own widget class, so the
need to subclass MailboxAction has disappeared. I've rewritten this
action to use the MessageListItem widget, and it works fine now.
Our mailbox actions (inbox and outbox) were doing their own display of
messages. This was causing issues with especially showmessage, which
since the more rigourous nickname checks were added, no longer works as
a mailbox subclass.
I've taken the time to rip out the message listing code from MailboxAction
and moved it to a MessageList widget. The different mailboxes now have their
own subclasses that show the correct profile in the list.
In order to apply to PHP's POST processing, the MAX_FILE_SIZE field must appear *before* the file upload field. They were incorrectly placed after, where they had no effect on POST processing.
Part of the reported issue was previuosly fixed by dc497ed0 (smaller size images being blanked).
This commit fixes the remaining bug with original-size avatars being left as BMP (which could include the 96px size for instance, which could cause problems in browsers not supporting BMP natively)
Added ImageFile::copyTo() as a convenient alias for resizeTo() when not resizing; this performs the BMP/XPM/XBM->PNG conversion if needed, or copies the original file.
Copying instead of using move_uploaded_file() is fine here since:
a) the files are cleaned up on script completion anyway (vs moving to remove it)
b) we're already performing getimagesize() and possibly load/resize on the file before this point (vs needing to move the file into a usable area to work with open_basedir restrictions that prevent working directly with uploaded files in the temp dir; since this would fail anyway, we lose nothing)
ImageFile::preferredType() now works on $this->type instead of asking for one, to make it handier to use from outside. (This is still needed in order for calling code to generate a target filename.)
Recommended for future:
* additional consolidation between the various ways of uploading avatars (touched avatarsettings, grouplogo, and apiaccountupdateprofileimage with similar minor changes)
* consolidate type checks and file naming into Avatar class
There's a new menu layout in this version of the software. It was
implemented as a plugin in 0.9.x to avoid clashes with existing themes,
but we're going to break that compatibility in this version, so we're just going for it.
This change involved moving all the changes in NewMenuPlugin into the
default code that was calling it. In addition, since
accountsettingsaction and connectsettingsaction differed only by menu,
I removed them, changed all references to them to the settingsmenu, and moved
the combined nav to its own class.
Let's put that episode behind us.
The CSS shim that was loaded by NewMenuPlugin for certain themes and certain actions
was removed.
L10n updates
Remove superfluous whitespace
Number parameters in message when two or more are used
ClientException and ServerException should end with a period
* 'testing' of gitorious.org:statusnet/mainline: (63 commits)
Add a scary 'experimental feture' warning & are-you-sure prompt on moveuser.php
fix wrong datatypes (saving string instead of array) in AtomPub notice processing
Account moving is a background activity
return a 409 Conflict when subscription already exists
OStatusPlugin does discovery in Profile::fromURI()
considerably more logging and error checking in AccountMover
add a log method to AccountMover
normalize accounts and check for return in HTTP for moving
move account-moving classes to their own libraries
execution protection on discovery.php
PHPCS discovery.php
Move discovery library from OStatus plugin to core
Revert "Revert "0.9.7alpha1""
first example of moving a user
Parse properties of links in XRD files
Add the Atom username to the XRD output
preserve activities in object
let callers pass in an XMLOutputter to output to
execution protection on discovery.php
PHPCS linkheader.php
...
Removed the free calls (unneeded since destructors now work), and added an error check w/ logging & an exception for future attempts to forward calls to nonexistent object.
* adds Right::CREATEGROUP
* logic in Profile::hasRight() checks for silencing
* NewgroupAction checks for the permission before letting you see or process the form in the UI
* User_group::register() logic does a low-level check on the specified initial group admin, and rejects creation if that user doesn't have the right; guaranteeing that API methods etc will also have this restriction applied sensibly.
* avoid PHP notice from using wrong variable
* show a visible error instead of blank screen if no file submitted with restore form
* avoid PHP strict warning from using calling "non-static" DOMDocument::loadXML statically
* suppress PHP warning from XML parse errors
The new DeleteaccountAction lets a user delete their own account
(subject to global rights set by the admin). It presents a form to
delete the account, with an "I am sure." text entry box.
It then schedules the account for deletion and logs the user out.
Feed for group memberships, in activity streams format.
Shows a feed; has proper pagination; accepts activitystreams "join"
activities to start a new membership.
common_shorten_links() can only access the web session's logged-in user, so never properly took user options into effect for posting via XMPP, API, mail, etc.
Adds an optional $user parameter on common_shorten_links(), and a $user->shortenLinks() as a clearer interface for that.
Tweaked some lower-level functions so $user gets passed down -- making the $notice_id param previously there for saving URLs at notice save time generalized a little.
Note also ticket #2919: there's a lot of duplicate code calling the shortening, checking the length, and reporting near-identical error messages. These should be consolidated to aid in code and translation maintenance.
Now, when you first come up the checkbox will most likely be off and the button to create an address is grayed out.
Checking the box enables use of the 'new' button to generate an email address -- it's left disabled until you check the box, so you can't accidentally trip it.
Actually adding the address now enables the post-by-mail option, as well, thus ensuring that it's saved. WARNING: OTHER CHANGES ON THE FORM WILL STILL BE LOST.
Removing the address now disables the post-by-mail option, so it's not sitting around confusingly enabled but useless.
You can still disable the checkbox manually without removing the address, in case you want to keep it for later.
It's also still possible to actually save it in the state where the option is enabled, but there's no configured address, but that shouldn't happen too often. Possibly that should be prevented outright though.
max_id is not yet implemented, as it'll need support added to the search backends. (since_id we get 'for free' by just cropping off the list, it'll do for now)
is specified in the request the page is probably meant to be displayed
in a small webview of another application, so suppress header, aside
and footer.
RedirectingAction->returnToArgs() has been renamed to returnToPrevious() to avoid confusion with the existing Action->returnToArgs which gives the arguments that would be used to pass to one of those. :)
Switching to the correct function call gets it working.
Thanks to Siebrand for catching it when I could debug it live!
Replace with a NoticeList to have output include avatar and username -- but CSS still hides them in default theme.
Event::addHandler('ShowStreamNoticeList', 'awesome');
function awesome($notice, $action, &$pnl)
{
$pnl = new NoticeList($notice, $action);
return false;
}
* now ignoring if-modified-since if we failed an etag if-none-match comparison, per spec
* now including a hash of user id/nickname in most etags, so we'll update the view properly after login/logout
For API methods, checking the API-auth'ed user. (Many change results to include things like 'you're subscribed to this user' or 'this is one of your favorites', so user info is again needed)
There'll still be some last-modified stamps that aren't including user info properly, probably.
Currently only one custom theme may be uploaded per site, saved with the name 'custom' and stored into the local/themes subdirectory.
Administrators can upload a .ZIP archive containing a theme through the design admin panel; its contents are validated to ensure that only legit files are saved, and a 5M size quota is enforced.
Theme upload requires the zip extension for PHP; if not present, theme uploading is disabled by default.
Uploading and the custom CSS can be controlled via $config['theme_upload']['enabled'] and $config['custom_css']['enabled'].
Configurable directory/path/server for 'local' subdirectory (currently only as used for themes; local plugins not yet switched over)
Can set $config['local']['dir'] etc; not currently exposed in the admin panels.
Per-site directories on a separate themes server could be set up such as:
$config['local']['dir'] = '/path/to/themes/local/' . $_nickname;
$config['local']['server'] = 'themes.example.com';
$config['local']['path'] = '/local/' . $_nickname;
$config['local']['ssl'] = 'never';
Fixed typo in RedirectingAction when no return-to data provided in form submission.
RedirectingAction::returnToArgs() has been renamed to returnToPrevious() to avoid conflict with Action::returnToArgs() which returns arguments to be passed to other actions as return-to arguments. All callers should now be updated.
More profile settings actions will now redirect through a login form if visited as a GET request, as would be expected from a bookmark, link sent in e-mail etc.
The forms would already submit to SSL, but people are happier if they start on a secure page!
Note: this really should be done for sensitive/all URLs in index.php, but it seems a bit awkward to reconstruct the SSL version of the link atm. Cleanup todo!
* Fake_XMPP back to Queued_XMPP, refactor how we use it and don't create objects and load classes until we need them.
* fix fatal error in IM settings while waiting for a Jabber confirmation.
* Caching fix for user_im_prefs
* fix for saving multiple transport settings
* some fixes for AIM & using normalized addresses for lookups
Users and administrators can set how long an URL can be before it's
shortened, and how long a notice can be before all its URLs are
shortened. They can also turn off shortening altogether.
Squashed commit of the following:
commit d136b39011
Author: Evan Prodromou <evan@status.net>
Date: Mon Apr 26 02:39:00 2010 -0400
use site and user settings to determine when to shorten URLs
commit 1e1c851ff3
Author: Evan Prodromou <evan@status.net>
Date: Mon Apr 26 02:38:40 2010 -0400
add a method to force shortening URLs
commit 4d29ca0b91
Author: Evan Prodromou <evan@status.net>
Date: Mon Apr 26 02:37:41 2010 -0400
static method for getting best URL shortening service
commit a9c6a3bace
Author: Evan Prodromou <evan@status.net>
Date: Mon Apr 26 02:37:11 2010 -0400
allow 0 in numeric entries in othersettings
commit 767ff2f7ec
Author: Evan Prodromou <evan@status.net>
Date: Mon Apr 26 02:36:46 2010 -0400
allow 0 or blank string in inputs
commit 1e21af42a6
Author: Evan Prodromou <evan@status.net>
Date: Mon Apr 26 02:01:11 2010 -0400
add more URL-shortening options to othersettings
commit 869a6be0f5
Author: Evan Prodromou <evan@status.net>
Date: Sat Apr 24 14:22:51 2010 -0400
move url shortener superclass to lib from plugin
commit 9c0c9863d5
Author: Evan Prodromou <evan@status.net>
Date: Sat Apr 24 14:20:28 2010 -0400
documentation and whitespace on UrlShortenerPlugin
commit 7a1dd5798f
Author: Evan Prodromou <evan@status.net>
Date: Sat Apr 24 14:05:46 2010 -0400
add defaults for URL shortening
commit d259c37ad2
Author: Evan Prodromou <evan@status.net>
Date: Sat Apr 24 13:40:10 2010 -0400
Add User_urlshortener_prefs
Add a table for URL shortener prefs, a corresponding class, and the
correct mumbo-jumbo in statusnet.ini to make everything work.
I added the statusnet: prefix to the xml output. This prefix should
be declared on the root element of all relevant XML output.
I also added two StatusNet-specific fields:
* statusnet:html - rendered HTML. Clients shouldn't have to guess
at the correct HTML rendering for notices, especially since some
of the links depend on context.
* statusnet:profile_url - profile URL for a user. You can't count
on a user being a local user in a distributed microblogging world.
So, this shows the explicit profile_url.
Fixed a few inconsistencies in the process. Apparently this code has been copied to imsettings.php, and some cases of 'email' had not been changed to 'IM'. Added FIXME for using globals/macros (?) in messages like "%%site.name%%". For easier/more consistent translation, this should preferably be plain %s parameters.
Note that much of that form is duplicated several times for Twitter, Facebook, and OpenID registrations -- these need to be refactored to avoid having multiple out-of-sync copies of code and messages.
Note that much of that form is duplicated several times for Twitter, Facebook, and OpenID registrations -- these need to be refactored to avoid having multiple out-of-sync copies of code and messages.
Refactored some of the returnto handling code. It looks like we have several different ways of handling this in the software, icky!
Marked the session-based functions with fixmes (they'll stomp on other forms when multiple tabs/windows are used) and combined some commonish bits of code between ProfileFormAction and the group block & makeadmin actions where they're using hidden form parameters. Extended that to allow passing dynamic parameters (eg 'page') as well as static ones (action, target user/group).
Setting focus into the form to make it easier to submit by keyboard. Enter or space will now trigger the button right off without having to mouse around to find the button after we've clicked into the conf form, and keyboarders can also get to the 'no' button more easily.
Setting focus into the form to make it easier to submit by keyboard. Enter or space will now trigger the button right off without having to mouse around to find the button after we've clicked into the conf form, and keyboarders can also get to the 'no' button more easily.
Conflicts:
actions/imsettings.php
lib/jabber.php
Made a quick attempt to merge the new JID validation into the XmppPlugin, have not had a chance to test that version live yet.
Should also move over the test cases.
This reverts commit 260f00d60b.
As mentioned in
260f00d60b (comment_8367)
Reverting this merge until the bugs are fixed and there is a general
agreement on the need for this enhancement.
Basic splitting/validation code submitted via http://status.net/wiki/XMPP/JID_validation -- Copyright 2009 Patrick Georgi <patrick@georgi-clan.de> Licensed under ISC-L, which is compatible with everything else that keeps the copyright notice intact.
Added PEAR Net_IDNA package to extlib to handle IDN normalization (also used by Validate's email verifier if present).
* added test suite, supplemented my own test cases with JID validation and normalization test cases from libpurple
* follows XMPP rules for validation of name part
* fixes for normalization with non-ASCII names
* will do domain checks if $config['email']['check_domain'] is on, checking for an XMPP-server SRV record or any lookup. (We don't actually need to ping those direct though.)
* some more obscure stringprep validation rules aren't quite followed yet, but we err on the side of permissiveness.
* we still don't actually let you save your address with a resource on it, as we strip resources when looking up users who've sent us presence or message updates. I would recommend saving the outgoing resource as a separate field if/when we add that..?
* respect count instead of listing all groups
* respect page parameter
* don't spew notice on undefined $id
* don't spew notice on undefined $group->homepage_url (dropped the element since there's nothing to go in it)
* respect count instead of listing all groups
* respect page parameter
* don't spew notice on undefined $id
* don't spew notice on undefined $group->homepage_url (dropped the element since there's nothing to go in it)
While deletion is in progress, the account is locked with the 'deleted' role, which disables all actions with rights control.
Todo:
* Pretty up the notice on the profile page about the pending delete. Show status?
* Possibly more thorough account disabling, such as disallowing all use for login and access.
* Improve error recovery; worst case is that an account gets left locked in 'deleted' state but the queue jobs have gotten dropped out. This would leave the username in use and any undeleted notices in place.
Now using the correct order consistently (URL, then URI if http/s), and as a niceness measure skipping the redirect if the only URL we have stored is the local one. (Could happen if remote OStatus feed has tag URIs and no alt link.)
- switch 'en_US' to 'en', fixes the "admin panel switches to Arabic" bug
- tweak setting descriptions to clarify that most of the time we'll be using browser language
- add a backend switch to disable language detection (should this be exposed to ui?)
* 'testing' of gitorious.org:statusnet/mainline:
init_conversation.php script to copy old notice conversations into the conversation table
cache results of webfinger lookups
Parse an hcard for hints, if available
use new hcard method for webfinger
add hkit for hCard parsing
use new dedicated hcard method for Webfinger profile
Add an hcard action
A dedicated hcard action for users. Our profile page includes
an hcard, but it's so full of other hcards that it's ambiguous which
one is the "real" one. So, this one make sense for meaning, "This is
my hcard."
- fix <activity:subject> generation so we get the profile info (what's available so far)
- use id instead of nickname for group join/leave forms so we can join/leave remote groups
while the rest of the groups UI remains limited to local groups
(plugins are responsible for making sure remote notifications and permission checks are done)
- fix remote notification when joining group through OStatus's remote subscribe form
I removed the check for local users in the subscribe button. I replaced
it with a more specific check for OMB 0.1 remote profiles, which you
can't use with this action.
I also took the opportunity to split the handle() method into
prepare() and handle(), and added PHPCS clean documentation.
- added rel="ostatus:attention" links for group delivery
- added events for plugins to override group profile/permalink pages
- pulled Notice::saveGroups up to save-time so we can override;
it's relatively cheap and gives us a clean list of target
groups for distrib time even with customized delivery.
- fixed notice::getGroups to return group objects as expected
- added some doc on new parameters to Notice::saveNew
- 'groups' list of group IDs to push to in place of parsing
- messages that come in via PuSH and contain local group targets
are delivered to local group members
- messages that come in via PuSH and contain remote group targets
are delivered to local members of the remote group
Todo:
- handle group posts that only come through Salmon
- handle conflicts in case something comes in both through Salmon and PuSH
- better source verification
- need a cleaner interface to look up groups by URI
- need a way to handle remote groups with conflicting names
I changed the way that tag: URIs are minted, so we now use the right
base. Ideally most of these would use HTTP URIs instead, but for
now at least they use the right base.
* detection of group feeds is currently a nasty hack based on presence of '/groups/' in URL -- should use some property on the feed?
* listing for the remote group is kinda cruddy; needs to be named more cleanly
* still need to establish per-author profiles (easier once we have the updated Atom code in)
* group delivery probably not right yet
* saving of group messages still triggering some weird behavior
Added support for since_id and max_id on group timeline feeds as a free extra. Enjoy!
* 'testing' of gitorious.org:statusnet/mainline:
Pull GeoRSS locations over OStatus feeds
Allow scripts/decache.php to blow out cache for objects that don't exist (anymore).
OStatus cleanup...
readme and version for beta5
Delete old Twitter user record when user changes screen name instead of updating. Simpler.
Store Twitter screen_name, not name, for foreign_user.nickname when saving Twitter user.
Actually store the timestamp on each nonce
OAuth app name should not be null
Fix issue with OAuth request parameters being parsed/stored twice when
- Fix cache handling in TwitterStatusFetcher
Added right margin for notice text. Helps Conversation notices look
Confirm dialog for reset OAuth consumer key and secret button
Always check for an OAuth request. This allows OAuth clients to set an
Linkify notice source when posting from registered OAuth apps
Suppress notice input box on OAuth authorization page
Better token revocation
Allow developers to delete OAuth applications
OAuth app names should be unique.
Prevents app statistic text from wrapping around avatar
Sentence case for app statistics
* Treat linkless feed posts as status updates; drop the "New post:" prefix and quotes on them.
* Use stable user IDs for atom/rss2 feed links instead of unstable nicknames
* Pull Atom feed preferentially when subscribing -- can now put the remote user's profile page straight into the feed subscription form and get to the right place.
* Clean up naming for push endpoints
* renamed FeedSub plugin to OStatus
* now setting avatar on subscriptions
* general fixes for subscription
* integrated PuSH hub to handle only user timelines on canonical ID url; sends updates directly
* set $config['feedsub']['nohub'] = true to test w/ foreign feeds that don't have hubs (won't actually receive updates though)
* a few bits of code documentation
* HMAC support for verified distributions (safest if sub setup is on HTTPS)
And a couple core changes:
* minimizing HTML output for exceptions in API requests to aid in debugging
* fix for rel=self link in apitimelineuser when id given
This does not not yet include any of the individual subscription management (Salmon notifications for sub/unsub, etc) nor a nice UI for user subscriptions.
Needs some further cleanup to treat posts as status updates instead of link references.
Defaulting to only looking at last 90 days of activity, can be adjusted up or down.
$config['tag']['cutoff'] = 86400 * 90;
$config['popular']['cutoff'] = 86400 * 90;
Per-user and per-group tag clouds do not use the cutoff (and it doesn't help with indexing on them).
Adds a robots.txt file to the site root. Defaults defined by
'robotstxt' section of config. New events StartRobotsTxt and
EndRobotsTxt to let plugins add information. Probably not
useful if path is not /, but won't hurt anything, either.
* testing: (130 commits)
HTTP auth provided is evaluated even if it's not required
Rename rc3to09.sql to rc3torc4.sql to avoid confusion if we add a last-minute change after this!
Add new oauth tables and modifications to 'consumer' table for rc4
Centred leaderboard ad
camelcase the uap param names
move leaderboard to after the header
Moved rectangle ad into aside and leaderboard to the right in header.
Aligning wide skyscraper to the right instead of left
CSS ids and classes fixed in UAPPlugin
wrong height for rectangle in BlankAd
Add the moved BlankAdPlugin
make BlankAd dir and change to use a 1x1 image
move BlankAdPlugin to its own dir
Add BlankAdPlugin to test ad layout in different themes
make uapplugin an abstract class
move UAP plugin to core
Lowercased switch cases in UAP Plugin
Plugin for Universal Ad Package. Outputs four most widely used ad types.
Add persistent:true property to Stomp messages so ActiveMQ doesn't decide to discard them even though persistence is enabled on the broker. :) (Thanks Aric!)
quick fix: use common_path() on realtime update JS so it works with the new JS path code (will pull from main server for now)
...
Conflicts:
actions/apioauthaccesstoken.php
actions/apioauthauthorize.php
actions/apioauthrequesttoken.php
actions/editapplication.php
actions/newapplication.php
lib/apiauth.php
lib/queuemanager.php
lib/router.php
We have about 10-12 JavaScript pages per Web page. They usually
are based on the same server as the Web pages, but since they're
static files, it makes sense to offload them to a lite server that
handles static files well.
This commit lets you set a separate Javascript server and path for the
default Javascript code in StatusNet.
Squashed commit of the following:
commit 139d1622fdafe5ad00c820224416d9021efc3234
Author: Evan Prodromou <evan@status.net>
Date: Wed Jan 27 11:30:24 2010 -0500
modules that call htmloutputter::script() don't prescribe js/ path
commit c6ca3174af73efed55eaed5ff1e2a3bdc77d2d87
Author: Evan Prodromou <evan@status.net>
Date: Wed Jan 27 11:28:07 2010 -0500
configurable server and path for javascript files
Moved much of the writing that happens when posting a notice to a new
queuehandler, distribqueuehandler. This updates tags, groups, replies
and inboxes at queue time (or at Web time, if queues are disabled).
To make this work well, I had to break up the monolithic
Notice::blowCaches() and make cache blowing happen closer to where
data is updated.
Squashed commit of the following:
commit 5257626c62750ac4ac1db0ce2b71410c5711cfa3
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 14:56:41 2010 -0500
slightly better handling of blowing tag memory cache
commit 8a22a3cdf6ec28685da129a0313e7b2a0837c9ef
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 01:42:56 2010 -0500
change 'distribute' to 'distrib' so not too long for dbqueue
commit 7a063315b0f7fad27cb6fbd2bdd74e253af83e4f
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 01:39:15 2010 -0500
change handle_notice() to handle() in distributqueuehandler
commit 1a39ccd28b9994137d7bfd21bb4f230546938e77
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 16:05:25 2010 -0500
error with queuemanager
commit e6b3bb93f305cfd2de71a6340b8aa6fb890049b7
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 01:11:34 2010 -0500
Blow memcache at different point rather than one big function for Notice class
commit 94d557cdc016187d1d0647ae1794cd94d6fb8ac8
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 00:48:44 2010 -0500
Blow memcache at different point rather than one big function for Notice class
commit 1c781dd08c88a35dafc5c01230b4872fd6b95182
Author: Evan Prodromou <evan@status.net>
Date: Wed Jan 20 08:54:18 2010 -0500
move broadcasting and distributing to new queuehandler
commit da3e46d26b84e4f028f34a13fd2ee373e4c1b954
Author: Evan Prodromou <evan@status.net>
Date: Wed Jan 20 08:53:12 2010 -0500
Move distribution of notices to new distribute queue handler
Queue handlers for XMPP individual & firehose output now send their XML stanzas
to another output queue instead of connecting directly to the chat server. This
lets us have as many general processing threads as we need, while all actual
XMPP input and output go through a single daemon with a single connection open.
This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues
Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob.
This is based on Craig Andrews' work branch to generalize queues to take any
object, but conservatively leaving out the serialization for now.
Table updater (preserves any existing queued items) in db/rc3to09.sql
Code changes to watch out for:
* Queue handlers should now define a handle() method instead of handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is queued
directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage at
each run through the event loop to watch for memory leaks
To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the actual
notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice objects
to queues.
* Find a way to improve interactive performance of the database-backed queue
handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager. The
grouping used to split out the XMPP output queue is a bit awkward.
Conflicts:
scripts/xmppdaemon.php
Queue handlers for XMPP individual & firehose output now send their XML stanzas
to another output queue instead of connecting directly to the chat server. This
lets us have as many general processing threads as we need, while all actual
XMPP input and output go through a single daemon with a single connection open.
This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues
Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob.
This is based on Craig Andrews' work branch to generalize queues to take any
object, but conservatively leaving out the serialization for now.
Table updater (preserves any existing queued items) in db/rc3to09.sql
Code changes to watch out for:
* Queue handlers should now define a handle() method instead of handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is queued
directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage at
each run through the event loop to watch for memory leaks
To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the actual
notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice objects
to queues.
* Find a way to improve interactive performance of the database-backed queue
handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager. The
grouping used to split out the XMPP output queue is a bit awkward.
Queue handlers for XMPP individual & firehose output now send their XML stanzas
to another output queue instead of connecting directly to the chat server. This
lets us have as many general processing threads as we need, while all actual
XMPP input and output go through a single daemon with a single connection open.
This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues
Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob.
This is based on Craig Andrews' work branch to generalize queues to take any
object, but conservatively leaving out the serialization for now.
Table updater (preserves any existing queued items) in db/rc3to09.sql
Code changes to watch out for:
* Queue handlers should now define a handle() method instead of handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is queued
directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage at
each run through the event loop to watch for memory leaks
To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the actual
notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice objects
to queues.
* Find a way to improve interactive performance of the database-backed queue
handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager. The
grouping used to split out the XMPP output queue is a bit awkward.
* Mostly punctuation updates so that the same message is used consistently in all of StatusNet.
* Some cases of "Title Case" removed, because that does not appear to be used consistently.
Consolidated several separate implementations of the same weighting algorithm into common_sql_weight() and fixed some bugs...
For MySQL, now using timestampdiff() instead of subtraction for the comparison, so we get sane results when the year doesn't match, and utc_timestamp() rather than now() so we don't get negative ages for recent items with local server timezone.
Unknown whether the same problems affect PostgreSQL, but note that it lacks the timestampdiff() SQL function.
Consolidated several separate implementations of the same weighting algorithm into common_sql_weight() and fixed some bugs...
For MySQL, now using timestampdiff() instead of subtraction for the comparison, so we get sane results when the year doesn't match, and utc_timestamp() rather than now() so we don't get negative ages for recent items with local server timezone.
Unknown whether the same problems affect PostgreSQL, but note that it lacks the timestampdiff() SQL function.
Moved the important parts of the location-argument-handling stuff
to a single function. Handles defaults and overrides correctly, and
easy to use. Changed Web and API channels to use it.
