UserActivityStream -- used to create a full activity stream including subscriptions, favorites, notices, etc -- normally buffers everything into memory at once. This is infeasible for accounts with long histories of serious usage; it can take tens of seconds just to pull all records from the database, and working with them all in memory is very likely to hit resource limits.
This commit adds an alternate mode for this class which avoids pulling notices until during the actual output. Instead of pre-sorting and buffering all the notices, empty spaces between the other activities are filled in with notices as we're making output. This means more smaller queries spread out during operations, and less stuff kept in memory.
Callers (backupaccount action, and backupuser.php) which can stream their output pass an $outputMode param of UserActivityStream::OUTPUT_RAW, and during getString() it'll send straight to output as well as slurping the notices in this extra funky fashion.
Other callers will let it default to the OUTPUT_STRING mode, which keeps the previous behavior.
There should be a better way to do this, swapping out the stringer output for raw output more consitently.
http://status.net/open-source/issues/2442
Notes:
* Mapstraction causes JavaScript errors in XHTML mode, breaking our code if we're run later so the link doesn't work to get back to Desktop.
* not 100% sure how safe feature detection is here?
* Currently will be useless but visible links if no JS available; need to fall back to server-side for limited browsers
Regressions caused by bad refactoring in commit 21feac3bea.
Test cases in tests/CommandInterpreterTest.php were made against the pre-refactoring code, and now check out with the fixed code.
Failures were caused by not changing logic structure when moving from multiple exit points (each if point would return directly with a null or an object) to setting a result variable and then falling through to a common exit point. Without the if statements being restructured, the result variable would just get overridden by the next case.
I've extended the rights framework (centering on the Right class and Profile::hasRight()) to cover
Web login and API use. This will make it possible to prevent login and API use by users.
I added two new Right constants to the Right class: WEBLOGIN and API. I check these rights using
Profile::hasRight() when initializing users. If the rights check fails, I throw an exception.
I created a new AuthorizationException class for this particular
exception, in order to allow a different UI for these kinds of exceptions (or whatever).
* 0.9.x:
* update translator documentation. * remove superfluous whitespace. * tab to spaces. * add FIXME for undocumented class.
* update translator documentation. * remove superfluous whitespace. * L10n updates. * small refactoring in publicrss.php. * remove PHP4-isms
Localisation updates from http://translatewiki.net.
Double quotes to single quotes.
* improve L10n consistency for English. For example proper punctuation for all button and label titles. * fix some i18n bugs (in-message variables). * update/add translator documentation. * remove superfluous whitespace.
add path separators for Plugin::path()
argument to send email summary to all users on all sites
fix indentation in siteemailsummaryhandler
fix indentation in sendemailsummary.php
fix indentation in Email_summary_status.php
fix indentation in EmailSummaryPlugin.php
fix indentation in usermailsummaryhandler.php
Our mailbox actions (inbox and outbox) were doing their own display of
messages. This was causing issues with especially showmessage, which
since the more rigourous nickname checks were added, no longer works as
a mailbox subclass.
I've taken the time to rip out the message listing code from MailboxAction
and moved it to a MessageList widget. The different mailboxes now have their
own subclasses that show the correct profile in the list.
This entry was using the strings 'true' and 'false' instead of literal booleans, which could confuse clients expecting literal booleans as in other places and on Twitter in this place.
Format's already available as a member variable, so use it!
Fixes some error reponses in api/statusnet/groups/leave.json which were coming through as XML.
May fix some others as well.
$config['site']['logperf'] = true; // to record & dump total hits of each type and the runtime to syslog
$config['site']['logperf_detail'] = true; // very verbose -- dump the individual cache keys and queries as they get used (may contain private info in some queries)
Seeing 180 cache gets on a timeline page seems not unusual currently; since these run in serial, even relatively small roundtrip times can add up heavily.
We should consider ways to reduce the number of round trips, such as more frequently storing compound objects or the output of processing in memcached.
Doing parallel multi-key lookups could also help by collapsing round-trip times, but might not be easy to fit into SN's object model. (For things like streams this should actually work pretty well -- grab the list, then when it's returned go grab all the individual items in parallel and return the list)
* dropped unnecessary join on notice table
* made the function actually static, since it makes no sense as an instance variable. The only caller (in AttachmentList) is updated.
In order to apply to PHP's POST processing, the MAX_FILE_SIZE field must appear *before* the file upload field. They were incorrectly placed after, where they had no effect on POST processing.
Part of the reported issue was previuosly fixed by dc497ed0 (smaller size images being blanked).
This commit fixes the remaining bug with original-size avatars being left as BMP (which could include the 96px size for instance, which could cause problems in browsers not supporting BMP natively)
Added ImageFile::copyTo() as a convenient alias for resizeTo() when not resizing; this performs the BMP/XPM/XBM->PNG conversion if needed, or copies the original file.
Copying instead of using move_uploaded_file() is fine here since:
a) the files are cleaned up on script completion anyway (vs moving to remove it)
b) we're already performing getimagesize() and possibly load/resize on the file before this point (vs needing to move the file into a usable area to work with open_basedir restrictions that prevent working directly with uploaded files in the temp dir; since this would fail anyway, we lose nothing)
ImageFile::preferredType() now works on $this->type instead of asking for one, to make it handier to use from outside. (This is still needed in order for calling code to generate a target filename.)
Recommended for future:
* additional consolidation between the various ways of uploading avatars (touched avatarsettings, grouplogo, and apiaccountupdateprofileimage with similar minor changes)
* consolidate type checks and file naming into Avatar class
There's a new menu layout in this version of the software. It was
implemented as a plugin in 0.9.x to avoid clashes with existing themes,
but we're going to break that compatibility in this version, so we're just going for it.
This change involved moving all the changes in NewMenuPlugin into the
default code that was calling it. In addition, since
accountsettingsaction and connectsettingsaction differed only by menu,
I removed them, changed all references to them to the settingsmenu, and moved
the combined nav to its own class.
Let's put that episode behind us.
The CSS shim that was loaded by NewMenuPlugin for certain themes and certain actions
was removed.
'admin' is a pretty common username that people try when installing;
it was blacklisted because all of our admin panels were at /admin/*,
which would conflict with the admin user's namespace.
Changed the location of all admin panels to /panel/*, blacklisted the
nickname 'panel', and allowed 'admin'. Tested with a fresh install;
seems to work great.
* 'testing' of gitorious.org:statusnet/mainline: (63 commits)
Add a scary 'experimental feture' warning & are-you-sure prompt on moveuser.php
fix wrong datatypes (saving string instead of array) in AtomPub notice processing
Account moving is a background activity
return a 409 Conflict when subscription already exists
OStatusPlugin does discovery in Profile::fromURI()
considerably more logging and error checking in AccountMover
add a log method to AccountMover
normalize accounts and check for return in HTTP for moving
move account-moving classes to their own libraries
execution protection on discovery.php
PHPCS discovery.php
Move discovery library from OStatus plugin to core
Revert "Revert "0.9.7alpha1""
first example of moving a user
Parse properties of links in XRD files
Add the Atom username to the XRD output
preserve activities in object
let callers pass in an XMLOutputter to output to
execution protection on discovery.php
PHPCS linkheader.php
...
Removed the free calls (unneeded since destructors now work), and added an error check w/ logging & an exception for future attempts to forward calls to nonexistent object.
We were checking the list as comma-delimited (per the description of it as comma-delimited), but in fact spaces are also accepted, and who knows what else.
We can make a lot of HTTP requests from the server side. This change
adds some configuration options for using an HTTP proxy, which can
cache hits from multiple sites (good for status.net-like services, for example).
Version 0.9.6 and below of StatusNet assume anything in <author> is a
Person. So, we include an <activity:subject> element, which will be
checked first by those versions of the code, only for group feeds.
At some point we'll take this out, but it's useful for now.
* adds Right::CREATEGROUP
* logic in Profile::hasRight() checks for silencing
* NewgroupAction checks for the permission before letting you see or process the form in the UI
* User_group::register() logic does a low-level check on the specified initial group admin, and rejects creation if that user doesn't have the right; guaranteeing that API methods etc will also have this restriction applied sensibly.
Moved most of the heavy-lifting for account restoration out of
restoreuser.php and into its own class, with the hope that we'll do
the work from the Web eventually.
The new DeleteaccountAction lets a user delete their own account
(subject to global rights set by the admin). It presents a form to
delete the account, with an "I am sure." text entry box.
It then schedules the account for deletion and logs the user out.
If a cache entry is dependent on the code that's running, upgrading
(or enabling/disabling plugins) can generate hard-to-track
inconsistencies.
This change adds a close-to-unique fingerprint of the running code to
some cache keys, so that if the fingerprint changes, the old values
are ignored and new values are used.
If the automated uniqueness fails, an administrator can add an extra
config value, $config['site']['build'], that's thrown into the key also.
If a cache entry is dependent on the code that's running, upgrading
(or enabling/disabling plugins) can generate hard-to-track
inconsistencies.
This change adds a close-to-unique fingerprint of the running code to
some cache keys, so that if the fingerprint changes, the old values
are ignored and new values are used.
If the automated uniqueness fails, an administrator can add an extra
config value, $config['site']['build'], that's thrown into the key also.
We had two ways to generate an activity entry from a notice; one through
Notice::asAtomEntry() and one through Notice::asActivity() and
Activity::asString(). The code paths had already diverged somewhat. I
took the conditions that were in Notice::asAtomEntry() and made sure
they were replicated in the other two functions. Then, I rewrote
Notice::asAtomEntry() to use the other two functions instead.
This change passes the ActivityGenerationTests unit tests, but there
may be some other stuff that's not getting covered.
common_shorten_links() can only access the web session's logged-in user, so never properly took user options into effect for posting via XMPP, API, mail, etc.
Adds an optional $user parameter on common_shorten_links(), and a $user->shortenLinks() as a clearer interface for that.
Tweaked some lower-level functions so $user gets passed down -- making the $notice_id param previously there for saving URLs at notice save time generalized a little.
Note also ticket #2919: there's a lot of duplicate code calling the shortening, checking the length, and reporting near-identical error messages. These should be consolidated to aid in code and translation maintenance.
commit 39fdd181d95d2c39a3ea1ca330b10a99a92b961f
Author: Evan Prodromou <evan@status.net>
Date: Mon Nov 29 10:37:49 2010 -0500
use cache key prefix for router cache key
commit 4cb9e56941922489b83d6425c059cf770991e68f
Author: Evan Prodromou <evan@status.net>
Date: Mon Nov 29 10:31:21 2010 -0500
use a unique hashkey based on the software version and loaded plugins
commit 44458b48aef719543e11f83b41fded65cbcb8be9
Author: Evan Prodromou <evan@status.net>
Date: Sat Nov 27 17:04:15 2010 -0500
cache the NUM object
commit 809c188307a9b4ada15f3d7fa573a6034341efef
Author: Evan Prodromou <evan@status.net>
Date: Sat Nov 27 15:44:12 2010 -0500
accelerate routing by pivoting paths on actions
commit 39fdd181d95d2c39a3ea1ca330b10a99a92b961f
Author: Evan Prodromou <evan@status.net>
Date: Mon Nov 29 10:37:49 2010 -0500
use cache key prefix for router cache key
commit 4cb9e56941922489b83d6425c059cf770991e68f
Author: Evan Prodromou <evan@status.net>
Date: Mon Nov 29 10:31:21 2010 -0500
use a unique hashkey based on the software version and loaded plugins
commit 44458b48aef719543e11f83b41fded65cbcb8be9
Author: Evan Prodromou <evan@status.net>
Date: Sat Nov 27 17:04:15 2010 -0500
cache the NUM object
commit 809c188307a9b4ada15f3d7fa573a6034341efef
Author: Evan Prodromou <evan@status.net>
Date: Sat Nov 27 15:44:12 2010 -0500
accelerate routing by pivoting paths on actions
This provides initial infrastructure for decoupling display names from internal canonical names, but continues to have us storing and using the canonical forms.
It should be/become possible to provide mixed-case and underscore-containing names in links, @-mention, !-group, etc, but we don't store those alternate forms generally.
max_id is not yet implemented, as it'll need support added to the search backends. (since_id we get 'for free' by just cropping off the list, it'll do for now)
- "To" drop-down list now defaults to showing "Select recipient:" instead of the first person on your list, reducing liklihood of accidentally sending a message to the wrong person.
- When there are no mutual subscribers to send to, instead of an empty list the list now shows 'No mutual subscribers.'
In both cases, attempting to send when the default is selected displays an error message.
I'm not disabling form elements in part because our themes right now don't show disabled button state correctly; we might want to tighten that up a bit more once fixed.
This option may be useful for intranet sites that don't have direct access to the internet, as they may be unable to successfully fetch those resources.
- workaround for providers that are skimpy on their data, such as missing width/height or thumbnail_width/thumbnail_height
- workaround for YFrog listing "image" instead of "photo" type
- generally more lax about formatting: if it comes back and looks kinda ok, we'll take it.
- discovery uses system HTML parser, should be more robust if the links include things like ampersands with proper HTML-level escaping
Newly supported:
- TwitPic: added a local function using TwitPic's API, since the oohembed implementation for TwitPic produced invalid output which Services_oEmbed rejects. (bug filed upstream)
Tweaked...
- Flickr: works, now using whitelist to use their endpoint directly instead of going through oohembed
- Youtube: worked around a bug in Services_oEmbed which broke the direct use of API discovery info, so we don't have to use oohembed.
Not currently working...
- YFrog: whitelisting their endpoint directly as the oohembed output is broken, but this doesn't appear to work currently as I think things are confused by YFrog's servers giving a '204 No Content' response on our HEAD checks on the original link.
