Brion Vibber 55b1f3d84c Scalability work on user backup stream generation. ... UserActivityStream -- used to create a full activity stream including subscriptions, favorites, notices, etc -- normally buffers everything into memory at once. This is infeasible for accounts with long histories of serious usage; it can take tens of seconds just to pull all records from the database, and working with them all in memory is very likely to hit resource limits. This commit adds an alternate mode for this class which avoids pulling notices until during the actual output. Instead of pre-sorting and buffering all the notices, empty spaces between the other activities are filled in with notices as we're making output. This means more smaller queries spread out during operations, and less stuff kept in memory. Callers (backupaccount action, and backupuser.php) which can stream their output pass an $outputMode param of UserActivityStream::OUTPUT_RAW, and during getString() it'll send straight to output as well as slurping the notices in this extra funky fashion. Other callers will let it default to the OUTPUT_STRING mode, which keeps the previous behavior. There should be a better way to do this, swapping out the stringer output for raw output more consitently.		2011-02-25 12:15:38 -08:00
..
allsites.php
apidocs.config	Some initial Doxygen-based documentation for the API	2010-04-22 13:11:49 -07:00
backupuser.php	Scalability work on user backup stream generation.	2011-02-25 12:15:38 -08:00
checkschema.php	make checkschema.php executable (which makes sense, cause the README says to run it :-) )	2009-11-30 17:00:00 -05:00
clear_jabber.php	clear_jabber.php script to clear confirmed jabber/xmpp addresses from one or more accounts	2010-11-15 13:26:42 -08:00
clearcache.php	clearcache.php column flag was conflicting with default flag	2010-02-03 12:13:20 -05:00
cleardb.sh
command.php	add an option to run a command as the site owner	2010-03-28 14:38:07 -04:00
commandline.inc	printfv and printfq functions in commandline.inc	2010-09-21 14:06:39 -04:00
console.php	quick fix to console.php: don't save blank lines into readline history	2010-01-25 09:41:40 -08:00
createsim.php	don't choke on dupelimit in createsim.php	2010-02-26 12:19:33 -05:00
decache.php	Allow scripts/decache.php to blow out cache for objects that don't exist (anymore).	2010-02-08 15:32:20 -08:00
delete_status_network.sh	Fixes for delete_status_network.sh:	2010-11-22 11:10:10 -08:00
deletegroup.php	scripts/deletegroup.php -- basic CLI script to delete a group by id or local nickname. Like deleteuser.php, this can be used in batch runs by providing the -y override.	2010-11-18 14:40:05 -08:00
deleteuser.php	fix long options on deleteuser.php	2010-01-11 13:24:40 -08:00
docgen.php	docgen.php no longer depends on having a working installation; uses Console_GetOpt directly and regexes the STATUSNET_VERSION from common.php	2010-04-21 20:20:43 +02:00
doxygen.tmpl	Add scripts/docgen.php to build basic doxygen HTML docs from doc comments, either for core or a given plugin.	2010-03-15 09:42:25 -07:00
fixup_blocks.php	fixup_blocks.php: finds any stray subscriptions in violation of blocks, and removes them.	2010-04-07 11:40:16 -07:00
fixup_conversations.php
fixup_deletions.php	fixup_deletions.php script to look for notices posted by now-deleted profiles and remove them.	2010-03-23 17:24:01 -07:00
fixup_files.php	Add forgotten scripts/fixup_files.php to clean up "the h bug"	2010-03-11 11:52:19 -08:00
fixup_group_uri.php	scripts/fixup_group_uri.php to fill in empty user_group.uri entries; needed before changing domain names on sites that are hosting groups for remote users	2010-10-04 15:13:27 -07:00
fixup_hashtags.php
fixup_inboxes.php
fixup_notices_rendered.php
fixup_replies.php
fixup_status_network.php	try/catch just in case..	2010-07-28 12:04:37 -04:00
fixup_utf8.php	Fix a few typos	2010-03-04 23:30:04 -05:00
flushrouter.php	a quick script to flush the router from cache if needed	2010-11-30 10:27:32 -05:00
flushsite.php	a script to flush site	2010-03-09 10:55:48 -05:00
getpiddir.php
getvaliddaemons.php	Major refactoring of queue handlers to support running multiple sites in one daemon.	2010-01-12 20:45:09 -08:00
handlequeued.php	XMPP queued output & initial retooling of DB queue manager to support non-Notice objects.	2010-01-21 22:40:35 -08:00
importtwitteratom.php	Always specify UTF-8 targt charset for html_entity_decode(); default is 8-bit ISO-8859-1 which causes things to break when we later pass them through things that expect to work with UTF-8. For instance, running through preg_replace() with the /u option results in NULL, leading to problems with OStatus and SubMirror generating their plaintext versions and doing length-cropping.	2010-09-30 11:29:31 -07:00
init_conversation.php	init_conversation.php script to copy old notice conversations into the conversation table	2010-02-25 17:56:01 -08:00
initializeinbox.php	actually use the user id instead of 0 ;)	2010-01-13 20:46:44 -08:00
install_cli.php	Always specify UTF-8 targt charset for html_entity_decode(); default is 8-bit ISO-8859-1 which causes things to break when we later pass them through things that expect to work with UTF-8. For instance, running through preg_replace() with the /u option results in NULL, leading to problems with OStatus and SubMirror generating their plaintext versions and doing length-cropping.	2010-09-30 11:29:31 -07:00
maildaemon.php	Add an IMAP daemon so StatusNet can process incoming user posts via catch-all mailbox (in addition to the pre-existing script alias method)	2010-01-08 18:52:43 -05:00
make-release.php	Quickie script to aid in building release tarballs -- encapsulates the archive-i18n gen-changelog gen-retar steps.	2010-10-29 14:34:27 -07:00
makegroupadmin.php	Revert "* [Cc]an't -> [Cc]annot"	2009-11-08 23:28:51 +01:00
moveuser.php	Add a scary 'experimental feture' warning & are-you-sure prompt on moveuser.php	2011-01-19 15:13:46 -08:00
queuectl.php	Control channel for queue daemons to request graceful shutdown, restart, or update to listen to a newly added or reconfigured site.	2010-01-26 11:49:49 -08:00
queuedaemon.php	Detect when queuedaemon/xmppdaemon parent processes die and kill the child processes.	2010-03-10 11:54:00 -08:00
rebuilddb_psql.sh
rebuilddb.sh
registeruser.php	Revert "* [Cc]an't -> [Cc]annot"	2009-11-08 23:28:51 +01:00
reportsnapshot.php
restoreuser.php	Make restoreuser use new FeedImporter queue handler	2010-12-17 16:27:20 -05:00
SearchMonkey-MrH.0.txt
SearchMonkey-Om3.0.txt
SearchMonkey-yQP.0.txt
sendemail.php	Add scripts/sendemail.php to send email to a user's address. Updated setup_status_network.sh to create a user with the site's nick, accept site tags, and send a mail to the user (if a template is set)	2010-01-27 15:12:19 -08:00
sessiongc.php
setconfig.php	Allow setconfig.php to Print Entire Config	2010-04-15 23:02:29 +02:00
setpassword.php
settag.php	normalizing tags for status_network	2010-07-20 17:34:58 -07:00
setup_status_network.sh	fix typo in the comment (for better copy & pasting)	2010-09-09 13:14:52 -04:00
setup.cfg.sample	add additional post-install shell script option for setup_status_network.sh to do any other site-specific setup	2010-01-27 15:37:18 -08:00
showcache.php	showcache.php column flag was conflicting with default flag	2010-02-03 12:13:07 -05:00
showplugins.php	showplugins.php: dump list of activated plugins and their settings on this site	2010-04-12 11:50:42 -07:00
showtable.php
sitemap.php	Revert "* [Cc]an't -> [Cc]annot"	2009-11-08 23:28:51 +01:00
startdaemons.sh
statusnet.spec
stopdaemons.sh	Add an RSSCloud queue handler daemon	2010-01-05 23:19:13 -08:00
strip_geo.php	Fix for strip_geo's decaching; also added --all option to run over all notices by given profile to help in fixing up cache inconsistencies	2010-04-09 11:37:29 -07:00
uncache_users.php
update_po_templates.php	Remove do_translatewiki_plugin(). No longer needed.	2010-09-18 00:23:38 +02:00
update_translations.php	Update group ID for translations export.	2010-09-18 23:50:36 +02:00
updateavatarurl_group.php	Add Script To Update Group Avatar URLs	2010-02-18 13:44:16 -05:00
updateavatarurl.php	Update Avatar URL Did Weird Stuff.	2010-02-18 12:14:34 -05:00
updatelocation.php	Script to update the location ID for users	2009-12-31 12:38:58 -10:00
updateprofileurl.php	Script to update profile URLs	2010-02-03 14:58:29 -05:00
useremail.php	Include Unconfirmed Addresses Too.	2010-01-13 10:27:50 -05:00
userrole.php	userrole.php will take a profile id for remote profiles	2010-10-26 11:19:47 -04:00
xmppdaemon.php	Runtime check for known bad PHP versions with 64-bit stream_select() bug in xmppdaemon.php.	2010-03-18 11:08:27 -07:00