Base problem is that our caching-on-insert interferes with relying on column default values; the cached object is missing those fields, so they appear to be empty (null) when the object is retrieved from cache.
Now explicitly setting them when inserting subscriptions, and cleaned up some code that had alternate code paths.
May also have made auto-subscription work for remote OStatus subscribers, but can't test until magic sigs are working again.
It's not currently used, and won't be efficient when we update the notice.profile_id_idx index to optimize for our id-based sorting when pulling user post lists for profile pages, feeds etc.
The subs_* functions in subs.php have made a lot of assumptions
about users versus profiles. I've refactored the functions to
be methods of the Subscription class instead, and to use Profile
objects throughout.
Some of the checks for blocks or existing subscriptions depended
on users or profiles, so I've moved those methods around a bit.
I've left stubs for the subs_* functions until we get time to replace
them.
No change in efficiency for the common case where nothing's deleted: does the same bulk fetch of just the notices we think we'll need as before, then if we turned up short keeps checking one by one until we've filled up to our $limit.
This can leave us with overlap between pages, but we already have that when new messages come in between clicks; seems to be the lesser of evils versus not getting a 'before' button.
More permanent fix for that will be to switch timeline paging in the UI to use notice IDs.
Moved much of the writing that happens when posting a notice to a new
queuehandler, distribqueuehandler. This updates tags, groups, replies
and inboxes at queue time (or at Web time, if queues are disabled).
To make this work well, I had to break up the monolithic
Notice::blowCaches() and make cache blowing happen closer to where
data is updated.
Squashed commit of the following:
commit 5257626c62750ac4ac1db0ce2b71410c5711cfa3
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 14:56:41 2010 -0500
slightly better handling of blowing tag memory cache
commit 8a22a3cdf6ec28685da129a0313e7b2a0837c9ef
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 01:42:56 2010 -0500
change 'distribute' to 'distrib' so not too long for dbqueue
commit 7a063315b0f7fad27cb6fbd2bdd74e253af83e4f
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 01:39:15 2010 -0500
change handle_notice() to handle() in distributqueuehandler
commit 1a39ccd28b9994137d7bfd21bb4f230546938e77
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 16:05:25 2010 -0500
error with queuemanager
commit e6b3bb93f305cfd2de71a6340b8aa6fb890049b7
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 01:11:34 2010 -0500
Blow memcache at different point rather than one big function for Notice class
commit 94d557cdc016187d1d0647ae1794cd94d6fb8ac8
Author: Evan Prodromou <evan@status.net>
Date: Mon Jan 25 00:48:44 2010 -0500
Blow memcache at different point rather than one big function for Notice class
commit 1c781dd08c88a35dafc5c01230b4872fd6b95182
Author: Evan Prodromou <evan@status.net>
Date: Wed Jan 20 08:54:18 2010 -0500
move broadcasting and distributing to new queuehandler
commit da3e46d26b84e4f028f34a13fd2ee373e4c1b954
Author: Evan Prodromou <evan@status.net>
Date: Wed Jan 20 08:53:12 2010 -0500
Move distribution of notices to new distribute queue handler
Sorting on notice.id when our primary selector was notice_inbox.user_id caused a filesort and table scan of the notice table.
Switchng to notice_inbox's notice_id means we can use our index, and everything comes right up.
Before:
mysql> explain SELECT notice.id AS id FROM notice JOIN notice_inbox ON notice.id = notice_inbox.notice_id WHERE notice_inbox.user_id = 18574 AND notice.repeat_of IS NULL ORDER BY notice.id DESC LIMIT 61 OFFSET 0;
+----+-------------+--------------+--------+------------------------------------+---------+---------+-------------------------------+--------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+--------+------------------------------------+---------+---------+-------------------------------+--------+----------------------------------------------+
| 1 | SIMPLE | notice_inbox | ref | PRIMARY,notice_inbox_notice_id_idx | PRIMARY | 4 | const | 102600 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | notice | eq_ref | PRIMARY | PRIMARY | 4 | stoica.notice_inbox.notice_id | 1 | Using index |
+----+-------------+--------------+--------+------------------------------------+---------+---------+-------------------------------+--------+----------------------------------------------+
After:
mysql> explain SELECT notice.id AS id FROM notice JOIN notice_inbox ON notice.id = notice_inbox.notice_id WHERE notice_inbox.user_id = 18574 AND notice.repeat_of IS NULL ORDER BY notice_id DESC LIMIT 61 OFFSET 0;
+----+-------------+--------------+--------+------------------------------------+---------+---------+-------------------------------+--------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+--------+------------------------------------+---------+---------+-------------------------------+--------+--------------------------+
| 1 | SIMPLE | notice_inbox | ref | PRIMARY,notice_inbox_notice_id_idx | PRIMARY | 4 | const | 102816 | Using where; Using index |
| 1 | SIMPLE | notice | eq_ref | PRIMARY,notice_repeatof_idx | PRIMARY | 4 | stoica.notice_inbox.notice_id | 1 | Using where |
+----+-------------+--------------+--------+------------------------------------+---------+---------+-------------------------------+--------+--------------------------+
self-subscription) via the API. Additionally, make it impossible
to block yourself or unsubscribe from yourself, period.
I also made User use the subs.php helper function for unsubscribing
during a block.
Hopefully, these changes will get rid of the problem of people
accidentally deleting their self-subscriptions once and for all
(knock on wood).
Added a right for new notices, realized that the hasRight() method
should be on the profile, and moved it.
Makes this a less atomic commit but that's the way it goes sometimes.
Added EmailAuthenticationPlugin
Added ReverseUsernameAuthenticationPlugin
Changed the StartChangePassword and EndChangePassword events to take a user, instead of a nickname
User::allowed_nickname was declared non-static, but used as if it was static, so I made the declaration static
The User_openid data object was explicitly listed as a related field to delete from in User::delete(); this class doesn't exist anymore by default since OpenID was broken out to a plugin.
Added UserDeleteRelated event for plugins to add related tables to delete from at user delete time.
This reverts commit 15f9c80c28.
So, so, elegant! And so, so, incorrect!
We can't have a user named 'notice' because that would interfere with
URLs like /notice/1234. However, there is no file named 'notice' in
the Web root.
If there were a way to automatically pull out the virtual paths in the
root dir, this may make sense. Until then, we keep track here.
Added a local directory for locally-installed software. This is where
you should put any code you write, themes, plugins, etc. so they don't
get stomped by upgrades.
"Rememberme" logins aren't allowed to make changes to an account
(since cookie-stealing is too easy). Users have to re-authenticate.
Previously, it was impossible to do so without having a username and
password; this change lets you do it with OpenID, too.
More PEAR coding standards global changes. Here, I've changed all
instances of TRUE to true and FALSE to false.
darcs-hash:20081223194428-84dde-cb1a1e6f679acd68e864545c4d4dd8752d6a6257.gz
Another huge change, for PEAR code standards compliance. Function
headers have to be in K&R style (opening brace on its own line),
instead of having the opening brace on the same line as the function
and parameters. So, a little perl magic found all the function
definitions and move the opening brace to the next line (properly
indented... usually).
darcs-hash:20081223193323-84dde-a28e36ecc66672c783c2842d12fc11043c13ab28.gz
Another global search-and-replace update. Here, I've replaced the PHP
keyword 'NULL' with its lowercase version. This is another PEAR code
standards change.
darcs-hash:20081223192129-84dde-4a0182e0ec16a01ad88745ad3e08f7cb501aee0b.gz
The PEAR coding standards decree: no tabs, but indent by four spaces.
I've done a global search-and-replace on all tabs, replacing them by
four spaces. This is a huge change, but it will go a long way to
getting us towards phpcs-compliance. And that means better code
readability, and that means more participation.
darcs-hash:20081223191907-84dde-21e8efe210e6d5d54e935a22d0cee5c7bbfc007d.gz
I moved the 4 streams for a user (with friends, faves, replies,
personal) into functions on the User object. Added a helper function
in Notice for making notice streams. Also, will fetch notice streams
out of the memcached server, if possible. Made the API, RSS, and HTML
output all use the same streams (hopefully cached).
Added some code to Notice to blow the cache when a notice is posted.
Also, added code to favor and disfavor actions to blow the faves
cache, too.
darcs-hash:20080928120119-5ed1f-ead542348bcd3cf315be6f42934353154402eb16.gz
I added a new class, Memcached_DataObject, that will (optionally)
fetch data out of a memcached server if it's available. This only
works on 'staticGet'.
Methods that write to the database (insert, update, delete) will clear
and set the cache correctly, too.
darcs-hash:20080926160941-5ed1f-922de078b4c1941853ad014edf9a17fae486f8cf.gz
Add the code to registration to handle invitation codes.
Some edge cases on invitations: is the user already subbed to this
person? Tell them. Is the person already on the system? Sub the user
to them, then, and tell the user.
Add some code to User to auto-sub invitees whenever the email address
changes. Call it from a new registration with an invite code, and also
from confirmaddress.
Some whitespace cleanup in the files touched.
darcs-hash:20080827001927-84dde-b50e5d921ca3f2fb894821730ff93cac09d2ba66.gz
noticesWithFriends is turning out to be one of our most expensive
queries. The join is costly, and this method is hit over and over and
over by desktop clients and other API users.
So, I've added a first pass at caching the results. I store a "window"
of notices -- equal to the first 3 pages of notices, plus one for
pagination -- in the memcached cache. If with-friends notices are
requests, I fetch the whole window out of the cache and grab the slice
requested. If the requested notices are outside the window, we just do
the query. If there's nothing in the cache, we request the window and
store it, then return a slice.
I had to add a NoticeWrapper class that works like DB_DataObject
(well, just the fetch() part...) but just holds an array of notices
instead of a DB cursor.
Finally, saving a new notice blows away the caches for subscribed users.
darcs-hash:20080915065616-84dde-1b1e814c2294498a10b763b779cbb62c3f96aa84.gz
