noticesWithFriends is turning out to be one of our most expensive
queries. The join is costly, and this method is hit over and over and
over by desktop clients and other API users.
So, I've added a first pass at caching the results. I store a "window"
of notices -- equal to the first 3 pages of notices, plus one for
pagination -- in the memcached cache. If with-friends notices are
requests, I fetch the whole window out of the cache and grab the slice
requested. If the requested notices are outside the window, we just do
the query. If there's nothing in the cache, we request the window and
store it, then return a slice.
I had to add a NoticeWrapper class that works like DB_DataObject
(well, just the fetch() part...) but just holds an array of notices
instead of a DB cursor.
Finally, saving a new notice blows away the caches for subscribed users.
darcs-hash:20080915065616-84dde-1b1e814c2294498a10b763b779cbb62c3f96aa84.gz
* No need to check $source's value before inserting
* No need to update the notice if the $uri was known in advance
darcs-hash:20080902173804-57fc3-496ceaf8192694db43e62f7af1f57785a1a16a01.gz
Make "#sanfrancisco", "#SanFrancisco", "#san_francisco", "#San.Francisco", and "#SAN-FRANCISCO" all link to http://identi.ca/tag/sanfrancisco but preserve appearance
darcs-hash:20080901025932-e3c0d-c0a939eaf7e242d88cbcb0d651c9d53718c60a9d.gz
Make "#test", "#Test", and "#tEsT" all preserve appearance but link to the same tag
darcs-hash:20080901001241-e3c0d-b466f35f4f023c6c90a6d2817487c97be9a1bbca.gz
Breaking up to use multiple queue handlers means we need multiple
queue items for the same notice. So, change the queue_item table to
have a compound pkey, (notice_id,transport).
darcs-hash:20080827211239-84dde-db118799bfd43be62fb02380829c64813c9334f8.gz
Eventually, the poor xmppdaemon has become overloaded with extra
tasks. So, I've broken it up. Now, we have 5 background scripts, and
more coming:
* xmppdaemon.php - handles incoming XMPP messages only.
* xmppqueuehandler.php - sends notices from the queue out through XMPP.
* smsqueuehandler.php - sends notices from the queue out over SMS
* ombqueuehandler.php - sends notices from the queue out over OMB
* xmppconfirmhandler.php - sends confirmation requests out over XMPP.
This is in addition to maildaemon.php, which takes incoming messages.
None of these are "true" daemons -- they don't daemonize themselves
automatically. Use nohup or another tool to background them. monit can
also be useful to keep them running.
At some point, these might become fork()'ing daemons, able to handle
more than one notice at a time. For now, I'm just running multiple
instances, hoping they don't interfere.
darcs-hash:20080827205407-84dde-97884a12f5f4e54c93bc785bd280683d1ee7e749.gz
