Among other things (such as permanent subscriptions), Pubsubhubbub 0.4
removed the "sync" verification method. This means that any incoming
PuSH subscription requests that follow the 0.4 spec won't really
_require_that we handle it as a background process, but if we were to
try direct verification of the subscription - and fail - there's no way
we could pick up the ball again. So _essentially_ we require background
processing with retries.
This means we must implement something like the "poorman cron" or
similar, so background processing can be handled
on-demand/on-site-visit. This is how Friendica, Drupal etc. handles it
and is necessary for environments where we can't run separate queue
daemons.
When the poorman-cron-ish thing is implemented, auto-renewal will work
for all users.
PuSH 0.4 spec:
https://pubsubhubbub.googlecode.com/git/pubsubhubbub-core-0.4.html
More on PuSH 0.4 release (incl. breaking changes):
https://groups.google.com/forum/#!msg/pubsubhubbub/7RPlYMds4RI/2mIHQTdV3aoJ
"[...] posts _local_ users (like you) make won't get data about "repeated by"
from federated users"
This was because the ActivityObject would processShare where the shared object
has a _local_ 'actor' URI. Ostatus_profile would complain this meant that a
"Local user cannot be referenced as remote.".
So we see if the shared activity object's id (URI) is in our Notice table, so
we don't have to processActivity - and can skip ensureActivityObjectProfile.
New plugins:
* LRDD
LRDD implements client-side RFC6415 and RFC7033 resource descriptor
discovery procedures. I.e. LRDD, host-meta and WebFinger stuff.
OStatus and OpenID now depend on the LRDD plugin (XML_XRD).
* WebFinger
This plugin implements the server-side of RFC6415 and RFC7033. Note:
WebFinger technically doesn't handle XRD, but we serve both that and
JRD (JSON Resource Descriptor), depending on Accept header and one
ugly hack to check for old StatusNet installations.
WebFinger depends on LRDD.
We might make this even prettier by using Net_WebFinger, but it is not
currently RFC7033 compliant (no /.well-known/webfinger resource GETs).
Disabling the WebFinger plugin would effectively render your site non-
federated (which might be desired on a private site).
Disabling the LRDD plugin would make your site unable to do modern web
URI lookups (making life just a little bit harder).
In commit e95f77d34c HubSub lost the
'staticGet' function in a consolidation into the Managed_DataObject class.
This was done carelessly by me as HubSub::staticGet was actually taking
two arguments, none of which was a key and merging them in HubSub::hashkey()
(staticGet was renamed getKV 2a4dc77a63).
NOTE: This complements commit 7e4718a4eb which
fixed a similar issue for the Magicsig class.
In commit e95f77d34c Magicsig lost the 'staticGet' function (later renamed to getKV in 2a4dc77a63 ), which was important to properly initialize the Magicsig object (fromString)
I used this hacky sed-command (run it from your GNU Social root, or change the first grep's path to where it actually lies) to do a rough fix on all ::staticGet calls and rename them to ::getKV
sed -i -s -e '/DataObject::staticGet/I!s/::staticGet/::getKV/Ig' $(grep -R ::staticGet `pwd`/* | grep -v -e '^extlib' | grep -v DataObject:: |grep -v "function staticGet"|cut -d: -f1 |sort |uniq)
If you're applying this, remember to change the Managed_DataObject and Memcached_DataObject function definitions of staticGet to getKV!
This might of course take some getting used to, or modification fo StatusNet plugins, but the result is that all the static calls (to staticGet) are now properly made without breaking PHP Strict Standards. Standards are there to be followed (and they caused some very bad confusion when used with get_called_class)
Reasonably any plugin or code that tests for the definition of 'GNUSOCIAL' or similar will take this change into consideration.
In some brief tests, this causes no problems.
In this state however, you would need to modify DB_DataObject to have a static declaration of staticget (and probably pkeyGet). The next commit will change the staticGet overload to a unique function name (like getKV for getKeyValue), which means we can properly call the function by PHP Strict Standards.
The switch to having a function in Managed_DataObject is now being applied
to plugins which can use this as well.
There are some plugins that still use Memcached_DataObject, but these will
be taken care of later.
Output from 0.9.6 PuSH feeds seems to have a rump <author> but no
<activity:actor>. It was overwriting valid and useful data set up at
subscribe time.
This fix tries to avoid overwriting data. However, it may prevent
updates that delete data.
Bug: 3028
StatusNet core code now sets the tooltip text on .attachment.more links when they receive their attachment-expansion magic; this will override the hardcoded tooltip text saved from OStatus plugin when displaying timelines in the web UI.
Changes in 2d4e0693c8 changed Ostatus_profile::filterReplies() (which sorts out the local, remote, and group recipients on incoming remote messages) from checking for remote profiles with a safe call to Ostatus_profile::staticGet() to calls through Ostatus_profile::ensureProfileURL() and Ostatus_profile::ensureWebfinger(), which throw exceptions and thus abort processing.
Since this was done before checking for local groups, the filter would fail when the ensure* functions determined it was looking at a local group and rightfully refused to create a remote group profile for it.
Changing the calls to the ensure* functions was done so we can record remote reply recipients for future reply-to-reply processing (the staticGet() call was a cheaper way to do a lookup when we knew we only actually had to process groups that somebody signed up to); most important fix is simply to actually check for the exception! :)
Here I'm changing the order of processing so we do the local group lookup first -- where it's nice and safe -- and then when we do the remote checks, we'll go ahead and gracefully skip that entry if the full remote lookup fails, so we'll still process any following recipients.
The original-size file is now forced to 0644 (all-readable), which should help. Not sure this is 100% ideal, but it's better than 0600!
(The other sizes were being created in other code and had sane read perms already.)
Watch out for similar issues in Twitter bridge etc; avatar import code should get cleaned up and consolidated.
If set up, this hub will be used to subscribe to feeds that don't specify a hub of their own.
Assumes that the fallback hub will, in fact, handle polling and updates for any feed we throw at it!
Authentication may be specified for the fallback hub.
Example:
$config['feedsub']['fallback_hub'] = 'https://superfeedr.com/hubbub';
$config['feedsub']['hub_user'] = 'abcd';
$config['feedsub']['hub_pass'] = 'ckcmdkmckdmkcdk';
Also:
* Fix for WordPress-RSS-via-Superfeedr-Atom; if we have <author> info but no ID from a native ActivityStreams actor, don't freak out in the low-level processing code that checks for identity matches.
* enhanced messages for low-level FeedSub exceptions if they make it to outside display
SubMirror: redid add-mirror frontend to accept a feed URL, then pass that on to OStatus, instead of pulling from your subscriptions.
Profile: tweaked subscriberCount() so it doesn't subtract 1 for foreign profiles who aren't subscribed to themselves; instead excludes the self-subscription in the count query.
Memcached_DataObject: tweak to avoid extra error spew in the DB error raising
Work in progress: tweaking feedsub garbage collection so we can count other uses
The ensure* family of functions will now return an OStatusShadowException in this case, which gives us a pleasant error message instead of a giant exception backtrace when you do 'sub somebody@this.local.server'.
Can be extended later to allow actually using the local profile, since we could figure it out.
Conflicts:
lib/attachmentlist.php
plugins/OStatus/classes/Ostatus_profile.php
Merge tried to delete things that it seems it shouldn't, very confusing order. Hope rest of the cherry-picking isn't a problem.
* drop OStatusPlugin::localProfileFromUrl(), we can just look up on user.uri
* clean up a few edge cases that returned null through Ostatus_profile::ensure* code paths, now throws clear exception when we can't find a feed from the given profile url
* add some doc comments on the ensure* methods
Superfeedr (sp.?) posts entries without author information. We can
assume that this is intended to be by the original author.
Re-structured the checks for entries that come in by PuSH so they can
either have no author or an empty author, but not a different author.
Base problem is that our caching-on-insert interferes with relying on column default values; the cached object is missing those fields, so they appear to be empty (null) when the object is retrieved from cache.
Now explicitly setting them when inserting subscriptions, and cleaned up some code that had alternate code paths.
May also have made auto-subscription work for remote OStatus subscribers, but can't test until magic sigs are working again.
Under MySQL, new tables will be created as InnoDB with UTF-8 (utf8/utf8_bin) same as core tables.
Existing plugin tables will have table engine and default charset/collation updated, and string columns will have charset updated, at checkschema time.
Switched from 'DESCRIBE' to INFORMATION_SCHEMA for pulling column information in order to get charset. A second hit to INFORMATION_SCHEMA is also needed to get table properties.
Indices were only being created at table creation time, which ain't so hot. Now also adding/dropping indices when they change.
Fixed up some schema defs in OStatus plugin that were a bit flaky, causing extra alter tables to be run.
TODO: Generalize this infrastructure a bit more up to base schema & pg schema classes.
Some stray shadow entries were ending up getting created, which would steal group posts from remote users.
Run plugins/OStatus/scripts/fixup-shadow.php for each site to remove any existing ones.
Also stripping id from foreign HTML messages (could interfere with UI) and disabled failing attachment popup for a.attachment links that don't have a proper id, so you can click through instead of getting an error.
Issues:
* any other links aren't marked and saved
* inconsistent behavior between local and remote attachments (local displays in lightbox, remote doesn't)
* if the enclosure'd object isn't referenced in the content, you won't be offered a link to it in our UI
* 'testing' of gitorious.org:statusnet/mainline:
OStatus: pull best-sized avatar image (96x96 if found, otherwise largest, otherwise if none labeled takes the first)
info about discovery in Ostatus_profile::ensureWebfinger()
cache Web responses in Webfinger library
* 'testing' of gitorious.org:statusnet/mainline:
OStatus: if no <link> available and no profileurl hint passed in, use object id if it's an HTTP(S) URL. Fixes profile link for Google accounts.
Tweak common_url_to_nickname to take the last path component; fixes pulling nicks from Google profile pages (path is "/profile/<nickname>")
Fix bug on subscribe/unsubscribe in profile lists. Bogus call to nonexisting profile->getProfile() was masked by DB_DataObject
- fix <activity:subject> generation so we get the profile info (what's available so far)
- use id instead of nickname for group join/leave forms so we can join/leave remote groups
while the rest of the groups UI remains limited to local groups
(plugins are responsible for making sure remote notifications and permission checks are done)
- fix remote notification when joining group through OStatus's remote subscribe form
* 'testing' of gitorious.org:statusnet/mainline:
Merge StatusNet core localization updates from 0.9.x branch
Fix update_po_templates.php to support the plural and context variants of _m() in plugins
Drop HTMLPurifier; we don't need its extra capabilities and we're already using htmLawed which is lighter-weight.
OStatus: handle update-profile Salmon pings
Revert "Updated jQuery Form Plugin from v2.17 to v2.36"
OStatus: disable HTMLPurify cache unless we've configured a writable path for it.
- add event hooks to profile update pings
- send Salmon pings with custom update-profile event to OStatus subscribees and groups (subscribers will see it on your next post)
- fix OStatus queues with overlong transport names, should work on DB queues now
- Ostatus_profile::notifyActivity() and ::notifyDeferred() now can take XML, Notice, or Activity for convenience
- added rel="ostatus:attention" links for group delivery
- added events for plugins to override group profile/permalink pages
- pulled Notice::saveGroups up to save-time so we can override;
it's relatively cheap and gives us a clean list of target
groups for distrib time even with customized delivery.
- fixed notice::getGroups to return group objects as expected
- added some doc on new parameters to Notice::saveNew
- 'groups' list of group IDs to push to in place of parsing
- messages that come in via PuSH and contain local group targets
are delivered to local group members
- messages that come in via PuSH and contain remote group targets
are delivered to local members of the remote group
Todo:
- handle group posts that only come through Salmon
- handle conflicts in case something comes in both through Salmon and PuSH
- better source verification
- need a cleaner interface to look up groups by URI
- need a way to handle remote groups with conflicting names
- hub now defers subscription state updates until after verification, per spec
- hub now supports synchronous verification when requested (if async is not requested after)
- client now requests synchronous verification (it's a bit safer)
- cleanup on subscription logging/error responses
- set minimal error page output on PuSH callback
- allow hub to retry ($config['ostatus']['hub_retries']), simplify internal iface a bit. Retries are pushed to end of queue but otherwise not delayed yet; makes delivery more robust to one-off transitory errors but not yet against downtime.
- actually udpate feedsub.last_update when we get a new PuSH update in
- move incoming PuSH processing to a queue handler to minimize time spent before POST return, as recommended by PuSH spec. When queues are disabled this'll still be handled immediately.
