Commit Graph

43 Commits

Author SHA1 Message Date
Alexei Sorokin
2b0251213f [DATABASE] Various table schema related fixes 2021-07-16 19:44:35 +01:00
Mikael Nordfeldth
9919ccb8b5 HubSub now remembers recent failures and counts them 2017-07-10 20:28:45 +02:00
Mikael Nordfeldth
74a60ab963 Rework the push mechanism a bit to a less DB dependant queue 2017-07-10 14:43:28 +02:00
Mikael Nordfeldth
f4d6710a0f Change mentions of PuSH to WebSub
WebSub is probably finalised before we make a release anyway. Here is
the official spec: https://www.w3.org/TR/websub/

Mostly just comments that have been changed. Some references to PuSH <0.4
are left because they actually refer to PuSH 0.3 and that's not WebSub...

The only actual code change that might affect anything is FeedSub->isPuSH()
but the only official plugin using that call was FeedPoller anyway...
2017-05-01 11:04:27 +02:00
Mikael Nordfeldth
0767bf487e Use the new onUpdateKeys in dataobject for tasks on-update of keys
sets the hashkey column of the row to sha1(topic + '|' + callback)
2016-03-23 15:22:34 +01:00
Mikael Nordfeldth
efe23ed404 updateWithKeys now understands multi-column keys
and automatically identifies _which_ columns are the right ones,
so for example 'uri' primary keys don't need to be explicitly set
2016-01-28 16:42:59 +01:00
Mikael Nordfeldth
c559b8ce2a bulkDistribute won't add empty lists to database 2016-01-16 17:34:27 +01:00
Mikael Nordfeldth
2b67b53112 In very specific circumstances we can bulkDistribute 0 notices
Seems to be what caused an infinite loop on quitter.es, or I guess so anyway.
2016-01-16 17:18:14 +01:00
Mikael Nordfeldth
59e75ef966 Incorrect use of getByHashKey in HubSub 2016-01-14 13:15:31 +01:00
Mikael Nordfeldth
bacd49a6a8 Don't try to replace http with https if https already exists 2016-01-14 13:06:37 +01:00
Mikael Nordfeldth
0482b7de8e Debugging by indexing callback column in HubSub 2016-01-13 22:57:42 +01:00
Mikael Nordfeldth
377947c57f s/getLease/getLeaseTime/ just to be a bit more clear 2016-01-13 19:55:17 +01:00
Mikael Nordfeldth
1d26fedf12 Don't store lease seconds, just sub start and end
The seconds can always be calculated from the dates!
2016-01-13 19:45:20 +01:00
Mikael Nordfeldth
494746e665 Minor PuSH comment and debug stuff 2016-01-13 19:25:39 +01:00
Mikael Nordfeldth
b38a789005 HubSub didn't save sub start and end datetimes 2016-01-13 19:23:34 +01:00
Mikael Nordfeldth
f4feef477b Don't follow redirects on PuSH POST. 2016-01-12 14:32:28 +01:00
Mikael Nordfeldth
c826fe0af4 $target was klantigtly copied from another debug message 2016-01-11 20:10:38 +01:00
Mikael Nordfeldth
bd6efa0e45 Update PuSH callback URL if remote side switched to HTTPS
See the comment in the source on why we're not following Location headers...
2016-01-11 19:55:02 +01:00
Mikael Nordfeldth
2f86cd8602 utf8mb4 conversion on database with index adjusts 2015-02-12 18:18:55 +01:00
Chimo
937adf05c9 Remove NOT NULL restriction on HubSub 'lease'
This fixes an issue where INSERTs in HubSub fail if MySQL/MariaDB is
running in "strict mode" since the default lease time in
PushHubAction::subunsub is null.

Permanent subscriptions have been removed in PuSH v0.4, but they are
being kept here for backward-compatibility with previous GS/SN versions.
2015-02-07 10:46:13 -05:00
Mikael Nordfeldth
ab4113168f PuSH 0.4: No outgoing 'sync' verifications. Feed renewal script. No auto-renewal.
Among other things (such as permanent subscriptions), Pubsubhubbub 0.4
removed the "sync" verification method. This means that any incoming
PuSH subscription requests that follow the 0.4 spec won't really
_require_that we handle it as a background process, but if we were to
try direct verification of the subscription - and fail - there's no way
we could  pick up the ball again. So _essentially_ we require background
processing with retries.

This means we must implement something like the "poorman cron" or
similar, so background processing can be handled
on-demand/on-site-visit. This is how Friendica, Drupal etc. handles it
and is necessary for environments where we can't run separate queue
daemons.

When the poorman-cron-ish thing is implemented, auto-renewal will work
for all users.

PuSH 0.4 spec:
    https://pubsubhubbub.googlecode.com/git/pubsubhubbub-core-0.4.html
More on PuSH 0.4 release (incl. breaking changes):
    https://groups.google.com/forum/#!msg/pubsubhubbub/7RPlYMds4RI/2mIHQTdV3aoJ
2013-11-02 20:02:28 +01:00
Mikael Nordfeldth
db889922ac We don't do local_push_bypass 2013-11-02 17:42:32 +01:00
Mikael Nordfeldth
3ba6374b9d Memcached_DataObject extensions got their update functions more consistent 2013-10-28 19:36:05 +01:00
Mikael Nordfeldth
3cab5b36c1 Replace common_good_random with common_random_hexstr 2013-10-21 13:20:30 +02:00
Mikael Nordfeldth
0bbcfa7bdf IMPORTANT - fixed HubSub to properly fetch primary keys
In commit e95f77d34c HubSub lost the
'staticGet' function in a consolidation into the Managed_DataObject class.
This was done carelessly by me as HubSub::staticGet was actually taking
two arguments, none of which was a key and merging them in HubSub::hashkey()
(staticGet was renamed getKV 2a4dc77a63).

NOTE: This complements commit 7e4718a4eb which
fixed a similar issue for the Magicsig class.
2013-08-21 11:25:08 +02:00
Mikael Nordfeldth
66997f2bec OStatus classes now has modern schemaDef 2013-08-21 11:01:31 +02:00
Mikael Nordfeldth
2a4dc77a63 The overloaded DB_DataObject function staticGet is now called getKV
I used this hacky sed-command (run it from your GNU Social root, or change the first grep's path to where it actually lies) to do a rough fix on all ::staticGet calls and rename them to ::getKV

   sed -i -s -e '/DataObject::staticGet/I!s/::staticGet/::getKV/Ig' $(grep -R ::staticGet `pwd`/* | grep -v -e '^extlib' | grep -v DataObject:: |grep -v "function staticGet"|cut -d: -f1 |sort |uniq)

If you're applying this, remember to change the Managed_DataObject and Memcached_DataObject function definitions of staticGet to getKV!

This might of course take some getting used to, or modification fo StatusNet plugins, but the result is that all the static calls (to staticGet) are now properly made without breaking PHP Strict Standards. Standards are there to be followed (and they caused some very bad confusion when used with get_called_class)

Reasonably any plugin or code that tests for the definition of 'GNUSOCIAL' or similar will take this change into consideration.
2013-08-18 13:13:56 +02:00
Mikael Nordfeldth
e95f77d34c Updating all Memcached_DataObject extended classes to Managed_DataObject
In some brief tests, this causes no problems.

In this state however, you would need to modify DB_DataObject to have a static declaration of staticget (and probably pkeyGet). The next commit will change the staticGet overload to a unique function name (like getKV for getKeyValue), which means we can properly call the function by PHP Strict Standards.
2013-08-18 12:32:32 +02:00
Brion Vibber
69b13cb279 Normalize execution guards on OStatus php files; mostly helps cut down on annoying 'class not found' errors when something spiders the dirs. :P 2010-10-08 10:42:59 -07:00
Siebrand Mazeland
5a6f616206 * i18n/L10n update
* translator comments added
* remove superfluous whitespace
2010-09-19 15:17:36 +02:00
Siebrand Mazeland
1bfbe9badf * i18n/L10n updates and FIXMEs added
* whitespace fixes
2010-09-03 01:35:04 +02:00
Brion Vibber
41e9dba729 OStatus plugin: Rolling batch queueing for PuSH output to >50 subscribing sites. Keeps latency down for other things enqueued while we work... 2010-06-07 10:03:43 -07:00
Brion Vibber
8b9436e8ae Option to divert PuSH items directly to the target site's queue when local 2010-06-03 18:06:55 -07:00
Brion Vibber
9e9ab23e1f Fixes for updating indices, charset/collation and engine type on plugin-created tables.
Under MySQL, new tables will be created as InnoDB with UTF-8 (utf8/utf8_bin) same as core tables.
Existing plugin tables will have table engine and default charset/collation updated, and string columns will have charset updated, at checkschema time.

Switched from 'DESCRIBE' to INFORMATION_SCHEMA for pulling column information in order to get charset. A second hit to INFORMATION_SCHEMA is also needed to get table properties.

Indices were only being created at table creation time, which ain't so hot. Now also adding/dropping indices when they change.

Fixed up some schema defs in OStatus plugin that were a bit flaky, causing extra alter tables to be run.

TODO: Generalize this infrastructure a bit more up to base schema & pg schema classes.
2010-03-12 13:16:32 -08:00
James Walker
7214db14fe wrong param order to strpos() 2010-03-09 01:24:21 -05:00
Brion Vibber
680ace1966 High-priority OStatus fixes:
* PuSHing out to multiple client services could fail; only first callback got reached
* Correction for re-sub request to a known sub
2010-03-01 18:46:34 -08:00
James Walker
6ee7660a58 should be sequenceKey (singular) 2010-02-26 16:51:50 -05:00
Brion Vibber
c36bdc1ba5 - break OMB profile update pings to a background queue
- add event hooks to profile update pings
- send Salmon pings with custom update-profile event to OStatus subscribees and groups (subscribers will see it on your next post)
- fix OStatus queues with overlong transport names, should work on DB queues now
- Ostatus_profile::notifyActivity() and ::notifyDeferred() now can take XML, Notice, or Activity for convenience
2010-02-24 20:36:36 +00:00
Brion Vibber
78ca45c7a0 OStatus PuSH fixes:
- hub now defers subscription state updates until after verification, per spec
- hub now supports synchronous verification when requested (if async is not requested after)
- client now requests synchronous verification (it's a bit safer)
- cleanup on subscription logging/error responses
2010-02-21 18:51:15 -08:00
Brion Vibber
aa0b2ce81a OStatus PuSH fixlets:
- set minimal error page output on PuSH callback
- allow hub to retry ($config['ostatus']['hub_retries']), simplify internal iface a bit. Retries are pushed to end of queue but otherwise not delayed yet; makes delivery more robust to one-off transitory errors but not yet against downtime.
2010-02-21 14:28:06 -08:00
Brion Vibber
22ff358ba8 OStatus sub/unsub updates:
- fix for PuSH unsub verification
- send Salmon notification on unsub
2010-02-18 18:20:48 +00:00
Brion Vibber
4ae760cb62 OStatus PuSH fixes:
* HMAC now calculated correctly - confirmed interop with Google's public hub
* Can optionally use an external PuSH hub, set URL in $config['ostatus']['hub']
  (may have issues in replication environment, and will ping the hub for every
  update rather than just those with subscribers) Internal hub will still function
  when this is set, but won't be advertised. Warning: setting this, then turning
  it off later will break subscriptions as that hub will no longer receive pings.
2010-02-10 22:58:39 +00:00
Brion Vibber
dc09453a77 First steps on converting FeedSub into the pub/sub basis for OStatus communications:
* renamed FeedSub plugin to OStatus
* now setting avatar on subscriptions
* general fixes for subscription
* integrated PuSH hub to handle only user timelines on canonical ID url; sends updates directly
* set $config['feedsub']['nohub'] = true to test w/ foreign feeds that don't have hubs (won't actually receive updates though)
* a few bits of code documentation
* HMAC support for verified distributions (safest if sub setup is on HTTPS)

And a couple core changes:
* minimizing HTML output for exceptions in API requests to aid in debugging
* fix for rel=self link in apitimelineuser when id given

This does not not yet include any of the individual subscription management (Salmon notifications for sub/unsub, etc) nor a nice UI for user subscriptions.
Needs some further cleanup to treat posts as status updates instead of link references.
2010-02-08 11:15:29 -08:00