Commit Graph

240 Commits

Author SHA1 Message Date
Brion Vibber
398e622fec Save attached URLs when importing a Twitter status: this lets our thumbnail detection handle photos and videos linked to by Twitter posters. 2010-11-12 17:40:34 -08:00
Brion Vibber
65eeb7cba5 Merge branch 'master' of gitorious.org:statusnet/mainline into 0.9.x
Conflicts:
	actions/newgroup.php
2010-11-12 13:32:48 -08:00
Brion Vibber
6291e8201f Fix for failure edge case in TwitterBridge outgoing repeat/retweets.
When the retweet failed with a 403 error (say due to it being a private tweet, which can't be retweeted) we would end up mishandling the return value from our internal error handling.
Instead of correctly discarding the message and closing out the queue item, we ended up trying to save a bogus twitter<->local ID mapping, which threw another exception and lead the queue system to re-run it.

- Fixed the logic check and return values for the retweet case in broadcast_twitter().
- Added doc comments explaining the return values on some functions in twitter.php
- Added check on Notice_to_status::saveNew() for empty input -- throw an exception before we try to actually insert into db. :)
2010-11-12 13:06:41 -08:00
Siebrand Mazeland
ca6d7f1042 Localisation updates from http://translatewiki.net. 2010-11-05 01:25:50 +01:00
Brion Vibber
9cbda32768 Pull out the 'tweetctl' queue for now; these should go over control signals, and actual handling isn't implemented yet anyway. 2010-11-02 16:51:07 -07:00
Brion Vibber
a2f0f68d75 fakestream.php can now take --all option to pull the latest messages from multiple locally-authed accounts when generating simulated sitestreams info 2010-11-02 16:43:01 -07:00
Brion Vibber
445b306b54 fakestream.php: script to build an emulated Twitter Site Stream from live Twitter data, for testing. 2010-11-02 16:27:14 -07:00
Brion Vibber
28703deb8f Allow custom apiroot for site streams testing on streamtest 2010-11-02 16:26:51 -07:00
Brion Vibber
62408fef09 Work in progress on twitter import daemon 2010-10-29 14:12:40 -07:00
Brion Vibber
d743539cf7 Fixups for twitter streaming daemon 2010-10-29 13:41:15 -07:00
Brion Vibber
47eada3a95 Work in progress on site streams-aware TwitterDaemon 2010-10-29 13:18:03 -07:00
Brion Vibber
86adc575ec TweetInQueueHandler: run incoming tweets through the queues to keep the Twitter streaming daemon clear. 2010-10-29 13:14:12 -07:00
Brion Vibber
15b108620e Fix a couple 'continue's from old looping code in Twitter importer (-> return null) 2010-10-29 13:06:32 -07:00
Brion Vibber
e76028b629 Work in progress: starting on new TwitterDaemon using the Site Streams API -- code is incomplete, pulling bits from streamtest.php pending a chance to test the actual site-streams mode 2010-10-28 18:26:48 -07:00
Brion Vibber
e62254f8cc Merge branch '0.9.x' into twitstream 2010-10-28 16:30:40 -07:00
Siebrand Mazeland
7a564276a9 Localisation updates from http://translatewiki.net. 2010-10-28 01:58:43 +02:00
Brion Vibber
8ff44a1fb9 Merge branch '0.9.x' into twitstream 2010-10-26 15:39:31 -07:00
Siebrand Mazeland
1545c1228b Localisation updates from http://translatewiki.net. 2010-10-23 20:50:55 +02:00
Brion Vibber
e292d8fb65 Fix for 140-char replies being unexpectedly cropped when bridged to Twitter.
This drops the '@' -> ' @' hack for CURL meta-chars in outgoing Twitter bridge, added in commit 04b95c25 back in the day.
The Twitter bridge has since been switched from using direct CURL calls to using HTTPClient, which even with the CURL backend enabled doesn't trigger this issue, as POST parameters are formatted directly.
Prepending the space before we did the message cropping was leading to 140-char messages getting cropped unnecessarily, which was confusing:

Examples of broken messages:
http://identi.ca/notice/57172587 vs http://twitter.com/marjoleink/status/28398050691
http://identi.ca/notice/57172878 vs http://twitter.com/marjoleink/status/28398492563
2010-10-22 12:10:11 -07:00
Siebrand Mazeland
e6df1b19df Localisation updates from http://translatewiki.net 2010-10-18 22:03:29 +02:00
Brion Vibber
22047f6412 Merge branch '0.9.x' into twitstream 2010-10-13 15:37:54 -07:00
Siebrand Mazeland
5215423ea9 Localisation updates from http://translatewiki.net. 2010-10-09 16:15:48 +02:00
Brion Vibber
67dc78b621 Merge branch 'fatals' into 0.9.x 2010-10-08 11:28:51 -07:00
Brion Vibber
5fe59322bc Check for Twitter foreign link actually existing before trying to delete it; friendlier error message in TwittersettingsAction 2010-10-08 11:12:36 -07:00
Brion Vibber
9c3fd10257 Prelim --all mode on streamtest.php to use site streams; doesn't use the destination user for import yet, and not actually tested yet until I'm whitelisted for the beta. :) 2010-10-05 17:08:04 -07:00
Brion Vibber
408483f771 Fix up Twitter JSON formatting to be consistent between the polling and streaming API interfaces; basic stream tester can now import your notices (ooooh) 2010-10-05 14:26:11 -07:00
Brion Vibber
0eaa26476c Split the guts of Twitter status -> notice import from twitterstatusfetcher daemon into TwitterImport class which can be called from other places, letting us reuse code for the streaming API. 2010-10-05 13:57:32 -07:00
Brion Vibber
dc6c0f325c Cleanup on input path 2010-10-05 13:41:15 -07:00
Brion Vibber
eb04df583a Buncha cleanup 2010-10-05 13:25:28 -07:00
Brion Vibber
76353ede54 Clean up event handling a bit 2010-10-05 12:42:55 -07:00
Brion Vibber
5058e8fd14 Twitter streaming API reader: Cleanup input handling & split from HTTP headers to body 2010-10-05 12:17:16 -07:00
Brion Vibber
3b304fc0ef Initial stub code for pulling data from Twitter's User Streams and Site Streams interfaces. This should allow us to make a much more efficient background importer which can use a relatively small number of connections getting push data for either a single user or for many users with credentials on the site. 2010-10-04 18:28:54 -07:00
Siebrand Mazeland
458512aafd Localisation updates from http://translatewiki.net 2010-10-03 22:46:38 +02:00
Siebrand Mazeland
08054e85fe Localisation updates from http://translatewiki.net. 2010-10-01 22:34:59 +02:00
Brion Vibber
1acc7d66c6 Always specify UTF-8 targt charset for html_entity_decode(); default is 8-bit ISO-8859-1 which causes things to break when we later pass them through things that expect to work with UTF-8. For instance, running through preg_replace() with the /u option results in NULL, leading to problems with OStatus and SubMirror generating their plaintext versions and doing length-cropping. 2010-09-30 11:29:31 -07:00
Siebrand Mazeland
ec7ab3af4d Localisation updates from http://translatewiki.net
* first export of L10n for languages other than English. Could use some testing/QA.
2010-09-28 01:09:29 +02:00
Siebrand Mazeland
870be512d2 Update POT files for plugins. 2010-09-23 00:37:16 +02:00
Evan Prodromou
78ed0348b0 Merge remote branch 'gitorious/0.9.x' into 0.9.x 2010-09-21 14:07:48 -04:00
Siebrand Mazeland
c932c0fdc1 Update pot files for plugins. 2010-09-19 17:44:29 +02:00
Siebrand Mazeland
ecafae8472 * Remove trailing newline in e-mail message.
* Superfluous whitespace removed.
2010-09-19 15:52:53 +02:00
Siebrand Mazeland
8135f7d960 Update pot files for plugins. 2010-09-19 00:24:48 +02:00
Siebrand Mazeland
02299d5bd7 * i18n/L10n updates.
* add FIXMEs for bad i18n.
* add translator hints.
2010-09-19 00:06:21 +02:00
Siebrand Mazeland
16d234f9af Remove superfluous whitespace 2010-09-18 23:49:49 +02:00
Siebrand Mazeland
ba6538ed78 * i18n/L10n updates
* whitespace updates.
2010-09-18 17:45:18 +02:00
Evan Prodromou
b2abae433d Merge remote branch 'gitorious/0.9.x' into 0.9.x 2010-09-15 15:57:41 -04:00
Siebrand Mazeland
db1d348135 Update/add pot files for plugins. 2010-09-15 01:16:45 +02:00
Evan Prodromou
398df867de use capturing parens for matching at-mentions in twitterstatusfetcher 2010-09-10 09:21:01 -04:00
Evan Prodromou
deb8f2a38b make our own twitter links if Twitter doesn't give us entities 2010-09-10 09:10:53 -04:00
Evan Prodromou
42f1ec85ea change the event we use for deleting a linked status on Twitter 2010-09-08 16:40:37 -04:00
Evan Prodromou
e40ed1fd56 move notice_to_status initialization code to run-once script 2010-09-08 16:10:07 -04:00
Evan Prodromou
5c98dfd58c text substitution is Unicode-aware for twitterstatusfetcher 2010-09-07 15:54:06 -04:00
Evan Prodromou
2d18963b4c Fix pkeyget call
Fix the pkeyget call in Twitter_synch_Status
2010-09-07 15:34:10 -04:00
Evan Prodromou
b6440a206d staticGet -> pkeyGet in Twitter_synch_status 2010-09-07 15:29:36 -04:00
Evan Prodromou
d37f0da077 debug statements for setting/getting last id 2010-09-07 11:53:26 -04:00
Evan Prodromou
9793e4ceb4 Store foreign ID for synch info, not user ID 2010-09-07 10:53:11 -04:00
Evan Prodromou
21bea14140 remove data dump of retrieved statuses 2010-09-07 04:42:23 -04:00
Evan Prodromou
8facfdcfee Correct DB_DataObject constant in Twitter_synch_status 2010-09-07 04:35:14 -04:00
Evan Prodromou
98e9fa19c8 Missing comma in column list in Twitter_synch_status 2010-09-07 04:33:28 -04:00
Evan Prodromou
49af35b44d add modified column to twitter_synch_status 2010-09-07 04:32:01 -04:00
Evan Prodromou
e53e152d6f use and save since_id parameter for home_timeline 2010-09-07 04:22:55 -04:00
Evan Prodromou
86b6d7b722 save synch status for different timelines 2010-09-07 04:17:36 -04:00
Evan Prodromou
a46ba0b548 save Twitter mentions as StatusNet replies 2010-09-07 04:00:59 -04:00
Evan Prodromou
c2c6acbfd2 add back in # and @ for links 2010-09-07 04:00:59 -04:00
Evan Prodromou
2de6d0b03d add Twitter-approved links to Twitter statuses 2010-09-07 04:00:59 -04:00
Evan Prodromou
28fecf70b9 better handling of params in oauthget 2010-09-07 04:00:59 -04:00
Evan Prodromou
a844aaaea6 better param handling in hometimeline 2010-09-07 04:00:59 -04:00
Evan Prodromou
86d37ba59c include entities in the results of home_timeline 2010-09-07 04:00:59 -04:00
Evan Prodromou
b059a19e0f delete Twitter notice if it was posted from here 2010-09-07 04:00:59 -04:00
Evan Prodromou
ba8a24073f Notify Twitter when StatusNet user faves/disfaves a Twitter notice 2010-09-07 04:00:59 -04:00
Evan Prodromou
5944fe57e6 using null variable in twitter_status_id() 2010-09-07 04:00:59 -04:00
Evan Prodromou
a5554bc11e check status_id before trying to retweet it 2010-09-07 04:00:59 -04:00
Evan Prodromou
702d36257c truncate retweeted stuff if it's too long 2010-09-07 04:00:58 -04:00
Evan Prodromou
9847bffe99 correctly mark repeats from Twitter as non-local 2010-09-07 04:00:58 -04:00
Evan Prodromou
e20dc9fec3 correctly check for local notice to status mappings when notice originated here 2010-09-07 04:00:58 -04:00
Evan Prodromou
df88624d49 use bigint for status_ids 2010-09-07 04:00:58 -04:00
Evan Prodromou
a9bb1c7528 debug output when saving new mapping 2010-09-07 04:00:58 -04:00
Evan Prodromou
d986537c43 do a uniqueness check before saving new notice-to-status mapping 2010-09-07 04:00:58 -04:00
Evan Prodromou
299da9e462 SQL syntax error when initializing notice_to_status table 2010-09-07 04:00:58 -04:00
Evan Prodromou
6c7262c8ca Delete Notice_to_status when a notice is deleted 2010-09-07 04:00:58 -04:00
Evan Prodromou
e9e06879bb use Notice_to_status to check for duplicates 2010-09-07 04:00:58 -04:00
Evan Prodromou
a1981770ce Save notice-to-status mapping in its own table
Introduce a table mapping notices to Twitter statuses. Initialize
this table at checkSchema() time. Save the mapping when we push
or pull statuses. Use the table to determine if a notice has a
Twitter equivalent.
2010-09-07 04:00:58 -04:00
Evan Prodromou
17a16026ff add in_reply_to_status_id if notice is from twitter 2010-09-07 04:00:58 -04:00
Evan Prodromou
928239ecf8 do our own repeating so we can pass in a uri 2010-09-07 04:00:58 -04:00
Evan Prodromou
1bf4972cc0 remove require_once for disappeared TwitterBasicAuthClient 2010-09-07 04:00:57 -04:00
Evan Prodromou
23dfadd9a6 remove basic auth code for Twitter since it's no longer supported 2010-09-07 04:00:57 -04:00
Evan Prodromou
e45b6df419 use home_timeline instead of friends_timeline 2010-09-07 04:00:57 -04:00
Evan Prodromou
a8c90d0243 switch twitterstatusfetcher from friends_timeline to home_timeline 2010-09-07 04:00:57 -04:00
Evan Prodromou
1a54291cc7 debug code to dump new status data 2010-09-07 04:00:57 -04:00
Evan Prodromou
184ee7fc1f move inbox insert code to status saver main loop 2010-09-07 04:00:57 -04:00
Evan Prodromou
20f6ea4840 some info notices in twitter status fetch for repeats and replies 2010-09-07 04:00:57 -04:00
Evan Prodromou
5651f79466 if something's a retweet, save it as a repeat in bridge 2010-09-07 04:00:57 -04:00
Evan Prodromou
d6719760d6 do a real retweet for a local repeat 2010-09-07 04:00:57 -04:00
Evan Prodromou
2b56d3b534 push notices through the bridge if they're in reply to a twitter notice 2010-09-07 04:00:57 -04:00
Evan Prodromou
4d70ba9597 save reply status for Twitter notices 2010-09-07 04:00:57 -04:00
Evan Prodromou
ed99fd51e8 don't save statuses from protected users 2010-09-07 04:00:57 -04:00
Brion Vibber
4cbbfdab84 Fix for #2635: use ssl-sometimes settings for Twitter settings & auth pages 2010-09-02 10:55:26 -07:00
Brion Vibber
c56939d596 Remove the 'Enable Twitter import' checkbox from Twitter admin panel by default; can be re-added with setting:
addPlugin('TwitterBridge', array('adminImportControl' => true, ....));

Added a note on the label that it requires manual daemon setup. (Note that by default the admin panel won't be shown, so it's no biggie to be hiding this for now.)
2010-08-02 13:00:50 -07:00
Zach Copley
4c5098cd32 Handle the case where a screen name has shifted from one Twitter ID to another 2010-07-08 21:17:11 +00:00
Brion Vibber
5f4c6ec626 Skip enqueueing to outgoing bridges on incoming remote messages. Twitter, Facebook, RSSCloud, and OStatus checks were enqueued on these when they'd never do anything but churn the queue servers.
Notice::isLocal() can replace a number of manual checks for $notice->is_local being LOCAL_PUBLIC or LOCAL_NONPUBLIC.
2010-06-03 16:58:45 -07:00
Brion Vibber
74a89b1fc3 Locale switch cleanup: use common_switch_locale() which is safer for updating gettext state. Also moved a few calls to reduce chance of hitting an exception before switching back.
Should help with problems where xmppdaemon would get stuck in wrong locale.
2010-05-19 10:10:55 -07:00