Merge branch 'yammer-master'

This commit is contained in:
Brion Vibber 2010-09-28 07:45:43 -07:00
commit 9a0027cab1
22 changed files with 3023 additions and 0 deletions

182
plugins/YammerImport/README Normal file
View File

@ -0,0 +1,182 @@
Yammer Import Plugin
====================
This plugin allows a one-time import pulling user accounts, groups, and
public messages from an existing Yammer instance, using Yammer's public API.
Requirements
------------
* An account on the Yammer network you wish to import from
* An administrator account on the target StatusNet instance, or
command-line administrative access
* This YammerImport plugin enabled on your StatusNet instance
Limitations
-----------
Yammer API key registrations only work for your own network unless you make
arrangements for a 'trusted app' key, so for now users will need to register
the app themselves. There is a helper in the admin panel for this.
In theory any number of users, groups, and messages should be supported, but
it hasn't been fully tested on non-trivial-sized sites.
No provision has yet been made for dealing with conflicting usernames or
group names, or names which are not considered valid by StatusNet. Errors
are possible.
Running via the web admin interface requires having queueing enabled, and is
fairly likely to have problems with the application key registration step in
a small installation at this time.
Web setup
---------
The import process is runnable through an administration panel on your
StatusNet site. The user interface is still a bit flaky, however, and if
errors occur during import the process may stop with no way to restart it
visible.
The admin interface will probably kinda blow up if JS/AJAX isn't working.
You'll be prompted to register the application and authenticate into Yammer,
after which a progress screen will display.
Two big warnings:
* The progress display does not currently auto-refresh.
* If anything fails once actual import has begun, it'll just keep showing
the current state. You won't see an error message, and there's no way
to reset or restart from the web UI yet.
You can continue or reset the import state using the command-line script.
CLI setup
---------
You'll need to register an application consumer key to allow the importer
to connect to your Yammer network; this requires logging into Yammer:
https://www.yammer.com/client_applications/new
Check all the 'read' options; no 'write' options are required, but Yammer
seems to end up setting them anyway.
You can set the resulting keys directly in config.php:
$config['yammer']['consumer_key'] = '#####';
$config['yammer']['consumer_secret'] = '##########';
Initiate authentication by starting up the importer script:
php plugins/YammerImport/scripts/yammer-import.php
Since you haven't yet authenticated, this will request an auth token and
give you a URL to open in your web browser. Once logged in and authorized
there, you'll be given a confirmation code. Pass this back:
php plugins/YammerImport/scripts/yammer-import.php --verify=####
If all is well, the import process will begin and run through the end.
In case of error or manual abort, you should be able to continue the
import from where you left off by running the script again:
php plugins/YammerImport/scripts/yammer-import.php
To reset the Yammer import state -- without removing any of the items
that have already been imported -- you can pass the --reset option:
php plugins/YammerImport/scripts/yammer-import.php --reset
This'll let you start over from the requesting-authentication stage.
Any users, groups, or notices that have already been imported will be
retained.
Subscriptions and group memberships
-----------------------------------
Yammer's API does not expose user/tag subscriptions or group memberships
except for the authenticating user. As a result, users will need to re-join
groups and re-follow their fellow users after the import.
(This limitation may be lifted in future for sites on the Silver or Gold
plans where the import is done by a verified admin, as it should be possible
to fetch the information for each user via the admin account.)
Authentication
--------------
Account passwords cannot be retrieved, but the primary e-mail address is
retained so users can reset their passwords by mail if you're not using a
custom authentication system like LDAP.
Private messages and groups
---------------------------
At this time, only public messages are imported; private direct and group
messages are ignored. (This may change with Silver and Gold plans in future.)
Yammer groups may be either public or private. Groups in StatusNet currently
have no privacy option, so any private groups will become public groups in the
imported site.
Attachments
-----------
Attached image and document files will be copied in as if they had been
uploaded to the StatusNet site. Currently images do not display inline like
they do on Yammer; they will be linked instead.
File type and size limitations on attachments will be applied, so beware some
attachments may not make it through.
Code structure
==============
Standalone classes
------------------
YammerRunner: encapsulates the iterative process of retrieving the various users,
groups, and messages via SN_YammerClient and saving them locally
via YammerImporter.
SN_YammerClient: encapsulates HTTP+OAuth interface to Yammer API, returns data
as straight decoded JSON object trees.
YammerImporter: encapsulates logic to pull information from the returned API data
and convert them to native StatusNet users, groups, and messages.
Web UI actions
-------------
YammeradminpanelAction: web panel for site administrator to initiate and monitor
the import process.
Command-line scripts
--------------------
yammer-import.php: CLI script to start a Yammer import run in one go.
Database objects
----------------
Yammer_state: data object storing YammerRunner's state between iterations.
Yammer_notice_stub: data object for temporary storage of fetched Yammer messages
between fetching them (reverse chron order) and saving them
to local messages (forward chron order).
Yammer_user,
Yammer_group,
Yammer_notice: data objects mapping original Yammer item IDs to their local copies.

View File

@ -0,0 +1,145 @@
<?php
/*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
/**
* @package YammerImportPlugin
* @maintainer Brion Vibber <brion@status.net>
*/
if (!defined('STATUSNET')) { exit(1); }
class YammerImportPlugin extends Plugin
{
/**
* Hook for RouterInitialized event.
*
* @param Net_URL_Mapper $m path-to-action mapper
* @return boolean hook return
*/
function onRouterInitialized($m)
{
$m->connect('admin/yammer',
array('action' => 'yammeradminpanel'));
$m->connect('admin/yammer/auth',
array('action' => 'yammerauth'));
return true;
}
/**
* Set up queue handlers for import processing
* @param QueueManager $qm
* @return boolean hook return
*/
function onEndInitializeQueueManager(QueueManager $qm)
{
$qm->connect('yammer', 'YammerQueueHandler');
return true;
}
/**
* Set up all our tables...
*/
function onCheckSchema()
{
$schema = Schema::get();
$tables = array('Yammer_state',
'Yammer_user',
'Yammer_group',
'Yammer_notice',
'Yammer_notice_stub');
foreach ($tables as $table) {
$schema->ensureTable(strtolower($table), $table::schemaDef());
}
return true;
}
/**
* If the plugin's installed, this should be accessible to admins.
*/
function onAdminPanelCheck($name, &$isOK)
{
if ($name == 'yammer') {
$isOK = true;
return false;
}
return true;
}
/**
* Add the Yammer admin panel to the list...
*/
function onEndAdminPanelNav($nav)
{
if (AdminPanelAction::canAdmin('yammer')) {
$action_name = $nav->action->trimmed('action');
$nav->out->menuItem(common_local_url('yammeradminpanel'),
_m('Yammer'),
_m('Yammer import'),
$action_name == 'yammeradminpanel',
'nav_yammer_admin_panel');
}
return true;
}
/**
* Automatically load the actions and libraries used by the plugin
*
* @param Class $cls the class
*
* @return boolean hook return
*
*/
function onAutoload($cls)
{
$base = dirname(__FILE__);
$lower = strtolower($cls);
switch ($lower) {
case 'sn_yammerclient':
case 'yammerimporter':
case 'yammerrunner':
case 'yammerapikeyform':
case 'yammerauthinitform':
case 'yammerauthverifyform':
case 'yammerprogressform':
case 'yammerqueuehandler':
require_once "$base/lib/$lower.php";
return false;
case 'yammeradminpanelaction':
$crop = substr($lower, 0, strlen($lower) - strlen('action'));
require_once "$base/actions/$crop.php";
return false;
case 'yammer_state':
case 'yammer_notice_stub':
case 'yammer_common':
case 'yammer_user':
case 'yammer_group':
case 'yammer_notice':
require_once "$base/classes/$cls.php";
return false;
default:
return true;
}
}
}

View File

@ -0,0 +1,174 @@
<?php
/**
* StatusNet, the distributed open-source microblogging tool
*
* Yammer import administration panel
*
* PHP version 5
*
* LICENCE: This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* @category Settings
* @package StatusNet
* @author Zach Copley <zach@status.net>
* @copyright 2010 StatusNet, Inc.
* @license http://www.fsf.org/licensing/licenses/agpl-3.0.html GNU Affero General Public License version 3.0
* @link http://status.net/
*/
if (!defined('STATUSNET')) {
exit(1);
}
class YammeradminpanelAction extends AdminPanelAction
{
private $runner;
/**
* Returns the page title
*
* @return string page title
*/
function title()
{
return _m('Yammer Import');
}
/**
* Instructions for using this form.
*
* @return string instructions
*/
function getInstructions()
{
return _m('This Yammer import tool is still undergoing testing, ' .
'and is incomplete in some areas. ' .
'Currently user subscriptions and group memberships are not ' .
'transferred; in the future this may be supported for ' .
'imports done by verified administrators on the Yammer side.');
}
function prepare($args)
{
$ok = parent::prepare($args);
$this->subaction = $this->trimmed('subaction');
$this->runner = YammerRunner::init();
return $ok;
}
function handle($args)
{
// @fixme move this to saveSettings and friends?
if ($_SERVER['REQUEST_METHOD'] == 'POST') {
$this->checkSessionToken();
if ($this->subaction == 'change-apikey') {
$form = new YammerApiKeyForm($this);
} else if ($this->subaction == 'apikey') {
if ($this->saveKeys()) {
$form = new YammerAuthInitForm($this, $this->runner);
} else {
$form = new YammerApiKeyForm($this);
}
} else if ($this->subaction == 'authinit') {
// hack
if ($this->arg('change-apikey')) {
$form = new YammerApiKeyForm($this);
} else {
$url = $this->runner->requestAuth();
$form = new YammerAuthVerifyForm($this, $this->runner);
}
} else if ($this->subaction == 'authverify') {
$this->runner->saveAuthToken($this->trimmed('verify_token'));
// Haho! Now we can make THE FUN HAPPEN
$this->runner->startBackgroundImport();
$form = new YammerProgressForm($this, $this->runner);
} else {
throw new ClientException('Invalid POST');
}
return $this->showAjaxForm($form);
}
return parent::handle($args);
}
function saveKeys()
{
$key = $this->trimmed('consumer_key');
$secret = $this->trimmed('consumer_secret');
Config::save('yammer', 'consumer_key', $key);
Config::save('yammer', 'consumer_secret', $secret);
return !empty($key) && !empty($secret);
}
function showAjaxForm($form)
{
$this->startHTML('text/xml;charset=utf-8');
$this->elementStart('head');
$this->element('title', null, _m('Yammer import'));
$this->elementEnd('head');
$this->elementStart('body');
$form->show();
$this->elementEnd('body');
$this->elementEnd('html');
}
/**
* Fetch the appropriate form for our current state.
* @return Form
*/
function statusForm()
{
if (!(common_config('yammer', 'consumer_key'))
|| !(common_config('yammer', 'consumer_secret'))) {
return new YammerApiKeyForm($this);
}
switch($this->runner->state())
{
case 'init':
return new YammerAuthInitForm($this, $this->runner);
case 'requesting-auth':
return new YammerAuthVerifyForm($this, $this->runner);
default:
return new YammerProgressForm($this, $this->runner);
}
}
/**
* Show the Yammer admin panel form
*
* @return void
*/
function showForm()
{
$this->elementStart('fieldset');
$this->statusForm()->show();
$this->elementEnd('fieldset');
}
function showStylesheets()
{
parent::showStylesheets();
$this->cssLink('plugins/YammerImport/css/admin.css', null, 'screen, projection, tv');
}
function showScripts()
{
parent::showScripts();
$this->script('plugins/YammerImport/js/yammer-admin.js');
}
}

View File

@ -0,0 +1,79 @@
<?php
/**
* StatusNet, the distributed open-source microblogging tool
*
* Yammer import administration panel
*
* PHP version 5
*
* LICENCE: This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* @category Settings
* @package StatusNet
* @author Zach Copley <zach@status.net>
* @copyright 2010 StatusNet, Inc.
* @license http://www.fsf.org/licensing/licenses/agpl-3.0.html GNU Affero General Public License version 3.0
* @link http://status.net/
*/
if (!defined('STATUSNET')) {
exit(1);
}
class YammerauthAction extends AdminPanelAction
{
/**
* Show the Yammer admin panel form
*
* @return void
*/
function prepare($args)
{
parent::prepare($args);
$this->verify_token = $this->trim('verify_token');
}
/**
* Handle request
*
* Does the subscription and returns results.
*
* @param Array $args unused.
*
* @return void
*/
function handle($args)
{
if ($this->verify_token) {
$runner->saveAuthToken($this->verify_token);
$form = new YammerAuthProgressForm();
} else {
$url = $runner->requestAuth();
$form = new YammerAuthVerifyForm($this, $url);
}
$this->startHTML('text/xml;charset=utf-8');
$this->elementStart('head');
$this->element('title', null, _m('Connect to Yammer'));
$this->elementEnd('head');
$this->elementStart('body');
$form->show();
$this->elementEnd('body');
$this->elementEnd('html');
}
}

View File

@ -0,0 +1,165 @@
<?php
/**
* Data class for remembering Yammer import mappings
*
* PHP version 5
*
* @category Data
* @package StatusNet
* @author Brion Vibber <brion@status.net>
* @license http://www.fsf.org/licensing/licenses/agpl.html AGPLv3
* @link http://status.net/
*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
if (!defined('STATUSNET')) {
exit(1);
}
/**
* Common base class for the Yammer import mappings for users, groups, and notices.
*
* Child classes must override these static methods, since we need to run
* on PHP 5.2.x which has no late static binding:
* - staticGet (as our other classes)
* - schemaDef (call self::doSchemaDef)
* - record (call self::doRecord)
*/
class Yammer_common extends Memcached_DataObject
{
public $__table = 'yammer_XXXX'; // table name
public $__field = 'XXXX_id'; // field name to save into
public $id; // int primary_key not_null
public $user_id; // int(4)
public $created; // datetime
/**
* @fixme add a 'references' thing for the foreign key when we support that
*/
protected static function doSchemaDef($field)
{
return array(new ColumnDef('id', 'bigint', null,
false, 'PRI'),
new ColumnDef($field, 'integer', null,
false, 'UNI'),
new ColumnDef('created', 'datetime', null,
false));
}
/**
* return table definition for DB_DataObject
*
* DB_DataObject needs to know something about the table to manipulate
* instances. This method provides all the DB_DataObject needs to know.
*
* @return array array of column definitions
*/
function table()
{
return array('id' => DB_DATAOBJECT_INT + DB_DATAOBJECT_NOTNULL,
$this->__field => DB_DATAOBJECT_INT + DB_DATAOBJECT_NOTNULL,
'created' => DB_DATAOBJECT_STR + DB_DATAOBJECT_DATE + DB_DATAOBJECT_TIME + DB_DATAOBJECT_NOTNULL);
}
/**
* return key definitions for DB_DataObject
*
* DB_DataObject needs to know about keys that the table has, since it
* won't appear in StatusNet's own keys list. In most cases, this will
* simply reference your keyTypes() function.
*
* @return array list of key field names
*/
function keys()
{
return array_keys($this->keyTypes());
}
/**
* return key definitions for Memcached_DataObject
*
* Our caching system uses the same key definitions, but uses a different
* method to get them. This key information is used to store and clear
* cached data, so be sure to list any key that will be used for static
* lookups.
*
* @return array associative array of key definitions, field name to type:
* 'K' for primary key: for compound keys, add an entry for each component;
* 'U' for unique keys: compound keys are not well supported here.
*/
function keyTypes()
{
return array('id' => 'K', $this->__field => 'U');
}
/**
* Magic formula for non-autoincrementing integer primary keys
*
* If a table has a single integer column as its primary key, DB_DataObject
* assumes that the column is auto-incrementing and makes a sequence table
* to do this incrementation. Since we don't need this for our class, we
* overload this method and return the magic formula that DB_DataObject needs.
*
* @return array magic three-false array that stops auto-incrementing.
*/
function sequenceKey()
{
return array(false, false, false);
}
/**
* Save a mapping between a remote Yammer and local imported user.
*
* @param integer $user_id ID of the status in StatusNet
* @param integer $orig_id ID of the notice in Yammer
*
* @return Yammer_common new object for this value
*/
protected static function doRecord($class, $field, $orig_id, $local_id)
{
$map = parent::staticGet($class, 'id', $orig_id);
if (!empty($map)) {
return $map;
}
$map = parent::staticGet($class, $field, $local_id);
if (!empty($map)) {
return $map;
}
common_debug("Mapping Yammer $field {$orig_id} to local $field {$local_id}");
$map = new $class();
$map->id = $orig_id;
$map->$field = $local_id;
$map->created = common_sql_now();
$map->insert();
return $map;
}
}

View File

@ -0,0 +1,79 @@
<?php
/**
* Data class for remembering Yammer import mappings
*
* PHP version 5
*
* @category Data
* @package StatusNet
* @author Brion Vibber <brion@status.net>
* @license http://www.fsf.org/licensing/licenses/agpl.html AGPLv3
* @link http://status.net/
*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
if (!defined('STATUSNET')) {
exit(1);
}
class Yammer_group extends Yammer_common
{
public $__table = 'yammer_group'; // table name
public $__field = 'group_id'; // field to map to
public $group_id; // int
/**
* Get an instance by key
*
* This is a utility method to get a single instance with a given key value.
*
* @param string $k Key to use to lookup
* @param mixed $v Value to lookup
*
* @return Yammer_group object found, or null for no hits
*
*/
function staticGet($k, $v=null)
{
return Memcached_DataObject::staticGet('Yammer_group', $k, $v);
}
/**
* Return schema definition to set this table up in onCheckSchema
*/
static function schemaDef()
{
return self::doSchemaDef('group_id');
}
/**
* Save a mapping between a remote Yammer and local imported group.
*
* @param integer $orig_id ID of the notice in Yammer
* @param integer $group_id ID of the status in StatusNet
*
* @return Yammer_group new object for this value
*/
static function record($orig_id, $group_id)
{
return self::doRecord('Yammer_group', 'group_id', $orig_id, $group_id);
}
}

View File

@ -0,0 +1,79 @@
<?php
/**
* Data class for remembering Yammer import mappings
*
* PHP version 5
*
* @category Data
* @package StatusNet
* @author Brion Vibber <brion@status.net>
* @license http://www.fsf.org/licensing/licenses/agpl.html AGPLv3
* @link http://status.net/
*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
if (!defined('STATUSNET')) {
exit(1);
}
class Yammer_notice extends Yammer_common
{
public $__table = 'yammer_notice'; // table name
public $__field = 'notice_id'; // field to map to
public $notice_id; // int
/**
* Get an instance by key
*
* This is a utility method to get a single instance with a given key value.
*
* @param string $k Key to use to lookup
* @param mixed $v Value to lookup
*
* @return Yammer_notice object found, or null for no hits
*
*/
function staticGet($k, $v=null)
{
return Memcached_DataObject::staticGet('Yammer_notice', $k, $v);
}
/**
* Return schema definition to set this table up in onCheckSchema
*/
static function schemaDef()
{
return self::doSchemaDef('notice_id');
}
/**
* Save a mapping between a remote Yammer and local imported notice.
*
* @param integer $orig_id ID of the notice in Yammer
* @param integer $notice_id ID of the status in StatusNet
*
* @return Yammer_notice new object for this value
*/
static function record($orig_id, $notice_id)
{
return self::doRecord('Yammer_notice', 'notice_id', $orig_id, $notice_id);
}
}

View File

@ -0,0 +1,183 @@
<?php
/**
* Data class for remembering Yammer import mappings
*
* PHP version 5
*
* @category Data
* @package StatusNet
* @author Brion Vibber <brion@status.net>
* @license http://www.fsf.org/licensing/licenses/agpl.html AGPLv3
* @link http://status.net/
*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
if (!defined('STATUSNET')) {
exit(1);
}
/**
* Temporary storage for imported Yammer messages between fetching and saving
* as local notices.
*
* The Yammer API only allows us to page down from the most recent items; in
* order to start saving the oldest notices first, we have to pull them all
* down in reverse chronological order, then go back over them from oldest to
* newest and actually save them into our notice table.
*/
class Yammer_notice_stub extends Memcached_DataObject
{
public $__table = 'yammer_notice_stub'; // table name
public $id; // int primary_key not_null
public $json_data; // text
public $created; // datetime
/**
* Get an instance by key
*
* This is a utility method to get a single instance with a given key value.
*
* @param string $k Key to use to lookup
* @param mixed $v Value to lookup
*
* @return Yammer_notice_stub object found, or null for no hits
*
*/
function staticGet($k, $v=null)
{
return Memcached_DataObject::staticGet('Yammer_notice_stub', $k, $v);
}
/**
* Return schema definition to set this table up in onCheckSchema
*/
static function schemaDef()
{
return array(new ColumnDef('id', 'bigint', null,
false, 'PRI'),
new ColumnDef('json_data', 'text', null,
false),
new ColumnDef('created', 'datetime', null,
false));
}
/**
* return table definition for DB_DataObject
*
* DB_DataObject needs to know something about the table to manipulate
* instances. This method provides all the DB_DataObject needs to know.
*
* @return array array of column definitions
*/
function table()
{
return array('id' => DB_DATAOBJECT_INT + DB_DATAOBJECT_NOTNULL,
'json_data' => DB_DATAOBJECT_STR + DB_DATAOBJECT_NOTNULL,
'created' => DB_DATAOBJECT_STR + DB_DATAOBJECT_DATE + DB_DATAOBJECT_TIME + DB_DATAOBJECT_NOTNULL);
}
/**
* return key definitions for DB_DataObject
*
* DB_DataObject needs to know about keys that the table has, since it
* won't appear in StatusNet's own keys list. In most cases, this will
* simply reference your keyTypes() function.
*
* @return array list of key field names
*/
function keys()
{
return array_keys($this->keyTypes());
}
/**
* return key definitions for Memcached_DataObject
*
* Our caching system uses the same key definitions, but uses a different
* method to get them. This key information is used to store and clear
* cached data, so be sure to list any key that will be used for static
* lookups.
*
* @return array associative array of key definitions, field name to type:
* 'K' for primary key: for compound keys, add an entry for each component;
* 'U' for unique keys: compound keys are not well supported here.
*/
function keyTypes()
{
return array('id' => 'K');
}
/**
* Magic formula for non-autoincrementing integer primary keys
*
* If a table has a single integer column as its primary key, DB_DataObject
* assumes that the column is auto-incrementing and makes a sequence table
* to do this incrementation. Since we don't need this for our class, we
* overload this method and return the magic formula that DB_DataObject needs.
*
* @return array magic three-false array that stops auto-incrementing.
*/
function sequenceKey()
{
return array(false, false, false);
}
/**
* Decode the stored data structure.
*
* @return mixed
*/
public function getData()
{
return json_decode($this->json_data, true);
}
/**
* Save the native Yammer API representation of a message for the pending
* import. Since they come in in reverse chronological order, we need to
* record them all as stubs and then go through from the beginning and
* save them as native notices, or we'll lose ordering and threading
* data.
*
* @param integer $orig_id ID of the notice on Yammer
* @param array $data the message record fetched out of Yammer API returnd data
*
* @return Yammer_notice_stub new object for this value
*/
static function record($orig_id, $data)
{
common_debug("Recording Yammer message stub {$orig_id} for pending import...");
$stub = new Yammer_notice_stub();
$stub->id = $orig_id;
$stub->json_data = json_encode($data);
$stub->created = common_sql_now();
$stub->insert();
return $stub;
}
}

View File

@ -0,0 +1,153 @@
<?php
/**
* Data class for remembering Yammer import state
*
* PHP version 5
*
* @category Data
* @package StatusNet
* @author Brion Vibber <brion@status.net>
* @license http://www.fsf.org/licensing/licenses/agpl.html AGPLv3
* @link http://status.net/
*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
if (!defined('STATUSNET')) {
exit(1);
}
class Yammer_state extends Memcached_DataObject
{
public $__table = 'yammer_state'; // table name
public $id; // int primary_key not_null
public $state; // import state key
public $oauth_token; // actual oauth token! clear when import is done?
public $oauth_secret; // actual oauth secret! clear when import is done?
public $users_page; // last page of users we've fetched
public $groups_page; // last page of groups we've fetched
public $messages_oldest; // oldest message ID we've fetched
public $messages_newest; // newest message ID we've imported
public $created; // datetime
public $modified; // datetime
/**
* Get an instance by key
*
* This is a utility method to get a single instance with a given key value.
*
* @param string $k Key to use to lookup
* @param mixed $v Value to lookup
*
* @return Yammer_state object found, or null for no hits
*
*/
function staticGet($k, $v=null)
{
return Memcached_DataObject::staticGet('Yammer_state', $k, $v);
}
/**
* Return schema definition to set this table up in onCheckSchema
*/
static function schemaDef()
{
return array(new ColumnDef('id', 'int', null,
false, 'PRI'),
new ColumnDef('state', 'text'),
new ColumnDef('oauth_token', 'text'),
new ColumnDef('oauth_secret', 'text'),
new ColumnDef('users_page', 'int'),
new ColumnDef('groups_page', 'int'),
new ColumnDef('messages_oldest', 'bigint'),
new ColumnDef('messages_newest', 'bigint'),
new ColumnDef('created', 'datetime'),
new ColumnDef('modified', 'datetime'));
}
/**
* return table definition for DB_DataObject
*
* DB_DataObject needs to know something about the table to manipulate
* instances. This method provides all the DB_DataObject needs to know.
*
* @return array array of column definitions
*/
function table()
{
return array('id' => DB_DATAOBJECT_INT + DB_DATAOBJECT_NOTNULL,
'state' => DB_DATAOBJECT_STR,
'oauth_token' => DB_DATAOBJECT_STR,
'oauth_secret' => DB_DATAOBJECT_STR,
'users_page' => DB_DATAOBJECT_INT,
'groups_page' => DB_DATAOBJECT_INT,
'messages_oldest' => DB_DATAOBJECT_INT,
'messages_newest' => DB_DATAOBJECT_INT,
'created' => DB_DATAOBJECT_STR + DB_DATAOBJECT_DATE + DB_DATAOBJECT_TIME + DB_DATAOBJECT_NOTNULL);
}
/**
* return key definitions for DB_DataObject
*
* DB_DataObject needs to know about keys that the table has, since it
* won't appear in StatusNet's own keys list. In most cases, this will
* simply reference your keyTypes() function.
*
* @return array list of key field names
*/
function keys()
{
return array_keys($this->keyTypes());
}
/**
* return key definitions for Memcached_DataObject
*
* Our caching system uses the same key definitions, but uses a different
* method to get them. This key information is used to store and clear
* cached data, so be sure to list any key that will be used for static
* lookups.
*
* @return array associative array of key definitions, field name to type:
* 'K' for primary key: for compound keys, add an entry for each component;
* 'U' for unique keys: compound keys are not well supported here.
*/
function keyTypes()
{
return array('id' => 'K');
}
/**
* Magic formula for non-autoincrementing integer primary keys
*
* If a table has a single integer column as its primary key, DB_DataObject
* assumes that the column is auto-incrementing and makes a sequence table
* to do this incrementation. Since we don't need this for our class, we
* overload this method and return the magic formula that DB_DataObject needs.
*
* @return array magic three-false array that stops auto-incrementing.
*/
function sequenceKey()
{
return array(false, false, false);
}
}

View File

@ -0,0 +1,79 @@
<?php
/**
* Data class for remembering Yammer import mappings
*
* PHP version 5
*
* @category Data
* @package StatusNet
* @author Brion Vibber <brion@status.net>
* @license http://www.fsf.org/licensing/licenses/agpl.html AGPLv3
* @link http://status.net/
*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
if (!defined('STATUSNET')) {
exit(1);
}
class Yammer_user extends Yammer_common
{
public $__table = 'yammer_user'; // table name
public $__field = 'user_id'; // field to map to
public $user_id; // int
/**
* Get an instance by key
*
* This is a utility method to get a single instance with a given key value.
*
* @param string $k Key to use to lookup
* @param mixed $v Value to lookup
*
* @return Yammer_user object found, or null for no hits
*
*/
function staticGet($k, $v=null)
{
return Memcached_DataObject::staticGet('Yammer_user', $k, $v);
}
/**
* Return schema definition to set this table up in onCheckSchema
*/
static function schemaDef()
{
return self::doSchemaDef('user_id');
}
/**
* Save a mapping between a remote Yammer and local imported user.
*
* @param integer $orig_id ID of the notice in Yammer
* @param integer $user_id ID of the status in StatusNet
*
* @return Yammer_user new object for this value
*/
static function record($orig_id, $user_id)
{
return self::doRecord('Yammer_user', 'user_id', $orig_id, $user_id);
}
}

View File

@ -0,0 +1,55 @@
.yammer-import {
padding: 16px;
}
.import-step {
padding: 8px;
}
.import-label {
font-weight: bold;
}
.import-status {
margin-left: 20px;
padding-left: 20px;
}
.waiting {
color: #888;
}
.progress {
background-color: white;
border: solid 1px blue;
border-radius: 8px;
-moz-border-radius: 8px;
-webkit-border-radius: 8px;
-opera-border-radius: 8px;
}
.progress .import-label {
color: blue;
}
.progress .import-status {
background-image: url(icon_processing.gif);
background-repeat: no-repeat;
}
.complete {
color: black;
}
.complete .import-status {
background-image: url(done.png);
background-repeat: no-repeat;
}
.import-step-done .import-status {
/* override */
background: none !important;
}
.magiclink {
margin-left: 40px;
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 991 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 673 B

View File

@ -0,0 +1,246 @@
<?php
/*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
/**
* Basic client class for Yammer's OAuth/JSON API.
*
* @package YammerImportPlugin
* @author Brion Vibber <brion@status.net>
*/
class SN_YammerClient
{
protected $apiBase = "https://www.yammer.com";
protected $consumerKey, $consumerSecret;
protected $token, $tokenSecret, $verifier;
public function __construct($consumerKey, $consumerSecret, $token=null, $tokenSecret=null)
{
$this->consumerKey = $consumerKey;
$this->consumerSecret = $consumerSecret;
$this->token = $token;
$this->tokenSecret = $tokenSecret;
}
/**
* Make an HTTP GET request with OAuth headers and return an HTTPResponse
* with the returned body and codes.
*
* @param string $url
* @return HTTPResponse
*
* @throws Exception on low-level network error
*/
protected function httpGet($url)
{
$headers = array('Authorization: ' . $this->authHeader());
$client = HTTPClient::start();
return $client->get($url, $headers);
}
/**
* Make an HTTP GET request with OAuth headers and return the response body
* on success.
*
* @param string $url
* @return string
*
* @throws Exception on low-level network or HTTP error
*/
public function fetchUrl($url)
{
$response = $this->httpGet($url);
if ($response->isOk()) {
return $response->getBody();
} else {
throw new Exception("Yammer API returned HTTP code " . $response->getStatus() . ': ' . $response->getBody());
}
}
/**
* Make an HTTP hit with OAuth headers and return the response body on success.
*
* @param string $path URL chunk for the API method
* @param array $params
* @return string
*
* @throws Exception on low-level network or HTTP error
*/
protected function fetchApi($path, $params=array())
{
$url = $this->apiBase . '/' . $path;
if ($params) {
$url .= '?' . http_build_query($params, null, '&');
}
return $this->fetchUrl($url);
}
/**
* Hit the main Yammer API point and decode returned JSON data.
*
* @param string $method
* @param array $params
* @return array from JSON data
*
* @throws Exception for HTTP error or bad JSON return
*/
public function api($method, $params=array())
{
$body = $this->fetchApi("api/v1/$method.json", $params);
$data = json_decode($body, true);
if ($data === null) {
common_log(LOG_ERR, "Invalid JSON response from Yammer API: " . $body);
throw new Exception("Invalid JSON response from Yammer API");
}
return $data;
}
/**
* Build an Authorization header value from the keys we have available.
*/
protected function authHeader()
{
// token
// token_secret
$params = array('realm' => '',
'oauth_consumer_key' => $this->consumerKey,
'oauth_signature_method' => 'PLAINTEXT',
'oauth_timestamp' => time(),
'oauth_nonce' => time(),
'oauth_version' => '1.0');
if ($this->token) {
$params['oauth_token'] = $this->token;
}
if ($this->tokenSecret) {
$params['oauth_signature'] = $this->consumerSecret . '&' . $this->tokenSecret;
} else {
$params['oauth_signature'] = $this->consumerSecret . '&';
}
if ($this->verifier) {
$params['oauth_verifier'] = $this->verifier;
}
$parts = array_map(array($this, 'authHeaderChunk'), array_keys($params), array_values($params));
return 'OAuth ' . implode(', ', $parts);
}
/**
* Encode a key-value pair for use in an authentication header.
*
* @param string $key
* @param string $val
* @return string
*/
protected function authHeaderChunk($key, $val)
{
return urlencode($key) . '="' . urlencode($val) . '"';
}
/**
* Ask the Yammer server for a request token, which can be passed on
* to authorizeUrl() for the user to start the authentication process.
*
* @return array of oauth return data; should contain nice things
*/
public function requestToken()
{
if ($this->token || $this->tokenSecret) {
throw new Exception("Requesting a token, but already set up with a token");
}
$data = $this->fetchApi('oauth/request_token');
$arr = array();
parse_str($data, $arr);
return $arr;
}
/**
* Get a final access token from the verifier/PIN code provided to
* the user from Yammer's auth pages.
*
* @return array of oauth return data; should contain nice things
*/
public function accessToken($verifier)
{
$this->verifier = $verifier;
$data = $this->fetchApi('oauth/access_token');
$this->verifier = null;
$arr = array();
parse_str($data, $arr);
return $arr;
}
/**
* Give the URL to send users to to authorize a new app setup.
*
* @param string $token as returned from accessToken()
* @return string URL
*/
public function authorizeUrl($token)
{
return $this->apiBase . '/oauth/authorize?oauth_token=' . urlencode($token);
}
/**
* High-level API hit: fetch all messages in the network (up to 20 at a time).
* Return data is the full JSON array returned, including meta and references
* sections.
*
* The matching messages themselves will be in the 'messages' item within.
*
* @param array $options optional set of additional params for the request.
* @return array
*
* @throws Exception on low-level or HTTP error
*/
public function messages($params=array())
{
return $this->api('messages', $params);
}
/**
* High-level API hit: fetch all users in the network (up to 50 at a time).
* Return data is the full JSON array returned, listing user items.
*
* The matching messages themselves will be in the 'users' item within.
*
* @param array $options optional set of additional params for the request.
* @return array of JSON-sourced user data arrays
*
* @throws Exception on low-level or HTTP error
*/
public function users($params=array())
{
return $this->api('users', $params);
}
/**
* High-level API hit: fetch all groups in the network (up to 20 at a time).
* Return data is the full JSON array returned, listing user items.
*
* The matching messages themselves will be in the 'users' item within.
*
* @param array $options optional set of additional params for the request.
* @return array of JSON-sourced user data arrays
*
* @throws Exception on low-level or HTTP error
*/
public function groups($params=array())
{
return $this->api('groups', $params);
}
}

View File

@ -0,0 +1,112 @@
<?php
class YammerApikeyForm extends Form
{
private $runner;
function __construct($out)
{
parent::__construct($out);
$this->runner = $runner;
}
/**
* ID of the form
*
* @return int ID of the form
*/
function id()
{
return 'yammer-apikey-form';
}
/**
* class of the form
*
* @return string of the form class
*/
function formClass()
{
return 'form_yammer_apikey form_settings';
}
/**
* Action of the form
*
* @return string URL of the action
*/
function action()
{
return common_local_url('yammeradminpanel');
}
/**
* Legend of the Form
*
* @return void
*/
function formLegend()
{
$this->out->element('legend', null, _m('Yammer API registration'));
}
/**
* Data elements of the form
*
* @return void
*/
function formData()
{
$this->out->hidden('subaction', 'apikey');
$this->out->elementStart('fieldset');
$this->out->elementStart('p');
$this->out->text(_m('Before we can connect to your Yammer network, ' .
'you will need to register the importer as an ' .
'application authorized to pull data on your behalf. ' .
'This registration will work only for your own network. ' .
'Follow this link to register the app at Yammer; ' .
'you will be prompted to log in if necessary:'));
$this->out->elementEnd('p');
$this->out->elementStart('p', array('class' => 'magiclink'));
$this->out->element('a',
array('href' => 'https://www.yammer.com/client_applications/new',
'target' => '_blank'),
_m('Open Yammer application registration form'));
$this->out->elementEnd('p');
$this->out->element('p', array(), _m('Copy the consumer key and secret you are given into the form below:'));
$this->out->elementStart('ul', array('class' => 'form_data'));
$this->out->elementStart('li');
$this->out->input('consumer_key', _m('Consumer key:'), common_config('yammer', 'consumer_key'));
$this->out->elementEnd('li');
$this->out->elementStart('li');
$this->out->input('consumer_secret', _m('Consumer secret:'), common_config('yammer', 'consumer_secret'));
$this->out->elementEnd('li');
$this->out->elementEnd('ul');
$this->out->submit('submit', _m('Save'), 'submit', null, _m('Save these consumer keys'));
$this->out->elementEnd('fieldset');
}
/**
* Action elements
*
* @return void
*/
function formActions()
{
}
}

View File

@ -0,0 +1,76 @@
<?php
class YammerAuthInitForm extends Form
{
/**
* ID of the form
*
* @return int ID of the form
*/
function id()
{
return 'yammer-auth-init-form';
}
/**
* class of the form
*
* @return string of the form class
*/
function formClass()
{
return 'form_yammer_auth_init form_settings';
}
/**
* Action of the form
*
* @return string URL of the action
*/
function action()
{
return common_local_url('yammeradminpanel');
}
/**
* Legend of the Form
*
* @return void
*/
function formLegend()
{
$this->out->element('legend', null, _m('Connect to Yammer'));
}
/**
* Data elements of the form
*
* @return void
*/
function formData()
{
$this->out->hidden('subaction', 'authinit');
$this->out->elementStart('fieldset');
$this->out->submit('submit', _m('Start authentication'), 'submit', null, _m('Request authorization to connect to Yammer account'));
$this->out->submit('change-apikey', _m('Change API key'));
$this->out->elementEnd('fieldset');
}
/**
* Action elements
*
* @return void
*/
function formActions()
{
}
}

View File

@ -0,0 +1,111 @@
<?php
class YammerAuthVerifyForm extends Form
{
private $runner;
function __construct($out, YammerRunner $runner)
{
parent::__construct($out);
$this->runner = $runner;
}
/**
* ID of the form
*
* @return int ID of the form
*/
function id()
{
return 'yammer-auth-verify-form';
}
/**
* class of the form
*
* @return string of the form class
*/
function formClass()
{
return 'form_yammer_auth_verify form_settings';
}
/**
* Action of the form
*
* @return string URL of the action
*/
function action()
{
return common_local_url('yammeradminpanel');
}
/**
* Legend of the Form
*
* @return void
*/
function formLegend()
{
$this->out->element('legend', null, _m('Connect to Yammer'));
}
/**
* Data elements of the form
*
* @return void
*/
function formData()
{
$this->out->hidden('subaction', 'authverify');
$this->out->elementStart('fieldset');
$this->out->elementStart('p');
$this->out->text(_m('Follow this link to confirm authorization at Yammer; you will be prompted to log in if necessary:'));
$this->out->elementEnd('p');
// iframe would be nice to avoid leaving -- since they don't seem to have callback url O_O
/*
$this->out->element('iframe', array('id' => 'yammer-oauth',
'src' => $this->runner->getAuthUrl()));
*/
// yeah, it ignores the callback_url
// soo... crappy link. :(
$this->out->elementStart('p', array('class' => 'magiclink'));
$this->out->element('a',
array('href' => $this->runner->getAuthUrl(),
'target' => '_blank'),
_m('Open Yammer authentication window'));
$this->out->elementEnd('p');
$this->out->element('p', array(), _m('Copy the verification code you are given below:'));
$this->out->elementStart('ul', array('class' => 'form_data'));
$this->out->elementStart('li');
$this->out->input('verify_token', _m('Verification code:'));
$this->out->elementEnd('li');
$this->out->elementEnd('ul');
$this->out->submit('submit', _m('Continue'), 'submit', null, _m('Save code and begin import'));
$this->out->elementEnd('fieldset');
}
/**
* Action elements
*
* @return void
*/
function formActions()
{
}
}

View File

@ -0,0 +1,456 @@
<?php
/*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
/**
* Basic client class for Yammer's OAuth/JSON API.
*
* @package YammerImportPlugin
* @author Brion Vibber <brion@status.net>
*/
class YammerImporter
{
protected $client;
function __construct(SN_YammerClient $client)
{
$this->client = $client;
}
/**
* Load or create an imported profile from Yammer data.
*
* @param object $item loaded JSON data for Yammer importer
* @return Profile
*/
function importUser($item)
{
$data = $this->prepUser($item);
$profileId = $this->findImportedUser($data['orig_id']);
if ($profileId) {
return Profile::staticGet('id', $profileId);
} else {
$user = User::register($data['options']);
$profile = $user->getProfile();
if ($data['avatar']) {
try {
$this->saveAvatar($data['avatar'], $profile);
} catch (Exception $e) {
common_log(LOG_ERR, "Error importing Yammer avatar: " . $e->getMessage());
}
}
$this->recordImportedUser($data['orig_id'], $profile->id);
return $profile;
}
}
/**
* Load or create an imported group from Yammer data.
*
* @param object $item loaded JSON data for Yammer importer
* @return User_group
*/
function importGroup($item)
{
$data = $this->prepGroup($item);
$groupId = $this->findImportedGroup($data['orig_id']);
if ($groupId) {
return User_group::staticGet('id', $groupId);
} else {
$group = User_group::register($data['options']);
if ($data['avatar']) {
try {
$this->saveAvatar($data['avatar'], $group);
} catch (Exception $e) {
common_log(LOG_ERR, "Error importing Yammer avatar: " . $e->getMessage());
}
}
$this->recordImportedGroup($data['orig_id'], $group->id);
return $group;
}
}
/**
* Load or create an imported notice from Yammer data.
*
* @param object $item loaded JSON data for Yammer importer
* @return Notice
*/
function importNotice($item)
{
$data = $this->prepNotice($item);
$noticeId = $this->findImportedNotice($data['orig_id']);
if ($noticeId) {
return Notice::staticGet('id', $noticeId);
} else {
$content = $data['content'];
$user = User::staticGet($data['profile']);
// Fetch file attachments and add the URLs...
$uploads = array();
foreach ($data['attachments'] as $url) {
try {
$upload = $this->saveAttachment($url, $user);
$content .= ' ' . $upload->shortUrl();
$uploads[] = $upload;
} catch (Exception $e) {
common_log(LOG_ERR, "Error importing Yammer attachment: " . $e->getMessage());
}
}
// Here's the meat! Actually save the dang ol' notice.
$notice = Notice::saveNew($user->id,
$content,
$data['source'],
$data['options']);
// Save "likes" as favorites...
foreach ($data['faves'] as $nickname) {
$user = User::staticGet('nickname', $nickname);
if ($user) {
Fave::addNew($user->getProfile(), $notice);
}
}
// And finally attach the upload records...
foreach ($uploads as $upload) {
$upload->attachToNotice($notice);
}
$this->recordImportedNotice($data['orig_id'], $notice->id);
return $notice;
}
}
/**
* Pull relevant info out of a Yammer data record for a user import.
*
* @param array $item
* @return array
*/
function prepUser($item)
{
if ($item['type'] != 'user') {
throw new Exception('Wrong item type sent to Yammer user import processing.');
}
$origId = $item['id'];
$origUrl = $item['url'];
// @fixme check username rules?
$options['nickname'] = $item['name'];
$options['fullname'] = trim($item['full_name']);
// Avatar... this will be the "_small" variant.
// Remove that (pre-extension) suffix to get the orig-size image.
$avatar = $item['mugshot_url'];
// The following info is only available in full data, not in the reference version.
// There can be extensive contact info, but for now we'll only pull the primary email.
if (isset($item['contact'])) {
foreach ($item['contact']['email_addresses'] as $addr) {
if ($addr['type'] == 'primary') {
$options['email'] = $addr['address'];
$options['email_confirmed'] = true;
break;
}
}
}
// There can be multiple external URLs; for now pull the first one as home page.
if (isset($item['external_urls'])) {
foreach ($item['external_urls'] as $url) {
if (common_valid_http_url($url)) {
$options['homepage'] = $url;
break;
}
}
}
// Combine a few bits into the bio...
$bio = array();
if (!empty($item['job_title'])) {
$bio[] = $item['job_title'];
}
if (!empty($item['summary'])) {
$bio[] = $item['summary'];
}
if (!empty($item['expertise'])) {
$bio[] = _m('Expertise:') . ' ' . $item['expertise'];
}
$options['bio'] = implode("\n\n", $bio);
// Pull raw location string, may be lookupable
if (!empty($item['location'])) {
$options['location'] = $item['location'];
}
// Timezone is in format like 'Pacific Time (US & Canada)'
// We need to convert that to a zone id. :P
// @fixme timezone not yet supported at registration time :)
if (!empty($item['timezone'])) {
$tz = $this->timezone($item['timezone']);
if ($tz) {
$options['timezone'] = $tz;
}
}
return array('orig_id' => $origId,
'orig_url' => $origUrl,
'avatar' => $avatar,
'options' => $options);
}
/**
* Pull relevant info out of a Yammer data record for a group import.
*
* @param array $item
* @return array
*/
function prepGroup($item)
{
if ($item['type'] != 'group') {
throw new Exception('Wrong item type sent to Yammer group import processing.');
}
$origId = $item['id'];
$origUrl = $item['url'];
$privacy = $item['privacy']; // Warning! only public groups in SN so far
$options['nickname'] = $item['name'];
$options['fullname'] = $item['full_name'];
$options['description'] = $item['description'];
$options['created'] = $this->timestamp($item['created_at']);
$avatar = $item['mugshot_url']; // as with user profiles...
$options['mainpage'] = common_local_url('showgroup',
array('nickname' => $options['nickname']));
// @fixme what about admin user for the group?
// bio? homepage etc? aliases?
$options['local'] = true;
return array('orig_id' => $origId,
'orig_url' => $origUrl,
'options' => $options,
'avatar' => $avatar);
}
/**
* Pull relevant info out of a Yammer data record for a notice import.
*
* @param array $item
* @return array
*/
function prepNotice($item)
{
if (isset($item['type']) && $item['type'] != 'message') {
throw new Exception('Wrong item type sent to Yammer message import processing.');
}
$origId = $item['id'];
$origUrl = $item['url'];
$profile = $this->findImportedUser($item['sender_id']);
$content = $item['body']['plain'];
$source = 'yammer';
$options = array();
if ($item['replied_to_id']) {
$replyTo = $this->findImportedNotice($item['replied_to_id']);
if ($replyTo) {
$options['reply_to'] = $replyTo;
}
}
$options['created'] = $this->timestamp($item['created_at']);
if ($item['group_id']) {
$groupId = $this->findImportedGroup($item['group_id']);
if ($groupId) {
$options['groups'] = array($groupId);
// @fixme if we see a group link inline, don't add this?
$group = User_group::staticGet('id', $groupId);
if ($group) {
$content .= ' !' . $group->nickname;
}
}
}
$faves = array();
foreach ($item['liked_by']['names'] as $liker) {
// "permalink" is the username. wtf?
$faves[] = $liker['permalink'];
}
$attachments = array();
foreach ($item['attachments'] as $attach) {
if ($attach['type'] == 'image' || $attach['type'] == 'file') {
$attachments[] = $attach[$attach['type']]['url'];
} else {
common_log(LOG_WARNING, "Unrecognized Yammer attachment type: " . $attach['type']);
}
}
return array('orig_id' => $origId,
'orig_url' => $origUrl,
'profile' => $profile,
'content' => $content,
'source' => $source,
'options' => $options,
'faves' => $faves,
'attachments' => $attachments);
}
private function findImportedUser($origId)
{
$map = Yammer_user::staticGet('id', $origId);
return $map ? $map->user_id : null;
}
private function findImportedGroup($origId)
{
$map = Yammer_group::staticGet('id', $origId);
return $map ? $map->group_id : null;
}
private function findImportedNotice($origId)
{
$map = Yammer_notice::staticGet('id', $origId);
return $map ? $map->notice_id : null;
}
private function recordImportedUser($origId, $userId)
{
Yammer_user::record($origId, $userId);
}
private function recordImportedGroup($origId, $groupId)
{
Yammer_group::record($origId, $groupId);
}
private function recordImportedNotice($origId, $noticeId)
{
Yammer_notice::record($origId, $noticeId);
}
/**
* Normalize timestamp format.
* @param string $ts
* @return string
*/
private function timestamp($ts)
{
return common_sql_date(strtotime($ts));
}
private function timezone($tz)
{
// Blaaaaaarf!
$known = array('Pacific Time (US & Canada)' => 'America/Los_Angeles',
'Eastern Time (US & Canada)' => 'America/New_York');
if (array_key_exists($tz, $known)) {
return $known[$tz];
} else {
return false;
}
}
/**
* Download and update given avatar image
*
* @param string $url
* @param mixed $dest either a Profile or User_group object
* @throws Exception in various failure cases
*/
private function saveAvatar($url, $dest)
{
// Yammer API data mostly gives us the small variant.
// Try hitting the source image if we can!
// @fixme no guarantee of this URL scheme I think.
$url = preg_replace('/_small(\..*?)$/', '$1', $url);
if (!common_valid_http_url($url)) {
throw new ServerException(sprintf(_m("Invalid avatar URL %s."), $url));
}
// @fixme this should be better encapsulated
// ripped from oauthstore.php (for old OMB client)
$temp_filename = tempnam(sys_get_temp_dir(), 'listener_avatar');
if (!copy($url, $temp_filename)) {
throw new ServerException(sprintf(_m("Unable to fetch avatar from %s."), $url));
}
$id = $dest->id;
// @fixme should we be using different ids?
$imagefile = new ImageFile($id, $temp_filename);
$filename = Avatar::filename($id,
image_type_to_extension($imagefile->type),
null,
common_timestamp());
rename($temp_filename, Avatar::path($filename));
// @fixme hardcoded chmod is lame, but seems to be necessary to
// keep from accidentally saving images from command-line (queues)
// that can't be read from web server, which causes hard-to-notice
// problems later on:
//
// http://status.net/open-source/issues/2663
chmod(Avatar::path($filename), 0644);
$dest->setOriginal($filename);
}
/**
* Fetch an attachment from Yammer and save it into our system.
* Unlike avatars, the attachment URLs are guarded by authentication,
* so we need to run the HTTP hit through our OAuth API client.
*
* @param string $url
* @param User $user
* @return MediaFile
*
* @throws Exception on low-level network or HTTP error
*/
private function saveAttachment($url, User $user)
{
// Fetch the attachment...
// WARNING: file must fit in memory here :(
$body = $this->client->fetchUrl($url);
// Save to a temporary file and shove it into our file-attachment space...
$temp = tmpfile();
fwrite($temp, $body);
try {
$upload = MediaFile::fromFileHandle($temp, $user);
fclose($temp);
return $upload;
} catch (Exception $e) {
fclose($temp);
throw $e;
}
}
}

View File

@ -0,0 +1,128 @@
<?php
class YammerProgressForm extends Form
{
/**
* ID of the form
*
* @return string ID of the form
*/
function id()
{
return 'yammer-progress';
}
/**
* class of the form
*
* @return string class of the form
*/
function formClass()
{
return 'form_settings';
}
/**
* Action of the form
*
* @return string URL of the action
*/
function action()
{
return common_local_url('yammeradminpanel');
}
/**
* Data elements of the form
*
* @return void
*/
function formData()
{
$runner = YammerRunner::init();
$userCount = $runner->countUsers();
$groupCount = $runner->countGroups();
$fetchedCount = $runner->countFetchedNotices();
$savedCount = $runner->countSavedNotices();
$labels = array(
'init' => array(
'label' => _m("Initialize"),
'progress' => _m('No import running'),
'complete' => _m('Initiated Yammer server connection...'),
),
'requesting-auth' => array(
'label' => _m('Connect to Yammer'),
'progress' => _m('Awaiting authorization...'),
'complete' => _m('Connected.'),
),
'import-users' => array(
'label' => _m('Import user accounts'),
'progress' => sprintf(_m("Importing %d user...", "Importing %d users...", $userCount), $userCount),
'complete' => sprintf(_m("Imported %d user.", "Imported %d users.", $userCount), $userCount),
),
'import-groups' => array(
'label' => _m('Import user groups'),
'progress' => sprintf(_m("Importing %d group...", "Importing %d groups...", $groupCount), $groupCount),
'complete' => sprintf(_m("Imported %d group.", "Imported %d groups.", $groupCount), $groupCount),
),
'fetch-messages' => array(
'label' => _m('Prepare public notices for import'),
'progress' => sprintf(_m("Preparing %d notice...", "Preparing %d notices...", $fetchedCount), $fetchedCount),
'complete' => sprintf(_m("Prepared %d notice.", "Prepared %d notices.", $fetchedCount), $fetchedCount),
),
'save-messages' => array(
'label' => _m('Import public notices'),
'progress' => sprintf(_m("Importing %d notice...", "Importing %d notices...", $savedCount), $savedCount),
'complete' => sprintf(_m("Imported %d notice.", "Imported %d notices.", $savedCount), $savedCount),
),
'done' => array(
'label' => _m('Done'),
'progress' => sprintf(_m("Import is complete!")),
'complete' => sprintf(_m("Import is complete!")),
)
);
$steps = array_keys($labels);
$currentStep = array_search($runner->state(), $steps);
$this->out->elementStart('fieldset', array('class' => 'yammer-import'));
$this->out->element('legend', array(), _m('Import status'));
foreach ($steps as $step => $state) {
if ($state == 'init') {
// Don't show 'init', it's boring.
continue;
}
if ($step < $currentStep) {
// This step is done
$this->progressBar($state,
'complete',
$labels[$state]['label'],
$labels[$state]['complete']);
} else if ($step == $currentStep) {
// This step is in progress
$this->progressBar($state,
'progress',
$labels[$state]['label'],
$labels[$state]['progress']);
} else {
// This step has not yet been done.
$this->progressBar($state,
'waiting',
$labels[$state]['label'],
_m("Waiting..."));
}
}
$this->out->elementEnd('fieldset');
}
private function progressBar($state, $class, $label, $status)
{
// @fixme prettify ;)
$this->out->elementStart('div', array('class' => "import-step import-step-$state $class"));
$this->out->element('div', array('class' => 'import-label'), $label);
$this->out->element('div', array('class' => 'import-status'), $status);
$this->out->elementEnd('div');
}
}

View File

@ -0,0 +1,58 @@
<?php
/*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
if (!defined('STATUSNET')) {
exit(1);
}
/**
* Queue handler for bumping the next chunk of Yammer import activity!
*
* @package YammerImportPlugin
* @author Brion Vibber <brion@status.net>
*/
class YammerQueueHandler extends QueueHandler
{
function transport()
{
return 'yammer';
}
function handle($notice)
{
$runner = YammerRunner::init();
if ($runner->hasWork()) {
if ($runner->iterate()) {
if ($runner->hasWork()) {
// More to do? Shove us back on the queue...
$runner->startBackgroundImport();
}
return true;
} else {
// Something failed?
// @fixme should we be trying again here, or should we give warning?
return false;
}
} else {
// We're done!
common_log(LOG_INFO, "Yammer import has no work to do at this time; discarding.");
return true;
}
}
}

View File

@ -0,0 +1,398 @@
<?php
/*
* StatusNet - the distributed open-source microblogging tool
* Copyright (C) 2010, StatusNet, Inc.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
if (!defined('STATUSNET')) {
exit(1);
}
/**
* State machine for running through Yammer import.
*
* @package YammerImportPlugin
* @author Brion Vibber <brion@status.net>
*/
class YammerRunner
{
private $state;
private $client;
private $importer;
/**
* Normalize our singleton state and give us a YammerRunner object to play with!
*
* @return YammerRunner
*/
public static function init()
{
$state = Yammer_state::staticGet('id', 1);
if (!$state) {
$state = self::initState();
}
return new YammerRunner($state);
}
private static function initState()
{
$state = new Yammer_state();
$state->id = 1;
$state->state = 'init';
$state->created = common_sql_now();
$state->modified = common_sql_now();
$state->insert();
return $state;
}
private function __construct($state)
{
$this->state = $state;
$this->client = new SN_YammerClient(
common_config('yammer', 'consumer_key'),
common_config('yammer', 'consumer_secret'),
$this->state->oauth_token,
$this->state->oauth_secret);
$this->importer = new YammerImporter($this->client);
}
/**
* Check which state we're in
*
* @return string
*/
public function state()
{
return $this->state->state;
}
/**
* Is the import done, finished, complete, finito?
*
* @return boolean
*/
public function isDone()
{
$workStates = array('import-users', 'import-groups', 'fetch-messages', 'save-messages');
return ($this->state() == 'done');
}
/**
* Check if we have work to do in iterate().
*
* @return boolean
*/
public function hasWork()
{
$workStates = array('import-users', 'import-groups', 'fetch-messages', 'save-messages');
return in_array($this->state(), $workStates);
}
/**
* Blow away any current state!
*/
public function reset()
{
$this->state->delete();
$this->state = self::initState();
}
/**
* Start the authentication process! If all goes well, we'll get back a URL.
* Have the user visit that URL, log in on Yammer and verify the importer's
* permissions. They'll get back a verification code, which needs to be passed
* on to saveAuthToken().
*
* @return string URL
*/
public function requestAuth()
{
if ($this->state->state != 'init') {
throw new ServerException("Cannot request Yammer auth; already there!");
}
$data = $this->client->requestToken();
$old = clone($this->state);
$this->state->state = 'requesting-auth';
$this->state->oauth_token = $data['oauth_token'];
$this->state->oauth_secret = $data['oauth_token_secret'];
$this->state->modified = common_sql_now();
$this->state->update($old);
return $this->getAuthUrl();
}
/**
* When already in requesting-auth state, grab the URL to send the user to
* to complete OAuth setup.
*
* @return string URL
*/
function getAuthUrl()
{
if ($this->state() == 'requesting-auth') {
return $this->client->authorizeUrl($this->state->oauth_token);
} else {
throw new ServerException('Cannot get Yammer auth URL when not in requesting-auth state!');
}
}
/**
* Now that the user's given us this verification code from Yammer, we can
* request a final OAuth token/secret pair which we can use to access the
* API.
*
* After success here, we'll be ready to move on and run through iterate()
* until the import is complete.
*
* @param string $verifier
* @return boolean success
*/
public function saveAuthToken($verifier)
{
if ($this->state->state != 'requesting-auth') {
throw new ServerException("Cannot save auth token in Yammer import state {$this->state->state}");
}
$data = $this->client->accessToken($verifier);
$old = clone($this->state);
$this->state->state = 'import-users';
$this->state->oauth_token = $data['oauth_token'];
$this->state->oauth_secret = $data['oauth_token_secret'];
$this->state->modified = common_sql_now();
$this->state->update($old);
return true;
}
/**
* Once authentication is complete, we need to call iterate() a bunch of times
* until state() returns 'done'.
*
* @return boolean success
*/
public function iterate()
{
switch($this->state())
{
case 'init':
case 'requesting-auth':
// Neither of these should reach our background state!
common_log(LOG_ERR, "Non-background YammerImport state '$state->state' during import run!");
return false;
case 'import-users':
return $this->iterateUsers();
case 'import-groups':
return $this->iterateGroups();
case 'fetch-messages':
return $this->iterateFetchMessages();
case 'save-messages':
return $this->iterateSaveMessages();
default:
common_log(LOG_ERR, "Invalid YammerImport state '$state->state' during import run!");
return false;
}
}
/**
* Trundle through one 'page' return of up to 50 user accounts retrieved
* from the Yammer API, importing them as we go.
*
* When we run out of users, move on to groups.
*
* @return boolean success
*/
private function iterateUsers()
{
$old = clone($this->state);
$page = intval($this->state->users_page) + 1;
$data = $this->client->users(array('page' => $page));
if (count($data) == 0) {
common_log(LOG_INFO, "Finished importing Yammer users; moving on to groups.");
$this->state->state = 'import-groups';
} else {
foreach ($data as $item) {
$user = $this->importer->importUser($item);
common_log(LOG_INFO, "Imported Yammer user " . $item['id'] . " as $user->nickname ($user->id)");
}
$this->state->users_page = $page;
}
$this->state->modified = common_sql_now();
$this->state->update($old);
return true;
}
/**
* Trundle through one 'page' return of up to 20 user groups retrieved
* from the Yammer API, importing them as we go.
*
* When we run out of groups, move on to messages.
*
* @return boolean success
*/
private function iterateGroups()
{
$old = clone($this->state);
$page = intval($this->state->groups_page) + 1;
$data = $this->client->groups(array('page' => $page));
if (count($data) == 0) {
common_log(LOG_INFO, "Finished importing Yammer groups; moving on to messages.");
$this->state->state = 'fetch-messages';
} else {
foreach ($data as $item) {
$group = $this->importer->importGroup($item);
common_log(LOG_INFO, "Imported Yammer group " . $item['id'] . " as $group->nickname ($group->id)");
}
$this->state->groups_page = $page;
}
$this->state->modified = common_sql_now();
$this->state->update($old);
return true;
}
/**
* Trundle through one 'page' return of up to 20 public messages retrieved
* from the Yammer API, saving them to our stub table for future import in
* correct chronological order.
*
* When we run out of messages to fetch, move on to saving the messages.
*
* @return boolean success
*/
private function iterateFetchMessages()
{
$old = clone($this->state);
$oldest = intval($this->state->messages_oldest);
if ($oldest) {
$params = array('older_than' => $oldest);
} else {
$params = array();
}
$data = $this->client->messages($params);
$messages = $data['messages'];
if (count($messages) == 0) {
common_log(LOG_INFO, "Finished fetching Yammer messages; moving on to save messages.");
$this->state->state = 'save-messages';
} else {
foreach ($messages as $item) {
Yammer_notice_stub::record($item['id'], $item);
$oldest = $item['id'];
}
$this->state->messages_oldest = $oldest;
}
$this->state->modified = common_sql_now();
$this->state->update($old);
return true;
}
private function iterateSaveMessages()
{
$old = clone($this->state);
$newest = intval($this->state->messages_newest);
$stub = new Yammer_notice_stub();
if ($newest) {
$stub->whereAdd('id > ' . $newest);
}
$stub->limit(20);
$stub->orderBy('id');
$stub->find();
if ($stub->N == 0) {
common_log(LOG_INFO, "Finished saving Yammer messages; import complete!");
$this->state->state = 'done';
} else {
while ($stub->fetch()) {
$item = $stub->getData();
$notice = $this->importer->importNotice($item);
common_log(LOG_INFO, "Imported Yammer notice " . $item['id'] . " as $notice->id");
$newest = $item['id'];
}
$this->state->messages_newest = $newest;
}
$this->state->modified = common_sql_now();
$this->state->update($old);
return true;
}
/**
* Count the number of Yammer users we've mapped into our system!
*
* @return int
*/
public function countUsers()
{
$map = new Yammer_user();
return $map->count();
}
/**
* Count the number of Yammer groups we've mapped into our system!
*
* @return int
*/
public function countGroups()
{
$map = new Yammer_group();
return $map->count();
}
/**
* Count the number of Yammer notices we've pulled down for pending import...
*
* @return int
*/
public function countFetchedNotices()
{
$map = new Yammer_notice_stub();
return $map->count();
}
/**
* Count the number of Yammer notices we've mapped into our system!
*
* @return int
*/
public function countSavedNotices()
{
$map = new Yammer_notice();
return $map->count();
}
/**
* Start running import work in the background queues...
*/
public function startBackgroundImport()
{
$qm = QueueManager::get();
$qm->enqueue('YammerImport', 'yammer');
}
}

View File

@ -0,0 +1,65 @@
<?php
if (php_sapi_name() != 'cli') {
die('no');
}
define('INSTALLDIR', dirname(dirname(dirname(dirname(__FILE__)))));
$longoptions = array('verify=', 'reset');
require INSTALLDIR . "/scripts/commandline.inc";
echo "Checking current state...\n";
$runner = YammerRunner::init();
if (have_option('reset')) {
echo "Resetting Yammer import state...\n";
$runner->reset();
echo "done.\n";
exit(0);
}
switch ($runner->state())
{
case 'init':
echo "Requesting authentication to Yammer API...\n";
$url = $runner->requestAuth();
echo "Log in to Yammer at the following URL and confirm permissions:\n";
echo "\n";
echo " $url\n";
echo "\n";
echo "Pass the resulting code back by running:\n";
echo "\n";
echo " php yammer-import.php --verify=####\n";
echo "\n";
break;
case 'requesting-auth':
if (!have_option('verify')) {
echo "Awaiting authentication...\n";
echo "\n";
echo "If you need to start over, reset the state:\n";
echo "\n";
echo " php yammer-import.php --reset\n";
echo "\n";
exit(1);
}
echo "Saving final authentication token for Yammer API...\n";
$runner->saveAuthToken(get_option_value('verify'));
// Fall through...
default:
while ($runner->hasWork()) {
echo "... {$runner->state()}\n";
if (!$runner->iterate()) {
echo "FAIL??!?!?!\n";
}
}
if ($runner->isDone()) {
echo "... done.\n";
} else {
echo "... no more import work scheduled.\n";
}
break;
}