[DOCUMENTATION] Three books instead of only one

The User one is intended to illustrate the various common use cases, possibilities regarding customization and introduce the existing functionalities.

The Administrator one explains the step by step of how to install and maintain a GNU social instance, be it as node of The Free Network or as an intranet social network in a company setting.

The Developer one starts by introducing the Modules system and architecture, then the plugin development process and finally the exciting internals of GNU social for those looking forward to make the most advanced contributions.

docs/administrator/book.toml

@@ -0,0 +1,6 @@
+[book]
+authors = ["Diogo Peralta Cordeiro", "Hugo Sales"]
+language = "en"
+multilingual = false
+src = "src"
+title = "GNU social for System Administrators"

docs/developer/book.toml

@@ -0,0 +1,6 @@
+[book]
+authors = ["Diogo Peralta Cordeiro", "Hugo Sales"]
+language = "en"
+multilingual = false
+src = "src"
+title = "GNU social for Developers"

docs/developer/src/SUMMARY.md

@@ -0,0 +1,14 @@
+# Summary
+
+- [Architecture and Paradigms](./architecture.md)
+- [Plugins](./plugins.md)
+    - [Event Handlers](./plugins/no_docker_shell.md)
+    - [Installation](./plugins/docker_web.md)
+    - [Configuration](./plugins/configuration.md)
+    - [Initialization and Clean Up](./plugins/lifetime.md)
+    - [Database](./plugins/database.md)
+    - [Routes and Controllers](./plugins/routes_and_controllers.md)
+    - [Events](./plugins/events.md)
+- [Sample Plugins](./sample_plugins.md)
+    - [Injecting Javascript](plugins/sample_plugins/Awesomeness.md)
+    - [Creating a block on the sidebar](plugins/sample_plugins/Sample.md)

docs/developer/src/backups.md

@@ -0,0 +1,6 @@
+# Backups
+
+There is no built-in system for doing backups in GNU social. You can make
+backups of a working StatusNet system by backing up the database and
+the Web directory. To backup the database use mysqldump <https://mariadb.com/kb/en/mariadb/mysqldump/>
+and to backup the Web directory, try tar.

docs/developer/src/i18n.md

@@ -0,0 +1,14 @@
+### Internationalization and localization
+
+For info on helping with translations, see the platform currently in use
+for translations: https://www.transifex.com/projects/p/gnu-social/
+
+Translations use the gettext system <http://www.gnu.org/software/gettext/>.
+If you for some reason do not wish to sign up to the Transifex service,
+you can review the files in the "locale/" sub-directory of GNU social.
+Each plugin also has its own translation files.
+
+To get your own site to use all the translated languages, and you are
+tracking the git repo, you will need to install at least 'gettext' on
+your system and then run:
+    $ make translations

docs/developer/src/install/bin-configure.md

@@ -0,0 +1,6 @@
+TODO more detail
+
+Run the `bin/configure` script and enter the information as asked.
+
+This will generate all the required `.env` files and (optionally) a
+`docker-compose.yaml` file.

docs/developer/src/install/dns.md

@@ -0,0 +1,12 @@
+### Configuring DNS
+
+In order for your GNU social node to be accessible with your chosen
+hostname, you can create an `A` or `AAAA` DNS record, with your
+server's fixed IP v4 or v6 respectively in your DNS provider
+(normally, your domain registrar); the `A` record doesn't need to be
+at the root of your domain, meaning it's name can be a subdomain. For
+dynamic IPs, create a `CNAME` record pointing to the hostname you
+created with your chosen Dynamic DNS host. A `CNAME` cannot normally be created
+for a domain root, so you must use a subdomain. Note that some DNS
+providers provide 'CNAME flattening', in which case you can use your
+root domain.

docs/developer/src/install/docker_shell.md

@@ -0,0 +1,63 @@
+# Docker Installation
+
+## Installation with Docker
+
+This installation method requires
+[Docker](https://docs.docker.com/engine/install/) and [Docker
+Compose](https://docs.docker.com/compose/install/). Use
+`bin/configure` and pick `docker`, which enables all needed services
+as containers, or `mixed` which lets you pick which services you'd
+like to create containers for. This way you can use services in the
+host machine, which may be useful if your host already has a
+webserver, for instance.
+
+If you elect to not use some service containers, check [Instal without
+Docker with shell access](./install/no_docker_shell.md) for details on
+the configuration of each service.
+
+Please remember that for the installation `configure` script to use docker,
+it is necessary that the executing user is in the docker group.
+
+## Prerequisites
+
+In order to host your GNU social instance, you'll need a domain:
+ - DNS domain
+ - `docker`
+ - `docker-compose` 
+
+If you don't have a fixed public IP, for local hosting or development,
+or if you're behind a NAT, use a dynamic DNS solutions. Search for
+`GnuDIP host` or `dynamic dns`. To use GnuDIP, [clone](https://notabug.org/someonewithpc/gnudip.git), then inspect and run
+the `./install.sh` script. This allows you to have a domain that
+dynamically points to your IP address.
+
+If you want to install locally for development or experimenting purposes,
+you can use `localhost` as the `root domain` while configuring the installation.
+If you then specify a subdomain, don't forget to add it in the `/etc/hosts` file.
+
+{{#include dns.md}}
+
+{{#include tls.md}}
+
+{{#include no_tls.md}}
+
+## Configuration
+
+{{#include bin-configure.md}}
+
+## Permissions
+
+The PHP docker container needs the GNU social folder to be owned by
+the group 82 (www-data).
+
+## Running
+
+If you elected to use all or some containers, run `docker-compose up`
+from the root of the project (the folder where the `.git` folder is).
+In this form, the application can be stopped by pressing `C-c` (`^C`,
+`CTRL + C`); pressing it again will force the containers to stop
+immediately. However, this form will show you all logs, but in most
+cases, you won't want to see those all the time. For that, run
+`docker-compose up -d` from the same directory; The application can
+then be stopped with `docker-compose down`.
+

docs/developer/src/install/no_docker_shell.md

@@ -0,0 +1,197 @@
+# No Docker and shell installation
+
+## Prerequisites
+
+The following software packages are *required* for this software to
+run correctly.
+
+ - PHP 8.0+
+ - Postgres 10+/MariaDB 10.2+
+ - Web server
+ - Mail server
+
+Apache, lighttpd and nginx will all work. CGI mode is recommended and
+also some variant of 'suexec' (or a properly setup php-fpm pool)
+NOTE: mod_rewrite or its equivalent is extremely useful.
+
+The mail server is used for sending notifications and password resets,
+among other things.
+
+### PHP modules
+
+Your PHP installation must include the following PHP extensions for a
+functional setup of GNU social:
+
+ - bcmath     Arbitrary Precision Mathematics
+ - ctype      Locale support
+ - curl       Fetching files by HTTP.
+ - exif       Exchangeable image information.
+ - gd         Image manipulation (scaling).
+ - gmp        For Salmon signatures (part of OStatus)
+ - iconv      Locale support
+ - intl       Internationalization support (transliteration et al).
+ - json       For WebFinger lookups and more.
+ - mbstring   String manipulation
+ - mysql      The native driver for MariaDB connections.
+ - opcache    Improved PHP performance by precompilation
+ - openssl    (compiled in for Debian, enabled manually in Arch Linux)
+ - pcre       Perl Compatible Regular Expression
+ - readline   For interactive scripts
+ - Session    User sessions
+ - SimpleXML  XML parser
+ - Tokenizer  Reflection and annotations
+
+NOTE: Some distros require manual enabling in the relevant php.ini for
+some modules, even if they're included in the main PHP package.
+
+#### Better performance
+
+For some functionality, you will also need the following extensions:
+
+ - opcache       Improves performance a _lot_. Included in PHP, must be
+                 enabled manually in php.ini for most distributions. Find
+                 and set at least:  opcache.enable=1
+ - mailparse     Efficient parsing of email requires this extension.
+                 Submission by email or SMS-over-email uses this.
+ - sphinx        A client for the sphinx server, an alternative to MySQL
+                 or Postgresql fulltext search. You will also need a
+                 Sphinx server to serve the search queries.
+ - gettext       For multiple languages. Default on many PHP installs;
+                 will be emulated if not present.
+ - exif          For thumbnails to be properly oriented.
+
+You may also experience better performance from your site if you configure
+a PHP cache/accelerator. Most distributions come with "opcache" support.
+Enable it in your php.ini where it is documented together with its settings.
+
+{{#include dns.md}}
+
+{{#include tls.md}}
+
+{{#include no_tls.md}}
+
+### Getting it up and running
+
+Installing the basic GNU Social web component is relatively easy,
+especially if you've previously installed PHP packages.
+
+ 1. Download and unpack the release tarball or clone the `git` repository on
+    your Web server. Usually a command like this will work:
+
+    ```
+    tar zxf gnusocial-*.tar.gz
+    ```
+
+   ...which will make a `gnusocial-x.y.z` directory in your current directory.
+   (If you don't have shell access on your Web server, you may have to unpack
+   the tarball on your local computer and FTP the files to the server. Checkout
+   [Instal without Docker with only web access](./install/no_docker_web.md))
+
+ 2. Move the tarball to a directory of your choosing in your Web root
+    directory. Usually something like this will work:
+
+    ```
+    mv gnusocial-x.y.z /var/www/gnusocial
+    ```
+
+    This will often make your GNU social instance available in the gnusocial
+    path of your server, like "http://example.net/gnusocial". "social" or
+    "blog" might also be good path names. If you know how to configure
+    virtual hosts on your web server, you can try setting up
+    "http://social.example.net/" or the like.
+ 
+    You need "rewrite" support on your webserver. This is used for "Fancy URL"
+    support, which you can read more about further down in this
+    document.
+ 
+ 3. Make your target directory writeable by the Web server, please note however
+    that 'a+w' will give _all_ users write access and securing the webserver is
+    not within the scope of this document, but reading more on this subject is
+    recommended.
+
+    ```
+    chmod a+w /var/www/gnusocial/
+    ```
+ 
+    On some systems, this will work as a more secure alternative:
+
+    ```
+    chgrp www-data /var/www/gnusocial/
+    chmod g+w /var/www/gnusocial/
+    ```
+ 
+    If your Web server runs as another user besides "www-data", try
+    that user's default group instead. As a last resort, you can create
+    a new group like "gnusocial" and add the Web server's user to the group.
+ 
+ 4. Create a database to hold your site data. Something like this
+    should work (you will be prompted for your database password):
+
+    ```
+    mysqladmin -u "root" -p create social
+    ```
+ 
+    Note that GNU social should have its own database; you should not share
+    the database with another program. You can name it whatever you want,
+    though.
+  
+    (If you don't have shell access to your server, you may need to use
+    a tool like phpMyAdmin to create a database. Check your hosting
+    service's documentation for how to create a new database.)
+  
+ 5. Create a new database account that GNU social will use to access the
+    database. If you have shell access, this will probably work from the
+    MariaDB/PostgreSQL shell:
+  
+        GRANT ALL on social.*
+        TO 'social'@'localhost'
+        IDENTIFIED BY 'agoodpassword';
+  
+    You should change the user identifier 'social' and 'agoodpassword'
+    to your preferred new database username and password. You may want to
+    test logging in to MariaDB/PostgreSQL as this new user.
+  
+ 6. Run `bin/configure`
+
+{{#include bin-configure.md}}
+  
+ 7. You should now be able to navigate to your social site's main directory
+    and see the "Public Timeline", which will probably be empty. You can
+    now register new user, post some notices, edit your profile, etc.
+
+### Fancy URLs
+
+By default, GNU social will use URLs that include the main PHP program's
+name in them. For example, a user's home profile might be found at either
+of these URLS depending on the webserver's configuration and capabilities:
+
+    https://social.example.net/index.php/fred
+    https://social.example.net/index.php?p=fred
+
+It's possible to configure the software to use fancy URLs so it looks like
+this instead:
+
+    https://social.example.net/fred
+
+These "fancy URLs" are more readable and memorable for users. To use
+fancy URLs, you must either have Apache 2.x with .htaccess enabled and
+mod_rewrite enabled, -OR- know how to configure "url redirection" in
+your server (like lighttpd or nginx).
+
+TODO Add webserver sample configs
+
+1. See the instructions for each respective webserver software
+
+ - For Apache, inspect the `docs/webserver/htaccess.sample` file and save it as
+   `.htaccess` after making any necessary modifications. Our sample
+   file is well commented.
+ - For lighttpd, inspect the `docs/webserver/lighttpd.conf.example` file and apply the
+   appropriate changes in your virtualhost configuration for lighttpd.
+ - For nginx, inspect the `docs/webserver/nginx.conf.sample` file and apply the appropriate
+   changes.
+ - For other webservers, we gladly accept contributions of
+   server configuration examples.
+
+2. Ensure your webserver is properly configured and has its settings
+applied (remember to reload/restart it)
+

docs/developer/src/install/no_tls.md

@@ -0,0 +1,7 @@
+## Without TLS/SSL
+
+This is not recommended unless you know what you're doing. One
+exception is if you want your node to be used with the Tor network.
+
+Pick 'mixed' and uncheck the `certbot` service
+to disable it, or `external`, if not using docker.

docs/developer/src/install/tls.md

@@ -0,0 +1,33 @@
+## Configuring TLS/SSL
+
+You should configure a valid certificate and use TLS/SSL in most cases,
+one exception being wanting to use the Tor network.
+
+The `bin/configure` script is capable of setting this up for you if you use a
+Docker container. Otherwise, using [certbot](https://certbot.eff.org/) and
+[Let's Encrypt](https://letsencrypt.org/) is recommended
+
+There are multiple approaches to achieve this, among which are using
+your own (non-self) signed certificate, or using a proxy service
+capable of either proxying an HTTP connection to HTTPS (not
+recommended) or an HTTPS connection to HTTPS. For this approach,
+follow the instructions of your proxy service provider, but generally
+you'll use a self signed certificate, which the configuration script
+can generate.
+
+TODO Mail server configuration (links below)
+
+GNU social can be configured to send emails for various reasons. See
+[mail server configuration](). You'll need a certificate for your web
+domain and your mail domain, which may or may not be the same (if you
+use the same hostname for both, or a certificate valid for both).
+
+TODO improve external certificate handling
+
+If you prefer to not use Let's Encrypt, or the docker container, pick
+`mixed` and uncheck the `certbot` service or pick `external`.
+
+Place your certificate in the folder
+`docker/certbot/.files/live/$HOSTNAME/`, where `$HOSTNAME` is the name
+where you want to host your node, such as `social.yourdomain`.
+Remember you also need a certificate for your mail server.

docs/developer/src/installation.md

@@ -0,0 +1,22 @@
+# Installation
+
+GNU social is intended to be easily installable in both a shared hosting environment or a private
+host with shell access, or just with PHP execution.
+
+If you need help, contact us on IRC on the `#social` room in freenode or XMPP at [xmpp:gnusocial@conference.bka.li](xmpp:gnusocial@conference.bka.li)
+
+The recommended way of installing is to use [Docker](https://www.docker.com/), as this simplifies
+configuration. GNU social is comprised of a variety of different services, such as a webserver, a
+PHP execution environment, a database, etc. You may choose to use all, some, or none of these
+services in Docker containers.
+
+Pick one of the following installation methods:
+
+ - [Instal with Docker with shell access](./install/docker_shell.md)
+ - [Instal without Docker with shell access](./install/no_docker_shell.md)
+ - [Instal with Docker with web access](./install/docker_web.md) (requires access to PHP's `system()`, which may be disabled)
+ - [Instal without Docker with only web access](./install/no_docker_web.md)
+
+Installation with Docker without shell access, such as in some shared hosting environments is
+possible by configuring social locally and copying the files over, however this is not a supported
+configuration.

docs/developer/src/plugins/sample_plugins/Awesomeness.md

@@ -0,0 +1 @@
+# Awesomeness

docs/developer/src/private.md

@@ -0,0 +1,30 @@
+### Private
+
+A GNU social node can be configured as "private", which means it will not
+federate with other nodes in the network. It is not a recommended method
+of using GNU social and we cannot at the current state of development
+guarantee that there are no leaks (what a public network sees as features,
+private sites will likely see as bugs).
+
+Private nodes are however an easy way to easily setup collaboration and
+image sharing within a workgroup or a smaller community where federation
+is not a desired feature. Also, it is possible to change this setting and
+instantly gain full federation features.
+
+Access to file attachments can also be restricted to logged-in users only:
+
+1. Add a directory outside the web root where your file uploads will be
+   stored. Use this command as an initial guideline to create it:
+
+       mkdir /var/www/gnusocial-files
+
+2. Make the file uploads directory writeable by the web server. An
+   insecure way to do this is (to do it properly, read up on UNIX file
+   permissions and configure your webserver accordingly):
+
+       chmod a+x /var/www/gnusocial-files
+
+3. Tell GNU social to use this directory for file uploads. Add a line
+   like this to your config.php:
+
+       $config['attachments']['dir'] = '/var/www/gnusocial-files';

docs/developer/src/queue.md

@@ -0,0 +1,102 @@
+## Queues and daemons
+
+Some activities that GNU social needs to do, like broadcasting with OStatus or
+ActivityPub, SMS, XMPP messages and TwitterBridge operations, can be 'queued'
+and done by off-line bots instead.
+
+Run the queue handler with:
+
+```sh
+php bin/console messenger:consume async --limit=10 --memory-limit=128M --time-limit=3600
+```
+
+GNU social uses Symfony, therefore the [documentation on
+queues](https://symfony.com/doc/current/messenger.html#deploying-to-production)
+might be useful.
+
+TODO queuing
+
+#### OpportunisticQM plugin
+
+This plugin is enabled by default. It tries its best to do background
+jobs during regular HTTP requests, like API or HTML pages calls.
+
+Since queueing system is enabled by default, notices to be broadcasted
+will be stored, by default, into DB (table queue_item).
+
+Whenever it has time, OpportunisticQM will try to handle some of them.
+
+This is a good solution whether you:
+
+* have no access to command line (shared hosting)
+* do not want to deal with long-running PHP processes
+* run a low traffic GNU social instance
+
+In other case, you really should consider enabling the queuedaemon for
+performance reasons. Background daemons are necessary anyway if you wish
+to use the Instant Messaging features such as communicating via XMPP.
+
+#### Queue deamon
+
+It's recommended you use the deamon, you must be able to run
+long-running offline processes, either on your main Web server or on
+another server you control. (Your other server will still need all the
+above prerequisites, with the exception of Apache.) Installing on a
+separate server is probably a good idea for high-volume sites.
+
+1. You'll need the "CLI" (command-line interface) version of PHP
+   installed on whatever server you use.
+
+   Modern PHP versions in some operating systems have disabled functions
+   related to forking, which is required for daemons to operate. To make
+   this work, make sure that your php-cli config (/etc/php5/cli/php.ini)
+   does NOT have these functions listed under 'disable_functions':
+
+       * pcntl_fork, pcntl_wait, pcntl_wifexited, pcntl_wexitstatus,
+         pcntl_wifsignaled, pcntl_wtermsig
+
+   Other recommended settings for optimal performance are:
+       * mysqli.allow_persistent = On
+       * mysqli.reconnect = On
+
+2. If you're using a separate server for queues, install StatusNet
+   somewhere on the server. You don't need to worry about the
+   .htaccess file, but make sure that your config.php file is close
+   to, or identical to, your Web server's version.
+
+3. In your config.php files (on the server where you run the queue
+    daemon), set the following variable:
+
+       $config['queue']['daemon'] = true;
+
+   You may also want to look at the 'Queues and Daemons' section in
+   this file for more background processing options.
+
+4. On the queues server, run the command scripts/startdaemons.sh.
+
+This will run the queue handlers:
+
+* queuedaemon.php - polls for queued items for inbox processing and
+  pushing out to OStatus, SMS, XMPP, etc.
+* imdaemon.php - if an IM plugin is enabled (like XMPP)
+* other daemons, like TwitterBridge ones, that you may have enabled
+
+These daemons will automatically restart in most cases of failure
+including memory leaks (if a memory_limit is set), but may still die
+or behave oddly if they lose connections to the XMPP or queue servers.
+
+It may be a good idea to use a daemon-monitoring service, like 'monit',
+to check their status and keep them running.
+
+All the daemons write their process IDs (pids) to /var/run/ by
+default. This can be useful for starting, stopping, and monitoring the
+daemons. If you are running multiple sites on the same machine, it will
+be necessary to avoid collisions of these PID files by setting a site-
+specific directory in config.php:
+
+       $config['daemon']['piddir'] = __DIR__ . '/../run/';
+
+It is also possible to use a STOMP server instead of our kind of hacky
+home-grown DB-based queue solution. This is strongly recommended for
+best response time, especially when using XMPP.
+

docs/developer/src/sms.md

@@ -0,0 +1,47 @@
+### SMS
+
+StatusNet supports a cheap-and-dirty system for sending update messages
+to mobile phones and for receiving updates from the mobile. Instead of
+sending through the SMS network itself, which is costly and requires
+buy-in from the wireless carriers, it simply piggybacks on the email
+gateways that many carriers provide to their customers. So, SMS
+configuration is essentially email configuration.
+
+Each user sends to a made-up email address, which they keep a secret.
+Incoming email that is "From" the user's SMS email address, and "To"
+the users' secret email address on the site's domain, will be
+converted to a notice and stored in the DB.
+
+For this to work, there *must* be a domain or sub-domain for which all
+(or most) incoming email can pass through the incoming mail filter.
+
+1. Run the SQL script carrier.sql in your StatusNet database. This will
+   usually work:
+
+       mysql -u "statusnetuser" --password="statusnetpassword" statusnet < db/carrier.sql
+
+   This will populate your database with a list of wireless carriers
+   that support email SMS gateways.
+
+2. Make sure the maildaemon.php file is executable:
+
+       chmod +x scripts/maildaemon.php
+
+   Note that "daemon" is kind of a misnomer here; the script is more
+   of a filter than a daemon.
+
+2. Edit /etc/aliases on your mail server and add the following line:
+
+       *: /path/to/statusnet/scripts/maildaemon.php
+
+3. Run whatever code you need to to update your aliases database. For
+   many mail servers (Postfix, Exim, Sendmail), this should work:
+
+       newaliases
+
+   You may need to restart your mail server for the new database to
+   take effect.
+
+4. Set the following in your config.php file:
+
+       $config['mail']['domain'] = 'yourdomain.example.net';

docs/developer/src/theme.md

@@ -0,0 +1,23 @@
+# Themes
+
+As of right now, your ability change the theme is limited to CSS
+stylesheets and some image files; you can't change the HTML output,
+like adding or removing menu items, without the help of a plugin.
+
+You can choose a theme using the $config['site']['theme'] element in
+the config.php file. See below for details.
+
+You can add your own theme by making a sub-directory of the 'theme'
+subdirectory with the name of your theme. Each theme can have the
+following files:
+
+display.css: a CSS2 file for "default" styling for all browsers.
+logo.png: a logo image for the site.
+default-avatar-profile.png: a 96x96 pixel image to use as the avatar for
+users who don't upload their own.
+default-avatar-stream.png: Ditto, but 48x48. For streams of notices.
+default-avatar-mini.png: Ditto ditto, but 24x24. For subscriptions
+listing on profile pages.
+
+You may want to start by copying the files from the default theme to
+your own directory.

docs/developer/src/upgrading.md

@@ -0,0 +1,5 @@
+# Upgrading
+
+Upgrading is strongly recommended to stay up to date with security fixes
+and new features. For instructions on how to upgrade GNU social code,
+please see the UPGRADE file.

docs/user/book.toml

@@ -3,4 +3,4 @@ authors = ["Diogo Peralta Cordeiro", "Hugo Sales"]
 language = "en"
 multilingual = false
 src = "src"
-title = "GNU social"
+title = "GNU social for Users"

docs/user/src/SUMMARY.md

@@ -0,0 +1,14 @@
+# Summary
+
+- [What is GNU social](./what_is.md)
+- [Getting Started]()
+  - [Choosing an instance](./getting_started/choosing_an_instance.md)
+  - [Registering an account](./getting_started/register.md)
+  - [Publishing a note](./getting_started/publish.md)
+  - [Favouriting, Repeating and Replying](./getting_started/note_interactions.md)
+  - [Hashtags and Mentions](./getting_started/hashtags_and_mentions.md)
+  - [Groups](./getting_started/groups.md)
+  - [Referring to your account](./getting_started/nickname_and_handle.md)
+- [Exploring the free network](./the_free_network.md)
+- [More on Groups](./groups.md)
+- [More on Tags](./tags.md)

docs/user/src/backups.md

@@ -0,0 +1,6 @@
+# Backups
+
+There is no built-in system for doing backups in GNU social. You can make
+backups of a working StatusNet system by backing up the database and
+the Web directory. To backup the database use mysqldump <https://mariadb.com/kb/en/mariadb/mysqldump/>
+and to backup the Web directory, try tar.

docs/user/src/getting_started/choosing_an_instance.md

@@ -0,0 +1,9 @@
+Getting Started
+Choosing a server
+
+You have to make one decision to get on GNU social: which server to use. If you’re technically inclined and want to host your own then you can do that. For everyone else there are many choices available. Many of them are listed on this webpage.
+
+There is a small advantage in choosing either a local server or a server where you already know people. Every server has a user directory that lists all the accounts registered there. This can make it easier for others to stumble across you.
+
+Although there are many servers not all of them allow members of the public to create new accounts. The administrator of each server can turn this feature on or off.
+

docs/user/src/i18n.md

@@ -0,0 +1,14 @@
+### Internationalization and localization
+
+For info on helping with translations, see the platform currently in use
+for translations: https://www.transifex.com/projects/p/gnu-social/
+
+Translations use the gettext system <http://www.gnu.org/software/gettext/>.
+If you for some reason do not wish to sign up to the Transifex service,
+you can review the files in the "locale/" sub-directory of GNU social.
+Each plugin also has its own translation files.
+
+To get your own site to use all the translated languages, and you are
+tracking the git repo, you will need to install at least 'gettext' on
+your system and then run:
+    $ make translations

docs/user/src/install/bin-configure.md

@@ -0,0 +1,6 @@
+TODO more detail
+
+Run the `bin/configure` script and enter the information as asked.
+
+This will generate all the required `.env` files and (optionally) a
+`docker-compose.yaml` file.

docs/user/src/install/dns.md

@@ -0,0 +1,12 @@
+### Configuring DNS
+
+In order for your GNU social node to be accessible with your chosen
+hostname, you can create an `A` or `AAAA` DNS record, with your
+server's fixed IP v4 or v6 respectively in your DNS provider
+(normally, your domain registrar); the `A` record doesn't need to be
+at the root of your domain, meaning it's name can be a subdomain. For
+dynamic IPs, create a `CNAME` record pointing to the hostname you
+created with your chosen Dynamic DNS host. A `CNAME` cannot normally be created
+for a domain root, so you must use a subdomain. Note that some DNS
+providers provide 'CNAME flattening', in which case you can use your
+root domain.

docs/user/src/install/docker_shell.md

@@ -0,0 +1,63 @@
+# Docker Installation
+
+## Installation with Docker
+
+This installation method requires
+[Docker](https://docs.docker.com/engine/install/) and [Docker
+Compose](https://docs.docker.com/compose/install/). Use
+`bin/configure` and pick `docker`, which enables all needed services
+as containers, or `mixed` which lets you pick which services you'd
+like to create containers for. This way you can use services in the
+host machine, which may be useful if your host already has a
+webserver, for instance.
+
+If you elect to not use some service containers, check [Instal without
+Docker with shell access](./install/no_docker_shell.md) for details on
+the configuration of each service.
+
+Please remember that for the installation `configure` script to use docker,
+it is necessary that the executing user is in the docker group.
+
+## Prerequisites
+
+In order to host your GNU social instance, you'll need a domain:
+ - DNS domain
+ - `docker`
+ - `docker-compose` 
+
+If you don't have a fixed public IP, for local hosting or development,
+or if you're behind a NAT, use a dynamic DNS solutions. Search for
+`GnuDIP host` or `dynamic dns`. To use GnuDIP, [clone](https://notabug.org/someonewithpc/gnudip.git), then inspect and run
+the `./install.sh` script. This allows you to have a domain that
+dynamically points to your IP address.
+
+If you want to install locally for development or experimenting purposes,
+you can use `localhost` as the `root domain` while configuring the installation.
+If you then specify a subdomain, don't forget to add it in the `/etc/hosts` file.
+
+{{#include dns.md}}
+
+{{#include tls.md}}
+
+{{#include no_tls.md}}
+
+## Configuration
+
+{{#include bin-configure.md}}
+
+## Permissions
+
+The PHP docker container needs the GNU social folder to be owned by
+the group 82 (www-data).
+
+## Running
+
+If you elected to use all or some containers, run `docker-compose up`
+from the root of the project (the folder where the `.git` folder is).
+In this form, the application can be stopped by pressing `C-c` (`^C`,
+`CTRL + C`); pressing it again will force the containers to stop
+immediately. However, this form will show you all logs, but in most
+cases, you won't want to see those all the time. For that, run
+`docker-compose up -d` from the same directory; The application can
+then be stopped with `docker-compose down`.
+

docs/user/src/install/no_docker_shell.md

@@ -0,0 +1,197 @@
+# No Docker and shell installation
+
+## Prerequisites
+
+The following software packages are *required* for this software to
+run correctly.
+
+ - PHP 8.0+
+ - Postgres 10+/MariaDB 10.2+
+ - Web server
+ - Mail server
+
+Apache, lighttpd and nginx will all work. CGI mode is recommended and
+also some variant of 'suexec' (or a properly setup php-fpm pool)
+NOTE: mod_rewrite or its equivalent is extremely useful.
+
+The mail server is used for sending notifications and password resets,
+among other things.
+
+### PHP modules
+
+Your PHP installation must include the following PHP extensions for a
+functional setup of GNU social:
+
+ - bcmath     Arbitrary Precision Mathematics
+ - ctype      Locale support
+ - curl       Fetching files by HTTP.
+ - exif       Exchangeable image information.
+ - gd         Image manipulation (scaling).
+ - gmp        For Salmon signatures (part of OStatus)
+ - iconv      Locale support
+ - intl       Internationalization support (transliteration et al).
+ - json       For WebFinger lookups and more.
+ - mbstring   String manipulation
+ - mysql      The native driver for MariaDB connections.
+ - opcache    Improved PHP performance by precompilation
+ - openssl    (compiled in for Debian, enabled manually in Arch Linux)
+ - pcre       Perl Compatible Regular Expression
+ - readline   For interactive scripts
+ - Session    User sessions
+ - SimpleXML  XML parser
+ - Tokenizer  Reflection and annotations
+
+NOTE: Some distros require manual enabling in the relevant php.ini for
+some modules, even if they're included in the main PHP package.
+
+#### Better performance
+
+For some functionality, you will also need the following extensions:
+
+ - opcache       Improves performance a _lot_. Included in PHP, must be
+                 enabled manually in php.ini for most distributions. Find
+                 and set at least:  opcache.enable=1
+ - mailparse     Efficient parsing of email requires this extension.
+                 Submission by email or SMS-over-email uses this.
+ - sphinx        A client for the sphinx server, an alternative to MySQL
+                 or Postgresql fulltext search. You will also need a
+                 Sphinx server to serve the search queries.
+ - gettext       For multiple languages. Default on many PHP installs;
+                 will be emulated if not present.
+ - exif          For thumbnails to be properly oriented.
+
+You may also experience better performance from your site if you configure
+a PHP cache/accelerator. Most distributions come with "opcache" support.
+Enable it in your php.ini where it is documented together with its settings.
+
+{{#include dns.md}}
+
+{{#include tls.md}}
+
+{{#include no_tls.md}}
+
+### Getting it up and running
+
+Installing the basic GNU Social web component is relatively easy,
+especially if you've previously installed PHP packages.
+
+ 1. Download and unpack the release tarball or clone the `git` repository on
+    your Web server. Usually a command like this will work:
+
+    ```
+    tar zxf gnusocial-*.tar.gz
+    ```
+
+   ...which will make a `gnusocial-x.y.z` directory in your current directory.
+   (If you don't have shell access on your Web server, you may have to unpack
+   the tarball on your local computer and FTP the files to the server. Checkout
+   [Instal without Docker with only web access](./install/no_docker_web.md))
+
+ 2. Move the tarball to a directory of your choosing in your Web root
+    directory. Usually something like this will work:
+
+    ```
+    mv gnusocial-x.y.z /var/www/gnusocial
+    ```
+
+    This will often make your GNU social instance available in the gnusocial
+    path of your server, like "http://example.net/gnusocial". "social" or
+    "blog" might also be good path names. If you know how to configure
+    virtual hosts on your web server, you can try setting up
+    "http://social.example.net/" or the like.
+ 
+    You need "rewrite" support on your webserver. This is used for "Fancy URL"
+    support, which you can read more about further down in this
+    document.
+ 
+ 3. Make your target directory writeable by the Web server, please note however
+    that 'a+w' will give _all_ users write access and securing the webserver is
+    not within the scope of this document, but reading more on this subject is
+    recommended.
+
+    ```
+    chmod a+w /var/www/gnusocial/
+    ```
+ 
+    On some systems, this will work as a more secure alternative:
+
+    ```
+    chgrp www-data /var/www/gnusocial/
+    chmod g+w /var/www/gnusocial/
+    ```
+ 
+    If your Web server runs as another user besides "www-data", try
+    that user's default group instead. As a last resort, you can create
+    a new group like "gnusocial" and add the Web server's user to the group.
+ 
+ 4. Create a database to hold your site data. Something like this
+    should work (you will be prompted for your database password):
+
+    ```
+    mysqladmin -u "root" -p create social
+    ```
+ 
+    Note that GNU social should have its own database; you should not share
+    the database with another program. You can name it whatever you want,
+    though.
+  
+    (If you don't have shell access to your server, you may need to use
+    a tool like phpMyAdmin to create a database. Check your hosting
+    service's documentation for how to create a new database.)
+  
+ 5. Create a new database account that GNU social will use to access the
+    database. If you have shell access, this will probably work from the
+    MariaDB/PostgreSQL shell:
+  
+        GRANT ALL on social.*
+        TO 'social'@'localhost'
+        IDENTIFIED BY 'agoodpassword';
+  
+    You should change the user identifier 'social' and 'agoodpassword'
+    to your preferred new database username and password. You may want to
+    test logging in to MariaDB/PostgreSQL as this new user.
+  
+ 6. Run `bin/configure`
+
+{{#include bin-configure.md}}
+  
+ 7. You should now be able to navigate to your social site's main directory
+    and see the "Public Timeline", which will probably be empty. You can
+    now register new user, post some notices, edit your profile, etc.
+
+### Fancy URLs
+
+By default, GNU social will use URLs that include the main PHP program's
+name in them. For example, a user's home profile might be found at either
+of these URLS depending on the webserver's configuration and capabilities:
+
+    https://social.example.net/index.php/fred
+    https://social.example.net/index.php?p=fred
+
+It's possible to configure the software to use fancy URLs so it looks like
+this instead:
+
+    https://social.example.net/fred
+
+These "fancy URLs" are more readable and memorable for users. To use
+fancy URLs, you must either have Apache 2.x with .htaccess enabled and
+mod_rewrite enabled, -OR- know how to configure "url redirection" in
+your server (like lighttpd or nginx).
+
+TODO Add webserver sample configs
+
+1. See the instructions for each respective webserver software
+
+ - For Apache, inspect the `docs/webserver/htaccess.sample` file and save it as
+   `.htaccess` after making any necessary modifications. Our sample
+   file is well commented.
+ - For lighttpd, inspect the `docs/webserver/lighttpd.conf.example` file and apply the
+   appropriate changes in your virtualhost configuration for lighttpd.
+ - For nginx, inspect the `docs/webserver/nginx.conf.sample` file and apply the appropriate
+   changes.
+ - For other webservers, we gladly accept contributions of
+   server configuration examples.
+
+2. Ensure your webserver is properly configured and has its settings
+applied (remember to reload/restart it)
+

docs/user/src/install/no_tls.md

@@ -0,0 +1,7 @@
+## Without TLS/SSL
+
+This is not recommended unless you know what you're doing. One
+exception is if you want your node to be used with the Tor network.
+
+Pick 'mixed' and uncheck the `certbot` service
+to disable it, or `external`, if not using docker.

docs/user/src/install/tls.md

@@ -0,0 +1,33 @@
+## Configuring TLS/SSL
+
+You should configure a valid certificate and use TLS/SSL in most cases,
+one exception being wanting to use the Tor network.
+
+The `bin/configure` script is capable of setting this up for you if you use a
+Docker container. Otherwise, using [certbot](https://certbot.eff.org/) and
+[Let's Encrypt](https://letsencrypt.org/) is recommended
+
+There are multiple approaches to achieve this, among which are using
+your own (non-self) signed certificate, or using a proxy service
+capable of either proxying an HTTP connection to HTTPS (not
+recommended) or an HTTPS connection to HTTPS. For this approach,
+follow the instructions of your proxy service provider, but generally
+you'll use a self signed certificate, which the configuration script
+can generate.
+
+TODO Mail server configuration (links below)
+
+GNU social can be configured to send emails for various reasons. See
+[mail server configuration](). You'll need a certificate for your web
+domain and your mail domain, which may or may not be the same (if you
+use the same hostname for both, or a certificate valid for both).
+
+TODO improve external certificate handling
+
+If you prefer to not use Let's Encrypt, or the docker container, pick
+`mixed` and uncheck the `certbot` service or pick `external`.
+
+Place your certificate in the folder
+`docker/certbot/.files/live/$HOSTNAME/`, where `$HOSTNAME` is the name
+where you want to host your node, such as `social.yourdomain`.
+Remember you also need a certificate for your mail server.

docs/user/src/installation.md

@@ -0,0 +1,22 @@
+# Installation
+
+GNU social is intended to be easily installable in both a shared hosting environment or a private
+host with shell access, or just with PHP execution.
+
+If you need help, contact us on IRC on the `#social` room in freenode or XMPP at [xmpp:gnusocial@conference.bka.li](xmpp:gnusocial@conference.bka.li)
+
+The recommended way of installing is to use [Docker](https://www.docker.com/), as this simplifies
+configuration. GNU social is comprised of a variety of different services, such as a webserver, a
+PHP execution environment, a database, etc. You may choose to use all, some, or none of these
+services in Docker containers.
+
+Pick one of the following installation methods:
+
+ - [Instal with Docker with shell access](./install/docker_shell.md)
+ - [Instal without Docker with shell access](./install/no_docker_shell.md)
+ - [Instal with Docker with web access](./install/docker_web.md) (requires access to PHP's `system()`, which may be disabled)
+ - [Instal without Docker with only web access](./install/no_docker_web.md)
+
+Installation with Docker without shell access, such as in some shared hosting environments is
+possible by configuring social locally and copying the files over, however this is not a supported
+configuration.

docs/user/src/private.md

@@ -0,0 +1,30 @@
+### Private
+
+A GNU social node can be configured as "private", which means it will not
+federate with other nodes in the network. It is not a recommended method
+of using GNU social and we cannot at the current state of development
+guarantee that there are no leaks (what a public network sees as features,
+private sites will likely see as bugs).
+
+Private nodes are however an easy way to easily setup collaboration and
+image sharing within a workgroup or a smaller community where federation
+is not a desired feature. Also, it is possible to change this setting and
+instantly gain full federation features.
+
+Access to file attachments can also be restricted to logged-in users only:
+
+1. Add a directory outside the web root where your file uploads will be
+   stored. Use this command as an initial guideline to create it:
+
+       mkdir /var/www/gnusocial-files
+
+2. Make the file uploads directory writeable by the web server. An
+   insecure way to do this is (to do it properly, read up on UNIX file
+   permissions and configure your webserver accordingly):
+
+       chmod a+x /var/www/gnusocial-files
+
+3. Tell GNU social to use this directory for file uploads. Add a line
+   like this to your config.php:
+
+       $config['attachments']['dir'] = '/var/www/gnusocial-files';

docs/user/src/queue.md

@@ -0,0 +1,102 @@
+## Queues and daemons
+
+Some activities that GNU social needs to do, like broadcasting with OStatus or
+ActivityPub, SMS, XMPP messages and TwitterBridge operations, can be 'queued'
+and done by off-line bots instead.
+
+Run the queue handler with:
+
+```sh
+php bin/console messenger:consume async --limit=10 --memory-limit=128M --time-limit=3600
+```
+
+GNU social uses Symfony, therefore the [documentation on
+queues](https://symfony.com/doc/current/messenger.html#deploying-to-production)
+might be useful.
+
+TODO queuing
+
+#### OpportunisticQM plugin
+
+This plugin is enabled by default. It tries its best to do background
+jobs during regular HTTP requests, like API or HTML pages calls.
+
+Since queueing system is enabled by default, notices to be broadcasted
+will be stored, by default, into DB (table queue_item).
+
+Whenever it has time, OpportunisticQM will try to handle some of them.
+
+This is a good solution whether you:
+
+* have no access to command line (shared hosting)
+* do not want to deal with long-running PHP processes
+* run a low traffic GNU social instance
+
+In other case, you really should consider enabling the queuedaemon for
+performance reasons. Background daemons are necessary anyway if you wish
+to use the Instant Messaging features such as communicating via XMPP.
+
+#### Queue deamon
+
+It's recommended you use the deamon, you must be able to run
+long-running offline processes, either on your main Web server or on
+another server you control. (Your other server will still need all the
+above prerequisites, with the exception of Apache.) Installing on a
+separate server is probably a good idea for high-volume sites.
+
+1. You'll need the "CLI" (command-line interface) version of PHP
+   installed on whatever server you use.
+
+   Modern PHP versions in some operating systems have disabled functions
+   related to forking, which is required for daemons to operate. To make
+   this work, make sure that your php-cli config (/etc/php5/cli/php.ini)
+   does NOT have these functions listed under 'disable_functions':
+
+       * pcntl_fork, pcntl_wait, pcntl_wifexited, pcntl_wexitstatus,
+         pcntl_wifsignaled, pcntl_wtermsig
+
+   Other recommended settings for optimal performance are:
+       * mysqli.allow_persistent = On
+       * mysqli.reconnect = On
+
+2. If you're using a separate server for queues, install StatusNet
+   somewhere on the server. You don't need to worry about the
+   .htaccess file, but make sure that your config.php file is close
+   to, or identical to, your Web server's version.
+
+3. In your config.php files (on the server where you run the queue
+    daemon), set the following variable:
+
+       $config['queue']['daemon'] = true;
+
+   You may also want to look at the 'Queues and Daemons' section in
+   this file for more background processing options.
+
+4. On the queues server, run the command scripts/startdaemons.sh.
+
+This will run the queue handlers:
+
+* queuedaemon.php - polls for queued items for inbox processing and
+  pushing out to OStatus, SMS, XMPP, etc.
+* imdaemon.php - if an IM plugin is enabled (like XMPP)
+* other daemons, like TwitterBridge ones, that you may have enabled
+
+These daemons will automatically restart in most cases of failure
+including memory leaks (if a memory_limit is set), but may still die
+or behave oddly if they lose connections to the XMPP or queue servers.
+
+It may be a good idea to use a daemon-monitoring service, like 'monit',
+to check their status and keep them running.
+
+All the daemons write their process IDs (pids) to /var/run/ by
+default. This can be useful for starting, stopping, and monitoring the
+daemons. If you are running multiple sites on the same machine, it will
+be necessary to avoid collisions of these PID files by setting a site-
+specific directory in config.php:
+
+       $config['daemon']['piddir'] = __DIR__ . '/../run/';
+
+It is also possible to use a STOMP server instead of our kind of hacky
+home-grown DB-based queue solution. This is strongly recommended for
+best response time, especially when using XMPP.
+

docs/user/src/sms.md

@@ -0,0 +1,47 @@
+### SMS
+
+StatusNet supports a cheap-and-dirty system for sending update messages
+to mobile phones and for receiving updates from the mobile. Instead of
+sending through the SMS network itself, which is costly and requires
+buy-in from the wireless carriers, it simply piggybacks on the email
+gateways that many carriers provide to their customers. So, SMS
+configuration is essentially email configuration.
+
+Each user sends to a made-up email address, which they keep a secret.
+Incoming email that is "From" the user's SMS email address, and "To"
+the users' secret email address on the site's domain, will be
+converted to a notice and stored in the DB.
+
+For this to work, there *must* be a domain or sub-domain for which all
+(or most) incoming email can pass through the incoming mail filter.
+
+1. Run the SQL script carrier.sql in your StatusNet database. This will
+   usually work:
+
+       mysql -u "statusnetuser" --password="statusnetpassword" statusnet < db/carrier.sql
+
+   This will populate your database with a list of wireless carriers
+   that support email SMS gateways.
+
+2. Make sure the maildaemon.php file is executable:
+
+       chmod +x scripts/maildaemon.php
+
+   Note that "daemon" is kind of a misnomer here; the script is more
+   of a filter than a daemon.
+
+2. Edit /etc/aliases on your mail server and add the following line:
+
+       *: /path/to/statusnet/scripts/maildaemon.php
+
+3. Run whatever code you need to to update your aliases database. For
+   many mail servers (Postfix, Exim, Sendmail), this should work:
+
+       newaliases
+
+   You may need to restart your mail server for the new database to
+   take effect.
+
+4. Set the following in your config.php file:
+
+       $config['mail']['domain'] = 'yourdomain.example.net';

docs/user/src/theme.md

@@ -0,0 +1,23 @@
+# Themes
+
+As of right now, your ability change the theme is limited to CSS
+stylesheets and some image files; you can't change the HTML output,
+like adding or removing menu items, without the help of a plugin.
+
+You can choose a theme using the $config['site']['theme'] element in
+the config.php file. See below for details.
+
+You can add your own theme by making a sub-directory of the 'theme'
+subdirectory with the name of your theme. Each theme can have the
+following files:
+
+display.css: a CSS2 file for "default" styling for all browsers.
+logo.png: a logo image for the site.
+default-avatar-profile.png: a 96x96 pixel image to use as the avatar for
+users who don't upload their own.
+default-avatar-stream.png: Ditto, but 48x48. For streams of notices.
+default-avatar-mini.png: Ditto ditto, but 24x24. For subscriptions
+listing on profile pages.
+
+You may want to start by copying the files from the default theme to
+your own directory.

docs/user/src/upgrading.md

@@ -0,0 +1,5 @@
+# Upgrading
+
+Upgrading is strongly recommended to stay up to date with security fixes
+and new features. For instructions on how to upgrade GNU social code,
+please see the UPGRADE file.

docs/user/src/what_is.md

@@ -0,0 +1,13 @@
+# What is GNU social
+
+GNU social is a social network for microblogging. It enables you to publish short notices including URLs and pictures. If you’re interested in what someone has to say you can follow them. When you log in to your account you see a timeline containing all of the notices from people that you follow. You can have a conversation with another person by replying to each other’s notices.
+
+You may have seen this sort of thing before. GNU social is special for two main reasons: it’s decentralised, and it’s free software.
+
+Being decentralised means that there is no single server that controls GNU social. Instead, many servers are run by different people around the world. These servers communicate with each other to form a federation. You can create an account on any one of them. Although the servers sometimes look different, ultimately it doesn’t matter which one you choose—you’re still part of the same network as everyone else.
+
+If one server suffers an outage it’s inconvenient for the people who have an account on that particular server. The rest of the network continues to operate as normal. This makes GNU social highly resilient. Censorship is difficult as servers can be located anywhere in the world.
+
+Because GNU social is free software it’s here to stay. A corporate social network might disappear or start running advertisements when the venture capital runs out. The GNU social code is available to everybody and there are many servers where it’s free to create an account.
+
+This is a social network that does what’s best for the people who use it—not what makes the most money.