Commit Graph

6117 Commits

Author SHA1 Message Date
Diogo Cordeiro 2a863ba746 [MODULES] Make default loading attributes available in global config 2021-07-16 19:44:38 +01:00
Alexei Sorokin caac2cea44 [SCHEMA] Better DBMS information fetching
On PostgreSQL:
  - Parse defaults for strings and booleans properly.
  - Parse the "serial" definition type properly.
  - Get information on the "enum" definition type.
  - Re-work getting information about keys/indices.

On MariaDB:
  - Get information about lengths in indices.
  - Get foreign key information separately from the rest as they can have
    colliding names.
2021-07-16 19:44:38 +01:00
Alexei Sorokin 573cd7d450 [SCHEMA] Fix a few mistakes 2021-07-16 19:44:38 +01:00
Alexei Sorokin bee3dea9c2 [DATABASE] Add explicit indices for all foreign keys
This adds a requirement for all definitions that have foreign keys to also
require indices for all source (local) attributes mentioned in foreign keys.

MariaDB/MySQL creates indices for source attributes automatically, so this
serves as a way to get rid of those automatic indices and create clean explicit
ones instead.

In PostgreSQL, most of the time, indices on the source are necessary to
decrease performance penalty of foreign keys (like in MariaDB), but they aren't
created automatically, so this serves to remove that difference between
PostgreSQL and MariaDB.
2021-07-16 19:44:38 +01:00
Alexei Sorokin 665e4574da [DATABASE] Fix index identifiers and clean up redundant ones 2021-07-16 19:44:38 +01:00
Alexei Sorokin b1b1d2af93 [DATABASE] Update "modified" in Managed_DataObject instead of a DBMS trigger
Instead of relying on the MariaDB's ON UPDATE CURRENT_TIMESTAMP trigger update
"modified" attributes in Managed_DataObject. Every raw query that needs
adjusting is adjusted, as they won't update "modified" automatically anymore.

The main goal behind this change is to fix "modified" updates on PostgreSQL.
2021-07-16 19:44:37 +01:00
Diogo Cordeiro c85feeaa1f [AuthCrypt] Update README and fix formatting 2021-07-16 19:44:37 +01:00
Diogo Cordeiro 360fc0262b [MODULES] Make settings great again
Plugin main class doesn't use this construct despite extending this class.
2021-07-16 19:44:37 +01:00
Alexei Sorokin 182f94cf6f [DATABASE][MariaDB] Fix index changes with foreign keys enabled 2021-07-16 19:44:37 +01:00
Alexei Sorokin 9ec1b667c0 [DATABASE][Schema_version] Change the hashing algorithm to SHA3-512 2021-07-16 19:44:37 +01:00
Alexei Sorokin 65b6a924bd [DATABASE] Make unprefixed schema.php a bit more DBMS-neutral 2021-07-16 19:44:37 +01:00
Diogo Cordeiro 2894eb9ed2 [MEDIA] Fix headers when using x-static-delivery 2021-07-16 19:44:37 +01:00
Alexei Sorokin e902a9bdfc [DATABASE] Check SQL boolean values with "IS TRUE"
This way UNKNOWN (NULL) explicitly turns to FALSE when three-valued logic is
reduced to binary.
In pgsqlschema, however, use "IS FALSE" as boolean attributes in pg_index are
non-nullable, there is no outer join and there's no clear preference for NULL
reduction.

Over-complicated constructions in TagCloud queries have been simplified, which
should not affect their performance.

Additionally, in TagCloud's lib/subscriptionspeopleselftagcloudsection.php
a typing mistake in an equi-join of "profile_tag" and "profile_list" on
"tagger" was fixed.
That regression was introduced in f446db8e2a
2021-07-16 19:44:37 +01:00
Hugo Sales 58af4799ee [NOTICE][UI] Fix bug: Option to message "Everyone at site" when using private instance 2021-07-16 19:44:37 +01:00
Alexei Sorokin 9ba9006014 [CORE] Avoid materialisation in the TaggedProfileNoticeStream query
This is analogous to c862589dcf
2021-07-16 19:44:37 +01:00
Alexei Sorokin cbb83a9752 [NOTICES] Revert "We want the profile stream to be as raw as possible!"
There are only three possible visible notice verbs: POST, SHARE and DELETE.
What including all verbs does is it makes limiting (FETCH FIRST) unreliable as
the query will fetch invisible stuff (like favourites) and count it in, but
nothing will be displayed.
NoticeStream only allows POST and SHARE, so this effectively removes tombstones
from the profile page like in 78a111b57d
2021-07-16 19:44:37 +01:00
Alexei Sorokin eb647f15a7 [NOTICES] Fix InboxNoticeStream caching
Introduce a new property in CachingNoticeStream for always checking if there
are any new elements in the stream.
It would be extremely hard to blow InboxNoticeStream, so instead the database
hit will still occur, but it is be much faster than starting fresh.

This fixes a regression introduced in 36a55d8436
2021-07-16 19:44:37 +01:00
Diogo Cordeiro 9f4c4edb02 [ActivityPub][Ostatus] Fetch avatars in accordance to the new media system 2021-07-16 19:44:37 +01:00
Alexei Sorokin e9d1c5593d [CORE] Avoid materialisation in the TagNoticeStream query
The notice.created sort forced the notice_tag by notice join plain to employ
materialisation, which can have a serious performance penalty depending on the
size of the database.
Sort by notice_tag.created instead, which should be exactly the same.
2021-07-16 19:44:37 +01:00
Alexei Sorokin d3e63bbac0 [CORE] Avoid the old "reply" relation in inboxnoticestream
All verbs for not visible notices are filtered out, so this should not
break the timeline.
Additionally, filter by profile outside of the derived relation as that shows
better performance in PostgreSQL and MariaDB both.
2021-07-16 19:44:37 +01:00
Alexei Sorokin 819ece93aa Revert "FullNoticeStream selects all verbs"
It appears this was added to display "tombstones" of deleted notices.
However, it has other side-effects and the concept of keeping them visible has
not been adopted by the wider fediverse.
2021-07-16 19:44:37 +01:00
Alexei Sorokin 26115482ef [SCHEMA] Improve timestamp storage
Avoid the use of deprecated MariaDB "zero dates" globally. If they're present
as attribute defaults somewhere, they will be replaced with NULL implicitly.
The existing "zero dates" in MariaDB storage will be left intact and this
should not present any issues.

The "timestamp" type in table definitions now corresponds to DATETIME in
MariaDB with "DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP", which
should be close enough to the original behaviour for compatibility purposes.
It is now the recommended type for "modified" attributes, because of the
update trigger on MariaDB. But there is no such trigger implemented on
PostgreSQL as of this moment.
2021-07-16 19:44:37 +01:00
Alexei Sorokin eb993df072 [DATABASE] Fix MariaDB schema verification 2021-07-16 19:44:37 +01:00
Hugo Sales ff75bb796b [MEDIA] Move AttachmentAction::sendFile to common_send_file
This fixed the wrong content type and status code returned by the Avatar action
2021-07-16 19:44:37 +01:00
Diogo Cordeiro e8a5347157 [UI] Make neo-quitter default theme 2021-07-16 19:44:37 +01:00
Diogo Cordeiro 7568789f62 [XML] Inscrease types strictness 2021-07-16 19:44:37 +01:00
Diogo Cordeiro 02055dee49 [AVATAR] Temporary ImageFile wasn't 2021-07-16 19:44:37 +01:00
Diogo Cordeiro 5439ff3ec5 [MEDIA] ImageFile fromUpload method wasn't ensuring uploaded file was an image 2021-07-16 19:44:37 +01:00
Alexei Sorokin d01f44ee99 [DATABASE] Some query improvements
Make common_sql_weight employ standard SQL functions for the timestamp
difference in seconds.
Also replace UTC_TIMESTAMP in the MariaDB-specific part with CURRENT_TIMESTAMP
as it is the only occurence and GNU social sets UTC as a default timezone.

In a delete_orphan_files.php script simplify the main query considerably.

In clean_profiles.php stop using COUNT as if it is ANY, that is unnecessary
punishment for the database. Instead implement the anti-join with a
left outer join.

In Autocomplete and Activitypub_profile use joins instead of a WHERE OR
anti-pattern for the semi-joins.

In lib/ui/galleryaction.php replace a CROSS JOIN with an INNER JOIN.

In actions/sup.php remove a redundant subquery: WHERE is applied before
grouping either way.
2021-07-16 19:44:36 +01:00
Alexei Sorokin a93c38a9af [CORE][DATABASE] Another approach to semi-join in the inboxnoticestream query
Compared to the solution with INNER JOIN this seems to have better performance
as there is no need to deduplicate the subquery result before use.
2021-07-16 19:44:36 +01:00
Alexei Sorokin 9ac632ca2d [DATABASE] Start transactions with START TRANSACTION
"BEGIN" is non-standard and unnecessary.
2021-07-16 19:44:36 +01:00
tenma bbd9044626 [TheFreeNetwork] Use config + module settings for initialization and online lookup
The Census event is now replaced with module settings for populating the
protocols array. With this we can shutdown some plugins and still make
them be checked by TFN.

The performance:high config is now added when deciding whether or not to
do online lookup after the offline lookup fails.

default:
- Add default values for the TFN protocol setting

EVENTS:
- Remove Cencus event

TheFreeNetworkModule
- Remove Census event handler, update protocols array to use module's settings
- Use performance:high config when deciding to do online lookup
2021-07-16 19:44:36 +01:00
Alexei Sorokin 9396539f58 [CORE] Re-format the inboxnoticestream query 2021-07-16 19:44:36 +01:00
Diogo Cordeiro 4caa394db6 [UI] Small improvement on attachment's visual 2021-07-16 19:44:36 +01:00
Hugo Sales 5b3105402e [SensitiveContent] Fix bug where having this plugin activated would not size thumbnails correctly 2021-07-16 19:44:36 +01:00
Hugo Sales b7ed15c865 [CORE] Make avatars be served with the same mechanism as attachments 2021-07-16 19:44:36 +01:00
tenma 1d7855a70d [INSTALL] Allow the setup of sslproxy during install time
install.php:
installer.php:
- add option field and handle logic

display.css:
- hardcode ssl's label margin-top value so all options are correctly displayed
2021-07-16 19:44:36 +01:00
tenma d369e807fb [DATABASE] Fix deprecated call to implode() 2021-07-16 19:44:36 +01:00
Diogo Cordeiro 0204ffb1fa [NOTICES] Add data-nosnippet attribute for notices by remote profiles 2021-07-16 19:44:36 +01:00
Diogo Cordeiro 684a1a4143 [FORMAT] Ran php-cs-fixer on the handlers of notice items and streams
This was to keep the following two commits readable
- actions/showstream.php
- actions/userbyid.php
- lib/modules/ActivityHandlerModule.php
- lib/modules/ActivityHandlerPlugin.php
- lib/notices/conversationnoticestream.php
- lib/notices/noticelistitem.php
- lib/notices/noticestream.php
- lib/notices/threadednoticelistitem.php
2021-07-16 19:44:36 +01:00
Alexei Sorokin 3951ccbef7 [CORE] Use monotonic time via hrtime() where applicable
The realtime clock is not reliable when calculating elapsed time.
2021-07-16 19:44:36 +01:00
Alexei Sorokin d467370efb [Embed][CORE] Validate the hexadecimal for hex2bin properly 2021-07-16 19:44:36 +01:00
Diogo Cordeiro 91869c78ac [CORE] Better queues defaults 2021-07-16 19:44:36 +01:00
Diogo Cordeiro 73af7be061 [ActivityPub] Properly handle Actor URIs by using events correctly
This should fix nulls on explorer lookups inputed by postman after generate_followers/getSubscribers, that I think were caused by calling common_profile_uri that, curiously, only handles local profiles
2021-07-16 19:44:36 +01:00
Diogo Cordeiro 165edc2609 [CORE] Add GNUSOCIAL_ENGINE_REPO_URL and increase usage of GNUSOCIAL_ENGINE_URL 2021-07-16 19:44:36 +01:00
Alexei Sorokin 7bd316b506 [API] Show a nicer no such user error on statusnet/groups/list_all 2021-07-16 19:44:36 +01:00
Diogo Cordeiro fbc6fe0818 [UI] Fix a small typo 2021-07-16 19:44:36 +01:00
Diogo Cordeiro 3b01aa31d3 [REALTIME] Reviewed both the superclass and its dist plugins 2021-07-16 19:44:36 +01:00
Alexei Sorokin eab5725698 [DATABASE] Disable 'NULL' strings evaluation as SQL NULLs
Use $object->sqlValue('NULL') (identical to DataObject_Cast'ing) instead and
fix related issues like (email|sms)settings considering these NULLs as a
false positive for the E-Mail address still being set when it's been removed.

There could also be security implications to the now-disabled approach of
considering 'NULL' strings as SQL NULLs.
2021-07-16 19:44:35 +01:00
tenma b19da881c7 [TheFreeNetwork] First code: Module class and census event 2021-07-16 19:44:35 +01:00
tenma c861eb7232 [CORE] Update common_user_uri
To be consistent between federated-protocols and maintain non-fancy URIs
2021-07-16 19:44:35 +01:00
Alexei Sorokin 01696246eb [Embed][CORE] hex2bin should always get an even amount of symbols 2021-07-16 19:44:35 +01:00
Diogo Cordeiro d69f479cba [CORE] Allow to force non-fancy URLs
Essential to allow toggling fancy urls later. In some cases it is
critical to keep the URL an unique URI.
2021-07-16 19:44:35 +01:00
Diogo Cordeiro 3dcedfa8bd [MEDIA] Simplify Attachment actions 2021-07-16 19:44:35 +01:00
Diogo Cordeiro 81711a18d9 [LIB_REFACTOR] Fix minor issues 2021-07-16 19:44:35 +01:00
Miguel Dantas e25803b537 [Embed][CORE] Fixes 'Invalid Filename' on Embed. Regex didn't get updated 2021-07-16 19:44:35 +01:00
Alexei Sorokin 3460c13535 [DAEMONS] Move resetDb to the Daemon class
As it is used inside the Daemon class now, it should always be available.
2021-07-16 19:44:35 +01:00
Alexei Sorokin ffef85414e [CORE] Fix logging of very early failures 2021-07-16 19:44:35 +01:00
Alexei Sorokin cfc93bacbb [DATABASE][MariaDB] Properly account for foreign keys 2021-07-16 19:44:35 +01:00
Diogo Cordeiro acd64ba54a [BugFix] Plugins ExtendedProfile and OverwriteThemeBackground - admin is identical to system path names. 2021-07-16 19:44:35 +01:00
Diogo Cordeiro ee405df000 [MODULES] Make disable in admin panel effective 2021-07-16 19:44:35 +01:00
Alexei Sorokin 16b5ddd230 [DATABASE] Re-introduce PostgreSQL support 2021-07-16 19:44:35 +01:00
Alexei Sorokin e807e3bf08 [DATABASE] Use "<>" as an SQL non-equality sign
"!=" is not SQL compliant.
2021-07-16 19:44:35 +01:00
Alexei Sorokin 0e0c375e65 [DATABASE] Re-introduce the enum type abstraction 2021-07-16 19:44:35 +01:00
Alexei Sorokin 3f17a0efea [DATABASE] Introduce a bool type in schema
PostgreSQL has a clear distinction between integers and booleans, so it makes
sense to draw a clear line.
2021-07-16 19:44:35 +01:00
Alexei Sorokin 9d87c37ac1 [DATABASE] Consistently use the "LIMIT $limit OFFSET $offset" syntax
The "LIMIT $offset, $limit" syntax is only supported by MySQL and MariaDB.
2021-07-16 19:44:35 +01:00
Alexei Sorokin 6095a6de64 [DATABASE] Only use single quotation marks for SQL strings
Double quotation marks are only used for identifiers in the SQL standard.
2021-07-16 19:44:35 +01:00
Alexei Sorokin d26aac77b3 [DATABASE] Always quote identifiers
The code used to operate under the assumption that MariaDB doesn't support
quoting identifiers. Not only is that not exactly true, but MariaDB has
reserved keywords that cannot be used as table or column names unquoted.
2021-07-16 19:44:35 +01:00
Alexei Sorokin 1b41a38719 [CORE] Another inboxnoticestream improvement 2021-07-16 19:44:35 +01:00
Alexei Sorokin 1b66209e53 [DAEMONS] Switch daemons to double-forking for daemonisation 2021-07-16 19:44:35 +01:00
Alexei Sorokin 72f5576795 [DAEMONS] Properly disconnect daemons from the database 2021-07-16 19:44:35 +01:00
Miguel Dantas 7cdd64f594 [PLUGINS] Added UnQueue, a new default plugin which does all actions immediately 2021-07-16 19:44:34 +01:00
Miguel Dantas 333b915740 [PLUGINS] Added DBQueue plugin 2021-07-16 19:44:34 +01:00
Miguel Dantas e3c34c4693 [PLUGINS] Added StompQueue plugin, based on the implementation in lib/queue/stompqueuemanager. Updated to new STOMP library version. Dropped liberalstomp.php and stompqueuemanager.php 2021-07-16 19:44:34 +01:00
Miguel Dantas 3fbf974dad [PLUGINS] Added RedisQueue 2021-07-16 19:44:34 +01:00
Miguel Dantas 44e96deecd [CORE][AUTOLOAD] Fix class OAuthRequest not found
Issue description as reported by aab:

2019-08-28 17:34:48 LOG_ERR: [khp.ignorelist.com:61055.f5f7f51c GET /api/statuses/show/178372.atom] ServerErrorAction: 500 Class 'OAuthRequest' not found
2019-08-28 17:39:50 LOG_ERR: [khp.ignorelist.com:65390.4483ff85 GET /api/statusnet/conversation/104672.rss] Handled serverError (500) but cannot output into desired format ('rss'): 'Class \'OAuthRequest\' not found'
2019-08-28 17:39:50 LOG_ERR: [khp.ignorelist.com:65390.4483ff85 GET /api/statusnet/conversation/104672.rss] ServerErrorAction: 500 Class 'OAuthRequest' not found
2019-08-28 17:40:49 LOG_ERR: [khp.ignorelist.com:65390.4c745f68 GET /api/statuses/show/18132.atom] Handled serverError (500) but cannot output into desired format ('atom'): 'Class \'OAuthRequest\' not found'
2019-08-28 17:40:49 LOG_ERR: [khp.ignorelist.com:65390.4c745f68 GET /api/statuses/show/18132.atom] ServerErrorAction: 500 Class 'OAuthRequest' not found
2019-08-28 17:47:41 LOG_ERR: [khp.ignorelist.com:65390.42ee9fd4 GET /api/statusnet/conversation/133023.as] Handled serverError (500) but cannot output into desired format ('as'): 'Class \'OAuthRequest\' not found'
2021-07-16 19:44:34 +01:00
Miguel Dantas 55f4ed036d [FRAMEWORK][AUTOLOAD] Fix autoloads 2021-07-16 19:44:34 +01:00
Miguel Dantas ab3e8ce21d [LIB_REFACTOR] Fix requires 2021-07-16 19:44:34 +01:00
Miguel Dantas 2ab2e22a36 [LIB_REFACTOR] Moving files into separate semantic categories 2021-07-16 19:44:34 +01:00
tenma 8f070f53ec [CORE] Fix undefined offset warnings in different files
This problem was presentend in the following issue:
https://notabug.org/diogo/gnu-social/issues/60

AcceptHeader/util:
- Perform isset before using the required array values
2021-07-16 19:44:34 +01:00
tenma 32ae48358d [RemoteFollow] Initial work in the RemoteFollow plugin
lib/default.php
- Add RemoteFollow to the list of default plugins

RemoteFollowPlugin:
- Subscribe events to add the remote-follow button

RemoteFollowInitAction:
- Handles the remote-follow form and getting the redirection url for follow completion

RemoteFollowSubAction:
- Handles the remote profile pulling and actual following
2021-07-16 19:44:34 +01:00
Miguel Dantas d9668a6d07 [URLMAPPER] Fix small bug in URLMapper: a dynamic route might not exist even if a static one does 2021-07-16 19:44:34 +01:00
Diogo Cordeiro 428878b6d8 [MODULES] Allow to delete third party plugins 2021-07-16 19:44:34 +01:00
Diogo Cordeiro 0d7c0069f2 [MODULES] Allow to upload third party plugins
Fixed some bugs
2021-07-16 19:44:34 +01:00
Diogo Cordeiro ca22279797 [MODULES] List all available plugins and allow enabling them via UI
Yet another revision of the Admin Plugin Management tool
2021-07-16 19:44:34 +01:00
Diogo Cordeiro 0c6d94fca8 [SamplePlugin] Review and update with the latest GNU social best practices 2021-07-16 19:44:34 +01:00
Diogo Cordeiro 2d7ca37036 [GroupFavorited] Fix plugin 2021-07-16 19:44:34 +01:00
Diogo Cordeiro fd1a7a5e68 [Plugins] Incorporated GNUsocialExtendedProfile as part of ExtendedProfile
Also improved a lot of the plugin and made things in a way it would make sense
2021-07-16 19:44:34 +01:00
Diogo Cordeiro c71fa9099f [GNUsocialProfileExtensions] Fixed plugin 2021-07-16 19:44:34 +01:00
Diogo Cordeiro 5a679ac561 [DEFAULT] Add 'DirectionDetector' as a default plugin 2021-07-16 19:44:33 +01:00
Diogo Cordeiro 5ca939fdd2 [DEFAULT] Add 'AccountManager' as a default plugin 2021-07-16 19:44:33 +01:00
Diogo Cordeiro 20a6f1d4f4 [CORE] Plugin API now extends a new Module API 2021-07-16 19:44:33 +01:00
Diogo Cordeiro 3821d2317e [PLUGIN API] Bug fixes and improvements 2021-07-16 19:44:33 +01:00
Diogo Cordeiro 99fe3e5a52 [CORE] Move plugin superclasses from /lib/ to /lib/modules/ 2021-07-16 19:44:33 +01:00
Diogo Cordeiro c18f26145c [CORE] Move core plugins to a new modules directory
For reference (raised by rozzin in IRC):

* http://foldoc.org/module
* http://foldoc.org/library
* http://foldoc.org/plugin

As noted by XRevan86, modules are not necessarily non-essential.
As we will keep the modules directory in GS root [therefore, near to
plugins/], it is evidenced the difference between both.

This is a simple yet fundamental structural change. It doesn't change
functionality but makes clearer the way we understand GNU social's
internals.
2021-07-16 19:44:33 +01:00
tenma 0b30d932fc [DirectMessage] Major plugin rework
This commit does the necessary rework to store private messages
as Notices and to support Federation. The plugin's README presents
some more detail about the changes and future work that is still
required to do.
2021-07-16 19:44:33 +01:00
tenma 23512bea14 [CORE][ROUTES] Update urlmapper to search dynamic routes before static ones when generating URLs.
This solves the problem of routes that differ only in having
or not $_GET params. The ones not having params (static) were
being matched first during URL generation.

The way this problem was solved was by separating the $reverse
array in both $reverse_statics and $reverse_dynamics and explicitly
traversing this last one first in the generation function. Note that
maintaining the $reverse array and unshifting dynamic routes to its
head ( and therefore to the front of the static ones ) doesn't work
since even among dynamic routes the order of arrival should be kept.
2021-07-16 19:44:33 +01:00
tenma 06fb856d24 [CORE] Add new Notice scope for private messaging
Notice:
- Add MESSAGE_SCOPE scope

lib/*.stream:
- Filter out notices with MESSAGE_SCOPE scope
2021-07-16 19:44:33 +01:00
Diogo Cordeiro 2f1ddd8280 [CORE] Add ActivityPub plugin
This is not the same as the one in https://notabug.org/diogo/gnu-social-activitypub-plugin
Differences to the first "release"
-> Doesn't use guzzle nor has any composer dependencies
-> Supports HTTP Signatures
-> Has basic l10n/i18n
-> Some minor bug fixes
2021-07-16 19:44:33 +01:00
Miguel Dantas 4f8ede8239 [CORE] Fixed small anti-pattern on cache code. Plugins had to throw EndCache events, when this should be done by the library 2021-07-16 19:44:33 +01:00