Commit Graph

6081 Commits

Author SHA1 Message Date
Alexei Sorokin
5c21816b22 [SEARCH][PostgreSQL] Make LIKE search case-insensitive 2020-08-16 19:05:47 +03:00
Alexei Sorokin
6c8cd4c9f8 [SEARCH][MariaDB] Change user-facing fulltext search syntax
Now analogous to the simple and safe PostgreSQL's websearch_to_tsquery syntax.
MariaDB's full-text boolean search queries will fail on input such as "@user"
(unquoted) which is particularly noticeable to the user.
2020-08-13 23:44:59 +03:00
Alexei Sorokin
255d395f1d [SEARCH] Exclude message scope 2020-08-13 22:59:41 +03:00
tenma
1371e3efb8 [MEDIA] Add WebP support
WebP format is now the default image format for anything that
is not an animated GIF. Image Intervention doesn't support
animated WebPs so we don't convert animated GIFs.

ImageFile:
MediaFile:
default:
- Add WebP support
2020-08-14 19:48:02 +01:00
Alexei Sorokin
8bc714a2b1 [DATABASE][MariaDB] Always use LONGBLOB for "blob"
"blob" is practically used with the expectation of unlimited length, which is
true with PostgreSQL's bytea, but not with MariaDB's BLOB, which is limited to
64KiB.
So instead use LONGBLOB, which has a maximum of 4GiB, effectively unlimited.
2020-08-10 10:55:59 +03:00
Alexei Sorokin
07b0aa8f52 [DATABASE] Fix remaining misuses of SQL's GROUP BY 2020-08-10 19:29:04 +03:00
Alexei Sorokin
8c41663175 Revert "[CORE] Avoid the old "reply" relation in inboxnoticestream"
It is needed for mentions.
2020-08-10 17:23:57 +03:00
Alexei Sorokin
47cacf5f1a [NOTICES] Restore FullNoticeStream
But only allow a select number of verbs.
This is to display optional subscription notices.
2020-08-09 21:00:14 +03:00
Alexei Sorokin
f84dbb369f [DATABASE] Enable fulltext search by default
Also rename fulltext indices to more fitting names
and move the check from classes into database/schema.php
2020-08-08 18:08:06 +03:00
Alexei Sorokin
34ec165bff [DATABASE][PostgreSQL] Add fulltext search support 2020-08-08 16:56:20 +03:00
Alexei Sorokin
46f788d1eb [UTIL] Fix up common_ensure_session()
Give priority to cookies over GET.

Make sure session ids have only expected characters
(PHP file session handler's limitation).

Replace a mostly useless log warning with a debug message.
2020-08-06 21:54:49 +03:00
Alexei Sorokin
5ea5d30075 [EXCEPTIONS] Inherit the Previous Exception parameter 2020-08-06 20:03:44 +03:00
Diogo Cordeiro
7a2bb38331 [MEDIA] fromUrl now supports using original file name 2020-08-05 17:53:31 +01:00
Alexei Sorokin
e206995268 Set HTTP status codes with http_​response_​code() 2020-08-04 14:12:17 +03:00
Diogo Cordeiro
a9c365a5eb [VersionBump] 2.0.0alpha0
Ran composer update and locale updater
2020-08-04 05:31:44 +01:00
Diogo Cordeiro
34c5be5c42 [MODULES] Make default loading attributes available in global config 2020-08-04 01:36:21 +01:00
Alexei Sorokin
a38c608420 [SCHEMA] Better DBMS information fetching
On PostgreSQL:
  - Parse defaults for strings and booleans properly.
  - Parse the "serial" definition type properly.
  - Get information on the "enum" definition type.
  - Re-work getting information about keys/indices.

On MariaDB:
  - Get information about lengths in indices.
  - Get foreign key information separately from the rest as they can have
    colliding names.
2020-08-01 19:05:48 +03:00
Alexei Sorokin
1870f38099 [SCHEMA] Fix a few mistakes 2020-07-31 18:24:12 +03:00
Alexei Sorokin
92e8c40c55 [DATABASE] Add explicit indices for all foreign keys
This adds a requirement for all definitions that have foreign keys to also
require indices for all source (local) attributes mentioned in foreign keys.

MariaDB/MySQL creates indices for source attributes automatically, so this
serves as a way to get rid of those automatic indices and create clean explicit
ones instead.

In PostgreSQL, most of the time, indices on the source are necessary to
decrease performance penalty of foreign keys (like in MariaDB), but they aren't
created automatically, so this serves to remove that difference between
PostgreSQL and MariaDB.
2020-07-31 16:36:40 +03:00
Alexei Sorokin
0bfa747382 [DATABASE] Fix index identifiers and clean up redundant ones 2020-07-31 16:12:48 +03:00
Alexei Sorokin
ec86de2bc4 [DATABASE] Update "modified" in Managed_DataObject instead of a DBMS trigger
Instead of relying on the MariaDB's ON UPDATE CURRENT_TIMESTAMP trigger update
"modified" attributes in Managed_DataObject. Every raw query that needs
adjusting is adjusted, as they won't update "modified" automatically anymore.

The main goal behind this change is to fix "modified" updates on PostgreSQL.
2020-07-27 19:10:33 +03:00
Diogo Cordeiro
0b947ce2c7 [AuthCrypt] Update README and fix formatting 2020-07-25 17:42:46 +01:00
Diogo Cordeiro
2222d6d173 [MODULES] Make settings great again
Plugin main class doesn't use this construct despite extending this class.
2020-07-25 17:47:53 +01:00
Alexei Sorokin
7e01fd9c38 [DATABASE][MariaDB] Fix index changes with foreign keys enabled 2020-07-23 19:09:41 +03:00
Alexei Sorokin
eefaf7a2b4 [DATABASE][Schema_version] Change the hashing algorithm to SHA3-512 2020-07-23 18:16:56 +03:00
Alexei Sorokin
31dcf99e61 [DATABASE] Make unprefixed schema.php a bit more DBMS-neutral 2020-07-23 18:12:01 +03:00
Diogo Cordeiro
9f72b6e2c0 [MEDIA] Fix headers when using x-static-delivery 2020-07-22 16:07:52 +01:00
Alexei Sorokin
7081720ecb [DATABASE] Check SQL boolean values with "IS TRUE"
This way UNKNOWN (NULL) explicitly turns to FALSE when three-valued logic is
reduced to binary.
In pgsqlschema, however, use "IS FALSE" as boolean attributes in pg_index are
non-nullable, there is no outer join and there's no clear preference for NULL
reduction.

Over-complicated constructions in TagCloud queries have been simplified, which
should not affect their performance.

Additionally, in TagCloud's lib/subscriptionspeopleselftagcloudsection.php
a typing mistake in an equi-join of "profile_tag" and "profile_list" on
"tagger" was fixed.
That regression was introduced in f446db8e2a
2020-07-21 18:06:39 +03:00
Hugo Sales
5e14f18c83 [NOTICE][UI] Fix bug: Option to message "Everyone at site" when using private instance 2020-07-16 22:29:59 +00:00
Alexei Sorokin
f51e5ba19d [CORE] Avoid materialisation in the TaggedProfileNoticeStream query
This is analogous to c862589dcf
2020-07-07 22:41:03 +03:00
Alexei Sorokin
89e84e9b1b [NOTICES] Revert "We want the profile stream to be as raw as possible!"
There are only three possible visible notice verbs: POST, SHARE and DELETE.
What including all verbs does is it makes limiting (FETCH FIRST) unreliable as
the query will fetch invisible stuff (like favourites) and count it in, but
nothing will be displayed.
NoticeStream only allows POST and SHARE, so this effectively removes tombstones
from the profile page like in 78a111b57d
2020-07-06 19:20:44 +03:00
Alexei Sorokin
9226cce151 [NOTICES] Fix InboxNoticeStream caching
Introduce a new property in CachingNoticeStream for always checking if there
are any new elements in the stream.
It would be extremely hard to blow InboxNoticeStream, so instead the database
hit will still occur, but it is be much faster than starting fresh.

This fixes a regression introduced in 36a55d8436
2020-07-06 18:12:50 +03:00
Diogo Cordeiro
fe4a9a6189 [ActivityPub][Ostatus] Fetch avatars in accordance to the new media system 2020-07-05 15:37:52 +01:00
Alexei Sorokin
c862589dcf [CORE] Avoid materialisation in the TagNoticeStream query
The notice.created sort forced the notice_tag by notice join plain to employ
materialisation, which can have a serious performance penalty depending on the
size of the database.
Sort by notice_tag.created instead, which should be exactly the same.
2020-07-01 18:20:44 +03:00
Alexei Sorokin
61765b0e33 [CORE] Avoid the old "reply" relation in inboxnoticestream
All verbs for not visible notices are filtered out, so this should not
break the timeline.
Additionally, filter by profile outside of the derived relation as that shows
better performance in PostgreSQL and MariaDB both.
2020-07-01 17:11:04 +03:00
Alexei Sorokin
78a111b57d Revert "FullNoticeStream selects all verbs"
It appears this was added to display "tombstones" of deleted notices.
However, it has other side-effects and the concept of keeping them visible has
not been adopted by the wider fediverse.
2020-06-30 23:03:41 +03:00
Alexei Sorokin
9a515b9234 [SCHEMA] Improve timestamp storage
Avoid the use of deprecated MariaDB "zero dates" globally. If they're present
as attribute defaults somewhere, they will be replaced with NULL implicitly.
The existing "zero dates" in MariaDB storage will be left intact and this
should not present any issues.

The "timestamp" type in table definitions now corresponds to DATETIME in
MariaDB with "DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP", which
should be close enough to the original behaviour for compatibility purposes.
It is now the recommended type for "modified" attributes, because of the
update trigger on MariaDB. But there is no such trigger implemented on
PostgreSQL as of this moment.
2020-06-29 01:41:46 +03:00
Alexei Sorokin
b924c180ae [DATABASE] Fix MariaDB schema verification 2020-06-28 20:05:11 +03:00
Hugo Sales
477c357f11 [MEDIA] Move AttachmentAction::sendFile to common_send_file
This fixed the wrong content type and status code returned by the Avatar action
2020-06-27 23:23:10 +01:00
Diogo Cordeiro
953e243639 [UI] Make neo-quitter default theme 2020-06-21 00:34:55 +01:00
Diogo Cordeiro
338d53c982 [XML] Inscrease types strictness 2020-06-21 00:34:54 +01:00
Diogo Cordeiro
37ebcc509b [AVATAR] Temporary ImageFile wasn't 2020-06-21 00:34:27 +01:00
Diogo Cordeiro
2cc2b5b856 [MEDIA] ImageFile fromUpload method wasn't ensuring uploaded file was an image 2020-06-20 14:47:33 +01:00
Alexei Sorokin
194976135f [DATABASE] Some query improvements
Make common_sql_weight employ standard SQL functions for the timestamp
difference in seconds.
Also replace UTC_TIMESTAMP in the MariaDB-specific part with CURRENT_TIMESTAMP
as it is the only occurence and GNU social sets UTC as a default timezone.

In a delete_orphan_files.php script simplify the main query considerably.

In clean_profiles.php stop using COUNT as if it is ANY, that is unnecessary
punishment for the database. Instead implement the anti-join with a
left outer join.

In Autocomplete and Activitypub_profile use joins instead of a WHERE OR
anti-pattern for the semi-joins.

In lib/ui/galleryaction.php replace a CROSS JOIN with an INNER JOIN.

In actions/sup.php remove a redundant subquery: WHERE is applied before
grouping either way.
2020-06-10 16:52:00 +03:00
Alexei Sorokin
ef17f3ea7d [CORE][DATABASE] Another approach to semi-join in the inboxnoticestream query
Compared to the solution with INNER JOIN this seems to have better performance
as there is no need to deduplicate the subquery result before use.
2020-06-10 14:21:47 +03:00
Alexei Sorokin
6c035d01d4 [DATABASE] Start transactions with START TRANSACTION
"BEGIN" is non-standard and unnecessary.
2020-06-08 12:25:01 +03:00
tenma
01dcaefcfb [TheFreeNetwork] Use config + module settings for initialization and online lookup
The Census event is now replaced with module settings for populating the
protocols array. With this we can shutdown some plugins and still make
them be checked by TFN.

The performance:high config is now added when deciding whether or not to
do online lookup after the offline lookup fails.

default:
- Add default values for the TFN protocol setting

EVENTS:
- Remove Cencus event

TheFreeNetworkModule
- Remove Census event handler, update protocols array to use module's settings
- Use performance:high config when deciding to do online lookup
2020-05-09 11:50:26 +01:00
Alexei Sorokin
42aa255152 [CORE] Re-format the inboxnoticestream query 2020-04-08 15:25:30 +03:00
Diogo Cordeiro
e58188d136 [UI] Small improvement on attachment's visual 2020-04-07 13:28:34 +01:00
Hugo Sales
13e8445083 [SensitiveContent] Fix bug where having this plugin activated would not size thumbnails correctly 2020-04-07 02:59:24 +01:00