This PR was squashed before being merged into the 5.1-dev branch.
Discussion
----------
Automatically provide Messenger Doctrine schema to "diff"
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | Alternative to #36629
| License | MIT
| Doc PR | TODO - WILL be needed
This follows this conversation: https://github.com/symfony/symfony/pull/36629#issuecomment-621745821 - it automatically adds SQL to Doctrine's migration/diff system when features are added the require a database table:
The new feature works for:
### A) Messenger Doctrine transport
**FULL support**
Works perfectly: configure a doctrine transport and run `make:migration`
**Note**: There is no current way to disable this. So if you have `auto_setup` ON and you
run `make:migration` before trying Messenger, it will generate the table SQL. Adding a
flag to disable it might be very complicated, because we need to know (in DoctrineBundle, at compile time) whether or not this feature is enabled/disabled so that we can decide *not* to add `messenger_messages` to the `schema_filter`.
### B) `PdoAdapter` from Cache
**FULL support**
Works perfectly: configure a doctrine transport and run `make:migration`
### C) `PdoStore` from Lock
**PARTIAL support**
I added `PdoStore::configureSchema()` but did NOT add a listener. While `PdoStore` *does* accept a DBAL `Connection`, I don't think it's possible via the `framework.lock` config to create a `PdoStore` that is passed a `Connection`. In other words: if we added a listener that called `PdoStore::configureSchema` if the user configured a `pdo` lock, that service will *never* have a `Connection` object... so it's kind of worthless.
**NEED**: A proper way to inject a DBAL `Connection` into `PdoStore` via `framework.lock` config.
### D) `PdoSessionHandler`
**NO support**
This class doesn't accept a DBAL `Connection` object. And so, we can't reliably create a listener to add the schema because (if there are multiple connections) we wouldn't know which Connection to use.
We could compare (`===`) the `PDO` instance inside `PdoSessionHandler` to the wrapped `PDO` connection in Doctrine. That would only work if the user has configured their `PdoSessionHandler` to re-use the Doctrine PDO connection.
The `PdoSessionHandler` *already* has a `createTable()` method on it to help with manual migration. But... it's not easy to call from a migration because you would need to fetch the `PdoSessionHandler` service from the container. Adding something
**NEED**: Either:
A) A way for `PdoSessionHandler` to use a DBAL Connection
or
B) We try to hack this feature by comparing the `PDO` instances in the event subscriber
or
C) We add an easier way to access the `createTable()` method from inside a migration.
TODOs
* [X] Determine service injection XML needed for getting all PdoAdapter pools
* [ ] Finish DoctrineBundle PR: https://github.com/doctrine/DoctrineBundle/pull/1163
Commits
-------
2dd9c3c3c8 Automatically provide Messenger Doctrine schema to "diff"
This PR was squashed before being merged into the 5.1-dev branch.
Discussion
----------
[ExpressionLanguage] Added expression language syntax validator
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | #35700
| License | MIT
| Doc PR | N/A <!-- required for new features -->
Proposal implementation #35700
The current solution is a compromise between support complexity and cleanliness.
I tried different solutions to the issue. A beautiful solution was obtained only with full duplication of the parser code. That is unacceptable because parser complexity is quite high.
The main problem in this solution is that nodes instances are created which are then not used. I do not think that linter can be a bottleneck and will greatly affect performance. If this is corrected, the parser code becomes a bunch of if's.
JFI: I did not added parsing without variable names, because this breaks caching and potential location for vulnerabilities.
Commits
-------
a5cd965494 [ExpressionLanguage] Added expression language syntax validator
This PR was merged into the 5.1-dev branch.
Discussion
----------
[Security/Core] Add CustomUserMessageAccountStatusException
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| License | MIT
| Doc PR | Not really needed
When implementing the `UserCheckerInterface`, we can throw `AccountStatusException`. Similar to `CustomUserMessageAuthenticationException`, this exception allow to throw an `AccountStatusException` with a custom message.
Commits
-------
9233efbe06 Add CustomUserMessageAccountStatusException
This PR was squashed before being merged into the 5.1-dev branch.
Discussion
----------
[Messenger] Add support for RecoverableException
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | N/A
| License | MIT
| Doc PR | N/A
The messenger supports the `UnrecoverableException` preventing the messenger retry mechanism
when the Handler will never be able to process the Message.
This PR adds the opposite behavior to always retry the message.
UseCase:
- High concurency Consumers use non-blocking lock
- 503/429 errors from 3rd party API
Commits
-------
e7c31675f7 [Messenger] Add support for RecoverableException
* 5.0: (26 commits)
[Filesystem] Handle paths on different drives
[WebProfiler] Do not add src-elem CSP directives if they do not exist
[Yaml] fix parse error when unindented collections contain a comment
Execute docker dependent tests with github actions
Update exception.html.php
[3.4][Inflector] Improve testSingularize() argument name
[Inflector] Fix testPluralize() arguments names
[PhpUnitBridge] fix PHP 5.3 compat again
Skip validation when email is an empty object
fix sr_Latn translation
[Validator] fix lazy property usage.
Fix annotation
[Debug][ErrorHandler] cleanup phpunit.xml.dist files
[Translation] Fix for translation:update command updating ICU messages
[PhpUnitBridge] fix compat with PHP 5.3
bumped Symfony version to 5.0.9
updated VERSION for 5.0.8
updated CHANGELOG for 5.0.8
bumped Symfony version to 4.4.9
updated VERSION for 4.4.8
...
* 4.4: (23 commits)
[Filesystem] Handle paths on different drives
[WebProfiler] Do not add src-elem CSP directives if they do not exist
[Yaml] fix parse error when unindented collections contain a comment
Execute docker dependent tests with github actions
Update exception.html.php
[3.4][Inflector] Improve testSingularize() argument name
[Inflector] Fix testPluralize() arguments names
[PhpUnitBridge] fix PHP 5.3 compat again
Skip validation when email is an empty object
fix sr_Latn translation
[Validator] fix lazy property usage.
Fix annotation
[Debug][ErrorHandler] cleanup phpunit.xml.dist files
[Translation] Fix for translation:update command updating ICU messages
[PhpUnitBridge] fix compat with PHP 5.3
bumped Symfony version to 4.4.9
updated VERSION for 4.4.8
updated CHANGELOG for 4.4.8
provide a useful message when extension types don't match
[Cache] Fixed not supported Redis eviction policies
...
* 3.4:
[Filesystem] Handle paths on different drives
[WebProfiler] Do not add src-elem CSP directives if they do not exist
[Yaml] fix parse error when unindented collections contain a comment
[3.4][Inflector] Improve testSingularize() argument name
[PhpUnitBridge] fix PHP 5.3 compat again
Skip validation when email is an empty object
fix sr_Latn translation
[Validator] fix lazy property usage.
Fix annotation
[PhpUnitBridge] fix compat with PHP 5.3
[DX] Show the ParseException message in YAML file loaders
This PR was merged into the 4.4 branch.
Discussion
----------
[Console] Default hidden question to 1 attempt for non-tty session
| Q | A
| ------------- | ---
| Branch? | 4.4
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | Fix#36565
| License | MIT
| Doc PR |
### Problem 1
`validateAttempts()` method repeats validation forever by default, until exception extending `RuntimeException` isn't thrown. This currently happens disregarding if user is in tty session where they can actually type input, or non-tty session. This presents a problem when user code throws custom exceptions for hidden questions -> loop doesn't stop. As far as I can tell this issue is in all Symfony versions, but it was uncovered only after we stopped marking interactive flag to false automatically ourselves. Actually, all 3 problems were already existing problems, just hidden until now.
### Problem 2
Infinite loop problem is related to hidden questions, but this one isn't. If validation fails, another attempt to read & validate happens. This means user will get two prompts: 2x same question with 2 different error messages. One error message coming from validator, second error message about inability to read input (because this loop repeats until this kind of error happens, so last output will always be this error). As an example, output in practice would look like following
```
What do you want to do:
>
[ERROR] Action must not be empty.
What do you want to do:
>
Aborted.
```
So even if loop stops, output is more than expected.
### Problem 3
This is purely cosmetic issue, but currently user gets `stty: stdin isn't a terminal` printed additionally when question helper tries to ask a hidden question without having tty. I have fixed this in same fashion as was already done for [getShell() method](ee7fc5544e/src/Symfony/Component/Console/Helper/QuestionHelper.php (L500)).
### More details
Well root of the first problem is that `\Symfony\Component\Console\Helper\QuestionHelper::getHiddenResponse` is inconsistent. In some cases it does throw `MissingInputException` (which extends `RuntimeException`), in others doesn't. This is because in others, `shell_exec` is used, which won't return `false` even in non-tty sessions. Initially I attempted to fix this and make them consistent by checking for empty result + `isTty` call, but during my testing I found that at least last, `bash -c` method returns `\n` as output both when passing empty input and when passing newline as input. This means we cannot differentiate with this technique when input is really empty, or at least I can't currently tell how, maybe someone does. I had also idea to use proc_open and check if `STDERR` cotains message about stdin not being a terminal, but I realized these functions might not be available. In future we should modernize this method to use less hacky techniques. Other solutions, eg. Inquirer.js or [hoa/console](https://github.com/hoaproject/Console/blob/master/Source/Readline/Readline.php) have much more elegant solutions. Anyway, since I encountered this issue and additionally this doesn't solve Problem 2, I stopped trying to fix this on this level.
### Alternative solution
Alternative solution to problem 1 and 3 would be to fallback to default in case of hidden questions when tty is missing. But this still doesn't solve problem 2 and I can't think about solution right now which would fix problem 2 separately. We also didn't really reach consensus if reading passwords via stdin is desired. I tried this in `Inquirer.js` and this library *does read password from stdin*
Commits
-------
ee7fc5544e [Console] Default hidden question to 1 attempt for non-tty session
This PR was squashed before being merged into the 3.4 branch.
Discussion
----------
[Filesystem] Handle paths on different drives
| Q | A
| ------------- | ---
| Branch? | 3.4
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| License | MIT
`makePathRelative` strips and ignores the drive letters given Windows paths on different drives, resulting in a relative path which does not resolve to the desired target.
This PR makes `makePathRelative` notice paths on different drives, and return the full (absolute) target path in case instead.
Commits
-------
00e727ae4e [Filesystem] Handle paths on different drives
This PR was merged into the 3.4 branch.
Discussion
----------
[WebProfiler] Do not add src-elem CSP directives if they do not exist
| Q | A
| ------------- | ---
| Branch? | 3.4, 4.4, 5.0
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | Fix#36643
| License | MIT
| Doc PR | n/a
In the latest 3.4.*, 4.4.* and 5.0.* branches the `script-src-elem` and `style-src-elem` directives are added to the Content-Security-Policy header if they don't exist by copying the `default-src`. This causes browsers to ignore the `script-src` and `style-src` directives which likely contain scripts and styles the developer wanted to allow.
As mentioned in the fixed ticket, we shouldn't be adding these directives if they don't exist because the browser will automatically fallback to `script-src` and `style-src` which we have already added `unsafe-inlen` and the `nonce-*` to.
This will need to be merged into 3.4, 4.4 and 5.0, but I was unsure which branch I am meant to base it off to start with. I've put it on 4.4 but can move it to another if required.
Commits
-------
d9c47087c9 [WebProfiler] Do not add src-elem CSP directives if they do not exist
This PR was merged into the 3.4 branch.
Discussion
----------
[DX] Show the ParseException message in all YAML file loaders
| Q | A
| ------------- | ---
| Branch? | 3.4
| Bug fix? | no
| New feature? | no
| Deprecations? | no
| Tickets | -
| License | MIT
| Doc PR | -
This PR synchronizes the exception message in the Routing, Validator and Translation YAML file loaders with the DependencyInjection YAML file loader behavior. Adding the ParseException message is a big DX gain because it highlights the problem directly instead of having to scroll down 7 previous exceptions.
I'm targetting 3.4 because DX can be considered as a bug fix AFAIK.
Commits
-------
fc6cf3d3c6 [DX] Show the ParseException message in YAML file loaders
This PR was merged into the 4.4 branch.
Discussion
----------
Execute docker dependent tests with github actions
| Q | A
| ------------- | ---
| Branch? | 4.4
| Bug fix? | no
| New feature? | no
| Deprecations? | no
| Tickets | Fixes#36427
| License | MIT
| Doc PR | -
* redis, memcached, rabbitmq and vulcain dependent tests moved to the github action
* run on PHP 7.1 and 7.4 only
* use the `integration` group for all tests that depend on docker services
* do not exclude the `integration` group on Travis, but make sure tests that depend on docker services are skipped properly
[<img width="1222" alt="image" src="https://user-images.githubusercontent.com/190447/80806323-48339100-8bb2-11ea-95cd-5ce773c74ce6.png">](https://github.com/jakzal/symfony/runs/636461875?check_suite_focus=true)
Commits
-------
d710c1b654 Execute docker dependent tests with github actions
This PR was merged into the 3.4 branch.
Discussion
----------
[Validator] Skip validation when email is an empty object
| Q | A
| ------------- | ---
| Branch? | 3.4
| Bug fix? | yes
| New feature? | no <!-- please update src/**/CHANGELOG.md files -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tickets | <!-- prefix each issue number with "Fix #", if any -->
| License | MIT
| Doc PR | <!-- required for new features -->
When the value passed to the email validator is an empty object the validator is still called and will mark the value as invalid. The object should be skipped in this case, as it is also done in the `UrlValidator`
bfdbb244fe/src/Symfony/Component/Validator/Constraints/UrlValidator.php (L59-L62)
<!--
Replace this notice by a short README for your feature/bugfix. This will help people
understand your PR and can be used as a start for the documentation.
Additionally (see https://symfony.com/releases):
- Always add tests and ensure they pass.
- Never break backward compatibility (see https://symfony.com/bc).
- Bug fixes must be submitted against the lowest maintained branch where they apply
(lowest branches are regularly merged to upper ones so they get the fixes too.)
- Features and deprecations must be submitted against branch master.
-->
Commits
-------
de5d68ef2a Skip validation when email is an empty object
This PR was merged into the 5.1-dev branch.
Discussion
----------
[FrameworkBundle] Allow configuring the default base URI with a DSN
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | Fixes#35121, replaces #35580, partially reverts #35281
| License | MIT
| Doc PR | -
Instead of defining 3-4 parameters, this PR enables using a single DSN to configure the default URL context (for commands mainly):
```
framework:
router:
base_uri: 'https://my.host:8443/base-path/'
```
When using parameters directly, one can now set the same absolute URI in the `router.request_context.base_url` parameter, this will provide the same benefit.
Commits
-------
250fa7e979 [FrameworkBundle] Allow configuring the default base URI with a DSN
This PR was merged into the 5.1-dev branch.
Discussion
----------
[Security] Added LDAP support to Authenticator system
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | -
| License | MIT
| Doc PR | -
The last missing authenticator in the new system 🎉
I have no experience with LDAP at all and I didn't succeed in setting up a server locally. So I can't test whether this works, but the unit test works (and also tested in a real app, while adding a `dd()` call in the listener).
---
I want to share with you the current state of Security LDAP, how this PR implements it and a possible other solution (which I think I would prefer most). Is there anyone who can share their opinions on this? (hopefully @weaverryan and @csarrazi can share their opinion, as they have most experience on this topic)
1. **Current Solution: An LDAP authentication provider + duplicated `SecurityFactory` classes**
LDAP is done in one centralized authentication provider. This provider is configured by security factories for each core factory (e.g. `form_login` becomes `form_login_ldap`, `http_basic` becomes `http_basic_ldap`).
2. **Implementation in this PR: A listener is executed before the default `VerifyCredentialsListener`, to verify `PasswordCredentials`**
This listener must be configured for each specific authenticator wanting to use LDAP. This is a technique similar to (1). It's a bit difficult to use this for your own authenticator (you need to configure a custom listener service) and still needs the duplicated factory classes
3. **Proposal: Introduce a `LdapCredentials` class and always register a listener**
If an authentictor returns `LdapCredentials`, it'll be checked using the LDAP verification listener. This is the easiest for custom authenticators and would remove the duplicated factories, I can imagine `form_login` getting a new `ldap` sub option to configure the settings.
The main disadvantage (I think) is that we would need to make `LdapCredentials` configure all options: ldap service, dnString, searchDn, searchPassword & queryString. Especially passing around the ldap service seems a bit weird. The main questions here are: Is it weird to pass all these things in the `LdapCredentials`? And, do we really need to support having multiple LDAP configuration sets for different authenticators? Or can we e.g. add a global `security.ldap` configuration, that registers the listener for all authenticators returning `LdapCredentials`?
Commits
-------
20962e604a [Security] Added LDAP support to Authenticator system
This PR was squashed before being merged into the 5.1-dev branch.
Discussion
----------
[Translations] Throw exception if xFileLoader dependencies don't exist.
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | Fix#36658
| License | MIT
| Doc PR |
`XliffFileLoader` & `QtFileLoader` both require `XmlUtils::class` from the `Config` component. This PR throws a friendly exception is the `Config` component does not exist.
Original idea by @xabbuh was to throw the exception from the __constructor. This PR throws the exception from the `load()` method to be consistent with the `YamlFileLoader::class`. But that can easily be changed.
Commits
-------
627e476eb4 [Translations] Throw exception if xFileLoader dependencies don't exist.
This PR was squashed before being merged into the 5.1-dev branch.
Discussion
----------
[AmazonSqsMessenger] Use AsyncAws to handle SQS communication
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | /
| License | MIT
| Doc PR | /
Similar to #35992 this PR use AsyncAws to handle Sqs messages sent/receive
It move complexity of authentication/streaming outside Symfony while keeping HttpClient integration.
Commits
-------
7c4888eed1 [AmazonSqsMessenger] Use AsyncAws to handle SQS communication
This PR was merged into the 5.1-dev branch.
Discussion
----------
[Messenger] Fix messenger:failed:remove can not remove single message
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | Fix#36659
| License | MIT
| Doc PR | -
Fix this error:
```
count(): Parameter must be an array or an object that implements Countable
```
When calling `messenger:failed:remove` command from other code with single id
Commits
-------
e66cd97ec3 [Messenger] Fix messenger:failed:remove can not remove single message
This PR was merged into the 5.1-dev branch.
Discussion
----------
Add support of PHP8 static return type for withers
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | https://github.com/orgs/symfony/projects/1#card-35852557
| License | MIT
| Doc PR |
Commits
-------
04fdf05cff Add support of PHP8 static return type for withers
This PR was merged into the 5.1-dev branch.
Discussion
----------
[DI] allow loading and dumping tags with an attribute named "name"
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | -
| License | MIT
| Doc PR | -
This is a minor feature added for consistency: using PHP, we can already define tags with an attribute named `"name"`. But then, we cannot dump such definitions in YAML nor XML since we don't have a syntax to declare such tags in these formats.
I spotted this while looking at a dumped container: we already use an attribute named `"name"` on two tags: `cache.pool` and `workflow.definition`. Currently, the dumped XML is wrong because of this.
This PR enables the following new syntaxes (the current style still works as usual):
- in YAML, consistently with the new syntax for method calls:
```yaml
tags:
- cache.pool: { name: my_cache_pool }
```
- in XML:
```xml
<tag name="my_cache_pool">cache.pool</tag>
```
Commits
-------
b023e4cac3 [DI] allow loading and dumping tags with an attribute named "name"
