This PR was squashed before being merged into the 5.2-dev branch.
Discussion
----------
[RateLimiter] Fix cache storage (use namespaced pool + remove \Serializable)
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | Fix#38365, fix#38338
| License | MIT
| Doc PR | -
Commits
-------
251c202874 Use a dedicated cache.rate_limiter cache pool
5dc562a318 Use __sleep/__wakeup instead of Serializable
This PR was merged into the 5.2-dev branch.
Discussion
----------
[Messenger] Added ErrorDetailsStamp
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | yes
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | #32311
| License | MIT
| Doc PR | No doc changes are needed
#SymfonyHackday
This PR is part of the work started in #32341. That PR has a workaround for showing exceptions added to a previous retry. This PR stores error messages in a separate stamp, so they're more easily accessed.
I also added the exceptionClass as a separate string (independant of FlattenException), so that information is always available, even if the trace is not (due to FlattenException not being available).
Duplicated exceptions (compared to the last one) are not stored separately.
**Questions:**
- Should we limit the total amount of exceptions (remove the oldest when adding a new one)?
- Yes, but in a new PR
- The current implementation adds this stamp in the Worker instead of the listeners to prevent duplicate code (due to the immutability of the envelope in the event). Is this the proper way to do this?
- No, create a new listener and a way to add stamps to the envelope inside the event.
- When adding details of a `HandlerFailedException`, should we add a stamp per wrapped `Throwable`? There can be multiple errors wrapped by a single `HandlerFailedException`.
- Yes, but in a later PR
**Todo:**
- [x] only add new information if it differs from the previous exception
- [x] add deprecations
- [x] fall back to old stamp data if no new stamp is available
- [x] rebase and retarget to master
- [x] update CHANGELOG.md
- [x] check for docs PR
**BC Breaks:**
When this PR is merged, RedeliveryStamps will no longer be populated with exception data. Any implementations that use `RedeliveryStamp::getExceptionMessage()` or `RedeliveryStamp::getFlattenedException()` will receive an empty string or `null` respectively for stamps added after this update. They should rely on `ErrorDetailsStamp` instead.
**New stuff:**
- New stamp `ErrorDetailsStamp`.
- New event subscriber `AddErrorDetailsStampListener`.
- New method `AbstractWorkerMessageEvent::addStamps`.
Commits
-------
cd27b863f9 [Messenger] Added FailedMessageErrorDetailsStamp
This PR was squashed before being merged into the 5.2-dev branch.
Discussion
----------
[Messenger] dispatch event when a message is retried
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| License | MIT
Hello,
i'm working on a bundle which helps to monitor messenger queues (add some stats for queues/transports + ability to manage failed messages from the browser)
https://github.com/SymfonyCasts/messenger-monitor-bundle/
and we're missing some hooks in the messaging system:
1. a way to know when a message has been retried (fixed by dispatching a new `WorkerMessageRetriedEvent` in `SendFailedMessageForRetryListener::onMessageFailed()`)
2. a way to update the enveloppe in worker message events (fixed by adding `AbstractWorkerMessageEvent::setEnvelope()`)
if needed i can provide some precise use cases.
thanks.
Commits
-------
55bddcb721 [Messenger] dispatch event when a message is retried
This PR was submitted for the 5.1 branch but it was merged into the 5.2-dev branch instead.
Discussion
----------
Can define ChatMessage transport to null
| Q | A
| ------------- | ---
| Branch? | 5.1
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | -
| License | MIT
I'm using notiier & messenger together to send notification asynchronously.
In order to have json messages in my broker, here is the messenger config I use:
```yaml
serializer:
default_serializer: messenger.transport.symfony_serializer
symfony_serializer:
format: json
context: { }
```
I send a new `ChatMessage`:
```php
use Symfony\Component\Notifier\ChatterInterface;
use Symfony\Component\Notifier\Message\ChatMessage;
$chatter->send(new ChatMessage('Using notifier & messenger together is amazing!');
```
Message is correctly sent to my broker, in json:
```json
{
"subject": "Using notifier & messenger together is amazing!",
"recipientId": null,
"options": null,
"transport": null,
"notification": null
}
```
But when I want to consume it, as the `transport` method used by the serializer to create the `ChatMessage` doesn't accept `null` value, I get the following error:
```
[Symfony\Component\Messenger\Exception\MessageDecodingFailedException]
Could not decode message: Failed to denormalize attribute "transport" value for class "Symfony\Component\Notifier\Message\ChatMessage": Expected argument of type "string", "null" given at property path "transport".
```
This PR correct the problem, even if I'm not sure it's the best way to go...
Commits
-------
09c9bde479 Can define ChatMessage transport to null
This PR was merged into the 5.2-dev branch.
Discussion
----------
[HttpClient] provide response body to the RetryDecider
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | yes but for not-yet released 5.2 feature
| Tickets | /
| License | MIT
| Doc PR | TODO
Some servers, like AWS, does not always return standard HTTP code. The strategy needs to parse the body to take a decision.
example:
```
400
x-amzn-requestid: XXXXX
content-type: application/x-amz-json-1.1
content-length: 58
date: Thu, 24 Sep 2020 11:17:35 GMT
connection: close
{"__type":"ThrottlingException","message":"Rate exceeded"}
````
This PR update the `RetryDeciderInterface` interface to let the decider notifying the Client when it need the body to take a decision. in that case, the Client, fetch te client, and call again the decider with the full body.
usage
```php
class Decider implements RetryDeciderInterface {
public function shouldRetry(string $requestMethod, string $requestUrl, array $requestOptions, int $responseCode, array $responseHeaders, ?string $responseContent, \Throwable $throwable = null): ?bool
{
if (null !== $throwable) {
return true;
}
if (in_array($responseCode, [423, 425, 429, 500, 502, 503, 504, 507, 510])) {
return true;
}
if (
$responseCode !== 400
|| $headers['content-type'][0] ?? null !== 'application/x-amz-json-1.1'
|| (int) $headers['content-length'][0] ?? '0' > 1024
) {
return false;
}
if (null === $responseContent) {
return null; // null mean no decision taken and need to be called again with the body
}
$data = json_decode($responseContent, true);
return $data['__type'] ?? '' === 'ThrottlingException';
}
}
```
Commits
-------
321be5884d [HttpClient] provide response body to the RetryDecider
This PR was merged into the 5.2-dev branch.
Discussion
----------
[Security][RateLimiter] Added request rate limiter to prevent breadth-first attacks
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | yes
| Deprecations? | no
| Tickets | -
| License | MIT
| Doc PR | -
This allows limiting on different elements of a request. The normal `CompoundLimiter` requires the same key for all its limiters.
This request limiter is useful to e.g. prevent breadth-first attacks, by allowing to enforce a limit on both IP and IP+username. It can also be useful for applications using some sort of API request limiting (or e.g. file upload limiting).
The default login throttling limiter will allow `max_attempts` (default: 5) attempts per minute for `username + IP` and `5 * max_attempts` for `IP`. Customizing this will require creating a new service that extends `AbstractRequestRateLimiter` and implementing `getLimiters(Request $request): LimiterInterface[]`.
Commits
-------
5d03afea99 Added request rate limiters and improved login throttling
This PR was squashed before being merged into the 5.2-dev branch (closes#38351).
Discussion
----------
[Console] clone stream on multiline questions so EOT doesn't close input
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | N/A
| License | MIT
| Doc PR | N/A
This fixes a bug in the multiline question feature that was introduced in #37683.
Today, @epitre commented on https://github.com/symfony/symfony/pull/37683#issuecomment-700666826:
> If I ask a question AFTER using the multiline option in a previous question, then it seems that the ctrl+d is kind of saved, and the command gets aborted.
I'm honestly not sure how I missed this while working on #37683, since I was testing it with multiple questions, but I think it might have resulted from some of the back-and-forth and the lack of ability to effectively test the EOT character from a unit test.
The solution was to _clone_ the input stream resource and use the clone to read the multiline input and capture the EOT byte. In this way, the primary input stream is not closed by the EOT.
This is similar to @epitre's solution in https://github.com/symfony/symfony/pull/38345, but I'm using the `uri` and `mode` from `stream_get_meta_data()` to create the new stream, and if the existing stream has any data and is seekable and writable (like the streams used in the tests), I add the data to the clone and seek to the same offset.
I've ensured that this solution works on a question that is in the middle of a series of other questions, and I've tested in on *nix and Windows. I've also improved the tests for multiline questions. While I'm unable to test (with a unit test) that an EOT character effectively stops reading from STDIN while continuing to the next question and prompt, I feel confident that the tests here provide sufficient coverage.
Commits
-------
ec688a361e [Console] clone stream on multiline questions so EOT doesn't close input
This allows limiting on different elements of a request. This is usefull to
e.g. prevent breadth-first attacks, by allowing to enforce a limit on both IP
and IP+username.
This PR was merged into the 5.2-dev branch.
Discussion
----------
[Security] Improve exception when rate-limiter is not installed and throttling enabled
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes-ish
| New feature? | no <!-- please update src/**/CHANGELOG.md files -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tickets | n/a <!-- prefix each issue number with "Fix #", no need to create an issue if none exist, explain below instead -->
| License | MIT
| Doc PR | n/a
When the rate-limiter component is not installed and if one enables login throttling, the error message is not easy to understand: `Service "limiter._login_local_main": Parent definition "limiter" does not exist.`.
This fixes the error message to be easier to understand.
/cc @wouterj
Commits
-------
186ecc108b [Security] Improve exceptionwhen rate-limiter is not installed and throttling enabled
This PR was merged into the 5.2-dev branch.
Discussion
----------
[RateLimiter] Call all compound limiters on failure and added IO blocking
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | -
| License | MIT
| Doc PR | -
Two small improvements that I missed in #38257:
* Added `wait()` method, to be able to do logic like this:
```php
while (!($limit = $limiter->consume())->isAccepted()) {
$limit->wait();
}
```
* Fixed the `CompoundLimiter` to always call all limiters (even when one already blocks). This was the original behavior of the compound limiter and imho the best behavior (at least, it's always consistent and doesn't rely on the order of limiters - which is unsorted).
Commits
-------
0279f88e6c Call all compound limiters on failure and added IO blocking
This PR was merged into the 5.2-dev branch.
Discussion
----------
[Validator] Constraints as php 8 Attributes
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | #38096
| License | MIT
| Doc PR | TODO
This is my attempt to teach the validator to load constraints from PHP attributes. Like we've done it for the `Route` attribute, I've hooked into the existing `AnnotationLoader`, so we can again mix and match annotations and attributes.
### Named Arguments
An attribute declaration is basically a constructor call. This is why, in order to effectively use a constraint as attribute, we need to equip it with a constructor that works nicely with named arguments. This way, IDEs like PhpStorm can provide auto-completion and guide a developer when declaring a constraint without the need for additional plugins. Right now, PhpStorm supports neither attributes nor named arguments, but I expect those features to be implemented relatively soon.
To showcase this, I have migrated the `Range` and `Choice` constraints. The example presented in #38096 works with this PR.
```php
#[Assert\Choice(
choices: ['fiction', 'non-fiction'],
message: 'Choose a valid genre.',
)]
private $genre;
```
A nice side effect is that we get a more decent constructor call in php 8 in situations where we directly instantiate constraints, for instance when attaching constraints to a form field via the form builder.
```php
$builder->add('genre, TextType::class, [
'constraints' => [new Assert\Choice(
choices: ['fiction', 'non-fiction'],
message: 'Choose a valid genre.',
)],
]);
```
The downside is that all those constructors generate the need for some boilerplate code that was previously abstracted away by the `Constraint` class.
The alternative to named arguments would be leaving the constructors as they are. That would basically mean that when declaring a constraint we would have to emulate the array that Doctrine annotations would crate. We would lose IDE support and the declarations would be uglier, but it would work.
```php
#[Assert\Choice([
'choices' => ['fiction', 'non-fiction'],
'message' => 'Choose a valid genre.',
])]
private $genre;
```
### Nested Attributes
PHP does not support nesting attributes (yet?). This is why I could not recreate composite annotations like `All` and `Collection`. I think it's okay if they're not included in the first iteration of attribute constraints and we can work on a solution in a later PR. Possible options:
* A later PHP 8.x release might give us nested attributes.
* We could find a way to flatten those constraints so we can recreate them without nesting.
* We could come up with a convention for a structure that lets us emulate nested attributes in userland.
### Repeatable attributes
In order to attach two instances of the same attribute class to an element, we explicitly have to allow repetition via the flag `Attribute::IS_REPEATABLE`. While we could argue if it really makes sense to do this for certain constraints (like `NotNull` for instance), there are others (like `Callback`) where having two instances with different configurations could make sense. On the other hand, the validator certainly won't break if we repeat any of the constraints and all other ways to configure constraints allow repetition. This is why I decided to allow repetition for all constraints I've marked as attributes in this PR and propose to continue with that practice for all other constraints.
### Migration Path
This PR only migrates a handful of constraints. My plan is to discuss the general idea with this PR first and use it as a blueprint to migrate the individual constraints afterwards. Right now, the migration path would look like this:
* Attach the `#[Attribute]` attribute.
* Recreate all options of the constraint as constructor arguments.
* Add test cases for constructor calls with named arguments to the test class of the constraint's validator.
Commits
-------
d1cb2d6354 [Validator] Constraints as php 8 Attributes.
This PR was merged into the 5.2-dev branch.
Discussion
----------
[Notifier] Add Dsn test case
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | no <!-- please update src/**/CHANGELOG.md files -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tickets | - <!-- prefix each issue number with "Fix #", no need to create an issue if none exist, explain below instead -->
| License | MIT
| Doc PR | - <!-- required for new features -->
<!--
Replace this notice by a short README for your feature/bugfix. This will help people
understand your PR and can be used as a start for the documentation.
Additionally (see https://symfony.com/releases):
- Always add tests and ensure they pass.
- Never break backward compatibility (see https://symfony.com/bc).
- Bug fixes must be submitted against the lowest maintained branch where they apply
(lowest branches are regularly merged to upper ones so they get the fixes too.)
- Features and deprecations must be submitted against branch master.
-->
Commits
-------
6a0510d859 Add Dsn test case
* 4.4:
Revert "bug #38063 [FrameworkBundle] generate preload.php in src/ to make opcache.preload predictable (nicolas-grekas)"
[PhpUnitBridge] Fix class_alias() for PHPUnit\Framework\Error\Error
This PR was merged into the 4.4 branch.
Discussion
----------
Revert "bug #38063 [FrameworkBundle] generate preload.php in src/ to make opcache.preload predictable"
| Q | A
| ------------- | ---
| Branch? | 4.4
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | Fix#38334
| License | MIT
| Doc PR | -
As discussed in the linked issue, let's replace this with a recipe:
https://github.com/symfony/recipes/pull/825
TL;DR, these PRs replace `src/.preload.php` (generated) by `config/preload.php` (committed).
Commits
-------
662fcff40f Revert "bug #38063 [FrameworkBundle] generate preload.php in src/ to make opcache.preload predictable (nicolas-grekas)"
This PR was squashed before being merged into the 5.2-dev branch.
Discussion
----------
[Console] Improve description for the help flag.
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | no
| Deprecations? | no
| Tickets | no
| License | MIT
I am currently trying to improve a CLI tool called `robo` (https://robo.li) that relies heavily on the console component.
First thing I did was running it without any argument like `./robo` to find the description for the help flag `-h` or `--help`
"Display this help message"
This is typical for any CLI tool out there except that when I run `./robo --help` I get the help for the `list` command (same output as `robo help list`).
One of the options I considered was to fix this behavior by showing the same output when no command is given disregarding the help flag (just for the case of no command given) but I actually like how symfony console handles the help flag.
I think it needs a clarification for the description given so I changed it to:
> Display help for the given command. When no command is given display help for the list command
Which is more descriptive about what actually happens in the code when that flag is given. Specially the "this" word can be a little confusing.
Commits
-------
c451c48258 [Console] Improve description for the help flag.
This PR was merged into the 5.2-dev branch.
Discussion
----------
[Validator] Add support for UUIDv6 in Uuid constraint
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | Fix#37751
| License | MIT
| Doc PR | -
Commits
-------
55c17e1af7 [Validator] Add support for UUIDv6 in Uuid constraint
This PR was merged into the 4.4 branch.
Discussion
----------
[PhpUnitBridge] Fixed class_alias() for PHPUnit\Framework\Error\Error
| Q | A
| ------------- | ---
| Branch? | 4.4
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | n/a
| License | MIT
| Doc PR | n/a
PHPUnit 6.x removed the PHPUnit_Framework_* classes in favor of PHP namespaces, but one error class did not map the same as the others: `PHPUnit_Framework_Error`.
Instead of mapping to `PHPUnit\Framework\Error` in the same way that `PHPUnit_Framework_Error_Warning` mapped to `PHPUnit\Framework\Error\Warning`, this base class was replaced with `PHPUnit\Framework\Error\Error`, so we cannot map it using simple string replacement like its descendants.
Commits
-------
8e607b58df [PhpUnitBridge] Fix class_alias() for PHPUnit\Framework\Error\Error
This PR was merged into the 5.2-dev branch.
Discussion
----------
[Contracts] add TranslatableInterface
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | #38319
| License | MIT
| Doc PR | -
An alternative to #38328:
- `TranslatableInterface` is added to symfony/contracts;
- `Translatable`, still in the component, implements it (and is not final as it makes no sense for a value-object)
- the `t()` function is kept in the component - it doesn't fit in the contracts IMHO;
- `Translatable::trans()` is not static anymore;
- the domain is nullable instead of defaulting to "messages";
- the `t()` function moved in the `Symfony\Component\Translation` namespace (for reference, the `s()` function from String is also namespaced.);
- the Twig extension is made strictly polymorphic: if you pass a Translatable, you cannot pass arguments/domain/count.
Commits
-------
9224f7ac5b [Contracts] add TranslatableInterface
This PR was squashed before being merged into the 5.2-dev branch.
Discussion
----------
[Validator] Add Ulid constraint and validator
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | Fix#38152
| License | MIT
ULID constraint and validator as mentioned in ticket #38103
I checked for it ulid specifications https://github.com/ulid/spec
Commits
-------
e36fd559da [Validator] Add Ulid constraint and validator
This PR was merged into the 5.2-dev branch.
Discussion
----------
[Uid] make UUIDv6 always return truly random nodes to prevent leaking the MAC of the host
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | -
| License | MIT
| Doc PR | -
As explained in http://gh.peabody.io/uuidv6/, the wording of the UUIDv1 spec suggests that using the MAC of the host is preferred to compute the "node" field of UUIDs. This is what the uuid extension does, and the reason why the 12 last chars of the UUIDv1 it generates are stable. But this is a privacy leak. There are stories in the wild about how knowing the MAC has been abused in the past.
UUIDv6 prefers putting a secure random number there.
So here is the PR to do so.
Commits
-------
b9c61ca86c [Uid] make UUIDv6 always return truly random nodes to prevent leaking the MAC of the host
PHPUnit 6.x removed the PHPUnit_Framework_* classes in favor of PHP namespaces, but one error class did not map the same as the others: `PHPUnit_Framework_Error`.
Instead of mapping to `PHPUnit\Framework\Error` in the same way that `PHPUnit_Framework_Error_Warning` mapped to `PHPUnit\Framework\Error\Warning`, this base class was replaced with `PHPUnit\Framework\Error\Error`.
This PR was merged into the 5.1 branch.
Discussion
----------
Fix flaky test possiblity on PHP >=7.1
| Q | A
| ------------- | ---
| Branch? | 5.1
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | Fix#38320
| License | MIT
| Doc PR | not needed
Commits
-------
f99256fec8 Fix flaky test possiblity on PHP >=7.1
