This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] bug fixes in Doctrine Transport
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets |
| License | MIT
| Doc PR |
Just tested the new Doctrine transport and I've see 3 bugs so far :
- [x] The message is not return by the transport
- [x] The headers column must be of type TEXT and not just STRING
- [ ] When using the PhpSerializer the message is truncated (PR: https://github.com/symfony/symfony/pull/30814)
The body in database looks like this :
```
O:36:"Symfony\Component\Messenger\Envelope":2:{s:44:"
```
The body given by the serializer is the following :
```
O:36:"Symfony\Component\Messenger\Envelope":2:{s:44:"Symfony\Component\Messenger\Envelopestamps";a:3:{s:49:"Symfony\Component\Messenger\Stamp\SerializerStamp";a:1:{i:0;O:49:"Symfony\Component\Messenger\Stamp\SerializerStamp":1:{s:58:"Symfony\Component\Messenger\Stamp\SerializerStampcontext";a:0:{}}}s:46:"Symfony\Component\Messenger\Stamp\BusNameStamp";a:1:{i:0;O:46:"Symfony\Component\Messenger\Stamp\BusNameStamp":1:{s:55:"Symfony\Component\Messenger\Stamp\BusNameStampbusName";s:21:"messenger.bus.default";}}s:43:"Symfony\Component\Messenger\Stamp\SentStamp";a:1:{i:0;O:43:"Symfony\Component\Messenger\Stamp\SentStamp":2:{s:56:"Symfony\Component\Messenger\Stamp\SentStampsenderClass";s:64:"Symfony\Component\Messenger\Transport\Doctrine\DoctrineTransport";s:56:"Symfony\Component\Messenger\Stamp\SentStampsenderAlias";s:16:"environment.stop";}}}s:45:"Symfony\Component\Messenger\Envelopemessage";O:34:"App\Message\EnvironmentStopMessage":1:{s:51:"App\Message\AbstractEnvironmentMessageenvironment";O:22:"App\Entity\Environment":5:{s:26:"App\Entity\Environmentid";s:36:"3bade252-b7a9-4188-82bd-3e68129e0da7";s:37:"App\Entity\EnvironmentrepositoryUrl";s:6:"string";s:30:"App\Entity\Environmentbranch";s:6:"string";s:33:"App\Entity\EnvironmenthostNames";a:1:{i:0;N;}s:27:"App\Entity\Environmentenv";a:2:{s:7:"APP_ENV";s:4:"prod";s:7:"APP_VAR";s:13:"example value";}}}}
```
Commits
-------
27466498d0 [Messenger] Fix get in Doctrine Transport
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] base64_encoding inside PhpSerializer to avoid null characters
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | #30805
| License | MIT
| Doc PR | not needed
Hi!
As pointed out in #30805, the `PhpSerializer` creates strings with null bytes. This apparently causes problems on at least some database systems (I didn't notice, but @vincenttouzet did). I also read that, for example, SQS doesn't like null characters. And, in general, because we're sending this data over a transport, `base64_encoding` data is pretty standard.
Does anyone see any downsides?
Cheers!
Commits
-------
fe7ad812c7 base64_encoding inside PhpSerializer to avoid null characters
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Process] Added more detail to the exception when the CWD is invalid
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets |
| License | MIT
| Doc PR |
---
When using panther with "special" directory structure, the message is
not really usefull. Let's add the CWD to the exception
Commits
-------
d27858f77b [Process] Added more detail to the exception when the CWD is invalid
This PR was merged into the 4.2 branch.
Discussion
----------
[Config] Improve PHPdoc / IDE autocomplete for config tree builder
| Q | A
| ------------- | ---
| Branch? | 4.2
| Bug fix? | no
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | -
| License | MIT
| Doc PR | -
follow up of #27476 and #21047
Commits
-------
21f797714e Improve PHPdoc / IDE autocomplete for config tree builder
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Inflector] remove "internal" marker from the component
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | https://github.com/symfony/symfony/issues/26903
| License | MIT
| Doc PR | -
This code works: making it non-internal won't increase the maintenance burden and can help others build on it.
Commits
-------
164b45b79c [Inflector] remove "internal" marker from the component
This PR was merged into the 4.2 branch.
Discussion
----------
[Bridge][Twig] DebugCommand - fix escaping and filter
| Q | A
| ------------- | ---
| Branch? | 4.2
| Bug fix? | no
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| License | MIT
cherry-pick on 4.2, conflicts resolved, original PR https://github.com/symfony/symfony/pull/30660#issuecomment-478468584
Commits
-------
b7120c5e49 [Bridge][Twig] DebugCommand - fix escaping and filter
This PR was merged into the 3.4 branch.
Discussion
----------
SCA: minor code tweaks
| Q | A
| ------------- | ---
| Branch? | 3.4
| Bug fix? | no
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | n/a
| License | MIT
| Doc PR | n/a
- minor code tweaks
- drop private properties, which used as local variables
Commits
-------
cc4529db51 SCA: minor code tweaks
This PR was squashed before being merged into the 4.3-dev branch (closes#30807).
Discussion
----------
[Messenger] Various minor fixes
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no <!-- don't forget to update src/**/CHANGELOG.md files -->
| BC breaks? | no <!-- see https://symfony.com/bc -->
| Deprecations? | no <!-- don't forget to update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tests pass? | yes <!-- please add some, will be required by reviewers -->
| Fixed tickets | n/a
| License | MIT
| Doc PR | n/a
Found while playing with the new Doctrine transport.
Commits
-------
96dee1ee20 [Messenger] fixed missing use statement
1044dfb93d [Messenger] simplified code
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] Remove unused option in the Doctrine transport
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets |
| License | MIT
| Doc PR |
This PR remove the unused option `loop_sleep` in the Messenger Doctrine transport
Commits
-------
4811400372 [Messenger] Remove unused option in the Doctrine transport
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Inflector] Support pluralization in the inflector
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | not yet
| Fixed tickets | N/A
| License | MIT
| Doc PR | Not Yet
At present the inflector only supports singularizing plural words, this PR adds the capability to pluralize singular words.
Commits
-------
06920a79c4 Support pluralization in the inflector
This PR was squashed before being merged into the 4.3-dev branch (closes#28637).
Discussion
----------
[Validator] add number constraints
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | #28608
| License | MIT
| Doc PR | tbd.
I added the following constraints:
* `Positive`
* `PositiveOrZero`
* `Negative`
* `NegativeOrZero`
Commits
-------
01870398eb [Validator] add number constraints
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] Fix the Doctrine transport to use the new interface
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | #29007
| License | MIT
| Doc PR | ø
Commits
-------
75e3355da5 Fix the Doctrine transport to use the new interface
This PR was squashed before being merged into the 3.4 branch (closes#30781).
Discussion
----------
[Intl] Update the ICU data to 64.1
| Q | A
| ------------- | ---
| Branch? | 3.4
| Bug fix? | no
| New feature? | no <!-- don't forget to update src/**/CHANGELOG.md files -->
| BC breaks? | no <!-- see https://symfony.com/bc -->
| Deprecations? | no <!-- don't forget to update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tests pass? | yes, including the intl-data group
| Fixed tickets | -
| License | MIT
| Doc PR | -
http://site.icu-project.org/download/64
Commits
-------
ae2cb6f5c5 [Intl] Update the ICU data to 64.1
This PR was squashed before being merged into the 4.3-dev branch (closes#30754).
Discussion
----------
[Messenger] New messenger:stop-workers Command
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | Kinda of #29451
| License | MIT
| Doc PR | symfony/symfony-docs#11236
o/ me again.
This requires and is built on top of #30708
When you deploy, all workers need to be stopped and restarted. That's not currently possible, unless you manually track the pids and send a SIGTERM signal. We can make that much easier :).
Now run:
```
bin/console messenger:stop-workers
```
And it will signal to all workers (even if they're distributed on other servers) that they should stop, once they finish processing their current message. This is done via a key in `cache.app`.
Cheers!
Commits
-------
58971627f5 [Messenger] New messenger:stop-workers Command
This PR was merged into the 4.3-dev branch.
Discussion
----------
[EventDispatcher] Fix BC/FC layer
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | n/a
| License | MIT
| Doc PR | n/a
Fixes
> TypeError: Argument 3 passed to Symfony\Component\EventDispatcher\EventDispatcher::doDispatch() must be an instance of Symfony\Component\EventDispatcher\Event
Spotted in https://github.com/lexik/LexikJWTAuthenticationBundle/pull/637
Commits
-------
caa0aded89 [EventDispatcher] Fix BC layer
This PR was merged into the 4.3-dev branch.
Discussion
----------
Changing to MessageDecodingFailedException so that invalid messages are rejected
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes <!-- please add some, will be required by reviewers -->
| Fixed tickets | #30649
| License | MIT
| Doc PR | not needed for bug fix
Bug fix if a message body is completely blank. I'm fixing this on master only, because in 4.2 and earlier, there is actually no system in place to fail serialization and cause the messages to be rejected. In 4.3, we just need to throw this exception.
Cheers!
Commits
-------
4be827d3ca Changing to MessageDecodingFailedException so that invalid messages are rejected
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] Updating SyncTransport for recent changes + tests
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | none
| License | MIT
| Doc PR | not needed
When making the `SyncTransport`, I neglected having at least one test in each of these classes, which allowed the test suite to pass, even after some interface changes made these classes fail. Fixed all of that :)
Commits
-------
2df023be46 Updating SyncTransport for recent changes + tests
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger][DX] Allow stamps to be passed directly to MessageBusInterface::dispatch()
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | yes
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | none
| License | MIT
| Doc PR | TODO
Me again o/!
This proposal is *purely* for DX. With `DelayStamp`, the proposal of QueueNameStamp and future things like `AmqpRoutingKeyStamp`, stamps are becoming more common for end users to use. This changes how it looks to use them:
```php
// before
$bus->dispatch(new Envelope(new SendSmsNotification('Hi!'), new DelayStamp(10), new QueueNameStamp('low')));
// after
$bus->dispatch(new SendSmsNotification('Hi!'), [new DelayStamp(10), new QueueNameStamp('low')]);
```
It's definitely a BC break, which is allowed because the component is experimental, though it should be minimized. This BC break shouldn't be felt by most end users, as creating your own bus is an advanced use-case. Even if you decorated it, you'll get an obvious error.
Commits
-------
e861de7e61 Allow stamps to be passed directly to MessageBusInterface::dispatch()
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] Add a Doctrine transport
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets |
| License | MIT
| Doc PR | symfony/symfony-docs#10616
| DoctrineBundle PR | doctrine/DoctrineBundle#868
As discussed with @sroze at PHPForum in Paris I've worked on adding a Doctrine transport to the Messenger component.
Actually `AMQP` is the only supported transport and it could be a good thing to support multiple transports. Having a Doctrine transport could help users to start using the component IMHO (Almost all projects use a database).
# How it works
The code is splitted betwwen this PR and the one on the DoctrineBundle : doctrine/DoctrineBundle#868
## Configuration
To configure a Doctrine transport the dsn MUST have the format `doctrine://<entity_manager_name>` where `<entity_manager_name>` is the name of the entity manager (usually `default`)
```yml
# config/packages/messenger.yaml
framework:
messenger:
transports:
my_transport: "doctrine://default?queue=important"
```
## Table schema
Dispatched messages are stored into a database table with the following schema:
| Column | Type | Options | Description |
|--------------|----------|--------------------------|-------------------------------------------------------------------|
| id | bigint | AUTO_INCREMENT, NOT NULL | Primary key |
| body | text | NOT NULL | Body of the message |
| headers | text | NOT NULL | Headers of the message |
| queue | varchar(32) | NOT NULL | Headers of the message |
| created_at | datetime | NOT NULL | When the message was inserted onto the table. (automatically set) |
| available_at | datetime | NOT NULL | When the message is available to be handled |
| delivered_at | datetime | NULL | When the message was delivered to a worker |
## Message dispatching
When dispatching a message a new row is inserted into the table. See `Symfony\Component\Messenger\Transport\Doctrine::publish`
## Message consuming
The message is retrieved by the `Symfony\Component\Messenger\Transport\Doctrine\DoctrineReceiver`. It calls the `Symfony\Component\Messenger\Transport\Doctrine::get` method to get the next message to handle.
### Getting the next message
* Start a transaction
* Lock the table to get the first message to handle (The lock is done with the `SELECT ... FOR UPDATE` query)
* Update the message in database to update the delivered_at columns
* Commit the transaction
### Handling the message
The retrieved message is then passed to the handler. If the message is correctly handled the receiver call the `Symfony\Component\Messenger\Transport\Doctrine::ack` which delete the message from the table.
If an error occured the receiver call the `Symfony\Component\Messenger\Transport\Doctrine::nack` method which update the message to set the delivered_at column to `null`.
## Message requeueing
It may happen that a message is stuck in `delivered` state but the handler does not really handle the message (Database connection error, server crash, ...). To requeue messages the `DoctrineReceiver` call the `Symfony\Component\Messenger\Transport\Doctrine::requeueMessages`. This method update all the message with a `delivered_at` not null since more than the "redeliver timeout" (default to 3600 seconds)
# TODO
- [x] Add tests
- [x] Create DOC PR
- [x] PR on doctrine-bundle for transport factory
- [x] Add a `available_at` column
- [x] Add a `queue` column
- [x] Implement the retry functionnality : See #30557
- [x] Rebase after #29476
Commits
-------
88d008c828 [Messenger] Add a Doctrine transport
