This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] Add WorkerStoppedEvent
| Q | A
| ------------- | ---
| Branch? | 4.3
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | n/a
| License | MIT
| Doc PR | n/a
In 4.2, one was able to decorate a transport and hook into `TransportInterface::stop()` to perform some business tasks when the `messenger:consume` process gets killed.
In 4.3 the `stop()` logic has been moved to `Worker` which cannot be decorated when used via `messenger:consume`.
This PR adds a marker event dispatched when the worker is stopped to provide the same capability.
Briefly discussed with @weaverryan.
My use case:
I have a temporary CSV file locally which grows while consuming jobs.
This file is uploaded to AWS S3 and squashed once a while (every 5minutes). It is also uploaded when the `messenger:consume` process gets stopped (to store the remaining lines).
For the former (time-based upload), I can listen on `WorkflowMessageHandledEvent` to make the upload happens in case the 5 minutes delay is elapsed. This solves the latter.
Commits
-------
0e7898b622 [Messenger] Add WorkerStoppedEvent
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] Add a redis stream transport
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | Yes
| Fixed tickets | #28681
| License | MIT
| Doc PR | https://github.com/symfony/symfony-docs/pull/11341
As discussed in #28681 this will refractor @soyuka implementation of redis using the redis stream features so we don't need to handle parking the messages ourself and redis is doing it for us.
Some interesting links about streams:
- https://redis.io/topics/streams-intro
- https://brandur.org/redis-streams
```
+-----------R
| GET | -> XREADGROUP
+-----------+
|
| handleMessage
V
+-----------+ No
| failed? |---------------------------+
+-----------+ |
| |
| Yes |
V |
+-----------+ No |
| retry? |---------------------------+
+-----------+ |
| |
| Yes |
V V
+-----------R +-----------R
| REJECT | -> XDEL | ACK | -> XACK
+-----------+ +-----------+
```
**GET**: Will use `XREADGROUP` to read the one message from the stream
**REJECT**: Reject will just remove the message with `XDEL` from the stream as adding it back to the stream is handled by symfony worker itself
**ACK**: Will use the `XACK` Method to ack the message for the specific group
The sender will still be simple by calling the `XADD` redis function.
#EU-FOSSA
Commits
-------
ff0b8554ea Refractor redis transport using redis streams
7162d2ec1d Implement redis transport
This PR was merged into the 4.3-dev branch.
Discussion
----------
Make error message more clear
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | no <!-- don't forget to update src/**/CHANGELOG.md files -->
| BC breaks? | no <!-- see https://symfony.com/bc -->
| Deprecations? | no <!-- don't forget to update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tests pass? | yes <!-- please add some, will be required by reviewers -->
| Fixed tickets | n/a
| License | MIT
| Doc PR | n/a
When you get the following error message `No transport supports the given DSN "...".`, you might think that this relates to a mailer transport or a messenger transport. Let's make this real clear.
<!--
Write a short README entry for your feature/bugfix here (replace this comment block.)
This will help people understand your PR and can be used as a start of the Doc PR.
Additionally:
- Bug fixes must be submitted against the lowest branch where they apply
(lowest branches are regularly merged to upper ones so they get the fixes too).
- Features and deprecations must be submitted against the master branch.
-->
Commits
-------
cdcb6c9969 made error message more clear
* 4.2:
Fix url matcher edge cases with trailing slash
[Form] Fix author tag + exception messages
[TwigBridge] Fix deprecation on twig 2.9
Fix left-associative ternary deprecation warnings for PHP 7.4
[Validator] Fixed imprecise translations
[Validator] Add Dutch translations
[Security] Cleanup "Digest nonce has expired." translation
Intercept redirections only for HTML format
[PhpUnitBridge] fix reading phpunit.xml on bootstrap
resolve class name parameters
Fix name and phpdoc of ContainerBuilder::removeBindings
[Intl] Update the ICU data to 64.2
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] ease testing and allow forking the middleware stack
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | #31179
| License | MIT
| Doc PR | -
A less radical alternative than #31185 that preserves laziness and addresses the linked issue.
Commits
-------
3bdf4b0e0f [Messenger] ease testing and allow forking the middleware stack
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] Uses an `AmqpStamp` to provide flags and attributes
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | #28885
| License | MIT
| Doc PR | ø
Using the `AmqpStamp` you can configure the flags and any attribute (such as `delivery_mode`).
Commits
-------
56fa574023 Uses an `AmqpStamp` to provide flags and attributes
This PR was merged into the 4.3-dev branch.
Discussion
----------
[messenger] Adds a stamp to provide a routing key on message publishing
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | #29950
| License | MIT
| Doc PR | symfony/symfony-docs#11236
Adds a stamp allowing to set a `routing_key` at `MessageBus::dispatch()` level.
```php
$message = (new Envelope('message'))->with(new RoutingKeyStamp('routing_key'));
$bus->dispatch($message);
```
Commits
-------
a515635f18 Simply code and rename "configuration" to "options"
3151b54b7a [messenger] AMQP configurable routing key & multiple queues
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] Make MessengerPass less strict when auto-register handlers
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| License | MIT
| Doc PR | not needed
This allows you to auto-register handlers that have more than one argument, which is useful when having custom middleware to pass more parameters.
#symfonyconhackday2018
Commits
-------
49ab2cd9d9 Make MessengerPass less strict when auto-register handlers
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] test DoctrineTransport on travis and appveyor
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes ? WIP
| Fixed tickets |
| License | MIT
| Doc PR |
Currently tests on the `Symfony\Component\Messenger\Tests\Transport\Doctrine\DoctrineIntegrationTest` are skipped because there is no `MESSENGER_DOCTRINE_DSN` environment variable is not defined.
This PR update the travis and AppVeyor configuration to run these tests.
This is a WIP. I'm not a Travis/AppVeyor user so this clearly need more work
Commits
-------
8f81f55a46 [Messenger] test DoctrineTransport on travis and appveyor
This PR was squashed before being merged into the 4.3-dev branch (closes#30757).
Discussion
----------
[Messenger] Adding MessageCountAwareInterface to get transport message count
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | none
| License | MIT
| Doc PR | symfony/symfony-docs#11236
This adds a new optional interface that receivers should implement to give an approximate number of the messages "waiting" to be handled. Why? Because, with this, you could design a system that dynamically adds/removes worker processes if a specific transport is getting slammed and needs help. Creating that system could be something we discuss for core later, but this at least makes it possible - and means it could be implemented by the user or in a bundle... which I might do if we don't get it in core ;).
Commits
-------
fc5b0cf570 [Messenger] Adding MessageCountAwareInterface to get transport message count
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] bug fixes in Doctrine Transport
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets |
| License | MIT
| Doc PR |
Just tested the new Doctrine transport and I've see 3 bugs so far :
- [x] The message is not return by the transport
- [x] The headers column must be of type TEXT and not just STRING
- [ ] When using the PhpSerializer the message is truncated (PR: https://github.com/symfony/symfony/pull/30814)
The body in database looks like this :
```
O:36:"Symfony\Component\Messenger\Envelope":2:{s:44:"
```
The body given by the serializer is the following :
```
O:36:"Symfony\Component\Messenger\Envelope":2:{s:44:"Symfony\Component\Messenger\Envelopestamps";a:3:{s:49:"Symfony\Component\Messenger\Stamp\SerializerStamp";a:1:{i:0;O:49:"Symfony\Component\Messenger\Stamp\SerializerStamp":1:{s:58:"Symfony\Component\Messenger\Stamp\SerializerStampcontext";a:0:{}}}s:46:"Symfony\Component\Messenger\Stamp\BusNameStamp";a:1:{i:0;O:46:"Symfony\Component\Messenger\Stamp\BusNameStamp":1:{s:55:"Symfony\Component\Messenger\Stamp\BusNameStampbusName";s:21:"messenger.bus.default";}}s:43:"Symfony\Component\Messenger\Stamp\SentStamp";a:1:{i:0;O:43:"Symfony\Component\Messenger\Stamp\SentStamp":2:{s:56:"Symfony\Component\Messenger\Stamp\SentStampsenderClass";s:64:"Symfony\Component\Messenger\Transport\Doctrine\DoctrineTransport";s:56:"Symfony\Component\Messenger\Stamp\SentStampsenderAlias";s:16:"environment.stop";}}}s:45:"Symfony\Component\Messenger\Envelopemessage";O:34:"App\Message\EnvironmentStopMessage":1:{s:51:"App\Message\AbstractEnvironmentMessageenvironment";O:22:"App\Entity\Environment":5:{s:26:"App\Entity\Environmentid";s:36:"3bade252-b7a9-4188-82bd-3e68129e0da7";s:37:"App\Entity\EnvironmentrepositoryUrl";s:6:"string";s:30:"App\Entity\Environmentbranch";s:6:"string";s:33:"App\Entity\EnvironmenthostNames";a:1:{i:0;N;}s:27:"App\Entity\Environmentenv";a:2:{s:7:"APP_ENV";s:4:"prod";s:7:"APP_VAR";s:13:"example value";}}}}
```
Commits
-------
27466498d0 [Messenger] Fix get in Doctrine Transport
This PR was merged into the 4.3-dev branch.
Discussion
----------
[Messenger] Remove unused option in the Doctrine transport
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets |
| License | MIT
| Doc PR |
This PR remove the unused option `loop_sleep` in the Messenger Doctrine transport
Commits
-------
4811400372 [Messenger] Remove unused option in the Doctrine transport
This PR was squashed before being merged into the 4.3-dev branch (closes#30754).
Discussion
----------
[Messenger] New messenger:stop-workers Command
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | Kinda of #29451
| License | MIT
| Doc PR | symfony/symfony-docs#11236
o/ me again.
This requires and is built on top of #30708
When you deploy, all workers need to be stopped and restarted. That's not currently possible, unless you manually track the pids and send a SIGTERM signal. We can make that much easier :).
Now run:
```
bin/console messenger:stop-workers
```
And it will signal to all workers (even if they're distributed on other servers) that they should stop, once they finish processing their current message. This is done via a key in `cache.app`.
Cheers!
Commits
-------
58971627f5 [Messenger] New messenger:stop-workers Command
