This PR was merged into the 5.2-dev branch.
Discussion
----------
[HttpClient] fix pausing AmpResponse
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | yes
| New feature? | no
| Deprecations? | no
| Tickets | None
| License | MIT
| Doc PR | None
This fixes the pause handler while streaming bodies in `AmpHttpClient`.
Needs to be rebased onto the target branch and needs some test fixes. @nicolas-grekas FYI.
Commits
-------
bf2d1cf6e7 Improve pause handler
Few years ago, we have introduced the Lock component. This is a very nice component, but sometime it is not enough. Sometime you need semaphore.
This is why I'm introducing this new component.
From wikipedia:
> In computer science, a semaphore is a variable or abstract data type used to control access to a common resource by multiple processes in a concurrent system such as a multitasking operating system. A semaphore is simply a variable. This variable is used to solve critical section problems and to achieve process synchronization in the multi processing environment. A trivial semaphore is a plain variable that is changed (for example, incremented or decremented, or toggled) depending on programmer-defined conditions.
This new component is more than a variable. This is an abstraction on top of different storage.
To make a quick comparison with a lock:
* A lock allows only 1 process to access a resource;
* A semaphore allow N process to access a resource.
Basically, a lock is a semaphore where `N = 1`.
PHP exposes some `sem_*` functions like [`sem_acquire`](http://php.net/sem_acquire). This module provides wrappers for the System V IPC family of functions. It includes semaphores, shared memory and inter-process messaging (IPC).
The Lock component has a storage that works with theses functions. It uses it with `N = 1`.
Wikipedia has some [examples](https://en.wikipedia.org/wiki/Semaphore_(programming)#Examples)
But I can add one more commun use case.
If you are building an async system that process user data, you may want to priorise all jobs. You can achieve that by running at maximum N jobs per user at the same time. If the user has more resources, you give him more concurrent jobs (so a bigger `N`).
Thanks to semaphores, it's pretty easy to know if a new job can be run.
I'm not saying the following services are using semaphore, but they may solve the previous problematic with semaphores. Here is some examples:
* services like testing platform where a user can test N projects concurrently (travis, circle, appveyor, insight, ...)
* services that ingest lots of data (newrelic, datadog, blackfire, segment.io, ...))
* services that send email in batch (campaign monitor, mailchimp, ...)
* etc...
To do so, since PHP is mono-threaded, you run M PHP workers. And in each worker, you look for for the next job. When you grab a job, you try to acquires a semaphore. If you got it, you process the job. If not you try another job.
FTR in other language, like Go, there are no need to run M workers, one is enough.
```php
<?php
use Symfony\Component\Lock\LockFactory;
use Symfony\Component\Lock\Store\RedisStore as LockRedisStore;
use Symfony\Component\Semaphore\SemaphoreFactory;
use Symfony\Component\Semaphore\Store\RedisStore;
require __DIR__.'/vendor/autoload.php';
$redis = new Redis();
$redis->connect('172.17.0.2');
// Internally, Semaphore needs a lock
$lock = (new LockFactory(new LockRedisStore($redis)))->createLock('test:lock', 1);
// Create a semaphore:
// * name = test
// * limit = 3 (it means only 3 process are allowed)
// * ttl = 10 seconds : Maximum expected semaphore duration in seconds
$semaphore = (new SemaphoreFactory($lock, new RedisStore($redis)))->createSemaphore('test', 3, 10);
if (!$semaphore->acquire()) {
echo "Could not acquire the semaphore\n";
exit(1);
}
// The semaphore has been acquired
// Do the heavy job
for ($i = 0; $i < 100; ++$i) {
sleep(1);
// Before the expiration, refresh the semaphore if the job is not finished yet
if ($i % 9 === 0) {
$semaphore->refresh();
}
}
// Release it when finished
$semaphore->release();
```
I looked at [packagist](https://packagist.org/?query=semaphore) and:
* most of packages are using a semaphore storage for creating a lock. So there are not relevant here;
* some packages need an async framework to be used (amphp for example);
* the only packages really implementing a semaphore, has a really low code quality and some bugs.
1. I initially copied the Lock component since the external API is quite similar;
1. I simplified it a lot for the current use case;
1. I implemented the RedisStorage according the [redis book](https://redislabs.com/ebook/part-2-core-concepts/chapter-6-application-components-in-redis/6-3-counting-semaphores/;)
1. I forced a TTL on the storage.
* 5.1:
Fix typo
Fix deprecated libxml_disable_entity_loader
Add Tagalog translations for validator messages 94, 95, 96 and 99
PHPUnit's assertContains() performs strict comparisons now.
[ClassLoader][Routing] Fix namespace parsing on php 8.
Fix deprecated libxml_disable_entity_loader
Made reference to PHPUnit\Util\XML::loadfile php5-compatible.
[Validator] Add missing translations for german and vietnamese
Modernized deprecated PHPUnit assertion calls
[Console] The message of "class not found" errors has changed in php 8.
The PHPUnit\Util\XML class has been removed in PHPUnit 9.3.
[Console] Make sure we pass a numeric array of arguments to call_user_func_array().
Remove outdated references from base_js.html.twig file
[String] We cannot have a "provides" function in test cases.
Typo: somes styles fixed
[Serializer] Fix that it will never reach DOMNode
[Validator] sync translations
[VarDumper] Improve previous fix on light array coloration
[Cache] Fix#37667
* 4.4:
Fix typo
Fix deprecated libxml_disable_entity_loader
Add Tagalog translations for validator messages 94, 95, 96 and 99
PHPUnit's assertContains() performs strict comparisons now.
[ClassLoader][Routing] Fix namespace parsing on php 8.
Fix deprecated libxml_disable_entity_loader
Made reference to PHPUnit\Util\XML::loadfile php5-compatible.
[Validator] Add missing translations for german and vietnamese
Modernized deprecated PHPUnit assertion calls
[Console] The message of "class not found" errors has changed in php 8.
The PHPUnit\Util\XML class has been removed in PHPUnit 9.3.
[Console] Make sure we pass a numeric array of arguments to call_user_func_array().
[Serializer] Fix that it will never reach DOMNode
[Validator] sync translations
[VarDumper] Improve previous fix on light array coloration
[Cache] Fix#37667
* 3.4:
Add Tagalog translations for validator messages 94, 95, 96 and 99
PHPUnit's assertContains() performs strict comparisons now.
[ClassLoader][Routing] Fix namespace parsing on php 8.
Fix deprecated libxml_disable_entity_loader
Made reference to PHPUnit\Util\XML::loadfile php5-compatible.
[Validator] Add missing translations for german and vietnamese
Modernized deprecated PHPUnit assertion calls
[Console] The message of "class not found" errors has changed in php 8.
The PHPUnit\Util\XML class has been removed in PHPUnit 9.3.
[Console] Make sure we pass a numeric array of arguments to call_user_func_array().
[Serializer] Fix that it will never reach DOMNode
[Validator] sync translations
[VarDumper] Improve previous fix on light array coloration
[Cache] Fix#37667
* 5.1: (28 commits)
[DI] fix
Use "composer/package-versions-deprecated" when possible
Fix
Small update in our internal terminology
Fix support for PHP8 union types
[VarDumper] fix typo
[Lock][Messenger] Fix precedence of DSN options for 5.1
Fix support for PHP8 union types
[FrameworkBundle] preserve dots in query-string when redirecting
[3.4] Fix support for PHP8 union types
[PhpUnitBridge] Streamline ansi/no-ansi of composer according to phpunit --colors option
[3.4] Small update in our internal terminology
[Cache] fix compat with DBAL v3
Remove unnecessary null check
[HttpFoundation] Allow `null` in InputBag@set
[HttpClient] Convert CurlHttpClient::handlePush() to instance method
Fix package rename when releasing
bumped Symfony version to 5.1.3
updated VERSION for 5.1.2
updated CHANGELOG for 5.1.2
...
* 5.1:
[Console] Reset question validator attempts only for actual stdin (bis)
Fix CookieClearingLogoutListener DI configuration
[HttpFoundation] use InputBag for Request::$request only if data is coming from a form
Make PhpDocExtractor compatible with phpDocumentor v5
fixed prototype block prefixes hierarchy of the CollectionType
Reset question validator attempts only for actual stdin
fixed block prefixes hierarchy of the CollectionType
bumped Symfony version to 5.0.11
updated VERSION for 5.0.10
updated CHANGELOG for 5.0.10
bumped Symfony version to 4.4.11
updated VERSION for 4.4.10
updated CHANGELOG for 4.4.10
* 5.0:
Make PhpDocExtractor compatible with phpDocumentor v5
Reset question validator attempts only for actual stdin
bumped Symfony version to 5.0.11
updated VERSION for 5.0.10
updated CHANGELOG for 5.0.10
bumped Symfony version to 4.4.11
updated VERSION for 4.4.10
updated CHANGELOG for 4.4.10
* 4.4:
Make PhpDocExtractor compatible with phpDocumentor v5
Reset question validator attempts only for actual stdin
bumped Symfony version to 4.4.11
updated VERSION for 4.4.10
updated CHANGELOG for 4.4.10
Version 5 of phpDocumentor introduced some changes to the
`getTagsByName()` method that break the `PhpDocExtractor`.
More specific, it now returns an instance of `InvalidTag` instead of
`null` when parsing an invalid tag.
* 4.4: (27 commits)
[Serializer] minor cleanup
fix merge
Run PHP 8 as 7.4.99
Remove calls to deprecated ReflectionParameter::getClass().
[VarDumper] fix PHP 8 support
Add php 8 to travis.
[Cache] Accessing undefined constants raises an Error in php8
[Cache] allow DBAL v3
Skip Doctrine DBAL on php 8 until we have a compatible version.
[DomCrawler] Catch expected ValueError.
Made method signatures compatible with their corresponding traits.
[ErrorHandler] Apply php8 fixes from Debug component.
[DomCrawler] Catch expected ValueError.
[Validator] Catch expected ValueError.
[VarDumper] ReflectionFunction::isDisabled() is deprecated.
[BrowserKit] Raw body with custom Content-Type header
[PropertyAccess] Parse php 8 TypeErrors correctly.
[Intl] Fix call to ReflectionProperty::getValue() for static properties.
[HttpKernel] Prevent calling method_exists() with non-string values.
Fix wrong roles comparison
...
* 5.1:
[PhpUnitBridge] fix leftover
[PhpUnitBridge] fix installing under PHP >= 8
Use ">=" for the "php" requirement
bump icu 67.1
[DI] Remove preload primitive types
[Validator] Add missing translations of nn locale
[HttpKernel] Fix that the `Store` would not save responses with the X-Content-Digest header present
[Intl] bump icu 67.1
[Validator] allow passing a validator to Validation::createCallable()
* 5.0:
[PhpUnitBridge] fix bad test
[4.4] CS fixes
[3.4] CS fixes
Disable phpunit verbosity
Queue name is a required parameter
[FrameworkBundle] display actual target for error in AssetsInstallCommand
Remove patches for Doctrine bugs and deprecations
[Mime] fix bad method call on "EmailAddressContains"
[DI][EventDispatcher] added contract for implementation
* 4.4:
[PhpUnitBridge] fix bad test
[4.4] CS fixes
[3.4] CS fixes
Disable phpunit verbosity
Queue name is a required parameter
[FrameworkBundle] display actual target for error in AssetsInstallCommand
Remove patches for Doctrine bugs and deprecations
[Mime] fix bad method call on "EmailAddressContains"
[DI][EventDispatcher] added contract for implementation
* 3.4:
[FrameworkBundle] display actual target for error in AssetsInstallCommand
Remove patches for Doctrine bugs and deprecations
[DI][EventDispatcher] added contract for implementation
* 5.0:
[PhpUnitBridge] add PolyfillTestCaseTrait::expectExceptionMessageMatches to provide FC with recent phpunit versions
[Messenger] Make sure redis transports are initialized correctly
Remove return type for Twig function workflow_metadata()
[DI] fix typo
* 4.4:
[PhpUnitBridge] add PolyfillTestCaseTrait::expectExceptionMessageMatches to provide FC with recent phpunit versions
[Messenger] Make sure redis transports are initialized correctly
Remove return type for Twig function workflow_metadata()
[DI] fix typo
* 5.0: (31 commits)
[HttpClient] NativeHttpClient should not send >1.1 protocol version
[HttpClient] fix support for non-blocking resource streams
[Mailer] Make sure you can pass custom headers to Mailgun
[Mailer] Remove line breaks in email attachment content
Update links to documentation
[Validator] Add the missing translations for the Arabic (ar) locale
ensure to expect no validation for the right reasons
[Security-Guard] fixed 35203 missing name tag in param docblock
[HttpClient] fix casting responses to PHP streams
[PhpUnitBridge] Add test case for @expectedDeprecation annotation
[PhpUnitBridge][SymfonyTestsListenerTrait] Remove $testsWithWarnings stack
[FrameworkBundle] Fix getUser() phpdoc in AbstractController
[Mailer] Fix addresses management in Sendgrid API payload
[Mailer][MailchimpBridge] Fix missing attachments when sending via Mandrill API
[Mailer][MailchimpBridge] Fix incorrect sender address when sender has name
[HttpClient] fix capturing SSL certificates with NativeHttpClient
Update year in license files
Update year in license files
[TwigBridge][Form] Added missing help messages in form themes
Update year in license files
...
* 5.0:
[Routing] fix tests
[DI] minor cleanup
[Form] group constraints when calling the validator
Remove wrong @group legacy annotations
[DependencyInjection] Fix dumping multiple deprecated aliases
allow button names to start with uppercase letter
Allow PHP ^7.2.5
States that the HttpClient provides a Http Async implementation
[Routing] Fix ContainerLoader and ObjectLoaderTest
[HttpKernel] Make ErrorListener::onKernelException()'s dispatcher argument explicit
[HttpKernel] Drop deprecated ExceptionListener
Removed extra whitespace
[Security] Fix best encoder not wired using migrate_from
* 4.4:
[travis] Fix build-packages script
Add types to constructors and private/final/internal methods (Batch III)
[HttpClient] Async HTTPlug client
[Messenger] Allow to configure the db index on Redis transport
[HttpClient] bugfix exploding values of headers
[VarDumper] Made all casters final
[VarDumper] Added a support for casting Ramsey/Uuid
Remove useless testCanCheckIfTerminalIsInteractive test case
[Validator] Add the missing translations for the Thai (\"th\") locale
[Routing] gracefully handle docref_root ini setting
[Validator] Fix ValidValidator group cascading usage
This PR was merged into the 5.0-dev branch.
Discussion
----------
[String] a new component for object-oriented strings management with an abstract unit system
| Q | A
| ------------- | ---
| Branch? | master
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| Tickets | -
| License | MIT
| Doc PR | -
This is a reboot of #22184 (thanks @hhamon for working on it) and a generalization of my previous work on the topic ([patchwork/utf8](https://github.com/tchwork/utf8)). Unlike existing libraries (including `patchwork/utf8`), this component provides a unified API for the 3 unit systems of strings: bytes, code points and grapheme clusters.
The unified API is defined by the `AbstractString` class. It has 2 direct child classes: `BinaryString` and `AbstractUnicodeString`, itself extended by `Utf8String` and `GraphemeString`.
All objects are immutable and provide clear edge-case semantics, using exceptions and/or (nullable) types!
Two helper functions are provided to create such strings:
```php
new GraphemeString('foo') == u('foo'); // when dealing with Unicode, prefer grapheme units
new BinaryString('foo') == b('foo');
```
`GraphemeString` is the most linguistic-friendly variant of them, which means it's the one ppl should use most of the time *when dealing with written text*.
Future ideas:
- improve tests
- add more docblocks (only where they'd add value!)
- consider adding more methods in the string API (`is*()?`, `*Encode()`?, etc.)
- first class Emoji support
- merge the Inflector component into this one
- use `width()` to improve `truncate()` and `wordwrap()`
- move method `slug()` to a dedicated locale-aware service class
- propose your ideas (send PRs after merge)
Out of (current) scope:
- what [intl](https://php.net/intl) provides (collations, transliterations, confusables, segmentation, etc)
Here is the unified API I'm proposing in this PR, borrowed from looking at many existing libraries, but also Java, Python, JavaScript and Go.
```php
function __construct(string $string = '');
static function unwrap(array $values): array
static function wrap(array $values): array
function after($needle, bool $includeNeedle = false, int $offset = 0): self;
function afterLast($needle, bool $includeNeedle = false, int $offset = 0): self;
function append(string ...$suffix): self;
function before($needle, bool $includeNeedle = false, int $offset = 0): self;
function beforeLast($needle, bool $includeNeedle = false, int $offset = 0): self;
function camel(): self;
function chunk(int $length = 1): array;
function collapseWhitespace(): self
function endsWith($suffix): bool;
function ensureEnd(string $suffix): self;
function ensureStart(string $prefix): self;
function equalsTo($string): bool;
function folded(): self;
function ignoreCase(): self;
function indexOf($needle, int $offset = 0): ?int;
function indexOfLast($needle, int $offset = 0): ?int;
function isEmpty(): bool;
function join(array $strings): self;
function jsonSerialize(): string;
function length(): int;
function lower(): self;
function match(string $pattern, int $flags = 0, int $offset = 0): array;
function padBoth(int $length, string $padStr = ' '): self;
function padEnd(int $length, string $padStr = ' '): self;
function padStart(int $length, string $padStr = ' '): self;
function prepend(string ...$prefix): self;
function repeat(int $multiplier): self;
function replace(string $from, string $to): self;
function replaceMatches(string $fromPattern, $to): self;
function slice(int $start = 0, int $length = null): self;
function snake(): self;
function splice(string $replacement, int $start = 0, int $length = null): self;
function split(string $delimiter, int $limit = null, int $flags = null): array;
function startsWith($prefix): bool;
function title(bool $allWords = false): self;
function toBinary(string $toEncoding = null): BinaryString;
function toGrapheme(): GraphemeString;
function toUtf8(): Utf8String;
function trim(string $chars = " \t\n\r\0\x0B\x0C\u{A0}\u{FEFF}"): self;
function trimEnd(string $chars = " \t\n\r\0\x0B\x0C\u{A0}\u{FEFF}"): self;
function trimStart(string $chars = " \t\n\r\0\x0B\x0C\u{A0}\u{FEFF}"): self;
function truncate(int $length, string $ellipsis = ''): self;
function upper(): self;
function width(bool $ignoreAnsiDecoration = true): int;
function wordwrap(int $width = 75, string $break = "\n", bool $cut = false): self;
function __clone();
function __toString(): string;
```
`AbstractUnicodeString` adds these:
```php
static function fromCodePoints(int ...$codes): self;
function ascii(array $rules = []): self;
function codePoint(int $index = 0): ?int;
function folded(bool $compat = true): parent;
function normalize(int $form = self::NFC): self;
function slug(string $separator = '-'): self;
```
and `BinaryString`:
```php
static function fromRandom(int $length = 16): self;
function byteCode(int $index = 0): ?int;
function isUtf8(): bool;
function toUtf8(string $fromEncoding = null): Utf8String;
function toGrapheme(string $fromEncoding = null): GraphemeString;
```
Case insensitive operations are done with the `ignoreCase()` method.
e.g. `b('abc')->ignoreCase()->indexOf('B')` will return `1`.
For reference, CLDR transliterations (used in the `ascii()` method) are defined here:
https://github.com/unicode-org/cldr/tree/master/common/transforms
Commits
-------
dd8745aced [String] add more tests
82a00956bc [String] add tests
012e92a772 [String] a new component for object-oriented strings management with an abstract unit system
* 4.4:
[Security/Http] fix typo in deprecation message
[Security] Deprecate isGranted()/decide() on more than one attribute
Fixed a minor typo in the UPGRADE to 5.0 guide
Various tweaks 3.4
Various tweaks 4.3
[Security] Make stateful firewalls turn responses private only when needed
[PhpUnit] Fix usleep mock return value
Revert \"feature #33507 [WebProfiler] Deprecated intercept_redirects in 4.4 (dorumd)\"
[TwigBundle] typo
[TwigBundle] fix test case
[Lock] use Predis\ClientInterface instead of Predis\Client
Allow Twig 3
Minor tweaks
Fix version typo in deprecation notice
[Form][SubmitType] Add "validate" option
hint to the --parse-tags when parsing tags fails
Make legacy "wrong" RFC2047 encoding apply only to one header
* 4.4:
Re-enable previously failing PHP 7.4 test cases
[PhpUnitBridge] fix uninitialized variable
[ErrorRenderer] fix Cannot use object of type ErrorException as array exception #33631
[Twig] Add missing check
Revert "bug #33618 fix tests depending on other components' tests (xabbuh)"
install from source to include components tests
Fix undefined constant and other minor issues
[Twig] Add NotificationEmail
ensure compatibility with type resolver 0.5
Call AssertEquals with proper parameters
[DependencyInjection] Allow binding iterable and tagged services
[Twig] Fix Twig config extra keys
fix tests depending on other components' tests
Fix lint commands frozen on empty stdin
* 4.4: (21 commits)
[appveyor] exclude tty group
[HttpFoundation] Add types to private/final/internal methods and constructors.
Add types to private/final/internal methods and constructors.
SCA: minor code tweaks
Tweak output
[FrameworkBundle] Added --sort option for TranslationUpdateCommand
[HttpClient] fallbackto CURLMOPT_MAXCONNECTS when CURLMOPT_MAX_HOST_CONNECTIONS is not available
[DI] generate preload.php file for PHP 7.4 in cache folder
Allow version 2 of the contracts package.
[Serializer] Allow multi-dimenstion object array in AbstractObjectNormalizer
fixed typo
[HttpKernel] Fix Apache mod_expires Session Cache-Control issue
deprecated not passing dash symbol (-) to STDIN commands
[VarDumper] display ellipsed FQCN for nested classes
[VarDumper] Display fully qualified title
[Mailer] Change the syntax for DSNs using failover or roundrobin
Removed workaround introduced in 4.3
[Console] Added support for definition list
[OptionsResolver] Display full nested options hierarchy in exceptions
New welcome page
...
* 4.4:
[MonologBridge] Bump min version for monolog ^1.25 and drop dead code
[Bridge/Twig] use tty group on testLintDefaultPaths
fix tests mocking final events
* 4.4:
[Routing] fix static route reordering when a previous dynamic route conflicts
conflict with HttpKernel 5
Return null as Expire header if it was set to null
bug #33370 Fix import statement typo in NullCache (adrienbrault)
[ProxyManager] remove ProxiedMethodReturnExpression polyfill
fix dumping not inlined scalar tag values
Fix import statement typo in NullCache
[DoctrineBridge] Allow configuring class names through methods instead of class parameters
* 4.3:
[Routing] fix static route reordering when a previous dynamic route conflicts
Return null as Expire header if it was set to null
bug #33370 Fix import statement typo in NullCache (adrienbrault)
[ProxyManager] remove ProxiedMethodReturnExpression polyfill
fix dumping not inlined scalar tag values
* 3.4:
Return null as Expire header if it was set to null
[ProxyManager] remove ProxiedMethodReturnExpression polyfill
fix dumping not inlined scalar tag values
Nicolas Grekas
Fabien Potencier
Grégoire Pineau
Niklas Keller
Christian Flothmann
Antonio Pauletich
Alexander M. Turek
DerManoMann
Grégoire Paris
Jérémy Derussé
soyuka
Nyholm