This PR was merged into the master branch.
Discussion
----------
New Component: Expression Language
| Q | A
| ------------- | ---
| Bug fix? | no
| New feature? | yes
| BC breaks? | no
| Deprecations? | no
| Tests pass? | yes
| Fixed tickets | #8850, #7352
| License | MIT
| Doc PR | not yet
TODO:
- [ ] write documentation
- [x] add tests for the new component
- [x] implement expression support for access rules in the security component
- [x] find a better character/convention for expressions in the YAML format
- [x] check the performance of the evaluation mode
- [x] better error messages in the evaluation mode
- [x] add support in the Routing
- [x] add support in the Validator
The ExpressionLanguage component provides an engine that can compile and
evaluate expressions.
An expression is a one-liner that returns a value (mostly, but not limited to, Booleans).
It is a strip-down version of Twig (only the expression part of it is
implemented.) Like Twig, the expression is lexed, parsed, and
compiled/evaluated. So, it is immune to external injections by design.
If we compare it to Twig, here are the main big differences:
* only support for Twig expressions
* no ambiguity for calls (foo.bar is only valid for properties, foo['bar'] is only valid for array calls, and foo.bar() is required for method calls)
* no support for naming conventions in method calls (if the method is named getFoo(), you must use getFoo() and not foo())
* no notion of a line for errors, but a cursor (we are mostly talking about one-liners here)
* removed everything specific to the templating engine (like output escaping or filters)
* no support for named arguments in method calls
* only one extension point with functions (no possibility to define new operators, ...)
* and probably even more I don't remember right now
* there is no need for a runtime environment, the compiled PHP string is self-sufficient
An open question is whether we keep the difference betweens arrays and hashes.
The other big difference with Twig is that it can work in two modes (possible
because of the restrictions described above):
* compilation: the expression is compiled to PHP and is self-sufficient
* evaluation: the expression is evaluated without being compiled to PHP (the node tree produced by the parser can be serialized and evaluated afterwards -- so it can be saved on disk or in a database to speed up things when needed)
Let's see a simple example:
```php
$language = new ExpressionLanguage();
echo $language->evaluate('1 + 1');
// will echo 2
echo $language->compile('1 + 2');
// will echo "(1 + 2)"
```
The language supports:
* all basic math operators (with precedence rules):
* unary: not, !, -, +
* binary: or, ||, and, &&, b-or, b-xor, b-and, ==, ===, !=, !==, <, >, >=, <=, not in, in, .., +, -, ~, *, /, %, **
* all literals supported by Twig: strings, numbers, arrays (`[1, 2]`), hashes
(`{a: "b"}`), Booleans, and null.
* simple variables (`foo`), array accesses (`foo[1]`), property accesses
(`foo.bar`), and method calls (`foo.bar(1, 2)`).
* the ternary operator: `true ? true : false` (and all the shortcuts
implemented in Twig).
* function calls (`constant('FOO')` -- `constant` is the only built-in
functions).
* and of course, any combination of the above.
The compilation is better for performances as the end result is just a plain PHP string without any runtime. For the evaluation, we need to tokenize, parse, and evaluate the nodes on the fly. This can be optimized by using a `ParsedExpression` or a `SerializedParsedExpression` instead:
```php
$nodes = $language->parse($expr, $names);
$expression = new SerializedParsedExpression($expr, serialize($nodes));
// You can now store the expression in a DB for later reuse
// a SerializedParsedExpression can be evaluated like any other expressions,
// but under the hood, the lexer and the parser won't be used at all, so it''s much faster.
$language->evaluate($expression);
```
That's all folks!
I can see many use cases for this new component, and we have two use cases in
Symfony that we can implement right away.
## Using Expressions in the Service Container
The first one is expression support in the service container (it would replace
#8850) -- anywhere you can pass an argument in the service container, you can
use an expression:
```php
$c->register('foo', 'Foo')->addArgument(new Expression('bar.getvalue()'));
```
You have access to the service container via `this`:
container.get("bar").getvalue(container.getParameter("value"))
The implementation comes with two functions that simplifies expressions
(`service()` to get a service, and `parameter` to get a parameter value). The
previous example can be simplified to:
service("bar").getvalue(parameter("value"))
Here is how to use it in XML:
```xml
<parameters>
<parameter key="value">foobar</parameter>
</parameters>
<services>
<service id="foo" class="Foo">
<argument type="expression">service('bar').getvalue(parameter('value'))</argument>
</service>
<service id="bar" class="Bar" />
</services>
```
and in YAML (I chose the syntax randomly ;)):
```yaml
parameters:
value: foobar
services:
bar:
class: Bar
foo:
class: Foo
arguments: [@=service("bar").getvalue(parameter("value"))]
```
When using the container builder, Symfony uses the evaluator, but with the PHP
dumper, the compiler is used, and there is no overhead as the expression
engine is not needed at runtime. The expression above would be compiled to:
```php
$this->get("bar")->getvalue($this->getParameter("value"))
```
## Using Expression for Security Access Control Rules
The second use case in Symfony is for access rules.
As we all know, the way to configure the security access control rules is confusing, which might lead to insecure applications (see http://symfony.com/blog/security-access-control-documentation-issue for more information).
Here is how the new `allow_if` works:
```yaml
access_control:
- { path: ^/_internal/secure, allow_if: "'127.0.0.1' == request.getClientIp() or has_role('ROLE_ADMIN')" }
```
This one restricts the URLs starting with `/_internal/secure` to people browsing from the localhost. Here, `request` is the current Request instance. In the expression, there is access to the following variables:
* `request`
* `token`
* `user`
And to the following functions:
* `is_anonymous`
* `is_authenticated`
* `is_fully_authenticated`
* `is_rememberme`
* `has_role`
You can also use expressions in Twig, which works well with the `is_granted` function:
```jinja
{% if is_granted(expression('has_role("FOO")')) %}
...
{% endif %}
```
## Using Expressions in the Routing
Out of the box, Symfony can only match an incoming request based on some pre-determined variables (like the path info, the method, the scheme, ...). But some people want to be able to match on more complex logic, based on other information of the Request object. That's why we introduced `RequestMatcherInterface` recently (but we no default implementation in Symfony itself).
The first change I've made (not related to expression support) is implement this interface for the default `UrlMatcher`. It was simple enough.
Then, I've added a new `condition` configuration for Route objects, which allow you to add any valid expression. An expression has access to the `request` and to the routing `context`.
Here is how one would configure it in a YAML file:
```yaml
hello:
path: /hello/{name}
condition: "context.getMethod() in ['GET', 'HEAD'] and request.headers.get('User-Agent') =~ '/firefox/i'"
```
Why do I keep the context as all the data are also available in the request? Because you can also use the condition without using the RequestMatcherInterface, in which case, you don't have access to the request. So, the previous example is equivalent to:
```yaml
hello:
path: /hello/{name}
condition: "request.getMethod() in ['GET', 'HEAD'] and request.headers.get('User-Agent') =~ '/firefox/i'"
```
When using the PHP dumper, there is no overhead as the condition is compiled. Here is how it looks like:
```php
// hello
if (0 === strpos($pathinfo, '/hello') && preg_match('#^/hello/(?P<name>[^/]++)$#s', $pathinfo, $matches) && (in_array($context->getMethod(), array(0 => "GET", 1 => "HEAD")) && preg_match("/firefox/i", $request->headers->get("User-Agent")))) {
return $this->mergeDefaults(array_replace($matches, array('_route' => 'hello')), array ());
}
```
Be warned that conditions are not taken into account when generating a URL.
## Using Expressions in the Validator
There is a new Expression constraint that you can put on a class. The expression is then evaluated for validation:
```php
use Symfony\Component\Validator\Constraints as Assert;
/**
* @Assert\Condition(condition="this.getFoo() == 'fo'", message="Not good!")
*/
class Obj
{
public function getFoo()
{
return 'foo';
}
}
```
In the expression, you get access to the current object via the `this` variable.
## Dynamic annotations
The expression language component is also very useful in annotations. the SensoLabs FrameworkExtraBundle leverages this possibility to implement HTTP validation caching in the `@Cache` annotation and to add a new `@Security` annotation (see sensiolabs/SensioFrameworkExtraBundle#238.)
Commits
-------
d4ebbfd [Validator] Renamed Condition to Expression and added possibility to set it onto properties
a3b3a78 [Validator] added a constraint that runs an expression
1bcfb40 added optimized versions of expressions
984bd38 mades things more consistent for the end user
d477f15 [Routing] added support for expression conditions in routes
86ac8d7 [ExpressionLanguage] improved performance
e369d14 added a Twig extension to create Expression instances
38b7fde added support for expression in control access rules
2777ac7 [HttpFoundation] added ExpressionRequestMatcher
c25abd9 [DependencyInjection] added support for expressions in the service container
3a41781 [ExpressionLanguage] added support for regexes
9d98fa2 [ExpressionLanguage] added the component
As explained in #6775, this has been done for the following reasons:
1. It's also Request::getHost()
2. The term hostname has been obsoleted in
http://tools.ietf.org/html/rfc3986#appendix-D.2 and uses the host only
3. hostname in the RFC was defined as the registered domain name, but we
probably also want to match IP-Adresses with the pattern which is the
host = IP-literal / IPv4address / reg-name for.
This PR was merged into the master branch.
Commits
-------
9fc7def added the UPGRADE file for Symfony 3.0
e84cad2 [Routing] updated CHANGELOG
65eca8a [Routing] added new schemes and methods options to the annotation loader
5082994 [Routing] renamed pattern to path
b357caf [Routing] renamed hostname pattern to just hostname
e803f46 made schemes and methods available in XmlFileLoader
d374e70 made schemes and methods available in YamlFileLoader
2834e7e added scheme and method setter in RouteCollection
10183de make scheme and method requirements first-class citizen in Route
Discussion
----------
Routing options
| Q | A
| ------------- | ---
| Bug fix? | no
| New feature? | no
| BC breaks? | no
| Deprecations? | yes
| Tests pass? | yes
| Fixed tickets | #5989, #5990, #6049
| License | MIT
In #5989, it has unanimously been decided to renamed `hostname_pattern` to `hostname` and `pattern` to `path`. That makes a lot of sense and I would like to do the renaming now as `hostname_pattern` is new in Symfony 2.2, so I'd like to avoid breaking BC just after the release. As we are modifying the route options, I've also included changes introduced by @Tobion in #6049 which were discussed in #5990.
As everything is BC, I think it's wise to include that in 2.2. What do you think?
---------------------------------------------------------------------------
by Tobion at 2013-01-14T18:25:53Z
I agree it should be done in 2.2. Thanks for working on it.
---------------------------------------------------------------------------
by vicb at 2013-01-14T23:11:12Z
@fabpot "Everything is BC" until it breaks BC in 3.0, that's why I'd like to see [deprecations in PR summary](https://github.com/symfony/symfony-docs/pull/2116) what do you think ?
---------------------------------------------------------------------------
by vicb at 2013-01-14T23:16:40Z
it would also be great to update the CHANGELOG with deprecations (it could also help people answering your question)
---------------------------------------------------------------------------
by fabpot at 2013-01-15T07:07:03Z
@vicb: I've just updated the CHANGELOG and created the UPGRADE file for 3.0.
---------------------------------------------------------------------------
by vicb at 2013-01-15T07:15:32Z
@fabpot thanks.
This PR was merged into the master branch.
Commits
-------
51223c0 added upgrade instructions
50e6259 adjusted tests
98f3ca8 [Routing] removed tree structure from RouteCollection
Discussion
----------
[Routing] removed tree structure from RouteCollection
BC break: yes (see below)
Deprecations: RouteCollection::getParent(); RouteCollection::getRoot()
tests pass: yes
The reason for this is so quite simple. The RouteCollection has been designed as a tree structure, but it cannot at all be used as one. There is no getter for a sub-collection at all. So you cannot access a sub-collection after you added it to the tree with `addCollection(new RouteCollection())`. In contrast to the form component, e.g. `$form->get('child')->get('grandchild')`.
So you can see the RouteCollection cannot be used as a tree and it should not, as the same can be achieved with a flat array!
Using a flat array removes all the need for recursive traversal and makes the code much faster, much lighter, less memory (big problem in CMS with many routes) and less error-prone.
BC break: there is only a BC break if somebody used the PHP API for defining RouteCollection and also added a Route to a collection after it has been added to another collection.
So
```
$rootCollection = new RouteCollection();
$subCollection = new RouteCollection();
$rootCollection->addCollection($subCollection);
$subCollection->add('foo', new Route('/foo'));
```
must be updated to the following (otherwise the 'foo' Route is not imported to the rootCollection)
```
$rootCollection = new RouteCollection();
$subCollection = new RouteCollection();
$subCollection->add('foo', new Route('/foo'));
$rootCollection->addCollection($subCollection);
```
Also one must call addCollection from the bottom to the top. So the correct sequence is the following (and not the reverse)
```
$childCollection->->addCollection($grandchildCollection);
$rootCollection->addCollection($childCollection);
```
Remeber, this is only needed when using PHP for defining routes and calling methods in a special order. There is no change required when using XML or YAML for definitions. Also, I'm pretty sure that neither the CMF, nor Drupal routing, nor Silex is relying on the tree stuff. So they should also still work.
cc @fabpot @crell @dbu
One more thing: RouteCollection wasn't an appropriate name for a tree anyway as a collection of routes (that it now is) is definitely not a tree.
Yet another point: The XML declaration of routes uses the `<import>` element, which is excatly what the new implementation of addCollection without the need of a tree does. So this is now also more analogous.
---------------------------------------------------------------------------
by Koc at 2012-11-26T17:34:15Z
What benefit of this?
---------------------------------------------------------------------------
by Tobion at 2012-11-26T17:56:53Z
@Koc Why did you not simply wait for the description? ^^
---------------------------------------------------------------------------
by dbu at 2012-11-26T18:33:09Z
i love PR that remove more code than they add whithout removing functionality.
---------------------------------------------------------------------------
by Crell at 2012-11-26T18:49:52Z
There's an issue somewhere in Drupal where we're trying to use addCollection() as a shorthand for iterating over one collection and calling add() on the other for each item. We can't do that, however, because the subcollections are not flattened properly when reading back and our current dumper can't cope with that. So this change would not harm Drupal at all, and would mean I don't have fix a bug in our dumper. :-) I cannot speak for any other projects, of course.
---------------------------------------------------------------------------
by Tobion at 2012-11-27T19:06:34Z
Ok, this is ready.
This PR was merged into the master branch.
Commits
-------
824a0f3 [Routing] compatibility with older PCRE (pre 8)
Discussion
----------
[Routing] compatibility with older PCRE (pre 8)
#6062 for master
we don't need the logic to merge numeric keys, as we don't have them. I could also improve the genrated code by PhpMatcherDumper a little by saving a function call.
This PR was merged into the master branch.
Commits
-------
c7a8f7a [Routing] fixed possible parameters conflict in apache url matcher
Discussion
----------
[Routing] fixed possible parameters conflict in apache url matcher
Bug fix: yes
Feature addition: no
Backwards compatibility break: no (as long as rewrite rules are generated after upgrading)
Symfony2 tests pass: yes
- This fixes a conflict in route parameters:
The rewrite rules currently pass route informations through environment variables:
`_ROUTING_DEFAULT_x`: passes the default value of parameter x
`_ROUTING__allow_x`: passes the information that method x was allowed for this route
`_ROUTING_x`: passes the value of parameter x
The problem is that naming a route parameter `DEFAULT_*` or `_allow_*` would not behave as expected.
I fixed this by namespacing all environment variables; e.g. parameters are in `_ROUTING_param_*`, defaults in `_ROUTING_default_*`, etc.
- The PR fixes a second issue: sometimes the variables are prefixed with multiple REDIRECT_. This PR handles this case by ignoring them all.
- This also improves performance a little:
Matching a route with two parameters and two default parameters 100K times: (`$_SERVER` was copied from a real request, so with many non `_ROUTING_` variables)
master: 6.6s
this branch: 4.7s
---------------------------------------------------------------------------
by fabpot at 2012-10-27T13:37:24Z
Any news on this PR? Is it mergeable?
---------------------------------------------------------------------------
by arnaud-lb at 2012-10-27T14:50:08Z
There is an issue with default parameter values, I can't find how to fix that in a simple way. Before this PR, default values are never used (if a parameter is an optional not present in the url, the parameter's value is the empty string); after this PR, when a parameter is present and empty (e.g. a requirement like `.*`), its value is set to its default value.
---------------------------------------------------------------------------
by Tobion at 2012-10-29T01:36:08Z
The problem is, it's not consistent with the default php matcher. So one cannot safely exchange it with the apache matcher because it behaves differently under some (special) circumstances.
---------------------------------------------------------------------------
by fabpot at 2012-11-05T08:05:54Z
We need to move forward as I want to merge the hostname support in the routing ASAP to have plenty of time for feedback before the 2.2 release.
Does it sound reasonable to merge this PR as is an open a ticket about the remaining issue (which should not occur that often anyways)?
---------------------------------------------------------------------------
by arnaud-lb at 2012-11-05T09:22:02Z
@fabpot it sounds reasonable to me. Also, I've the hostname support branch is currently rebased so that it can be merged without this one.
---------------------------------------------------------------------------
by Tobion at 2012-11-11T21:50:20Z
Btw, does the ApacheMatcherDumper handle the _scheme requirement? It doesn't look like it. This would be another bug.
Anyway, we can probably merge this PR and open new issues for the remaining bugs.
* 2.1:
[ClassLoader] fixed unbracketed namespaces (closes#5747)
slight refactoring in UrlMatcher
[Form] Created test for DoctrineOrmTypeGuesser see #5790
[Form] Fixed DoctrineOrmTypeGuesser to guess the "required" option for to-one associations
Commits
-------
7464dcd added phpdoc
c413e7b [Routing] remove RequestContextAwareInterface from RequestMatcherInterface
921be34 [Routing] fix phpdoc
Discussion
----------
[Routing] RequestMatcherInterface doesn't need context
Matchers that implement RequestMatcherInterface should match a Request, thus they don't need the request context.
---------------------------------------------------------------------------
by travisbot at 2012-06-14T21:39:48Z
This pull request [fails](http://travis-ci.org/symfony/symfony/builds/1624496) (merged f5ff1fe0 into 7c91ee57).
---------------------------------------------------------------------------
by schmittjoh at 2012-06-15T13:32:59Z
I think it makes sense to remove the RequestContext from the RequestMatcher.
---------------------------------------------------------------------------
by travisbot at 2012-06-15T15:54:28Z
This pull request [passes](http://travis-ci.org/symfony/symfony/builds/1628931) (merged 7464dcd2 into f881d282).
---------------------------------------------------------------------------
by Tobion at 2012-06-26T12:32:06Z
Anything missing?
