Commits
-------
41b7a19 Updated the tests so that tests will be marked as skipped when there is no MongoDB server present!
233c7db Updated the code to follow the symfony coding standards
7b24de5 Updated the code to follow the symfony coding standard using stof his remarks
fbcbdde - Fixed a small bug - Updated some phpdoc
00fdfec Added a MongoDbProfilerStorage engine
Discussion
----------
[2.1] [HttpKernel] MongoDb storage for Profiler
As a documentbased database like MongoDB is [supposedly fantastic in logging](http://blog.mongodb.org/post/172254834/mongodb-is-fantastic-for-logging) I implemented a storage engine for the profiler that should enable us to use this database as storage for this.
Activate it using this way:
framework:
profiler:
dsn: mongodb://user:pass@location/database/collection
---------------------------------------------------------------------------
by stof at 2011/07/24 11:23:06 -0700
btw, the MongoDB session storage has already be rejected from the core so this should probably be moved to the DoctrineMongoDBBundle (even if it uses only the PHP extension and not Doctrine). @fabpot thoughts about this ?
---------------------------------------------------------------------------
by Wotre at 2011/07/24 11:52:56 -0700
Just my personal opinion, if it is prefered that way I will move this into the DoctrineMongoDBBundle.
While it is reasonable to bundle all Mongo related things together, I do believe that in the case of logging we want to avoid as many depencies as possible. Some exceptions can occur pretty early inside the framework, and it would be a shame if those aren't logged because this layer is written on top of doctrine. I'm not exactly familliar enough with the symfony internals as I only started using it a few days ago, but I can imagine that this can make a difference with some exceptions.
---------------------------------------------------------------------------
by stof at 2011/07/24 11:59:10 -0700
I don't ask you to use Doctrine in this code. It is fine to use the extension directly if it is enough.
Btw, the profiler is *not* used early. :)
---------------------------------------------------------------------------
by Wotre at 2011/07/26 10:45:05 -0700
So... Any final remark whether this should be moved to [the DoctrineMongoDBBundle](https://github.com/symfony/DoctrineMongoDBBundle) or not?
If it has to be moved, any comment on where in that bundle this should be put?
Also, if it has to be moved, how can we arrange the configuration using DI? Currently I've put a line in the FrameworkExtension file to use this engine for everything with a $dsn starting with mongodb; I imagine this kind of ugly depency can't really exist between the FrameworkBundle and another one.
Although it seems completely illogical to me, I will move it, but I do need some directions on how to elegantly do this...
---------------------------------------------------------------------------
by stof at 2011/07/26 11:03:04 -0700
@fabpot what do you think ?
---------------------------------------------------------------------------
by stof at 2011/09/04 01:28:48 -0700
@fabpot what do you think about the place where this should be done ?
Commits
-------
9f0bd03 [HttpKernel] Update tests for FileProfilerStorage
b7032bc [HttpKernel] Update FileProfileStorage to search from EOF
188a5fa [HttpKernel] Override the existing tokens in FileProfilerStorage
b1b1424 [HttpKernel] Delete folders in the profiler cache
88bc3ec [HttpKernel] Fixes standards of FileProfilerStorage
affe66c Merge remote-tracking branch 'origin/master' into new-profiler-storage
ea916c3 [HttpKernel] Coding convention for the file profiler storage
9ae2c8d [HttpKernel] CS in file storage
b415efd [HttpKernel] Add a test for semicolon in file storage test
1c1215f [HttpKernel] Use subfolders for better storage in file storage of profiler
4b1dc1f [HttpKernel] Fix the folder attribute of file storage to private
70f73e1 [HttpKernel] Fix tests for the file storage of profiler
d5313d9 [HttpKernel] Add tests for the file profiler storage
09fc0a2 [HttpKernel] Add Symfony credits to the file storage class for the profiler
d1d5892 [HttpKernel] Finalize the file storage for the profiler
2f65cf2 Add POC for file storage system
Discussion
----------
[2.1] [HttpKernel] File storage for profiler
Symfony2 has some problems when dealing with multiple concurrency queries in the SQLite storage, resulting in a timeout error or terrible lack.
I've implemented after discussions with @fabpot a filesystem storage.
Enable it in your project with :
framework:
profiler:
dsn: "file:%kernel.cache_dir%/profiler"
I also studied the possibility to store only big data string in files and rest in the SQLite, but not concluant.
Results of my measures (4 concurrency, 120 total) :
* SQLite with data : 1057ms
* SQLite without data : 615ms
* MySQL : 40ms
* This File storage : 54ms
---------------------------------------------------------------------------
by alexandresalome at 2011/07/22 12:01:10 -0700
An idea for the find method : a csv file containing ip;url;token
The iteration could be done over a big file, without loading the whole file in memory.
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:22:32 -0700
OK new version, with as explained previously : a CSV file containing the index + file for each profile.
The speed is similar to the speed of MySQL, and no memory overhead should occur with this solution.
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:37:14 -0700
Hm... Created tests, duplicated from SqliteProfilerStorageTest.
Any idea on how to put this code in common ? Is it usual to create a base class for 2 tests ?
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:48:39 -0700
Just tested with 24.000 requests, the 24.001'th request still takes less than 50ms to execute.
The index file is about 2Mb, and iterating the whole file is fast.
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:53:19 -0700
I've filled the file with 120Mb of data, requests are still less than 50ms for executing.
Iterating the index takes more than 30s (so it crashed), but it's because of the amount of lines. 30 seconds = 1,400,000 lines in this computer. The file = 1,500,000 lines
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:56:54 -0700
I've tested it with Linux, is someone can test with Windows
---------------------------------------------------------------------------
by stloyd at 2011/07/24 00:32:32 -0700
IMO to speedup it a bit more and not end up with "crash" (to not end with "limit" of files per directory, also to many files in dir slow down every OS) you should use same method to write as Twig, split up files in to directories. If you do this you can speed up index more, because you can create smaller one per directory.
Also you should fix CS (coding standards).
---------------------------------------------------------------------------
by stloyd at 2011/07/24 02:10:20 -0700
Tested on Win 7, seems ok. Similar speed to sqlite, dunno why ;-) but used a bit less of memory.
---------------------------------------------------------------------------
by alexandresalome at 2011/07/24 02:13:21 -0700
Did you tried with concurrent requests ? It makes sense when you use assetic
and your browser hits the application 4 times simultaneously for CSS
generation
---------------------------------------------------------------------------
by alexandresalome at 2011/07/24 02:17:23 -0700
I used Apache Benchmark for producing results :
ab -c 4 -n 120 URL
---------------------------------------------------------------------------
by alexandresalome at 2011/07/24 02:56:55 -0700
OK I used subfolders, based on last characters (because the first part of token is mostly the same between queries.
---------------------------------------------------------------------------
by stof at 2011/09/04 01:27:15 -0700
@fabpot any news about it ? Can it be merged ?
This commit also fixes exception pages when Twig is not enabled as a templating engine.
Instead of just displaying the raw Twig template as before, we now fallback to the default
exception handler introduced some time ago.
Commits
-------
9bcce9f fix tests
fc4787a fix non-extensible router
Discussion
----------
Router fix
Right now, the router is hard to overwrite (you need always a compiler pass). This commit fixes this.
---------------------------------------------------------------------------
by fabpot at 2011/07/18 01:15:36 -0700
Why do you need a complier pass to override the router?
---------------------------------------------------------------------------
by schmittjoh at 2011/07/18 01:47:47 -0700
How would you suggest to overwrite it?
Basically, I want to do something like this:
```yml
services:
router:
parent: router.default
class: MyClass
calls:
- [moreDeps, []]
```
---------------------------------------------------------------------------
by Seldaek at 2011/07/18 05:07:19 -0700
Then maybe we should somehow support redefining services with the same name while keeping the old one as parent, otherwise we need this foo.default for every service out there?
---------------------------------------------------------------------------
by fabpot at 2011/07/18 06:30:34 -0700
as @Seldeak said, why do that for the router and not all services?
---------------------------------------------------------------------------
by schmittjoh at 2011/07/18 06:38:39 -0700
I have designed the SecurityBundle this way where extension is encouraged.
---------------------------------------------------------------------------
by schmittjoh at 2011/07/18 11:15:57 -0700
I should add that this is mainly a problem for services where you still want to use the semantic configuration that is provided by the bundle. For services which are not configured by the extension, this is not so much of an issue.
Anyway, if you don't want to merge it, just close the PR. I have no problem with using a compiler pass.
---------------------------------------------------------------------------
by fabpot at 2011/07/18 11:55:11 -0700
We already have such a case with translator and translator.real. I will review the existing services to see where it makes sense to implement the same strategy.
---------------------------------------------------------------------------
by Seldaek at 2011/07/18 12:20:55 -0700
I guess you'd do it anyway, but we should pick a winner between .real and .default
---------------------------------------------------------------------------
by lsmith77 at 2011/07/18 12:26:52 -0700
I would prefer ".default" as ".real" always confused me.
The current implementation is not ready for inclusion in 2.0. It has several
known problems (security, not possible to disable it, not "cloud-compatible",
...) and it's not a must have feature anyway.
Some references:
* Security issue in FileType: https://github.com/symfony/symfony/issues/1001
* Validation fails on file, still stored in TemporaryStorage: https://github.com/symfony/symfony/issues/908
* Add a size argument & ability to configure TemporaryStorage: https://github.com/symfony/symfony/pull/748
This feature should be reworked and discussed for inclusion in 2.1.
Here are the new simplified rules:
* Required cache warmers are *always* executed when the Kernel boots for the first time;
* Optional cache warmers are *only* executed from the CLI via cache:warmup
These new rules means that all the configuration settings for the cache
warmers have been removed. So, if you want the best performance, remember to
warmup the cache when going to production.
This also fixed quite a few bugs.
* Seldaek/events:
[EventDispatcher] Removed temporary code
[FrameworkBundle] Improved code readability
[FrameworkBundle] Clarified code and fixed regression
Update Core and Security events to latest model
[EventDispatcher] Allow registration of arbitrary callbacks
[EventDispatcher] Remove useless code
[EventDispatcher] Minor memory optimization to getListeners()
[FrameworkBundle] Small optimization, remove some function calls
This in effect removes the direct link between event name and the method name on the handler.
Any callback can be given as a handler and the event name becomes an arbitrary string. Allowing for easier namespacing (see next commit)
