Commits
-------
41b7a19 Updated the tests so that tests will be marked as skipped when there is no MongoDB server present!
233c7db Updated the code to follow the symfony coding standards
7b24de5 Updated the code to follow the symfony coding standard using stof his remarks
fbcbdde - Fixed a small bug - Updated some phpdoc
00fdfec Added a MongoDbProfilerStorage engine
Discussion
----------
[2.1] [HttpKernel] MongoDb storage for Profiler
As a documentbased database like MongoDB is [supposedly fantastic in logging](http://blog.mongodb.org/post/172254834/mongodb-is-fantastic-for-logging) I implemented a storage engine for the profiler that should enable us to use this database as storage for this.
Activate it using this way:
framework:
profiler:
dsn: mongodb://user:pass@location/database/collection
---------------------------------------------------------------------------
by stof at 2011/07/24 11:23:06 -0700
btw, the MongoDB session storage has already be rejected from the core so this should probably be moved to the DoctrineMongoDBBundle (even if it uses only the PHP extension and not Doctrine). @fabpot thoughts about this ?
---------------------------------------------------------------------------
by Wotre at 2011/07/24 11:52:56 -0700
Just my personal opinion, if it is prefered that way I will move this into the DoctrineMongoDBBundle.
While it is reasonable to bundle all Mongo related things together, I do believe that in the case of logging we want to avoid as many depencies as possible. Some exceptions can occur pretty early inside the framework, and it would be a shame if those aren't logged because this layer is written on top of doctrine. I'm not exactly familliar enough with the symfony internals as I only started using it a few days ago, but I can imagine that this can make a difference with some exceptions.
---------------------------------------------------------------------------
by stof at 2011/07/24 11:59:10 -0700
I don't ask you to use Doctrine in this code. It is fine to use the extension directly if it is enough.
Btw, the profiler is *not* used early. :)
---------------------------------------------------------------------------
by Wotre at 2011/07/26 10:45:05 -0700
So... Any final remark whether this should be moved to [the DoctrineMongoDBBundle](https://github.com/symfony/DoctrineMongoDBBundle) or not?
If it has to be moved, any comment on where in that bundle this should be put?
Also, if it has to be moved, how can we arrange the configuration using DI? Currently I've put a line in the FrameworkExtension file to use this engine for everything with a $dsn starting with mongodb; I imagine this kind of ugly depency can't really exist between the FrameworkBundle and another one.
Although it seems completely illogical to me, I will move it, but I do need some directions on how to elegantly do this...
---------------------------------------------------------------------------
by stof at 2011/07/26 11:03:04 -0700
@fabpot what do you think ?
---------------------------------------------------------------------------
by stof at 2011/09/04 01:28:48 -0700
@fabpot what do you think about the place where this should be done ?
Commits
-------
9f0bd03 [HttpKernel] Update tests for FileProfilerStorage
b7032bc [HttpKernel] Update FileProfileStorage to search from EOF
188a5fa [HttpKernel] Override the existing tokens in FileProfilerStorage
b1b1424 [HttpKernel] Delete folders in the profiler cache
88bc3ec [HttpKernel] Fixes standards of FileProfilerStorage
affe66c Merge remote-tracking branch 'origin/master' into new-profiler-storage
ea916c3 [HttpKernel] Coding convention for the file profiler storage
9ae2c8d [HttpKernel] CS in file storage
b415efd [HttpKernel] Add a test for semicolon in file storage test
1c1215f [HttpKernel] Use subfolders for better storage in file storage of profiler
4b1dc1f [HttpKernel] Fix the folder attribute of file storage to private
70f73e1 [HttpKernel] Fix tests for the file storage of profiler
d5313d9 [HttpKernel] Add tests for the file profiler storage
09fc0a2 [HttpKernel] Add Symfony credits to the file storage class for the profiler
d1d5892 [HttpKernel] Finalize the file storage for the profiler
2f65cf2 Add POC for file storage system
Discussion
----------
[2.1] [HttpKernel] File storage for profiler
Symfony2 has some problems when dealing with multiple concurrency queries in the SQLite storage, resulting in a timeout error or terrible lack.
I've implemented after discussions with @fabpot a filesystem storage.
Enable it in your project with :
framework:
profiler:
dsn: "file:%kernel.cache_dir%/profiler"
I also studied the possibility to store only big data string in files and rest in the SQLite, but not concluant.
Results of my measures (4 concurrency, 120 total) :
* SQLite with data : 1057ms
* SQLite without data : 615ms
* MySQL : 40ms
* This File storage : 54ms
---------------------------------------------------------------------------
by alexandresalome at 2011/07/22 12:01:10 -0700
An idea for the find method : a csv file containing ip;url;token
The iteration could be done over a big file, without loading the whole file in memory.
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:22:32 -0700
OK new version, with as explained previously : a CSV file containing the index + file for each profile.
The speed is similar to the speed of MySQL, and no memory overhead should occur with this solution.
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:37:14 -0700
Hm... Created tests, duplicated from SqliteProfilerStorageTest.
Any idea on how to put this code in common ? Is it usual to create a base class for 2 tests ?
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:48:39 -0700
Just tested with 24.000 requests, the 24.001'th request still takes less than 50ms to execute.
The index file is about 2Mb, and iterating the whole file is fast.
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:53:19 -0700
I've filled the file with 120Mb of data, requests are still less than 50ms for executing.
Iterating the index takes more than 30s (so it crashed), but it's because of the amount of lines. 30 seconds = 1,400,000 lines in this computer. The file = 1,500,000 lines
---------------------------------------------------------------------------
by alexandresalome at 2011/07/23 14:56:54 -0700
I've tested it with Linux, is someone can test with Windows
---------------------------------------------------------------------------
by stloyd at 2011/07/24 00:32:32 -0700
IMO to speedup it a bit more and not end up with "crash" (to not end with "limit" of files per directory, also to many files in dir slow down every OS) you should use same method to write as Twig, split up files in to directories. If you do this you can speed up index more, because you can create smaller one per directory.
Also you should fix CS (coding standards).
---------------------------------------------------------------------------
by stloyd at 2011/07/24 02:10:20 -0700
Tested on Win 7, seems ok. Similar speed to sqlite, dunno why ;-) but used a bit less of memory.
---------------------------------------------------------------------------
by alexandresalome at 2011/07/24 02:13:21 -0700
Did you tried with concurrent requests ? It makes sense when you use assetic
and your browser hits the application 4 times simultaneously for CSS
generation
---------------------------------------------------------------------------
by alexandresalome at 2011/07/24 02:17:23 -0700
I used Apache Benchmark for producing results :
ab -c 4 -n 120 URL
---------------------------------------------------------------------------
by alexandresalome at 2011/07/24 02:56:55 -0700
OK I used subfolders, based on last characters (because the first part of token is mostly the same between queries.
---------------------------------------------------------------------------
by stof at 2011/09/04 01:27:15 -0700
@fabpot any news about it ? Can it be merged ?
Commits
-------
cc098a3 [HttpKernel] Add support for xdebug.file_link_format to Debug\ExceptionHandler.php
Discussion
----------
[HttpKernel] Add support for xdebug.file_link_format to Debug\ExceptionHandler
Format file and line as url, if xdebug.file_link_format is set. Inspired by #1893
If not, as classes can be loaded during the boot, there is no way to be sure that
a class will not be already loaded by a third party bundle.
If the Kernel is already booted, we don't included the compiled classes.
Commits
-------
5e80c68 fixes a naming inconsistency
8cfca15 added change to upgrade file
4123ec4 updated some missing references
Discussion
----------
Fix inconsistent naming
---------------------------------------------------------------------------
by jalliot at 2011/07/15 08:15:01 -0700
I think you forgot one commit (the one effectively changing Session and that you reverted in the main repo)
---------------------------------------------------------------------------
by schmittjoh at 2011/07/15 09:07:17 -0700
You're right, fixed now.
Commits
-------
11369eb Fixed phpdoc
dbe1854 Added a AccessDeniedHttpException to wrap the AccessDeniedException.
Discussion
----------
Added a AccessDeniedHttpException to wrap the AccessDeniedException.
This is a proposal to fix#1631
It wraps the AccessDeniedException in an AccessDeniedHttpException when the firewall is not able to handle it itself. This allows getting a 403 response using the standard exception listener in this case.
Note that the app should not throw the AccessDeniedHttpException itself but keep using the AccessDeniedException to let the Security component check if the user is already fully authenticated or if it should give a chance to authenticate.
---------------------------------------------------------------------------
by fabpot at 2011/07/11 07:10:12 -0700
For reference, I've tried something more radical some time ago here: https://github.com/symfony/symfony/pull/369.
---------------------------------------------------------------------------
by stof at 2011/07/11 07:22:07 -0700
my implementation is what @schmittjoh suggested in the comments on your PR.
Commits
-------
f7d0f65 RFC2616 changes
b9a218a [HttpFoundation] set Content-Length header to the length of content
Discussion
----------
[HttpFoundation] set Content-Length header to the length of content
I can't think of why this could be bad but if somebody knows please chime in.
The good thing is that with this change keepalive will work out of the box.
---------------------------------------------------------------------------
by Seldaek at 2011/07/06 05:34:51 -0700
That sounds like a great change. I think it might explain/fix the issues I've encountered with AppCache on my production box. Never had time to look into it, but IIRC I noticed the missing Content-Length, and it seemed to load forever.
---------------------------------------------------------------------------
by fabpot at 2011/07/06 06:46:50 -0700
The `Content-Length` is automatically added by servers like Apache. Moreover, sometimes, you should not add it: http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4
---------------------------------------------------------------------------
by lenar at 2011/07/06 07:54:45 -0700
It is not added automatically by default. Yes, in case of Apache it is actually added if deflate module is enabled and if that module decides to compress the content (decision based on content-type).
About RFC2616: I will read it and add changes to this PR if applicable.
---------------------------------------------------------------------------
by fabpot at 2011/07/06 08:38:14 -0700
e943fde2ef
---------------------------------------------------------------------------
by Seldaek at 2011/07/06 08:45:22 -0700
@lenar all you have to do is skip setting the Content-Length for `1xx`, `204`, and `304` responses I believe.
---------------------------------------------------------------------------
by Seldaek at 2011/07/06 08:46:54 -0700
But this should maybe be done in sendHeaders() à la `fixContentType`, because you can't be sure about the statusCode before that.
---------------------------------------------------------------------------
by lenar at 2011/07/06 13:55:33 -0700
I propose this based on what I read and understood from RFC2616.
---------------------------------------------------------------------------
by mheleniak at 2011/07/10 03:57:26 -0700
+1
Commits
-------
6786e81 [HttpFoundation] code factorization in UploadedFile
Discussion
----------
[HttpFoundation] code factorization in UploadedFile
As both #1542 and #1544 have been merged.
Commits
-------
5b0f1da [HttpKernel] made WebTestCase methods static
Discussion
----------
[HttpKernel] made WebTestCase methods static
This makes it possible to load fixture data in `::setUpBeforeClass()` which makes tests run much faster.
Also, `createClient()` is not protected instead of public; I'm not sure why it was public in the first place.
Commits
-------
cdf4b6a Checked log levels
a45d3ee Reverted last commit
529381b ControllerNotFound: Changed log level from info to error. Also moved throw exception code block up, to prevent the message from beeing logged multiple times.
7c29e88 Changed log level of "Matched route ..." message from info to debug
dca09fd Changed log level of "Using Controller ..." message from info to debug
Discussion
----------
Log levels
Just wanted to ask if the log level INFO is still correct for these messages?
As there are only four log levels left (DEBUG, INFO, WARNING, ERROR), DEBUG might be the more appropriate level for these messages now.
Let me give an example: An application is logging user actions (maybe to database) in order to assure comprehensibility, e. g. "User %s deleted post %d", "User %s written a message to user %s". These are not warnings of course, so the only suitable log level is INFO.
But they will be thrown together with these very common (at least two per request?) "Using controller..." and "Matched route..." messages when choosing INFO as log level.
---------------------------------------------------------------------------
by Seldaek at 2011/05/24 07:13:18 -0700
Agreed, this stuff is framework debug information.
---------------------------------------------------------------------------
by fabpot at 2011/05/24 08:53:24 -0700
Why do you want to change these two specific ones? The framework uses the INFO level at other places too. Is it a good idea to say that the framework only logs with DEBUG?
---------------------------------------------------------------------------
by stof at 2011/05/24 09:12:53 -0700
Doctrine logs at the INFO level too and I think it is useful to keep it as INFO. Being able to see the queries without having all DEBUG messages of the event dispatcher and security components is useful IMO.
---------------------------------------------------------------------------
by Seldaek at 2011/05/25 02:30:24 -0700
Yeah, that's true, maybe we just need to reintroduce (again, meh:) NOTICE between INFO and WARNING.
@kaiwa Of course the other way could be that you just add your DB handler to the app logger stack. That could be done in a onCoreRequest listener or such, basically you'd have to call `->pushHandler($yourDBHandler)` on the `monolog.logger.app` service. That way your messages will flow to it, but it won't receive noise from the framework stuff since those log on monolog.logger.request and other log channels.
---------------------------------------------------------------------------
by fabpot at 2011/05/25 02:48:26 -0700
@Seldaek: I don't think we need another level. We just need to come up with a standard rules about the usage of each level. Adapted from log4j:
* ERROR: Other runtime errors or unexpected conditions.
* WARN: Use of deprecated APIs, poor use of API, 'almost' errors, other runtime that are undesirable or unexpected, but not necessarily "wrong" (unable to write to the profiler DB, ).
* INFO: Interesting runtime events (security infos like the fact the user is logged-in or not, SQL logs, ...).
* DEBUG: Detailed information on the flow through the system (route match, security flow infos like the fact that a token was found or that remember-me cookie is found, ...).
What do you think?
---------------------------------------------------------------------------
by stloyd at 2011/05/25 02:53:38 -0700
+1 for this standard (also this PR can be merged then), but we should review code for other "wrong" log levels usage (if everyone accept this standard)
---------------------------------------------------------------------------
by fabpot at 2011/05/25 02:55:07 -0700
I won't merge this PR before all occurrences of the logger calls have been reviewed carefully and changed to the right level.
---------------------------------------------------------------------------
by kaiwa at 2011/05/25 02:58:44 -0700
@fabpot: Just noticed these two occurring for every request in my log file. You are right, there are other places where this changes must be applied if we will change the log level.
@stof: Hmm, i see. It is not possible to set the logger separately for each bundle, is it? That maybe would solve the problem. If somebody is interested in seeing the queries, he could set the log handler level to DEBUG for doctrine bundle, but still use INFO for the framwork itself. Plus he could even define a different output file or a completely different handler.
I'm not sure if something like that is possible already (?) or realizable at all... just came into my mind.
---------------------------------------------------------------------------
by Seldaek at 2011/05/25 03:01:07 -0700
Just FYI, from Monolog\Logger (which has CRITICAL and ALERT):
* Debug messages
const DEBUG = 100;
* Messages you usually don't want to see
const INFO = 200;
* Exceptional occurences that are not errors
* This is typically the logging level you want to use
const WARNING = 300;
* Errors
const ERROR = 400;
* Critical conditions (component unavailable, etc.)
const CRITICAL = 500;
* Action must be taken immediately (entire service down)
* Should trigger alert by sms, email, etc.
const ALERT = 550;
The values kind of match http error codes too, 4xx are expected errors that are not really important (404s etc) and 5xx are server errors that you'd better fix ASAP. I'm ok with the descriptions, but I think alert and critical should be included too. I'll probably update Monolog docblocks to match whatever ends up in the docs.
---------------------------------------------------------------------------
by Seldaek at 2011/05/25 03:03:21 -0700
@kaiwa you can do a lot, but not from the default monolog configuration entry, I'm not sure if we can really make that fully configurable without having a giant config mess. Please refer to my [comment above](https://github.com/symfony/symfony/pull/1073#issuecomment-1234316) to see how you could solve it. Maybe @fabpot has an idea how to make this more usable though.
---------------------------------------------------------------------------
by stof at 2011/05/25 03:19:43 -0700
@Seldaek the issue is that the different logging channels are only know in the compiler pass, not in the DI extension. So changing the level in the extension is really hard IMO.
Thus, the handlers are shared between the different logging channels (needed to open the log file only once for instance, or to send a single mail instead of one per channel) and the level is handled in the handlers, not the logger.
I'm +1 for the standard, by adding the distinction between 400 and 500 status calls using ERROR and CRITICAL (which is already the case in the code).
@kaiwa do you have time to review the calls to the logger between DEBUG and INFO or do you prefer I do it ? For instance, the Security component currently logs all message at DEBUG level and some of them should be INFO.
---------------------------------------------------------------------------
by kaiwa at 2011/05/25 04:31:04 -0700
@stof ok i'll do that
---------------------------------------------------------------------------
by kaiwa at 2011/05/25 12:22:51 -0700
Need some help :) I came across `ControllerNameParser::handleControllerNotFoundException()` which leads to redundant log messages currently:
>[2011-05-25 20:53:16] request.INFO: Unable to find controller "AppBaseBundle:Blog" - class "App\BaseBundle\Controller\BlogController" does not exist.
>[2011-05-25 20:53:16] request.ERROR: InvalidArgumentException: Unable to find controller "AppBaseBundle:Blog" - class "App\BaseBundle\Controller\BlogController" does not exist. (uncaught exception) at /home/ruth/symfony3/src/Symfony/Bundle/FrameworkBundle/Controller/ControllerNameParser.php line 87
Is it necessary to call `$this->logger->info($log);` if the InvalidArgumentException will be logged anyway?
---------------------------------------------------------------------------
by stof at 2011/05/25 12:39:22 -0700
Well, the issue is that the ControllerNameParser logs messages and then uses them to throw an exception. I guess the logging call should be removed as it is redundant with the one of the ExceptionListener. @fabpot thoughts ?
---------------------------------------------------------------------------
by kaiwa at 2011/05/27 11:39:25 -0700
I checked all debug, info and log calls. Sometimes it is hard to distinguish between the levels, so it would be great if someone reviews @cdf4b6a. @stof, maybe you want to take a look?
---------------------------------------------------------------------------
by kaiwa at 2011/05/31 12:52:07 -0700
@stof, thanks for your comments. I added some replies above, please let me know your suggestions.
---------------------------------------------------------------------------
by stof at 2011/05/31 14:04:22 -0700
@kaiwa As I said before, all the security logging calls should be DEBUG (most of them) or INFO (the one syaing that authentication succeeded for instance), but not WARN or ERROR as the exception don't go outside the firewall.
