Symfony, PHP, Life, Parenting, Management, etc :)
Posts tagged symfony1
Culling your module/actions (Symfony1)
09 years ago
by Sarel
in Development
This post is centered around Symfony1 – but it translates to other frameworks quite easily if you have filter chains. The reason we needed to do this, was to figure out which of our old legacy symfony1 module/actions are still in use, how heavy the usage is on each, and which ones are not being used anymore – so we can delete them, yay fun 🙂
Just some background: We’ve run Symfony 1.4 for a long time, and most of our legacy systems were written in that. Then when Symfony2 was launched, we started to write new code in there, and slowly started to move things from the old legacy system into the new system – especially things we wanted to make changes to. We’re a point now, that we would like to get rid of all the front-end module/actions in our old system.
Here’s the approach that seems to be working rather well…
Figuring out what module/actions are still in use:
A filter does the job nicely. We wrote a small filter that which merely logs to a file the time and what module/action was called. (lib/myDeprecationFilter.class.php)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
class myDeprecationFilter extends sfFilter { public function execute($filterChain) { $actionName = $this->getContext()->getController()->getActionStack()->getFirstEntry()->getActionName(); $moduleName = $this->getContext()->getController()->getActionStack()->getFirstEntry()->getModuleName(); $logger = new Monolog\Logger(__CLASS__); $stream = new \Monolog\Handler\StreamHandler(sfConfig::get('sf_log_dir') . '/' . __CLASS__ . '.log', \Monolog\Logger::INFO); $stream->setFormatter(new \Monolog\Formatter\LogstashFormatter('module_action_deprecation')); $logger->pushHandler($stream); $logger->addCritical(__CLASS__, array( 'parsed_url'=>parse_url($_SERVER['REQUEST_URI']), 'action'=>$actionName, 'module'=>$moduleName )); $filterChain->execute(); } } |
(Please ensure that you install Monolog using composer – but you really can log it any other way you want as long as your parser then knows how to read that)
You then have to put this in your filter.yml in your app folder (somewhere similar to apps/yourAppName/config/filters.yml)
|
1 2 3 4 5 6 |
rendering: ~ security: ~ myDeprecationFilter: class: myDeprecationFilter cache: ~ execution: ~ |
(the order in which you put this really doesn’t matter)
The log file:
When you make this live (remember to clear cache), you’ll notice that the log file is created and starts to get entries written to it, here’s an extract of how it looks like:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
{ "@timestamp": "2014-09-25T04:18:07.816809+02:00", "@message": "myDeprecationFilter", "@tags": [ "myDeprecationFilter" ], "@source": "yoursite.com", "@fields": { "channel": "myDeprecationFilter", "level": 500, "ctxt_parsed_url": { "path": "/", "query": "src=website_nav" }, "ctxt_action": "viewClientContact", "ctxt_module": "contactdetails_primary" }, "@type": "module_action_deprecation" } |
Monolog adds a nice bunch of information bits to the logfile, which we use for other things – but the two that you’re concerned about is the action and the module that’s being logged, called “ctxt_action” and “ctxt_module” respectively.
It may be prudent to logrotate this file, as it will grow fast (very fast for our site) and we don’t want it to be the cause for a filled up disk error ;). Merely add something like this in your logrotate.d:
|
1 2 3 4 5 6 7 |
/home/public_html/yoursite.com/log/*.log { daily minsize 1M create 0660 afrihost apache compress rotate 30 } |
(note the rotate 30 – we thus only keep 30 days worth of logs)
Reading / Parsing the log
This is not nearly as elegant as it can be, but who cares – it’s only for us internally.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
$doGZfiles = true; // hardcoded for the post, but that's from the command line $moduleonly = false; // also taken from the command line - if set to true it'll only show the modules and not drill down into the actions $glob = glob(sfConfig::get('sf_log_dir') . '/myDeprecation'.$context.'Filter.log*'); foreach ($glob as $filename) { Tools::SaveToLog('Parsing file "'.$filename.'"', LOG_FILENAME, VERBOSE); if ($doGZfiles && substr($filename, -3) == '.gz') { $fp = gzopen($filename, 'r'); while ($line = json_decode(gzgets($fp, 4096), true)) { if ($moduleonly) { @$counterARR[$line['@fields']['ctxt_module']]++; } else { @$counterARR[$line['@fields']['ctxt_module']][$line['@fields']['ctxt_action']]++; } } } else { $fp = fopen($filename, 'r'); while ($line = json_decode(fgets($fp), true)) { if ($moduleonly) { @$counterARR[$line['@fields']['ctxt_module']]++; } else { @$counterARR[$line['@fields']['ctxt_module']][$line['@fields']['ctxt_action']]++; } } } } asort($counterARR); print_r($counterARR); |
As you can see above (please excuse the hardcoding – no need to put the CLI handler in here), we then run through all the files (including the compressed .gz ones that logrotate create) and reads every single line (nice and inefficient, right? 😉 ). We then do counts into an array (using @, lol… I mentioned this is an internal tool right?), and then output the contents of the array.
It looks something similar to the below extract:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
$ php myDeprecationFilterLogParser.php 2014-09-25 11:13:55: Parsing file "/home/public_html/yoursite.com/log/myDeprecationFilter.log" 2014-09-25 11:13:55: Parsing file "/home/public_html/yoursite.com/log/myDeprecationFilter.log-20140827.gz" 2014-09-25 11:13:57: Parsing file "/home/public_html/yoursite.com/log/myDeprecationFilter.log-20140828.gz" ... 2014-09-25 11:14:10: Parsing file "/home/public_html/yoursite.com/log/myDeprecationFilter.log-20140925.gz" Array ( [signup] => Array ( [index] => 2 ) [some_random_module_name] => Array ( [some_random_action_name] => 2 ) [tiny] => Array ( [index] => 6877 ) [status] => Array ( [error404] => 535 ) [support] => Array ( [index] => 94 ) [orderPromotions] => Array ( [index] => 40 [process] => 38 [refreshBreadCrumb] => 46 [reLoadBilling] => 1 ) ) |
From the above, you can see that we have a multi-dimensional-array that has the module name and then the actions inside it which was called.
The above then shows you which modules (for a start) have been called in the last 30 days (remember our logrotate?)
For our systems, it’s safe to assume that if somebody have not looked at a module/action in 30 days we have created and linked the new one in our new Symfony2 install and that clients/customers cannot get to the old one anymore (except off course if they have it bookmarked)
From here it’s easy to extract the module names only (with a simple regex) and put that into a file.
Then you list the modules in your apps/appName/modules folder, and put that in another file.
And then simply run a diff between the two files.
The ones that’s in the directory listing but not in the called list – those are the ones you want to delete / cull off your server and get rid of in your code 🙂
Luckily for us we don’t really dig into the actions as we rewrite the entire module into the new code and only then do we remove it from the old code, so we don’t have to overcomplicate it.
Have fun and let me know if you have something to add to this 🙂
