Archive for September, 2014

Culling your module/actions (Symfony1)

0

This post is centered around Symfony1 – but it translates to other frameworks quite easily if you have filter chains. The reason we needed to do this, was to figure out which of our old legacy symfony1 module/actions are still in use, how heavy the usage is on each, and which ones are not being used anymore – so we can delete them, yay fun :)

Just some background: We’ve run Symfony 1.4 for a long time, and most of our legacy systems were written in that. Then when Symfony2 was launched, we started to write new code in there, and slowly started to move things from the old legacy system into the new system – especially things we wanted to make changes to. We’re a point now, that we would like to get rid of all the front-end module/actions in our old system.

Here’s the approach that seems to be working rather well…

Figuring out what module/actions are still in use:

A filter does the job nicely. We wrote a small filter that which merely logs to a file the time and what module/action was called. (lib/myDeprecationFilter.class.php)

(Please ensure that you install Monolog using composer – but you really can log it any other way you want as long as your parser then knows how to read that)

You then have to put this in your filter.yml in your app folder (somewhere similar to apps/yourAppName/config/filters.yml)

(the order in which you put this really doesn’t matter)

The log file:

When you make this live (remember to clear cache), you’ll notice that the log file is created and starts to get entries written to it, here’s an extract of how it looks like:

Monolog adds a nice bunch of information bits to the logfile, which we use for other things – but the two that you’re concerned about is the action and the module that’s being logged, called “ctxt_action” and “ctxt_module” respectively.

It may be prudent to logrotate this file, as it will grow fast (very fast for our site) and we don’t want it to be the cause for a filled up disk error ;). Merely add something like this in your logrotate.d:

(note the rotate 30 – we thus only keep 30 days worth of logs)

Reading / Parsing the log

This is not nearly as elegant as it can be, but who cares – it’s only for us internally.

As you can see above (please excuse the hardcoding – no need to put the CLI handler in here), we then run through all the files (including the compressed .gz ones that logrotate create) and reads every single line (nice and inefficient, right? 😉 ). We then do counts into an array (using @, lol… I mentioned this is an internal tool right?), and then output the contents of the array.

It looks something similar to the below extract:

From the above, you can see that we have a multi-dimensional-array that has the module name and then the actions inside it which was called.

The above then shows you which modules (for a start) have been called in the last 30 days (remember our logrotate?)

For our systems, it’s safe to assume that if somebody have not looked at a module/action in 30 days we have created and linked the new one in our new Symfony2 install and that clients/customers cannot get to the old one anymore (except off course if they have it bookmarked)

From here it’s easy to extract the module names only (with a simple regex) and put that into a file.

Then you list the modules in your apps/appName/modules folder, and put that in another file.

And then simply run a diff between the two files.

The ones that’s in the directory listing but not in the called list – those are the ones you want to delete / cull off your server and get rid of in your code :)

Luckily for us we don’t really dig into the actions as we rewrite the entire module into the new code and only then do we remove it from the old code, so we don’t have to overcomplicate it.

Have fun and let me know if you have something to add to this :)

database1_s600x600

Know your IDE (Data Sources in PhpStorm 7+)

0

Sometimes it’s the small things that makes a huge difference to your day. It’s those tiny things that makes a developer grin – smile even! :)

Like, for instance, setting up your Data Source so that SQL statements not only look cool, but actually give you some added productivity.

Having a look at PhpStorm when putting a simple SQL statement in, it’s got a rather ugly colour depicting that you don’t have a data-source set up. On top of that, it actually displays a little light bulb when clicking on it indicating that something is wrong, but the IDE knows how to fix it if you just gave it a little bit of into.

Screenshot 2014-09-05 23.19.17

 

 

 

 

 

 

If you look at the screen grab above, you’ll note the colour of the string containing the SQL, as well as the light bulb, which I clicked on.

Clicking on the “Configure Data Source” gives us the ability to set up the data source, and for good measure which database to connect to as a default:

PS: You may need to have PhpStorm install the JDBC first, check at the bottom of the pop-up window for details on that.

Screenshot 2014-09-05 23.24.20

 

 

 

 

 

 

 

 

 

 

 

It’s always a good idea to test the connection first, but if all goes well and you save and apply… your SQL statement inside your file suddenly changes colour and you have some extra functionality there:

Screenshot 2014-09-05 23.27.50

 

 

 

Note that it highlights the column name in a bold and slightly different colour. This indicates that the column actually exists.

Also, when working with complex databases, it’s not always possible (or even necessary) for developers to memorize all the fields – and their spelling – by heart. Now, because you set up your data-source, you get code-assist!!! Awesome, right!

Screenshot 2014-09-05 23.31.13

 

 

 

 

 

 

 

As you can see, not only do you get code-assist on the column names – but it also tells you which is the primary key with the little key-icon, it tells you what the data type is of the column, and it indicates that the column is referenced to another table (in the case of the project_id field with the little highlighted part of the icon).

Pretty awesome, hey!

So go ahead, learn your IDE and set this kind of thing up.

You won’t be disappointed!

 

 

database1_s600x600

alter table … change column… NOT NULL DEFAULT 2 — weird behaviour

2

>> EDIT:

So, thanks to @morgo and @cmptrwizard I now know what was going on :)

Turns out, old decisions do sometimes come back and bite later on hehe… who knew self-deprecating-sarcasm.

We turned STRICT_TRANS_TABLES off way back when we upgraded to MySQL 5.6 – meaning that the error MySQL was supposed to through due to the alter not going to do what I was hoping (my mistake), was not thrown. This resulted it the failure being silent – which is what bugged me for days :)

Thanks guys for the assiste.

<< EDIT

Original post follows…

This is just weird, and it’s bugging me. It appears when MySQL does the alter table operation on a previously nullable column it does run through it changing the current null values to the new default – but instead of using the default specified it uses the base default value of the core.

Have a look at this to reproduce it:

Ok, let’s add some rows, so we have a nice view of things:

Looking at the data, we find it looks like this:

Ok so far so good. Now, let’s alter the table’s “some_value” column to be a default of “2”.

Now, when doing this on a large table it’s pretty clear that MySQL actually runs through the table entries and changes their values (as it takes minutes to run on a big enough table). So we know it’s not just changing the metadata of the table.

Right, let’s look at the data again:

Do you see what I see? The “some_value” of row with id 101 was changed to “0” – zero. But the alter table statement said the default should be “2” – two.

Ok, did I break the table? Let’s test inserting a row without the “some_value” filled in – which in the past would’ve made it null… but now that I altered the definition it should make it “2”.

Ok cool, so let’s retrieve the data again:

Anybody know why the heck this is?

Go to Top