Posts tagged symfony

Stop child-processes from keeping lock (LockHandler)

8 years ago

by Sarel in Development

This kept me busy for a while. I have a queueing system that run constantly on our servers – built in PHP because, why now 🙂

Basically I have the controller, run as: php app/console aqueue:controller --bootstrap – this merely starts, and one by one run the child-processes and disowns them, like so:

<?php
foreach ($this->queue_repo->findAll() as $Queue) {
    /** @var Queue $Queue */

    $cmd = 'php app/console aqueue:parent --queue-name=' . $Queue->getName() . ' --env=' . $input->getOption('env') . ' > /dev/null 2>&1 < /dev/null';
    echo $cmd . PHP_EOL;

    $p = new Process($cmd);
    $p->start();

    // TODO: Figure out why when we don't dump it afterward, it doesn't remain running after this Command quites:
}

<?php

foreach ($this->queue_repo->findAll() as $Queue) {

/** @var Queue $Queue */

$cmd = 'php app/console aqueue:parent --queue-name=' . $Queue->getName() . ' --env=' . $input->getOption('env') . ' > /dev/null 2>&1 < /dev/null';

echo $cmd . PHP_EOL;

$p = new Process($cmd);

$p->start();

// TODO: Figure out why when we don't dump it afterward, it doesn't remain running after this Command quites:

}

However, the problem is – in order to make sure somebody doesn’t run the bootstrap script more than one at the same time, I get a lock at the top of the console command before continuing – and if I can’t get the lock, it exits immediately. It’s a safety measure.

I get the lock like this:

protected function initialize(InputInterface $input, OutputInterface $output)
{

    parent::initialize($input, $output);

    // Lock (so we don't run multiple of these at the same time):
    $this->lockFile = $this->getName() . '.lock';
    $this->lockHandler = new LockHandler($this->lockFile);
    if (!$this->lockHandler->lock()) {
        echo 'Sorry, can\'t get the lock. Bailing out!' . PHP_EOL;
        exit;
    }

}

protected function initialize(InputInterface $input, OutputInterface $output)

{

parent::initialize($input, $output);

// Lock (so we don't run multiple of these at the same time):

$this->lockFile = $this->getName() . '.lock';

$this->lockHandler = new LockHandler($this->lockFile);

if (!$this->lockHandler->lock()) {

echo 'Sorry, can\'t get the lock. Bailing out!' . PHP_EOL;

exit;

}

The problem is, once you start the child-process – each of them ALSO holds the lock that the aqueue:controller created above. Thus, when the bootstrapping controller is done, but the child processes are still running (as they should be) – it can’t start the bootstrapper again because the lockfile is held by other processes.

To fix this: Simple, release the lock.

So to do this easily and elegantly, you merely create a destructor that releases the lock in the bootstrapping command, like so:

    public function __destruct()
    {
        /**
         * This is important to do, because the controller --boostrap intention is to run the aqueue:parents commands and dies,
         * and then run again to make sure they are still there through crontab
         */
        if (isset($this->lockHandler)) {
            $this->lockHandler->release();
        }
    }

public function __destruct()

{

/**

* This is important to do, because the controller --boostrap intention is to run the aqueue:parents commands and dies,

* and then run again to make sure they are still there through crontab

if (isset($this->lockHandler)) {

$this->lockHandler->release();

}

Once this is done, the lockfile inheritance problem goes away 🙂

Symfony2 Auto ACL / permissions bash script

11 years ago

by Sarel in Development

Recently I had to install Symfony2 from scratch. Being a company that works on one install for years, we don’t always get acquainted with the install phase that often.

Every time I do a personal install just to screw around I have to go read how to do the ACL / Permissions for the cache and logs folders in /app/*

I decided a quick shell script to keep in my toolbox would be a good use of my time this evening. Here it is:

#!/bin/bash

USER=$(whoami)
APACHE_USER=$(ps axho user,comm|grep -E "[h]ttpd|[a]pache"|uniq|grep -v "root"|awk 'END {print $1}')

rm -rf app/cache/*
rm -rf app/logs/*

sudo chmod +a "$APACHE_USER allow delete,write,append,file_inherit,directory_inherit" app/cache app/logs
sudo chmod +a "$USER allow delete,write,append,file_inherit,directory_inherit" app/cache app/logs

php app/console cache:clear

echo "You're good to go... load localhost/web/config.php"

#!/bin/bash

USER=$(whoami)

APACHE_USER=$(ps axho user,comm|grep -E "[h]ttpd|[a]pache"|uniq|grep -v "root"|awk 'END {print $1}')

rm -rf app/cache/*

rm -rf app/logs/*

sudo chmod +a "$APACHE_USER allow delete,write,append,file_inherit,directory_inherit" app/cache app/logs

sudo chmod +a "$USER allow delete,write,append,file_inherit,directory_inherit" app/cache app/logs

php app/console cache:clear

echo "You're good to go... load localhost/web/config.php"

This quick script just follows a few steps that is done by the documentation at the Symfony2 Installation Documentation

Copy this code into a file called “set_permissions.sh” in your project root.
Add executable permission to the script, by running chmod +x set_permissions.sh
Run the script:
./set_permissions.sh
If you’d like, delete the script afterward, alternatively you can keep it for when you do an update to your code / mess up your permissions (like I do from time to time 🙂 )

Have fun, hope it’s useful.

Grateful to Matthias Noback for his post on How to Install Sismo which is what gave me the idea of creating the shell script.

bash dev symfony

Human Readable bytes (Twig Filter Extension)

11 years ago

by Sarel in Development

Recently we needed a Twig Filter that takes bytes (as an integer) and translates that to the nearest human readable version of it. For example… if you have 100 bytes, that’s easy, it’s written as in “100 B”. But, what if you have more than 1024 bytes? Then, for example of having 2000 bytes, it is “1.953125 KB” (i.e. the first 1024 bytes is a kilobyte and the remaining 976 bytes is 0.953125 kilobytes).

You can read more here: http://en.wikipedia.org/wiki/Byte

What we actually want to do here is “filter” the bytes and translate that to a human readable, close to the biggest denominator, string that makes it easy for us to see the size of data. For instance, it’s very very difficult to see at a glance how big 1234567890 Bytes are, but if you break it down to the closest 1024 multiple, you note that 1234567890 Bytes = 1.15 GB (rounded).

The easy way for us to solve this was to use a Twig Filter Extension. Our extension is used as below:

{{ 123456789012345 | bytesToHuman }}

1	{{ 123456789012345 \| bytesToHuman }}

… this will take 123456789012345 and divide it by 1024 until it can go no further, and will then eventually print out 112.283 TB

How do we do this? We take advantage of some math and PHP:

$unit = array();
$unit[0] = 'B';
$unit[1] = 'KB';
$unit[2] = 'MB';
$unit[3] = 'GB';
$unit[4] = 'TB';
$unit[5] = 'PB';
$unit[6] = 'EB';
$unit[7] = 'ZB';
$unit[8] = 'YB';

$bytes = 123456789012345;
$precision = 3;

echo round($bytes / pow(1024, ($i = floor(log($bytes, 1024)))), $precision) . ' ' . $unit[$i];

$unit = array();

$unit[0] = 'B';

$unit[1] = 'KB';

$unit[2] = 'MB';

$unit[3] = 'GB';

$unit[4] = 'TB';

$unit[5] = 'PB';

$unit[6] = 'EB';

$unit[7] = 'ZB';

$unit[8] = 'YB';

$bytes = 123456789012345;

$precision = 3;

echo round($bytes / pow(1024, ($i = floor(log($bytes, 1024)))), $precision) . ' ' . $unit[$i];

The above code is common use for doing this calculation so I won’t go into this too much. You can read more on this at http://php.net/manual/en/function.log.php in the comments section.

Putting it into a Twig Extension is pretty easy, there’s mainly 3 steps:

Create the Twig Extension (anywhere, really, we put it in our “util” bundle in Symfony2)
Register the extension as a service
Use it 🙂 – anywhere in your project

To create a Twig Extension in Symfony2 read: http://symfony.com/doc/current/cookbook/templating/twig_extension.html

We went one step further though, in order to make the extension a little more flexible. We used a wildcard. In the section where you register your twig filters you may use wildcards, here’s our getFilters method code:

public function getFilters()
{
    $filters = array();
    $filters['bytesTo*'] = new Twig_Filter_Method($this, 'bytes2Filter');

    return $filters;
}

public function getFilters()

{

$filters = array();

$filters['bytesTo*'] = new Twig_Filter_Method($this, 'bytes2Filter');

return $filters;

}

The wildcard at the end of ‘bytesTo*’ means we can put just about anything we want there. This means we can have a filter saying, even though this might not resolve to the closest GB, we’d like to force it to resolve to GB and end up with something like 0.005 GB or 4000.123 GB even.

Full source of our Extension Class to see it all work together (with some sanity checks) can be seen below:

class OurCoolTwigExtension extends \Twig_Extension
{
    protected $unit = array();

    public function getFilters()
    {
        $filters = array();
        $filters['bytesTo*'] = new Twig_Filter_Method($this, 'bytes2Filter');

        return $filters;
    }

    public function __construct()
    {
        $this-&gt;unit = array();
        $this-&gt;unit['B'] = 'Bytes'; // 8 bits
        $this-&gt;unit['KB'] = 'Kilobytes'; // 1024 bytes
        $this-&gt;unit['MB'] = 'Megabytes'; // 1048576 bytes
        $this-&gt;unit['GB'] = 'Gigabytes'; // 1073741824 bytes
        $this-&gt;unit['TB'] = 'Terabytes'; // 1099511627776 bytes
        $this-&gt;unit['PB'] = 'Petabytes'; // 1.12589991e15 bytes
        $this-&gt;unit['EB'] = 'Exabytes'; // 1.1529215e18 bytes
        $this-&gt;unit['ZB'] = 'Zettabytes'; // 1.18059162e21 bytes
        $this-&gt;unit['YB'] = 'Yottabytes'; // 1.20892582e24 bytes
    }

    public function bytes2Filter($suffix, $bytes, $precision = 2)
    {
        $auto = array('Human', 'Auto');
        $unit = array_keys($this-&gt;unit);

        if ($bytes &lt;= 0) {
            return '0 ' . $unit[0];
        }

        if ($suffix == '') {
            $suffix = 'Auto';
        }

        if (array_search($suffix, array_merge($auto, $unit)) === false) {
            throw new \Exception('Sorry, you have to specify a legal Byte value or "Human" for automatic. Legal options are: Human, ' . implode(', ', $unit));
        }

        switch ($suffix) {
            case '':
            case 'Human':
            case 'Auto':
                return round($bytes / pow(1024, ($i = floor(log($bytes, 1024)))), $precision) . ' ' . $unit[$i];
            break;
            default: {
                $i = array_search($suffix, $unit);
                return round($bytes / pow(1024, $i), $precision) . ' ' . $unit[$i];
            }
            break;
        }

    }
}

class OurCoolTwigExtension extends \Twig_Extension

{

protected $unit = array();

public function getFilters()

{

$filters = array();

$filters['bytesTo*'] = new Twig_Filter_Method($this, 'bytes2Filter');

return $filters;

}

public function __construct()

{

$this->unit = array();

$this->unit['B'] = 'Bytes'; // 8 bits

$this->unit['KB'] = 'Kilobytes'; // 1024 bytes

$this->unit['MB'] = 'Megabytes'; // 1048576 bytes

$this->unit['GB'] = 'Gigabytes'; // 1073741824 bytes

$this->unit['TB'] = 'Terabytes'; // 1099511627776 bytes

$this->unit['PB'] = 'Petabytes'; // 1.12589991e15 bytes

$this->unit['EB'] = 'Exabytes'; // 1.1529215e18 bytes

$this->unit['ZB'] = 'Zettabytes'; // 1.18059162e21 bytes

$this->unit['YB'] = 'Yottabytes'; // 1.20892582e24 bytes

}

public function bytes2Filter($suffix, $bytes, $precision = 2)

{

$auto = array('Human', 'Auto');

$unit = array_keys($this->unit);

if ($bytes <= 0) {

return '0 ' . $unit[0];

}

if ($suffix == '') {

$suffix = 'Auto';

}

if (array_search($suffix, array_merge($auto, $unit)) === false) {

throw new \Exception('Sorry, you have to specify a legal Byte value or "Human" for automatic. Legal options are: Human, ' . implode(', ', $unit));

}

switch ($suffix) {

case '':

case 'Human':

case 'Auto':

return round($bytes / pow(1024, ($i = floor(log($bytes, 1024)))), $precision) . ' ' . $unit[$i];

break;

default: {

$i = array_search($suffix, $unit);

return round($bytes / pow(1024, $i), $precision) . ' ' . $unit[$i];

}

break;

}

This gives us the following ways of using it:

{{ 123456789012345 | bytesTo }}
{{ 123456789012345 | bytesToHuman }}
{{ 123456789012345 | bytesToAuto }}
{{ 123456789012345 | bytesToKB }}
{{ 123456789012345 | bytesToMB }}
{{ 123456789012345 | bytesToGB }}
{{ 123456789012345 | bytesToTB }}
{{ 123456789012345 | bytesToPB }}
{{ 12345678901234567890 | bytesToEB }} (note I used a longer number, otherwise this would've just been 0 EB)

{{ 12345678901234567890 | bytesToEB }} (note I used a longer number, otherwise this would've just been 0 EB)

… yielding the following results:

112.283 TB
112.283 TB
112.283 TB
120563270519.87 KB
117737568.88 MB
114978.095 GB
112.283 TB
0.11 PB
0.0708 EB (note I used a longer number, otherwise this would've just been 0 EB)

112.283 TB

120563270519.87 KB

117737568.88 MB

114978.095 GB

112.283 TB

0.11 PB

0.0708 EB (note I used a longer number, otherwise this would've just been 0 EB)

Hope this is helpful and somebody can use this! n-Joy!

dev php symfony twig

Migrating from Symfony 1.4 to Symfony 2 iteratively via Apache Rewrites

11 years ago

by Sarel in Development

A problem that we have currently is that our entire system is written on Symfony 1.4 (well, it’s still the bridging 1.3 actually, but who’s counting). Migrating a system that’s been in development for 4+ years with thousands of lines of code and a lot of not-so-MVC-code is a nightmare, and in fact in the modern fast-to-market environment it is impossible to spend a year, or even 6 months, re-developing a system like this only to launch it and find it’s not only outdated once again but you missed a years worth of opportunities.

The solution? Run two systems next to each other! Easier said than done though, because when we look at the software, it is our customer zone and runs off a subdomain. So how do you run zone.domain.com and use both Symfony 1.4 (old code) and write new code in Symfony 2?

The solution? Fool the world a little bit! Put something into your Symfony 2 URL’s that uniquely identify them. Let’s fool the world into thinking we are going to be moving to internationalization. Let’s say all Symfony 2 URL’s will have a …./en/… between the domain name, and the route. So, URL’s will look like this in future:

zone.domain.com – goes to front page of Symfony 1.4 system
zone.domain.com/products/list – goes to page listing the clients products Symfony 1.4 style
zone.domain.com/en/products/list – goes to new-school and new-style Symfony 2 listing of the products (the cool thing about this is, that we can test the new products listing page extensively – even live! In fact, we can even start a beta program where certain clients/customers (say 5%) are given the opportunity to see new things before it’s launched)

Now, how do we do this? Options, anybody?

Option 1: PHP Hacking
Option 2: Apache Virtual Host Hacking
Option 3: .htaccess hacking – this is the option we went with

In short, PHP Hacking involves us taking our app.php controller file on Symfony 1.4 (which everything is linked to) and checking for the /en/ at the start of the route, if we find it we go and include the Symfony 2 app.php file instead. This gets messy, because you now have to change current-working-directories, you have to keep track of things to avoid infinite looping, etc. It generally sucked when we tried it.

Option 2 and 3 is basically the same from what we did, but hacking away at the vhost means each change you’ll have to restart apache – not a great idea if you have to make a change on a live system with thousands of users logged in and trying to do something at that time.

Let’s then look at what we did with Option 3: .htaccess hacking:

First, let’s explain the old and new directory structures:

/home/public_html
- symfony1.4projectname
  - apps/
  - cache/
  - config/
    - databases.yml – config file holding database names, server addresses, usernames and passwords!
  - lib/
  - log/
  - web/
    - app.php
    - images/
    - js/
    - css/
- symfony2projectname
  - app/
    - config/
  - bin/
  - src/
  - web/
    - app.php

The old system, had a virtual host set up for zone.domain.com to point to /home/public_html/symfony1.4projectname/web/app.php. Thus, if you go to say zone.domain.com/products/list it would translate into loading the php file at /home/public_html/symfony1.4projectname/web/app.php and appending to it the URI of /products/list. We want to keep the system the same and have it still use that controller file for the old-school system.

With the new system, we’d like the exact same to happen, but instead of pointing it to ..symfony1.4projectname/web/app.php we want to point it to ..symfony2projectname/web/app.php (note the 2 instead of 1.4).

Easy (and wrong) way of doing this, is to have the vhost point not to /home/public_html/symfony1.4projectname/web/ but to /home/public_html/ and then have the .htaccess file make the call of where it’s going. This, however, opens up your entire folder structure to the world (if you’re not very very careful) which kind of voids one of the big reasons for using frameworks like Symfony with a single controller file for your application inside a single non-connected web folder.

Better: Create a “switching” folder, and point vhost there!

We created the following new folder structure (new is in green):

/home/public_html
- symfony1.4projectname/
- symfony2projectname/
- switch/
  - s1.4/
    - Symbolic link to: ../../symfony1.4projectname/web
  - s2/
    - Symbolic link to: ../../symfony2projectname/web
- .htaccess

Now, with our vhost pointing to /home/public_html/switch/ we can safely say that we are still secure, however nothing is working 🙂 This is where the .htaccess file comes in:

The .htaccess file now has to figure out where to send us, first let’s make it work as it did in the past, here’s the file then:

DirectoryIndex app.php
RewriteEngine on
RewriteCond %{REQUEST_URI} !^/(s1.4)
RewriteRule ^/?(.*)$ /s1.4/web/$1 [L]

DirectoryIndex app.php

RewriteEngine on

RewriteCond %{REQUEST_URI} !^/(s1.4)

RewriteRule ^/?(.*)$ /s1.4/web/$1 [L]

Let’s look at this, all we’re basically saying is, whatever URI we call, send it to the destination of the symbolic link of 1.4 and load the app.php file with whatever parameters was given afterward.

Now that that’s working, let’s put in the /en/ rules:

The .htaccess file now changes to this (green is new code):

DirectoryIndex app.php
RewriteEngine on
RewriteCond %{REQUEST_URI} ^/en
RewriteRule ^/?(.*)$ /s2/web/app.php/$1 [L]
RewriteCond %{REQUEST_URI} !^/(s1.4)
RewriteRule ^/?(.*)$ /s1.4/web/$1 [L]

DirectoryIndex app.php

RewriteEngine on

RewriteCond %{REQUEST_URI} ^/en

RewriteRule ^/?(.*)$ /s2/web/app.php/$1 [L]

RewriteCond %{REQUEST_URI} !^/(s1.4)

RewriteRule ^/?(.*)$ /s1.4/web/$1 [L]

Here you can see, that the rule states that “if the URI begins with /en, then send it to the symbolic destination folder sitting at s2/web and load the app.php controller file, appending whatever parameters was given at the end”.

That worked great, except none of the images, css and js was loading. The great thing about Symfony 2 is that it has a /bundles folder inside web which holds all the assets. This is nice and identifiable of symfony 2 (granted that you don’t have that foldername in Symfony 1.4 though). Therefore, we just have to add another rule in for that. Here’s the new .htaccess file (again, green is new code):

DirectoryIndex app.php
RewriteEngine on
RewriteCond %{REQUEST_URI} ^/en
RewriteRule ^/?(.*)$ /s2/web/app.php/$1 [L]
RewriteCond %{REQUEST_URI} ^/bundles RewriteRule ^/?(.*)$ /s2/web/$1 [L] RewriteCond %{REQUEST_URI} !^/(s1.4)
RewriteRule ^/?(.*)$ /s1.4/web/$1 [L]

DirectoryIndex app.php

RewriteEngine on

RewriteCond %{REQUEST_URI} ^/en

RewriteRule ^/?(.*)$ /s2/web/app.php/$1 [L]

RewriteCond %{REQUEST_URI} ^/bundles RewriteRule ^/?(.*)$ /s2/web/$1 [L] RewriteCond %{REQUEST_URI} !^/(s1.4)

RewriteRule ^/?(.*)$ /s1.4/web/$1 [L]

And that, as they say, is that! Now all css, images, js etc is loading fine in Symfony 2 as well as the old-school Symfony 1.4 project is still working, and the best thing: Your PHP code doesn’t even know it’s been fooled! 🙂 Neither does the public users of your system, by the way.

The only drawback of this is, that all your routes in Symfony 2 now has to be prefixed by /en/. The solution to this, though, sits in the word “prefix” – the great thing about symfony2 is that it has that ability! Open your routing file (that should be in app/config/routing.yml) and merely change the line that says:

prefix: /

prefix: /

… to be:

prefix: /en/

1	prefix: /en/

Done! 🙂 Enjoy and please comment if you have some insight to give on this matter!

apache dev htaccess rewrite symfony

Posts tagged symfony

Stop child-processes from keeping lock (LockHandler)

Like this:

Symfony2 Auto ACL / permissions bash script

Like this:

Human Readable bytes (Twig Filter Extension)

Like this:

Migrating from Symfony 1.4 to Symfony 2 iteratively via Apache Rewrites

Like this:

Archives

My Stackoverflow Stats

Posts tagged symfony

Stop child-processes from keeping lock (LockHandler)

Share this:

Like this:

Symfony2 Auto ACL / permissions bash script

Share this:

Like this:

Human Readable bytes (Twig Filter Extension)

Share this:

Like this:

Migrating from Symfony 1.4 to Symfony 2 iteratively via Apache Rewrites

Share this:

Like this:

Archives

Tags

My Stackoverflow Stats