Posts tagged php

Visualising the Evolution of our Source Code

10 years ago

by Sarel in Development

We have well over 100k lines of code in the core system right now, that’s not even counting all the supporting templates, images etc that is all over the place.

We run a hybrid system between Symfony 1.3 and Symfony 2.1, parts of our system is in the old Symfony and parts in the new Symfony2.

People always likes cool pictures (especially the kind that move), so I set off on a path to take our SVN repository of everything and visualise it.

Luckily for me, there’s already a tool to do this, called Gource. I have Gource 0.38 but it’s already at version 0.40 already with some nifty extra features.

Here’s the youtube video of the code as it progressed from 2007 up to August 2013. Can’t believe it’s been 6 years worth of development! http://www.youtube.com/watch?v=DojMulQ8CcI (Best viewed at HD-720p)

In order to do this, you go into your base-SVN folder and do the following commands:

svn log -r 1:HEAD --xml --verbose --quiet > gource-project-log.xml
gource -1280x720 \
 --hash-seed 16 \
 --highlight-all-users \
 -seconds-per-day 0.01 \
 --title "Afrihost SVN Repository" \
 --hide dirnames,filenames,root \
 --camera-mode overview \
 --max-files 3000 \
 --key \
 --file-filter 'images|cache|log|/web' \
 --bloom-intensity 0.2 \
 -highlight-users \
 --highlight-colour FF3030 \
 --logo your_company_logo.png \
 gource-project-log.xml \
 -o - | ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i - -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 -bf 0 your-video.mp4

svn log -r 1:HEAD --xml --verbose --quiet > gource-project-log.xml

gource -1280x720 \

--hash-seed 16 \

--highlight-all-users \

-seconds-per-day 0.01 \

--title "Afrihost SVN Repository" \

--hide dirnames,filenames,root \

--camera-mode overview \

--max-files 3000 \

--key \

--file-filter 'images|cache|log|/web' \

--bloom-intensity 0.2 \

-highlight-users \

--highlight-colour FF3030 \

--logo your_company_logo.png \

gource-project-log.xml \

-o - | ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i - -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 -bf 0 your-video.mp4

As you can see, it’s rather straight forward.

Firstly you get the SVN log – this takes a while especially on a large project like this.
Secondly you run the log (in XML format) through gource to create the video – you pipe the video to ffmpeg to make the mp4
Lastly if you’d like, open it in iMovie and add sound to it, like I did.

Important to notice that you shouldn’t fiddle with the video while gource is running, because everything you do to the app while it runs is saved to the mp4. Just… be patient 🙂

You can find gource here: https://code.google.com/p/gource/

I just installed it with brew (in OSX), instructions can be found here: https://code.google.com/p/gource/wiki/MacSupport

Well, it was fun 🙂 And it looks pretty, so I guess it was worth the hour it took me 🙂

Human Readable bytes (Twig Filter Extension)

11 years ago

by Sarel in Development

Recently we needed a Twig Filter that takes bytes (as an integer) and translates that to the nearest human readable version of it. For example… if you have 100 bytes, that’s easy, it’s written as in “100 B”. But, what if you have more than 1024 bytes? Then, for example of having 2000 bytes, it is “1.953125 KB” (i.e. the first 1024 bytes is a kilobyte and the remaining 976 bytes is 0.953125 kilobytes).

You can read more here: http://en.wikipedia.org/wiki/Byte

What we actually want to do here is “filter” the bytes and translate that to a human readable, close to the biggest denominator, string that makes it easy for us to see the size of data. For instance, it’s very very difficult to see at a glance how big 1234567890 Bytes are, but if you break it down to the closest 1024 multiple, you note that 1234567890 Bytes = 1.15 GB (rounded).

The easy way for us to solve this was to use a Twig Filter Extension. Our extension is used as below:

{{ 123456789012345 | bytesToHuman }}

1	{{ 123456789012345 \| bytesToHuman }}

… this will take 123456789012345 and divide it by 1024 until it can go no further, and will then eventually print out 112.283 TB

How do we do this? We take advantage of some math and PHP:

$unit = array();
$unit[0] = 'B';
$unit[1] = 'KB';
$unit[2] = 'MB';
$unit[3] = 'GB';
$unit[4] = 'TB';
$unit[5] = 'PB';
$unit[6] = 'EB';
$unit[7] = 'ZB';
$unit[8] = 'YB';

$bytes = 123456789012345;
$precision = 3;

echo round($bytes / pow(1024, ($i = floor(log($bytes, 1024)))), $precision) . ' ' . $unit[$i];

$unit = array();

$unit[0] = 'B';

$unit[1] = 'KB';

$unit[2] = 'MB';

$unit[3] = 'GB';

$unit[4] = 'TB';

$unit[5] = 'PB';

$unit[6] = 'EB';

$unit[7] = 'ZB';

$unit[8] = 'YB';

$bytes = 123456789012345;

$precision = 3;

echo round($bytes / pow(1024, ($i = floor(log($bytes, 1024)))), $precision) . ' ' . $unit[$i];

The above code is common use for doing this calculation so I won’t go into this too much. You can read more on this at http://php.net/manual/en/function.log.php in the comments section.

Putting it into a Twig Extension is pretty easy, there’s mainly 3 steps:

Create the Twig Extension (anywhere, really, we put it in our “util” bundle in Symfony2)
Register the extension as a service
Use it 🙂 – anywhere in your project

To create a Twig Extension in Symfony2 read: http://symfony.com/doc/current/cookbook/templating/twig_extension.html

We went one step further though, in order to make the extension a little more flexible. We used a wildcard. In the section where you register your twig filters you may use wildcards, here’s our getFilters method code:

public function getFilters()
{
    $filters = array();
    $filters['bytesTo*'] = new Twig_Filter_Method($this, 'bytes2Filter');

    return $filters;
}

public function getFilters()

{

$filters = array();

$filters['bytesTo*'] = new Twig_Filter_Method($this, 'bytes2Filter');

return $filters;

}

The wildcard at the end of ‘bytesTo*’ means we can put just about anything we want there. This means we can have a filter saying, even though this might not resolve to the closest GB, we’d like to force it to resolve to GB and end up with something like 0.005 GB or 4000.123 GB even.

Full source of our Extension Class to see it all work together (with some sanity checks) can be seen below:

class OurCoolTwigExtension extends \Twig_Extension
{
    protected $unit = array();

    public function getFilters()
    {
        $filters = array();
        $filters['bytesTo*'] = new Twig_Filter_Method($this, 'bytes2Filter');

        return $filters;
    }

    public function __construct()
    {
        $this-&gt;unit = array();
        $this-&gt;unit['B'] = 'Bytes'; // 8 bits
        $this-&gt;unit['KB'] = 'Kilobytes'; // 1024 bytes
        $this-&gt;unit['MB'] = 'Megabytes'; // 1048576 bytes
        $this-&gt;unit['GB'] = 'Gigabytes'; // 1073741824 bytes
        $this-&gt;unit['TB'] = 'Terabytes'; // 1099511627776 bytes
        $this-&gt;unit['PB'] = 'Petabytes'; // 1.12589991e15 bytes
        $this-&gt;unit['EB'] = 'Exabytes'; // 1.1529215e18 bytes
        $this-&gt;unit['ZB'] = 'Zettabytes'; // 1.18059162e21 bytes
        $this-&gt;unit['YB'] = 'Yottabytes'; // 1.20892582e24 bytes
    }

    public function bytes2Filter($suffix, $bytes, $precision = 2)
    {
        $auto = array('Human', 'Auto');
        $unit = array_keys($this-&gt;unit);

        if ($bytes &lt;= 0) {
            return '0 ' . $unit[0];
        }

        if ($suffix == '') {
            $suffix = 'Auto';
        }

        if (array_search($suffix, array_merge($auto, $unit)) === false) {
            throw new \Exception('Sorry, you have to specify a legal Byte value or "Human" for automatic. Legal options are: Human, ' . implode(', ', $unit));
        }

        switch ($suffix) {
            case '':
            case 'Human':
            case 'Auto':
                return round($bytes / pow(1024, ($i = floor(log($bytes, 1024)))), $precision) . ' ' . $unit[$i];
            break;
            default: {
                $i = array_search($suffix, $unit);
                return round($bytes / pow(1024, $i), $precision) . ' ' . $unit[$i];
            }
            break;
        }

    }
}

class OurCoolTwigExtension extends \Twig_Extension

{

protected $unit = array();

public function getFilters()

{

$filters = array();

$filters['bytesTo*'] = new Twig_Filter_Method($this, 'bytes2Filter');

return $filters;

}

public function __construct()

{

$this->unit = array();

$this->unit['B'] = 'Bytes'; // 8 bits

$this->unit['KB'] = 'Kilobytes'; // 1024 bytes

$this->unit['MB'] = 'Megabytes'; // 1048576 bytes

$this->unit['GB'] = 'Gigabytes'; // 1073741824 bytes

$this->unit['TB'] = 'Terabytes'; // 1099511627776 bytes

$this->unit['PB'] = 'Petabytes'; // 1.12589991e15 bytes

$this->unit['EB'] = 'Exabytes'; // 1.1529215e18 bytes

$this->unit['ZB'] = 'Zettabytes'; // 1.18059162e21 bytes

$this->unit['YB'] = 'Yottabytes'; // 1.20892582e24 bytes

}

public function bytes2Filter($suffix, $bytes, $precision = 2)

{

$auto = array('Human', 'Auto');

$unit = array_keys($this->unit);

if ($bytes <= 0) {

return '0 ' . $unit[0];

}

if ($suffix == '') {

$suffix = 'Auto';

}

if (array_search($suffix, array_merge($auto, $unit)) === false) {

throw new \Exception('Sorry, you have to specify a legal Byte value or "Human" for automatic. Legal options are: Human, ' . implode(', ', $unit));

}

switch ($suffix) {

case '':

case 'Human':

case 'Auto':

return round($bytes / pow(1024, ($i = floor(log($bytes, 1024)))), $precision) . ' ' . $unit[$i];

break;

default: {

$i = array_search($suffix, $unit);

return round($bytes / pow(1024, $i), $precision) . ' ' . $unit[$i];

}

break;

}

This gives us the following ways of using it:

{{ 123456789012345 | bytesTo }}
{{ 123456789012345 | bytesToHuman }}
{{ 123456789012345 | bytesToAuto }}
{{ 123456789012345 | bytesToKB }}
{{ 123456789012345 | bytesToMB }}
{{ 123456789012345 | bytesToGB }}
{{ 123456789012345 | bytesToTB }}
{{ 123456789012345 | bytesToPB }}
{{ 12345678901234567890 | bytesToEB }} (note I used a longer number, otherwise this would've just been 0 EB)

{{ 12345678901234567890 | bytesToEB }} (note I used a longer number, otherwise this would've just been 0 EB)

… yielding the following results:

112.283 TB
112.283 TB
112.283 TB
120563270519.87 KB
117737568.88 MB
114978.095 GB
112.283 TB
0.11 PB
0.0708 EB (note I used a longer number, otherwise this would've just been 0 EB)

112.283 TB

120563270519.87 KB

117737568.88 MB

114978.095 GB

112.283 TB

0.11 PB

0.0708 EB (note I used a longer number, otherwise this would've just been 0 EB)

Hope this is helpful and somebody can use this! n-Joy!

dev php symfony twig

%d bloggers like this:

Go to Top

Posts tagged php

Visualising the Evolution of our Source Code

Like this:

Human Readable bytes (Twig Filter Extension)

Like this:

Archives

My Stackoverflow Stats

Posts tagged php

Visualising the Evolution of our Source Code

Share this:

Like this:

Human Readable bytes (Twig Filter Extension)

Share this:

Like this:

Archives

Tags

My Stackoverflow Stats