PHP Generators: the power of yielding

  • Sharebar

 

PHP generators are awesome

I’m not sure if you heard but PHP 5.5 is going to support generators. This is fantastic news if you ask me.

Here is some of the syntax you will be seeing in future releases, taken from the link the above:

Yield syntax

The newly introduced yield keyword (T_YIELD) is used both for sending and receiving values inside the generator. There are three basic forms of the yield expression:

yield $key => $value: Yields the value $value with key $key.
yield $value: Yields the value $value with an auto-incrementing integer key.
yield: Yields the value null with an auto-incrementing integer key.
The return value of the yield expression is whatever was sent to the generator using send(). If nothing was sent (e.g. during foreach iteration) null is returned.

To avoid ambiguities the first two yield expression types have to be surrounded by parenthesis when used in expression-context. Some examples when parentheses are necessary and when they aren’t

Example:

function getLinesFromFile($fileName) {
    if (!$fileHandle = fopen($fileName, 'r')) {
        return;
    }
    while (false !== $line = fgets($fileHandle)) {
        yield $line;
    }
    fclose($fileHandle);
}
$lines = getLinesFromFile($fileName);
foreach ($lines as $line) {
    // do something with $line
}

Pretty awesome right? Now here comes the problem… although PHP 5.3 has been out a while, few hosting providers have fully adopted it so I have little hope getting this awesomeness on the burner just yet – so I put together a little script to emulate some of the yield functionality that will be implemented by our good friends at PHP.

For those python/Vala/Ruby/Java people out there this may not look like a big deal but it is… It shows that PHP, with all of its faults, is becoming a better language every day.

So in short what’s coming is great but what we have is not as great? Not quite.

This script by no means optimized and should be seen only as a way to possibly implement Yielding in PHP as it is now and how to use in Magento for a finer and better data interaction with exports and imports. Notice that I unset the data as much as possible even tough the garbage collector should take care of it. However the only similarity between PHP’s yield and this script is that they are both named yield.

Make no mistake… to really wield the power of yielding you will need something that goes beyond the basic functionality this script provides.

As you can see there are 2 different versions, one for < 5.3 and one for 5.3 and greater. Since I can’t do math in my head I added an extra convert function and you should know that little function adds more memory to the mix… so take it out if you think your data is skewed. Try reading a file with this approach or a big array

<?php
echo "Current Memory: " . convert(memory_get_usage(true)), "<br/>";
function convert($size) {
    $unit = array('b', 'kb', 'mb', 'gb', 'tb', 'pb');
    return @round($size / pow(1024, ($i = floor(log($size, 1024)))), 2) . ' ' . $unit[$i];
}
function yield() {
    //echo 'Memory before processing the data: ', memory_get_usage(),'  in ',__FUNCTION__, ':';
    $args = func_get_args();
    //do something with data
    //echo '';
    //var_dump($args);
    // echo '';
    // echo 'Memory after processing the data: ', memory_get_usage(),'  in ',__FUNCTION__, ':';
    unset($args);
    return;
}
function process(array $someData = array()) {
    // do some computations that generates $data
    foreach ($someData as $data) {
        // echo 'Memory before yielding: ', memory_get_usage(),'  in ',__FUNCTION__, ':';
        call_user_func('yield', $data);
        // echo 'Memory after yielding: ', memory_get_usage(),'  in ',__FUNCTION__, ':';
    }
}
function process_53(array $someData = array()) {
    // do some computations that generates $data
    foreach ($someData as $data) {
        // echo 'Memory before yielding: ', memory_get_usage(),'  in ',__FUNCTION__, ':';
        call_user_func(function() {
                    //echo 'Memory before processing the data: ', memory_get_usage(),'  in ',__FUNCTION__, ':';
                    $args = func_get_args();
                    //do something with data
                    //echo '';
                    //var_dump($args);
                    // echo '';
                    // echo 'Memory after processing the data: ', memory_get_usage(),'  in ',__FUNCTION__, ':';
                    unset($args);
                    return;
                }, $data);
        // echo 'Memory after yielding: ', memory_get_usage(),'  in ',__FUNCTION__, ':';
    }
}
echo 'Let\'s do something fun';
echo 'Starting memory: ', convert(memory_get_usage(true)), '';
$longString = str_repeat(md5('php yield'), 2000); //heavy operation
$someData = array_fill(5, 60000, $longString); // this alone will raise your memory usage to the moon
echo 'Memory after getting the data: ', convert(memory_get_usage(true)), '';
process_53($someData);
unset($someData);
echo 'End memory: ', convert(memory_get_usage(true)), '';

Run that script and you will experience the power of php generators.

How is this useful?

To answer that you need to understand what a garbage collector is and how system memory is freed – by passing control back and forth to different functions, the memory usage by each function is freed up. Cool right? What’s better is the time it takes. Note that in real magentoland this script will use more memory than when you run it straight up.

I will do a follow up article with vagrant to install PHP 5.5 and use Magento with the Generators and maybe Zend Framework 2 since it is officially out. Stay tuned for more.

VN:F [1.9.22_1171]
Rating: 6.8/10 (10 votes cast)
VN:F [1.9.22_1171]
Rating: +2 (from 4 votes)
PHP Generators: the power of yielding, 6.8 out of 10 based on 10 ratings

Author: Luis Tineo

Husband, Father, performance improvement junkie, biker and video gamer, Linux user and in my day job I'm a Systems Architect at Blue Acorn.

Share This Post On
  • Pierre du Plessis

    Generators are included in the upcoming PHP 5.5, not 5.4

    VA:F [1.9.22_1171]
    Rating: 0.0/5 (0 votes cast)
    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • letas

      Pierre, thanks for message and sorry for the typo – it has been fixed now :-D

      VA:F [1.9.22_1171]
      Rating: 0.0/5 (0 votes cast)
      VA:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
  • ircmaxell

    This has nothing to do with generators. This is continuation passing syntax pure and simple… Please reword your post, as it confuses the issue with proper generators…

    VA:F [1.9.22_1171]
    Rating: 0.0/5 (0 votes cast)
    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • letas

      ircmaxell,

      I agree with you… this is not php yield and might not even be similar as PHP’s yield function is more robust and powerful. This is just an attempt at showing the world how to use a small fraction of the yield functionality right now. How would you name this article though? I can do that as long as it promotes the php’s yielding functionality

      VA:F [1.9.22_1171]
      Rating: 0.0/5 (0 votes cast)
      VA:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
  • Guest

    guytuutyuyuytuyu

    VA:F [1.9.22_1171]
    Rating: 0.0/5 (0 votes cast)
    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)