Precog: A code generation tool for PHP

Jon Ramsey

<jonathon.ramsey@gmail.com>

PHP London

What is code generation?

A method for transforming text from a single representation to another (which may or may not be code) programatically.

So does viewing this script count?:
<?php
echo "Hello world!";
?>
Well, kind of, but not really.

What is code generation?

If however we ran the above script like so:

$ php helloworld.php > hello.txt

… so that we could make later use of the hello.txt file, then we'd have more of a case.

This may or may not be useful.

What is code generation?

More generally we want to generate a number of artifacts from a single representation of a piece of knowledge; eg. for a CRUD app we'd like to generate as much as possible from a single representation of the database structure:

Why generate code?

Passive code generation

Passive code generation means generating skeleton code that needs some kind of human intervention before it is really useful, ie. using something like a wizard.

This means that every time you run the generator you will need to update the code – it's a bad smell.

Active code generation

Active code generators output fully formed code every time that they are run; the generated code requires no extra effort to function.

Active code generators can (and probably should) be run as part of a build process.

Cog: a Python code generator

Cog is a Python code generation tool created by Ned Batchelder.

Cog works in an appealingly simple way; chunks of python code embedded in files are executed and the output inserted into the original file, replacing the output from any previous runs.

I used Cog for a while on a project and grew to really like it — I've been itching to port it to PHP ever since …

Precog: a PHP code generator

A pure rip of Ned's excellent work ;)

Pretty simple to implement in PHP (so far), with a bit of output buffering and a soupçon of nasty eval().

Still needs a bit of work, but the basics are there.

Example usage

The basic idea:
<?php
/* A simple example */
/* [[[precog
for ($i = 1; $i < 11; $i++) {
    echo 'echo ' . $i . ";\n";
}
]]] */
/* [[[end]]] */
?>
when run through precog
$ precog example.php

Example usage

Generates the output:
<?php
/* A simple example */
/* [[[precog
for ($i = 1; $i < 11; $i++) {
    echo 'echo ' . $i . ";\n";
}
]]] */
echo 1;
echo
2;
echo
3;
echo
4;
echo
5;
echo
6;
echo
7;
echo
8;
echo
9;
echo
10;
/* [[[end]]] */
?>
Which is nice.

Real world usage

You aren't, of course, limited to generating echos of ints.

There is no need for the input file to be PHP, or for a cog section to output PHP code:

-- [[[precog
-- foreach (array('table1', 'table2', 'table3') as $table) {
--     echo "DROP TABLE $table;\n";
-- }
-- ]]]
DROP TABLE table1;
DROP TABLE table2;
DROP TABLE table3;
-- [[[end]]]

Real world usage

You can use whatever php you want within your cog sections:

<html>
  <body>
    <h1>blah blah blah </h1>
    <p>
<!-- [[[precog
echo "<p>\n";
include_once 'doc/generated.content.php';
echo "</p>\n";
class TestClass {
  var $a;
  function TestClass() {
    $this->a = 'my variable';
  }
}
$obj =& new TestClass();
echo "<p><strong>" . $obj->a . "</strong></p>";
]]] -->
<!-- [[[end]]] -->
  </body>
</html>

Real world usage

doc/generated.content.php:

<?php
echo str_repeat('blah ', 10);
?>

Real world usage

The output:

<html>
  <body>
    <h1>blah blah blah </h1>
    <p>
<!-- [[[precog
echo "<p>\n";
include_once 'doc/generated.content.php';
echo "</p>\n";
class TestClass {
  var $a;
  function TestClass() {
    $this->a = 'my variable';
  }
}
$obj =& new TestClass();
echo "<p><strong>" . $obj->a . "</strong></p>";
]]] -->
<p>
blah blah blah blah blah blah blah blah blah blah </p>
<p><strong>my variable</strong></p>
<!-- [[[end]]] -->
  </body>
</html>

Real world usage

You can also use multiple cog sections within the same file, and reuse the variables, functions etc. defined in one section within another:

<?php
/* [[[precog
$test = array('a', 'b', 'c');
function testing($in_array) {
    echo '$toast = "eggs";' . "\n";
    foreach ($in_array as $item) {
        echo 'echo "' . $item . "\";\n";
    }
}
]]]
[[[end]]]
*/
echo "blah!\n";
/* [[[precog
testing($test);
]]] */
$toast = "eggs";
echo
"a";
echo
"b";
echo
"c";
/* [[[end]]] */
?>

Real world usage

View source …

(Originally the file had a .php extension and had to run under a webserver running PHP — highlight_string() was run on the code examples directly. Now the file is run thru Precog, and viewing via a webserver is no longer required to see the code examples.)

A question from the audience …

(This question arose during the presentation at PHP London — it totally threw me and I provided no useful answer …)

Surely this is just what PHP does anyway? Couldn't we achieve exactly the same by embedding PHP directly in a file and running it through the PHP cli?

(Me, thrown): Errmmmm, yes, errmmm … (loses all confidence, aaargh, it was pointless writing it, aaarghh, it's so obvious, aarghhh)

Continued …

A question from the audience …

(Me, on later reflection, and borrowing from Ned again): Yes, we could of course.

It's not obvious from these slides, but Precog can (and typically would) write it's results back to the original file, rather than having separate generator and output files.

Because the code generating code is commented for the desired output, the file can be checked directly into source control systems and used without further modification; whenever the generating code is updated, Precog can be run on the file and the result checked in.

Continued …

A question from the audience …

So, to recap, benefits of Precog over straight PHP templating:

Continued …

A question from the audience …

Of course, I'm probably missing the point again ;)

ps. Both methods still require you to remember to run the code generation on the right files when the source data changes; hence the need to integrate either approach with a build system.

Wrapping up

A few resources