It is rarely the case that your unit tests actually test your database. It is however not so rare that the code your unit test tries to cover needs a database connection of some sorts.

In symfony 1.x you will find quite some tightly coupled code. Together with Doctrines (1.x) implementation of the active record pattern you will get a lot of exceptions complaining about no open database connection.

So if your tests need a database connection how should you do that?

I created a little experiment to test setting up database connections. If you’re interested you can reproduce those experiments with the sources on GitHub.

I basically tried a few combinations of the following code.

Setting up a database connection using the settings in the fixture projects databases.yml

Setting up a database connection using Doctrines mock adapter

Findings

You should always set up a database connection using the setUp() method of your testcase or a test method itself if it is the only method requiring a database connection. This way you only use the database when you actually need it.

I am still pretty amazed that using the mock adapter shows no functional difference to sqlite::memory: and is using up the same amount of memory.

Using –process-isolation on PHPUnit when executing a testcase or testsuite will drastically reduce the memory consumption but at the time of writing this (using PHPUnit 3.5RC1) it will produce a strange runtime exception when the database connection is made in the bootstrap file.

Oh and on the GitHub page you can also see the results of the experiments.

If you have a look at the symfony and Doctrine documentation you will notice that whenever you want to get the table object for a model you will call Doctrine_Core::getTable(‘ModelName’).

Apparently this is considered a best practice however I came to thing quite the opposite.

As you probably know I am quite keen on the topic of unit testing and that is exactly where Doctrine_Core::getTable() gets annoying.

It’s a static method that will be called in many methods of your classes. Mostly actions and components I suppose but also in other places.

Now when you want to test those methods that include a static call like this you will notice that you need to bootstrap a lot of symfony and Doctrine. Many times you will also need to open a database connection. But the method you want to test only calls to database related methods.

I would like to be able to mock all these database related methods in order to optimise execution speed of my unit tests. However you can not mock static methods (with PHP3.5 and PHPUnit 3.5 you will be able to) which means that you need to connect the database.

This may all sound a bit loopy but I just spend several hours optimising unit tests replacing calls to Doctrine_Core::getTable(‘BarFoo’) with new BarFooTable().

By now I have been writing quite a bit about how to utilise PHPUnit when developing symfony plugins. But symfony is of course still bundled with lime so before starting right away every plugin needs to be prepared with small changes like adjusting the bootstrap.

And because all these little changes have to be repeated over and over again for every plugin you start they tend to get tedious.

But I’ve got an idea!

Well credit where credit is due: it was actually Stefan Koopmanschap (skoop) who initially gave me the idea of putting my findings and best practices into a plugin.

So what is to be expected?

Well so far I plan the following three features:

1. A symfony task to change the default test bootstrap to work with PHPUnit
2. A symfony task  to create a best practice phpunit.xml.dist
3. A symfony task to create skeleton test cases for classes in your project

Of course the plugin itself will be PHPUnit tested (eating my own dog food).

What would you like to see implemented? Feel free to comment on quora.

PHPUnit offers quite a lot options to be set as arguments on the commandline. However this is tedious when typing over and over again.

For this reason you can create an XML configuration file phpunit.xml that will automatically be used by the phpunit binary.

Now you may want to include this configuration file in your projects sourcecode to be used by all participating developers and your continuous integration server. But how?

Well, you could just commit the phpunit.xml to you repository right?

Well, yes. But then all developers who want to checkout and work on your project will have to use these settings or going through the trouble of maintaining local changes to the configuration or specifying a different one using the commandline.

No! Use phpunit.xml.dist instead.

Sebastian made me aware of this and he is right. If you provide a phpunit.xml.dist(ribution) instead then every developer can chose whether to use your configuration or his/her own.

It is also wise to ignore the real phpunit.xml for your repository using .gitignore or svn:ignore or similar. This way everybody can use the settings he/she wants without forcing them onto others.

How can my continuous integration server use the config?

Simple! the phpunit binary is clever enough to use any phpunit.xml.dist file if no phpunit.xml is available.

I must admit that I have been lazy with my efforts on continuous integration lately. Eventually my server crashed unnoticed and I didn’t get any emails about broken builds anymore and by now I think I’ve stacked up some work to do.

First of course I’ve got to get my CI server up and running again, that’s why I installed Hudson again.

But there is room for improvement too. Sebastian Bergmann of PHPUnit fame spent some time on a template Hudson job for PHP projects that includes much more than PHPUnit. So I decided to use that!

Goal

What I want to achieve is to setup a project for my symfony plugin sfImageTransformExtraPlugin using the Sebastians template.

Prerequesites

I am assuming that you already have a working Hudson installation and the following plugins installed:

• Checkstyle (for processing PHP_CodeSniffer logfiles in Checkstyle format)
• DRY (for processing phpcpd logfiles in PMD-CPD format)
• HTML Publisher (for publishing the PHPUnit code coverage report, for instance)
• JDepend (for processing PHP_Depend logfiles in JDepend format)
• PMD (for processing phpmd logfiles in PMD format)
• Template Project (for using php-hudson-template as a template for Hudson jobs)
• Violations (for processing various logfiles)
• xUnit (for processing PHPUnit logfiles in JUnit format)

Setting up the template job

Just as described in the README on GitHub we just checkout the project in to our Hudson jobs folder.

$ cd /home/of/hudson/jobs
$ git clone git://github.com/sebastianbergmann/php-hudson-template.git

And then we have to restart Hudson to take notice of this new job.

The job itself is disabled as it has no subject.

Setting up the real job

So what I want to do is to checkout the current development version of my symfony plugin from GitHub. I also have to checkout it dependencies which are symfony itself and sfImageTransformPlugin which both are available only via Subversion.

I start by adding a new “free-style software project” job with the name “sfImageTransformExtraPlugin”.

First I configure the GitHub source and under the advanced options I configure it to be checked out into a local subdirectory “sfImageTransformExtraPlugin”.

Next I add a build step “Invoke Ant” and under advanced options I point it to the build.xml at “sfImageTransformExtraPlugin/build.xml”.

Last thing to configure is to “Use publishers from another project” and I enter “php-hudson-template” as the template project.

And that all you have to do to setup the project in Hudson.

But of course that’s not all. Two produce all the reports that the template should process and to checkout and update the dependencies we need two more configuration files which are both part of the plugin.

This is more or less a copy of the example from Sebastians GitHub Page with soem slight modifications.

The first target “update” will check out the dependencies from symfonys Subversion repositories in parallel to the plugin itself. The plugin is configured to assume symfony and sfImageTransformPlugin at these locations.

All artifacts such as coverage reports and metrics will be written to a directory “build” which also lies in parallel to the plugin source. All together will produce a directory structure like this.

/path/to/hudson/jobs/sfImageTransformExtraPlugin/build/
/path/to/hudson/jobs/sfImageTransformExtraPlugin/sfImageTransformExtraPlugin/
/path/to/hudson/jobs/sfImageTransformExtraPlugin/sfImageTransformPlugin/
/path/to/hudson/jobs/sfImageTransformExtraPlugin/symfony/

But not all reports that are expected by the template are being produced by this build.xml. The PHPUnit specific reports are configured in the phpunit.xml file.

Adjust these two configuration files for your plugin to your needs and you’re ready to go.

In the following weeks I will write some posts explaining all the results you can now see in your Hudson installation (or in mine).

Conclusion

This template makes setting up Hudson for PHP development so much easier! Great thanks to Sebastian for his awesome work!

The feature discussed below has been removed from PHPUnit due to community feedback.

Over the years, I have gotten quite a few "complaints" from PHPUnit users that they do not like typing $this-> as often as they have to:

<?php
class StackTest extends PHPUnit_Framework_TestCase
{
    public function testPushAndPop()
    {
        $stack = array();
        $this->assertEquals(0, count($stack));
 
        array_push($stack, 'foo');
        $this->assertEquals('foo', $stack[count($stack)-1]);
        $this->assertEquals(1, count($stack));
 
        $this->assertEquals('foo', array_pop($stack));
        $this->assertEquals(0, count($stack));
    }
}
?>

As of PHPUnit 3.5, they can write test code that requires less $this-> statements:

<?php
require_once 'PHPUnit/Framework/Assert/Functions.php';
 
class StackTest extends PHPUnit_Framework_TestCase
{
    public function testPushAndPop()
    {
        $stack = array();
        assertEquals(0, count($stack));
 
        array_push($stack, 'foo');
        assertEquals('foo', $stack[count($stack)-1]);
        assertEquals(1, count($stack));
 
        assertEquals('foo', array_pop($stack));
        assertEquals(0, count($stack));
    }
}
?>

Here's hoping that this makes some people happy :-)

I have been invited to attend the 5th Annual Google Test Automation Conference, better known as GTAC, in Hyderabad, India in October. I am very much looking forward to discuss cutting edge challenges in test automation and evaluate potential solutions, especially with this year's focus on testability.

Another thing I like about this year's GTAC is that the participants are responsible for selecting the presentations for the conference. Here is my submission:

Challenges in Unit Testing PHP Applications

According to TIOBE, PHP is the most popular programming language after C/C++ and Java. The language has made strong inroads into large-scale, business-critical Web systems. In the six years since the release of PHP 5 -- which not only kickstarted the development of PHP-based frameworks for Web development but also the development of tools for dynamic and static testing techniques -- the PHP community as a whole has developed an increasing interest in developing software that delivers the best possible quality.

When PHP developers start to write unit tests they rarely find themselves without any constraints that are imposed by prior work of less than optimal quality. It is a well-known fact that writing unit tests for legacy code is hard. In the case of PHP it can be even harder: the legacy code has not only been written without testability in mind, but it may have been written for earlier versions of PHP that encouraged practices that make the code next to impossible to unit test.

PHPUnit, the de-facto standard framework for unit-testing PHP code, has some unique features not found in other xUnit test frameworks that allow the testing of untestable code. While developers should not use these features (as they are not required when writing tests for testable code), these features ease the pain of writing tests for legacy code and thus help developers get started with unit testing before they refactor the code for testability.

This session, presented by the creator of PHPUnit, highlights the challenges developers are facing when unit testing legacy PHP code. Some of these challenges will be familiar to developers that use other programming languages such as Java but they will see a new perspective on the problem and different approaches to solve it.

Although I am hoping, of course, that my submission will be accepted by my peers, I know that GTAC will be valuable for me even if I do not get to present: the "hallway track" of GTAC 2008 was amazing.

When you look at the list of changes for PHPUnit 3.5, you will see that many of them deal with refactoring to components. Here is an overview of these new components:

• PHP_CodeCoverage

  The collection, processing, and rendering of code coverage information has been factored out into a separate component. A bit more information can be found here.

• PHPUnit_MockObject

  The functionality to automatically generate an object that can act as a test double for a specified original class has been factored out into a separate component. Do not worry, getMock() and related methods will work just as they did in previous versions of PHPUnit. The refactoring, however, makes the usage of other mock object libraries (such as Mockery or Phake, for instance) easier.

• DbUnit

  The database testing functionality that is provided by the DbUnit extension and the respective DatabaseTestCase class has been moved to a separate component. Michael Lively Jr, the author of DbUnit, is now able to make releases of the database testing extension on a release schedule that is separate from PHPUnit itself, meaning that I will be even less involved in its development than I have before.

• PHPUnit_Selenium

  SeleniumTestCase, the Selenium RC integration for PHPUnit, has been moved to a separate component to allow a release cycle separate from PHPUnit itself.

• File_Iterator, PHP_Timer, Text_Template

  Utility methods to collect files and report resource usage as well as a simple templating mechanism have been moved to separate components so that they can be reused by other tools such as phpcb, phpcs, phpcpd, and pdepend.

I was asked to speak at the symfony day 2010 in Cologne and I happily agreed to it. My session will be about Unit testing symfony plugins with PHPUnit.

While I have a clear view what to talk about within the one hour I have I wonder what your expectations are?

So instead of writing down what I want to present I ask you to have your say to give a good user experience.

• What are the topics you want to hear about?
• What is the level of knowledge I should assume the audience to have?
• Do you expect a presentation or hands on examples?
• Theory or practice?
• What should I wear?

Comments please!

Yesterday I wrote a lot of functional tests for a symfony plugin that provides and XML web service interface.

Using symfonys functional testing classes this is actually quite easy to do.

But wouldn’t it be cool if you could integrate these tests into your continuous integration service just like PHPUnit tests? Wouldn’t it be cool to be able to generate PHPUnit coverage reports?

Well I thought so too that’s why I decided to write a very simple PHPUnit test case that executes the functional tests (using lime) and runs asserts on the output of those.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

<?php
 
/** central bootstrap for unit tests */
require_once dirname(__FILE__).'/bootstrap/functional.php';
/** PHPUnit Framework */
require_once 'PHPUnit/Framework.php';
 
class yourPluginFunctionalTest extends PHPUnit_Framework_Testcase
{
  public function testIndex()
  {
    $this->assertLimeReport('Index');
  }
 
  public function testShow()
  {
    $this->assertLimeReport('Show');
  }
 
  private function assertLimeReport($action)
  {
    ob_start();
    include(dirname(__FILE__).'/functional/yourModuleActions'.$action.'Test.php');
    $output = ob_get_clean();
    $lines = explode("\n", $output);
    foreach($lines as $line)
    {
      if(trim($line) && '#' != $line[0] && '>' != $line[0])
      {
        $this->assertEquals('ok', substr($line, 0, 2), $line);
      }
    }
  }
}

You can see that I used a simple convention how to name the functional tests. There is actually one file per action with the action name encoded in the file name. In the above example I run the tests for executeIndex() and executeShow().

The rest is simple as well (and can for sure be done far more elegant). I just iterate over the lime output and check for result lines that start with ok.

I also wrote a small phpunit.functional.xml configuration file so the coverage reports only considers my module code (all other code is unit tested of course).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

<?xml version="1.0" encoding="UTF-8"?>
 
<phpunit backupGlobals="false"
         backupStaticAttributes="false"
         colors="false"
         convertErrorsToExceptions="true"
         convertNoticesToExceptions="true"
         convertWarningsToExceptions="true"
         processIsolation="false"
         stopOnFailure="false"
         syntaxCheck="false">
 
  <filter>
    <whitelist addUncoveredFilesFromWhitelist="true">
      <directory suffix=".class.php">./modules/yourModule/actions/</directory>
      <directory suffix=".class.php">./modules/yourModule/lib/</directory>
    </whitelist>
  </filter>
</phpunit>

Running this test case is easy.

$ phpunit --configuration=phpunit.functional.xml --coverage-html=/path/to/your/report test/yourPluginFunctionalTests.php

The coverage report comes in quite handy as I can now see which methods are not executed and are probably not needed any longer (I did some heavy refactoring on this).

The only thin I can not get rid of is that lime will only have the last word telling me “# Looks like everything went fine.“.

This is due to limes use of register_shutdown_function() which unfortunately does not have a counter part to unregister..

Yesterday I attended the The State of Symfony 2 online conference. Actually my first online conference so it was quite exciting.

One of the most exciting questions though was: What is this new killer feature? Well, I made my guess before but lets see if I was right.

A few minutes before the first session there were already more than 100 people connected to the collaboration platform (http://www.elluminate.com/). Not a bad turnout for “just an online conference”.

The tool features moderation, listening as well as speaking, voting, a whiteboard, file transfers, a group chat and probably a lot more. It seemed a suitable choice.

Please excuse this article to be somewhat bullet-point-ish but I wrote most of it while listening to the sessions and I guess you will get the meaning anyway.

Miscellaneous enhancements Fabien Potencier

The philosophy of Symfony 2 is to be as flexible as possible but with sensible defaults so it can be used right out of the box but possible to be changed to everything you need. Symfony 2 now also adapts PEAR standards to allow easy integration of non-symfony libraries rather than setting its own standards.

As much as possible went was put into the components to make sure they don’t depend on the MVC framework and can run standalone. This as well eases the use by Symfony newbies as well as experts.

Fabien introduced the latest additions to the components library.

• CssSelector developed from scratch converts CSS selector to their XPath equivalents.
• DomCrawler was based on symfony 1 browser class and can navigate the DOM. Its methods are chainable, it can utilise CssSelector but uses XPath internally and its API feels jQuery like.
• Browser is an HTTP client implemented in PHP that can even manage cookies.
• Finder is completely rewritten from its symfony 1 version and now implements PHPs Iterator interface. It’s chainable and provides easy to use date filters. The method signatures are standardised and instead of file handles it returns SplFileInfo instances. It can also work with streams like on the Zend FWs Amazon S3 Stream.
• HttpKernel is a framework construction kit and replaces the former RequestHandler. It has a very simple interface with only two methods getRequest() and handle(). It also includes a Profiler and the WebDebugToolbar functionalities.

All components are decoupled and therefore fully testable.

With these and the already known components Symfony 2 is not yet ready but the main concepts and overall philosophy will not change anymore.

From the Q&A at the end of the session we learned that at this point there will be no migration tool from symfony 1 to Symfony 2 but there will probably be an intermediate layer once Symfony 2 is ready.

However the API can change at any time! So for production use sf1.4 is still recommended!

The first alpha version planned for September.

Symfony 2 Meets Propel 1.5 François Zaninotto

I don’t really use Propel but still I was interested in its latest developments.

Propel 1.5 is fully backwards compatible with Propel 1.3 and 1.4. It’s faster, very IDE friendly, better documented, has three times more unit tests as Propel 1.3 and is fully integrated into symfony 1.4 using the sfPropel15Plugin.

Announced as one of the killer features come the model queries which are supposed to be to SQL what ActiveRecord is to the table row. To me it looked like a clone of Doctrines DQL but with domain specific method names..

Next killer feature and missing in Doctrine 1.2 is concrete table inheritance! But on the database level it is actually denormalised meaning that columns defined on the parent model are stored in the child tables as well for performance reasons. I wouldn’t really consider this to be concrete but composite and I remember having done something similar in Doctrine as well..

The third killer feature still to come will be the aggregate_column behaviour which utilises aggregation functions and stores and maintains their results in extra table columns.

Well, Propel is not dead but I am not convinced of its killer instinct and frankly I am annoyed by all this Doctrine bashing. If Propel is a good ORM (what it is to many users) then I should be able co-exist with other solutions! I think the major difference between Propel 1.5 and Doctrine 2 is that Propel is still an ActiveRecord which some might prefer. Not me though.

Funnily after the presentation a poll was setup whether to prefer Propel or Doctrine and when I had a look it was 76% favouring Doctrine. http://twtpoll.com/r/545mw5. Nowhere near representative but entertaining.

What’s new in the Doctrine 2 Symfony Integration Jonathan Wage

Jonathans session didn’t bring too many news for you when you followed his talks and tweets regularly but it was a good and up to date wrap up of the current implementation.

Next to the DoctrineBundle for Symfony 2 there now is also a MondoDB Object Document Mapper bundle (ODM not ORM!) and a bundle for the beloved Doctrine Migrations with unchanged functionality.

Here’s Doctrines Architecture in a nutshell.

• The DBAL is responsible for managing connections and creating/dropping databases and running dql from the command line.
• The ORM has the EntitiyManager which persists and retrieves entities. You can have multiple instances (i.e. per database connection) of it.
• An Entity is a regular PHP object, no more magic, clean and testable, fast and only limited by PHP and now you can use the constructor as you like to.

One nice little thing I like is the new command line task that ensures your production settings! But also tasks for cache control are a nice addition.

DQL now uses a proper lexer/parser and has a strict BNF which allows for helpful syntax errors rather than side effects!

During development you can now change your schema without throwing away your database. Instead a schema update compares your current database layout and your current schema changes to update only what is necessary.

The MongoDB ODM shares the same architecture as ORM only with a DocumentManager instead of an EntitiyManager. Its DocumentManager makes full use of the MongoDB API.

You can even remap objects from Document to Entity and back during development and even during runtime. So fetching a model from the database and saving it to MongoDB should be possible.

Currently only MongoDB is supported and there is no support for CouchDB yet.

Finally Doctrines ORM supports class table inheritance but after seen the Propel version of it I have to take deeper look before I’m amazed.

Unit & Functional Tests Fabien Potencier

Symfony 2 switched from lime to PHPUnit 3.5 because of standardisation (top prio as the framework is easier to learn that way). lime at the time was an easy option to PHP testing when unit testing had not yet arrived in the PHP world. By using PHPUnit Symfony can now benefit from yet another community while focussing on the framework and some improvements went back to PHPUnit already.

While this is great in itself it gets better when Fabien shares two best practices.

• There should be no more AllTests.php. Instead you can use phpunit.xml (it has a bootstrap attribute!).

• Don’t unit test your controllers! Controllers are not unit tested because they should only contain glue code and no business logic.

The functional test framework now has a cleaner API and utilises DomCrawler and Browser components.

Functional tests now only test the response object, which is exactly what you receive then the request is handled by your controller.

You can have multiple instances of the Browser so that you can even test the interaction of multiple clients. Even when you use a library with static states this will work as for each Browser a new PHP process will be created.

Because the functional tests are using the Browser component they can even be run on your production environment for monitoring purposes utilising the profiler of the request. Even testing of non-symfony websites is possible!

The new form framework Bernhard Schussek

After all the experience with the form framework of symfony 1 the new incarnation now got a whole new architecture and was completely redesigned for simplicity and reusability.

“Symfony 2 embraces your domain model”

Validators now validate domain models and forms.

Form fields are fully culture aware (localised) and can be grouped to “lightweight subforms” that become a widget like i.e. a google maps locator widget.

CollectionFields can be used to add lists of fields for 1-n relations.

The form framework is of course also a standalone library that does not depend on without Symfony 2.

Constraints can be added using annotations or XML or Yaml even using callbacks on your models so you can easily implement dynamic constraints.

If forms are now directly modelled on your domain objects then how can you do embedded forms? Well you can’t or at least not as you do in symfony 1. For example if you want to have a form for a user registration based on the user model but with additional fields for terms and conditions or newsletter subscriptions then you have to create a new model Registration first. This may seem like additional effort but if you think about it it makes a lot more sense.

Constraints can also be sequenced i.e. to save on resources when validating cheap constraints first before validating more costly ones that may contain a web service call.

HTML 5 input fields are not yet supported but the forms and validators and fields and everything is fully internationalisationable.

This sounds like something to really look forward to as the forms of symfony 1.2+ were certainly better than previous versions but had a lot of design flaws which make them feel unintuitive.

The Symfony 2 Killer Feature… Fabien Potencier

I was right!

“Caching on the Edge”

According to yet another benchmark test Symfony 2 will be much faster than symfony 1 and even faster with more concurrent requests. All this is caused by the new killer feature: the cache.

Just like Varnish, Squid or Akamai Symfony 2 provides an HTTP cache or reverse proxy implemented in PHP. Symfony 2 also implements the related HTTP specifications and uses the cache headers for controlling the cache behaviour. This means that you can easily replace the PHP cache with Varnish or Akamai without the need to change.

And the cache classes are very easily integrated as they decorate the kernel. They can therefore be activated even for functional tests.

What about when parts of the page must be cached less?

Yes the best bit is still to come. Symfony 2 also implements the Edge Side Include (ESI) specifications written by Akamai which can be used to cache parts of the pages differently from the rest. This is accomplished by the view part of the MVC pattern which can return every partial separately if I understood correctly. You can even use other webpages as partials. Well, I have to digg into this at some point but I am utterly amazed of the completeness of the design.

I can see that it forces you to better separation of concerns as you have to think about your cache times early in the process. Which is good.

To sum it all up: the cache layer is separated from the framework. This does not only mean that you can exchange it with other reverse proxies it only means that for each page request that can be answered by the cache layer the kernel of the framework is not even booted! Symfony 2 also implements stale-while-revalidate and stale-if-error. Go and google it if you don’t know what this is, it is absolutely fantastic!

Another best practice advise:

If you have to think about invalidating the cache then you are most likely using expiration instead of validation. In other words: for dynamic parts you should let the cache validate the cache instead of setting an expire time. Cache validation is soon going to be automatically optimised. I’m curious how this will work.

However great this all sounds for now you have to remember that everything is still experimental.

Conclusion

Well what can I say except that Symfony 2 feels so right on so many levels. This might even have a big impact on the whole PHP community. Can’t wait until it’s stable in 2011!

Today I have released the second beta version of phpUnderControl. Beside several minor tweaks and bug fixes, this release contains one additional feature I was asked for during the IPC in Berlin. This feature allows you to specify a maximum number of log entries that will be shown in the generated metric charts. This can be very useful once you have a project with a great amount of builds and the chart rendering gets slower and slower.

This feature adds a new option --max-number to phpUnderControl's chart command. To rebuild all your charts you can call phpUnderControl's shell script with the following command:

  mapi@arwen ~ $ phpuc --force-update --max-number 42 \
                 /opt/cruisecontrol/logs/phpUnderControl \
                 /opt/cruisecontrol/logs/phpUnderControl

You can get the latest release of phpUnderControl through its pear channel:

  mapi@arwen ~ $ pear upgrade --alldeps phpuc/phpUnderControl-0.6.0beta2
  Starting to download phpUnderControl-0.6.0beta2.tgz (546,314 bytes)
  ...................................................................
  .............................done: 546,314 bytes

or you can get the latest development version on github.

As I mentioned before I am working on a small framework to implement custom stream wrappers.

For example you can later use this to easily wrap access to a web service or similar.

As of today I reached a 90+% code coverage and everything passes. Check out the sources and see if some test does not work for you.

I already got the feedback that running Windows there are a few minor issues that I will take care of soon.

The unit tests themselves might not be the easiest on the eye right now but I am getting there. Also the documentation is not yet written.. remember this is still alpha!

So feel free to try and break and contribute by checking out or forking StreamHitching on GitHub.

To celebrate the 15th birthday of PHP, I have released PHPUnit 3.5 Beta 1 today. The refactoring towards components is a "new feature" that is already visible upon installation:

sb@vmware ~ % pear install phpunit/PHPUnit-beta
phpunit/PHPUnit can optionally use PHP extension "dbus"
downloading PHPUnit-3.5.0beta1.tgz ...
Starting to download PHPUnit-3.5.0beta1.tgz (105,588 bytes)
........................done: 105,588 bytes
downloading DbUnit-1.0.0beta1.tgz ...
Starting to download DbUnit-1.0.0beta1.tgz (38,209 bytes)
...done: 38,209 bytes
downloading File_Iterator-1.2.1.tgz ...
Starting to download File_Iterator-1.2.1.tgz (3,225 bytes)
...done: 3,225 bytes
downloading Text_Template-1.0.0.tgz ...
Starting to download Text_Template-1.0.0.tgz (2,493 bytes)
...done: 2,493 bytes
downloading PHP_CodeCoverage-1.0.0beta1.tgz ...
Starting to download PHP_CodeCoverage-1.0.0beta1.tgz (108,640 bytes)
...done: 108,640 bytes
downloading PHP_Timer-1.0.0.tgz ...
Starting to download PHP_Timer-1.0.0.tgz (2,536 bytes)
...done: 2,536 bytes
downloading PHPUnit_MockObject-1.0.0beta1.tgz ...
Starting to download PHPUnit_MockObject-1.0.0beta1.tgz (15,816 bytes)
...done: 15,816 bytes
downloading PHPUnit_Selenium-1.0.0beta1.tgz ...
Starting to download PHPUnit_Selenium-1.0.0beta1.tgz (15,298 bytes)
...done: 15,298 bytes
downloading PHP_TokenStream-1.0.0beta1.tgz ...
Starting to download PHP_TokenStream-1.0.0beta1.tgz (7,023 bytes)
...done: 7,023 bytes
install ok: channel://pear.phpunit.de/File_Iterator-1.2.1
install ok: channel://pear.phpunit.de/Text_Template-1.0.0
install ok: channel://pear.phpunit.de/PHP_Timer-1.0.0
install ok: channel://pear.phpunit.de/PHP_TokenStream-1.0.0beta1
install ok: channel://pear.phpunit.de/PHP_CodeCoverage-1.0.0beta1
install ok: channel://pear.phpunit.de/PHPUnit-3.5.0beta1
install ok: channel://pear.phpunit.de/DbUnit-1.0.0beta1
install ok: channel://pear.phpunit.de/PHPUnit_MockObject-1.0.0beta1
install ok: channel://pear.phpunit.de/PHPUnit_Selenium-1.0.0beta1

Happy Birthday PHP! And have fun testing (with) PHPUnit 3.5!

This question came up in the comments to yesterdays post.

The commenter WC summed it up pretty well I think but I am going to summarise it anyway.

I a lot of best practice books or blog posts you can read that only a single assertion is advised per test case.

The reason is simple.

A test case will return failed as soon as one of its assertions fails. The consequence of this is that any succeeding asserts are no longer executed so you wont know if they pass or fail until you fixed the first assertion.

There are exceptions to this rule though.

I just recently had to test a stream wrapper and one of the methods to test was stream_stat() which is called by PHP in return of calling fstat().

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

  public function testFstat()
  {
    $fh = fopen($this->url, 'r');
    $stat = fstat($fh);
    $this->assertType(PHPUnit_Framework_Constraint_IsType::TYPE_ARRAY, $stat, 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('dev', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('ino', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('mode', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('nlink', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('uid', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('gid', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('rdev', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('size', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('atime', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('mtime', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('ctime', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('blksize', $stat), 'fstat — Gets information about a file using an open file pointer');
    $this->assertTrue(array_key_exists('blocks', $stat), 'fstat — Gets information about a file using an open file pointer');
    fclose($fh);
  }

This might not be the prettiest test case but you can see a lot of succeeding assertions.

The reason why I think this is a valid exception to the otherwise reasonable rule is that all of these assertions test for a detail of a single operation.
Especially the TYPE_ARRAY test would make no real sense alone it would just blow up the code.

Any thoughts?

In the last couple of months I have occupied myself a lot with unit testing. Once you get the hand of it it starts to feel natural and easy and you don’t want to miss it any more. Testing rocks!

Another thing I worked with are PHP streams which IMHO is one of the most underrated features of PHP.

Naturally I want to test my stream wrappers with PHPUnit just as I would test everything else. But it’s not that easy..

When you develop a software component you design it to be used and with your unit tests you simulate this usage as a warranty that your changes to the component won’t break the code using it.

With stream wrappers this becomes a bit tricky as you don’t have an easy access to the code calling your component.

For example you will most likely implement stream_open() which is called internally by fopen() and that’s the difficulty.

This is a part your stream wrapper:

  public function stream_open ( string $path , string $mode , int $options , string &$opened_path )
  {
    ..
    // Returns TRUE on success or FALSE on failure.
  }

And this will be the part of the code using it:

..
$fh = fopen ('yourprotocol://some/path/and/file.name' , 'r', false, $context);
..

Usually I would attempt to test stream_open() with a test like this:

  public function testStreamOpen()
  {
    $wrapper = new MyStreamWrapper();
    $this->assertTrue($wrapper->stream_open('myprotocol://some/path/and/file.name' , 'r'));
  }

But have you noticed that the signatures of  stream_open() and fopen() is different after the second parameter?

Somewhere within the C code of PHP the call to fopen() will be implemented and eventually it will make a call to your stream_open(). In the meantime it will have passed the context (which needs to be a public attribute of your stream wrapper and it will calculate some options and an opened path.

Writing a test as I did above will probably work in most cases. But how can you be sure that every possible call to fopen() is working well when you don’t know what happens between the two methods?

You could have a thorough look at the sources of the PHP binary of course and I won’t stop you as this will be an interesting read. But I think an easier approach would be to change the unit test to something like this:

  public function testStreamOpen()
  {
    stream_wrapper_register('myprotocol', 'MyStreamWrapper');
    $this->assertNotFalse(fopen('yourprotocol://some/path/and/file.name' , 'r'));
    $this->assertNotFalse(fopen('yourprotocol://some/path/and/file.name' , 'w'));
    $this->assertNotFalse(fopen('yourprotocol://some/path/and/file.name' , 'r', true));
    $this->assertNotFalse(fopen('yourprotocol://some/path/and/file.name' , 'r', false, stream_context_create(array('yourprotocol'=>array(...)))));
    $this->assertFalse(fopen('yourprotocol://some/path/and/file.name' , 'r', true, stream_context_create(array('yourprotocol'=>array(...)))));
    ..
    stream_wrapper_unregister('myprotocol');
  }

So instead of testing your stream wrappers methods directly you test if all defined usage of your stream wrapper is functioning by calling the appropriate PHP filesystem functions.

In my current case I have a stream wrapper that represents a single read only file. So I wrote tests for each PHP filesystem function that attempts to work on a file assuming that write access fails while read access passes.

Additionally I wrote unit tests calling the stream wrapper methods directly just to make sure.

It’s been a while since David Sklar called out to let a thousand string concatenations bloom. That discussion produced some entertaining suggestions for putting strings together such as using preg_replace and calling out to MySQL with SELECT CONCAT.

Here’s an approach that uses filesystem functions. When combined with some lesser-known PHP streams functionality, it has several practical applications.

Opening Files for Reading and Writing

There are several modes for opening a file that will allow you to seek to arbitrary positions in the file and read or write at those positions. One of the most frequently used of these modes is w+, which the tmpfile function uses automatically.

We can repeatedly write, then rewind the pointer and read:

$f = tmpfile();
fwrite($f, 'foo');
fwrite($f, 'bar');
 
rewind($f);
$contents = stream_get_contents($f);  //=> "foobar"
fclose($f);

When writing to the filesystem, the above provides yet another inefficient solution to David’s exercise. Now let’s take it a bit further to see how this can be useful.

In-Memory Streams

PHP 5.1 introduced two new in-memory streams: php://memory and php://temp. The php://memory stream operates entirely in memory. The php://temp stream operates in memory until it reaches a given size, then transparently switches to the filesystem.

We can modify the above example to use the php://memory stream instead of hitting the filesystem:

$f = fopen('php://memory', 'w+');
fwrite($f, 'foo');
fwrite($f, 'bar');
 
rewind($f);
$contents = stream_get_contents($f);  //=> "foobar"
fclose($f);

Putting a string inside a fast temporary stream can be very useful. For example, we can then attach filters to that stream.

Testing

Temporary streams are also handy for testing. There are some rather elaborate virtual file system libraries out there but many times a stream is all you need.

Zend_Log has a log handler for streams that accepts either a filename or a stream resource. We can configure it with a php://memory stream for testing:

$f = fopen('php://memory', 'w+');
$writer = new Zend_Log_Writer_Stream($f);
$logger = new Zend_Log($writer);

Assuming your well-designed application has a convenient injection point for the logger instance, your test can pass it in before your test activates some action which should result in logging:

$logger->crit('critical message worth testing');

When the action has completed, the test can rewind $f and use it as a test spy.

rewind($f);
$contents = stream_get_contents($f);
$this->assertRegExp('/message worth testing/i', $contents);  // PHPUnit

Not surprisingly, my unit tests for Zend_Log use this same technique.

Application Usage

Not long ago, we built a custom document storage system for a client. One of its more interesting features is that it integrates with an internet fax service so users can select documents in the system and then fax them. For each fax, the software automatically generates a cover page.

To make the cover page, I first made a nice template using a drawing tool and then saved it in PDF format. I then used Zend_Pdf to write over the template with dynamic content. I first demonstrated this technique in this article.

Since the cover page is only used once (during transmission) and easy to regenerate, I don’t save the output to the filesystem. Instead, I create a php://temp stream. The instance method Zend_Pdf->render() writes the PDF output only to that stream, which is then rewound. Functions like stream_copy_to_stream or fpassthru can then be used to send the final output where it needs to go, and the whole process normally never needs to use the disk.

Slides from my two talks at OSCON 2008 are now available:

• Integration Testing PHP Applications
• Supervisor as a Platform

Both talks were well attended had great audience participation. Thanks to everyone that attended and I hope you enjoyed them.

About seven people came up after the Integration Testing talk with good questions and feedback. I think that 45 minutes was enough to provide good starting points for the topics covered but it was clear that these users were engaged and ready to dig into it more. If I give this talk again, it is going to be in a tutorial format so we can get much deeper into the material and have some exercises.

After my Supervisor talk at the end of the day, I had the pleasure of going out to dinner with Roger Hoover and some other Supervisor users. It’s been very exciting to see Supervisor picking up traction over the past year. Besides the work that Chris McDonough and I are doing, Supervisor is being used in several large architectures of companies whose names you know. If you’re using Supervisor and wouldn’t mind us telling others about it, please contact me.

The OSCON 2008 website has published its talk schedule. I’ll be giving two talks at OSCON this year; one on the Python track and one on the PHP track.

Supervisor as a Platform

I will quickly introduce you to Supervisor and the immediate benefits of running your server processes under it. We will then dive into how applications written specifically for Supervisor can take advantage of it as a platform — writing your own event listeners to observe Supervisor and the process lifecycle, controlling with XML-RPC, and extending the Supervisor core with your own Python extensions.

This will be an expanded version of the talk I gave with Chris McDonough at PyCon 2008. Since PyCon, there’s been quite a few interesting developments in Supervisor like the ability to extend supervisorctl and progress made on configuration reloading. We’ll touch on these as well, so if you attended the PyCon talk there will still be new and interesting material in this talk for you.

Integration Testing PHP Applications

While more PHP developers are recognizing the importance and benefits of unit testing, the uptake of PHP developers using automated integration or acceptance testing is relatively slow. This testing is equally crucial to maintaining the integrity of applications.

I’m going to help get you started testing at the application level with practical tips and source code. We’ll look at how to structure your HTML markup so it’s more easily testable, making tests easier to write and maintain with CSS selectors, organizing your tests, and testing with or without a browser.

We write a lot of PHP applications with this kind of testing at Maintainable. Before the conference, we plan to release some open source PHP code to help you test that we’ll cover in the talk as well. I’d also suggest you check out Sebastian Bergmann’s tutorial session on PHPUnit’s integration with Selenium RC.

I started this blog to occupy myself with topics like unit testing, continuous integration and code quality in general. As a subject for my studies I quickly released a symfony plugin (sfImageTransformExtraPlugin) that I already worked on for a while and decided to excercise all my learnings on it. So you can peek at the sources to see more details.

As all the early posts of this blog were kept in German I am going to summarise the best practice of PHPUnit testing a symfony 1.4 plugin in this post.

Of course I have to say that the term best practice refers to my current state of knowledge.

Why PHPUnit?

Actually when I started this blog I was determined to use symfonys own lime unit test implementation because it is integrated and documented within the symfony documentation. But the more I worked with it the more I was put off by it.

lime is ok for being run during development by the developer itself but it can not easily be used in continuous integration as it is not fully xUnit compliant and does not provide all infos you want to see there. lime is also very procedual and the more complex your application become the more spaghetti code you will have to maintain. And foremost lime is only used in the symfony world and learning it will not have any benefit for you in other projects.

PHPUnit on the other hand is the de facto standard for PHP. It is object oriented as well as well documented and maintained and it is fully xUnit compliant thus can easily be integrated into your continuous integration workflow.

Prerequisites / Assumptions about your plugin

In the following I assume that you generated your plugin by running the symfony task generate:plugin (provided by sfTaskExtraPlugin) and that you did not change the files and directories inside your plugins /test directory.

This is important as there are many files that were generated for you that may look curious to you but will make a lot more sense once you learned how to use them.

Of course these files were created with lime in mind and in the following I will show you what you need to change to use the same structure with PHPUnit.

Bootstrapping

In your plugins /test/bootstrap you will find a script for bootstrapping for your unit tests unit.php.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

// /plugins/myTestPlugin/test/bootstrap/unit.php
<?php
 
if (!isset($_SERVER['SYMFONY']))
{
  throw new RuntimeException('Could not find symfony core libraries.');
}
 
require_once $_SERVER['SYMFONY'].'/autoload/sfCoreAutoload.class.php';
sfCoreAutoload::register();
 
$configuration = new sfProjectConfiguration(dirname(__FILE__).'/../fixtures/project');
require_once $configuration->getSymfonyLibDir().'/vendor/lime/lime.php';
 
function myTestPlugin_autoload_again($class)
{
  $autoload = sfSimpleAutoload::getInstance();
  $autoload->reload();
  return $autoload->autoload($class);
}
spl_autoload_register('myTestPlugin_autoload_again');
 
if (file_exists($config = dirname(__FILE__).'/../../config/myTestPluginConfiguration.class.php'))
{
  require_once $config;
  $plugin_configuration = new myTestPluginConfiguration($configuration, dirname(__FILE__).'/../..', 'myTestPlugin');
}
else
{
  $plugin_configuration = new sfPluginConfigurationGeneric($configuration, dirname(__FILE__).'/../..', 'myTestPlugin');
}

There are two things to notice here:

1. The use of $_SERVER['SYMFONY']

When executing unit tests you have to tell the bootstrap where to look for the symfony core libraries. For lime this is actually done by the symfony test tasks but for PHPUnit we don’t have a task available so we need to run the following command first (only once per console session).

$ export SYMFONY="/absolute/path/to/lib/vendor/symfony/lib"

I will explain the reason behind this in a moment.

2. The project configuration is looaded from a fixture project

If you have a look at your plugins /test/fixtures/project directory you will recognise a complete symfony project just like you run generate:project.

This is going to be your test environment.

The reason for both of these two features is this:

The whole point of a symfony plugin is to be reusable and to be reusable it needs to be as project independant as possible.
Therefor it will not be unit testet in a project where the project developers can easily create situations where your tests will fail but in a fixture project you as a plugin developer can completely control.

And because this fixture project is included in your project we don’t want to also include the whole symfony code.
But a symfony plugin will always have at least one dependancy and that is towards symfony itself, so we have to make the location of it known to our plugin before running the tests.

Now for PHPUnit we need to remove the following line from the code above.

13

require_once $configuration->getSymfonyLibDir().'/vendor/lime/lime.php';

The purpose of this bootstrap file is to create the environment our plugin expects. So if all of your plugins functionalities need an instance of sfContext you would create it here or if they would all depend on the database you would create a connection here as well.

But remember there should be as few dependencies as possible!

How to write your test cases?

This is the easies part of all. Just refer to the PHPUnit documentation and see how to write the tests themselves and add everything to classes like this:

1
2
3
4
5
6
7
8
9
10
11
12

// /test/unit/lib/yourPluginClassTest.php
<?php
require_once dirname(__FILE__).'/../bootstrap/unit.php';
require_once 'PHPUnit/Framework.php';
 
class yourPluginClassTest extends PHPUnit_Framework_TestCase
{
  public function testYourMethod()
  {
    // .. implement your test here
  }
}

The above example will test your method yourPluginClass::yourMethod() so you see the conventions of appending “Test” to the class and prepending “test” to the methods your going to test.

I found it a good practice to organise my test cases the same way as the tested classes.

For example:
/lib/routing/yourOwnRoute.class.php (classname: yourOwnRoute)
would be tested in:
/test/unit/lib/routing/yourOwnRouteTest.php (classname: yourOwnRouteTest)

Another good practice is to have one test case per class so each of your classes has exactly one test class.

You can run your test case like this.

$ phpunit /test/unit/lib/routing/yourOwnRouteTest.php

How to write a test suite?

Often you want to run all your test cases at once. In fact for continuous integration purposes this is always the case. This can be achieved using a PHPUnit test suite that can run multiple test cases.

A good naming convention is to create such a suite in the /test directory of your plugin and to call it like you called you plugin.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

// /test/yourPluginTests.php
<?php
class yourPluginTests
{
  public static function suite()
  {
    global $configuration, $plugin_configuration;
    $suite = new PHPUnit_Framework_TestSuite('yourPlugin');
 
    // loading plugin configurations
    $configuration = ProjectConfiguration::getActive();
    $pluginConfig = $configuration->getPluginConfiguration('yourPlugin');
 
    // instantiate a fake symfony unit test task to retrieve all connected tests for this plugin
    $task = new sfTestUnitTask($configuration->getEventDispatcher(), new sfFormatter());
    $event = new sfEvent($task, 'task.test.filter_test_files', array('arguments' => array('name' => array()), 'options' => array()));
    $files = $pluginConfig->filterTestFiles($event, array());
    $suite->addTestFiles($files);
 
    return $suite;
  }
}

In this example I collected all test cases from the fixture project just like the symfony test task for lime does it. This is very convenient as you don’t have to manually add new test cases. Refer to the AllTests section of the PHPUnit documentation for further details of a test suite.

Next we need to make all our test cases known to the fixture project by editing the ProjectConfiguration class.

1
2
3
4
5
6
7
8
9
10
11

// taken from http://trac.symfony-project.org/browser/plugins/sfImageTransformExtraPlugin/trunk/test/fixtures/project/config/ProjectConfiguration.class.php
<?php
class ProjectConfiguration extends sfProjectConfiguration
{
  // ...
 
  public function setupPlugins()
  {
    $this->pluginConfigurations['sfImageTransformExtraPlugin']->connectTests();
  }
}

Now you can run all your test cases like this.

$ phpunit /test/yourPluginTests.php

The fixture project

I already mentioned the dependency towards symfony but there is more. There is also the dependency of your plugin towards being properly installed.
This dependency has to be reflected in the fixture project.

For example if you have any schema defined for Doctrine or Propel then the classes generated from these schema will be generated outside your plugin except for the classes prefixed with “Plugin”. So you have to create these generated classes inside of the fixture project as well.
Make sure not to modify these classes they have to be absolutely pristine!

If your plugin needs an application youn have to create one in the fixtrue project and if certain settings in the app.yml or setting.yml or elsewhere are mandatory you will have to create them in the fixture project as well.

Just be aware that you don’t try to fix anything by only adding more to the fixture project. There should be as few changes to it as possible and all changes you have made should be properly documented as they are necessary for anybody who wants to use your plugin.

Dependencies to other plugins

Sometimes your plugin will depend on the existence of other plugins. For sfImageTransformExtraPlugin this was the case as it required sfImageTransformPlugin to be installed. In your fixture project this can be solved by editing the ProjectConfiguration class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

// taken from http://trac.symfony-project.org/browser/plugins/sfImageTransformExtraPlugin/trunk/test/fixtures/project/config/ProjectConfiguration.class.php
<?php
class ProjectConfiguration extends sfProjectConfiguration
{
  public function setup()
  {
    $this->setPlugins(array('sfImageTransformExtraPlugin', 'sfImageTransformPlugin', 'sfDoctrinePlugin'));
 
    $this->setPluginPath('sfImageTransformExtraPlugin', dirname(__FILE__).'/../../../..');
    $this->setPluginPath('sfImageTransformPlugin', dirname(__FILE__).'/../../../../../sfImageTransformPlugin');
  }
 
  // ...
}

This expects sfImageTransformPlugin to be located right next to sfImageTransformExtraPlugin which comes in handy if integrated with continuous integration services.

What to test?

For quite some time I kept asking this question. My answer is this:

• Don’t test module code! Module code in this case means actions or components. They have far too many dependencies in symfony 1.x and they should be very slim anyway. Test them with functional tests.
• Don’t test tests! This should be self explaining really..
• Test everything else! This leaves you with the /lib and /config folders only.

And to keep a good overview you should create a coverage report from time to time. Using the following configuration file I use to create mine.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

<?xml version="1.0" encoding="UTF-8"?>
 
<phpunit backupGlobals="false"
         backupStaticAttributes="false"
         colors="false"
         convertErrorsToExceptions="true"
         convertNoticesToExceptions="true"
         convertWarningsToExceptions="true"
         processIsolation="false"
         stopOnFailure="false"
         syntaxCheck="false">
 
  <testsuites>
    <testsuite name="yourPlugin Suite">
      <directory>./test/unit/</directory>
    </testsuite>
  </testsuites>
 
  <filter>
    <whitelist addUncoveredFilesFromWhitelist="true">
      <directory suffix=".class.php">./lib/</directory>
      <directory suffix=".class.php">./config/</directory>
    </whitelist>
  </filter>
</phpunit>

Change into the root directory of your plugin to execute phpunit other wise the relative paths will not work as they are relative to the execution dir not the configuration file!

Conclusion

PHPUnit testing of symfony plugins is tricky as it is not documented anywhere. But it is still very possible and once you got the hang of it feels very natural.

I hope this will help some of you t get started.

Things like mocking database connections or all the hazzle with sfContext might be subject to another post.

This blog post is in German as the mentioned article was published in German.

In der heute erschienenen Ausgabe 4.10 des PHP Magazins ist auch ein Artikel von mir enthalten, und wie sollte es wohl anders sein *Trommelwirbel*, beschäftigt der sich mit dem Thema Softwaremetriken In diesem Artikel gibt es eine sehr ausführliche Einführung in die Thematik, mit eine Reihe von praktischen Beispielen und den daraus ableitbaren Aussagen über den untersuchten Quelltext.

Über Kommentare, Anregungen und Kritik rund um den Artikel würde ich mich sehr freuen. Und sollte euch der Artikel gefallen haben, empfehle ich die Internationale PHP Conference 2010, die vom 30. Mai bis 2. Juni in Berlin stattfindet, an der ich teilnehmen und zwei Vorträge halten werden.

• Von Continuous Integration zu Continuous Deployment
• Statische Codeanalyse wirklich effektiv nutzen

This blog post is in German as the mentioned article was published in German.

In der heute erschienenen Ausgabe 4.10 des PHP Magazins ist auch ein Artikel von mir enthalten, und wie sollte es wohl anders sein *Trommelwirbel*, beschäftigt der sich mit dem Thema Softwaremetriken In diesem Artikel gibt es eine sehr ausführliche Einführung in die Thematik, mit eine Reihe von praktischen Beispielen und den daraus ableitbaren Aussagen über den untersuchten Quelltext.

Über Kommentare, Anregungen und Kritik rund um den Artikel würde ich mich sehr freuen. Und sollte euch der Artikel gefallen haben, empfehle ich die Internationale PHP Conference 2010, die vom 30. Mai bis 2. Juni in Berlin stattfindet, an der ich teilnehmen und zwei Vorträge halten werden.

• Von Continuous Integration zu Continuous Deployment
• Statische Codeanalyse wirklich effektiv nutzen

Yesterday I wrote about setting up a symfony plugin project in phpUnderControl. However so far I’ve been using Hudson as my Continuous Integration server. So before comparing the two I will explain how to achieve yesterdays setup using Hudson.I assume that you have a running Hudson installation ready.

Hudson is – as you will notice – a web dialog driven tool. You can of course edit the underlying configuration files (mostly XML) but I never experienced the need as the web interface is just as powerful.

Creating the project

In Hudson projects are called jobs. You create one by clicking on “New Job“.

Hudson dialog: New Job

What we want is a free-style software project.

This will directly lead us to the Job Configuration dialog.

Checking out the sources

In the Source Code Management section of the Job Configuration dialog we select Subversion for our version control system (There is a Git plugin for Hudson available) and configure the three sources for symfony, sfImageTransformPlugin and sfImageTransformExtraPlugin.

Hudson dialog: Job Configuration

This is self explaining really.

Preparing the build

Next we want the project to be build whenever there is a commit to one of the sources.

Hudson dialog: Job Configuration

The syntax for the schedule is cronjob like. In this example Subversion will be polled every 15 minutes.

Running the tests

Now we have to configure everything for Hudson to know how to run the PHPUnit tests. For this we need to add a build step.

Hudson dialog: Job Configuration

There are several ways to do this, some are build-in such as using ant scripts while some require a plugin like using a phing script.

I found it easiest to use a shell command. For this you have to select Execute shell from the dropdown.

Hudson dialog: Job Configuration

export SYMFONY=$WORKSPACE/symfony
mkdir -p $WORKSPACE/logs
rm -rf $WORKSPACE/logs/*
cd $WORKSPACE/sfImageTransformExtraPlugin/
phpunit --log-xml=$WORKSPACE/logs/phpunit-results.xml test/sfImageTransformExtraPluginTests.php

As you can see this is a small bash script that sets the SYMFONY variable prepares a location for the PHPUnit report and runs the test suite.

The last thing we need to do is to tell Hudson what to do with the test results.

For this we add a Post-build Action and select PHPUnit.

Hudson dialog: Job Configuration

Then we enter the location of the result XML generated by the previous build step.

Hudson dialog: Job Configuration

For this last step you will need the xUnit Plugin installed.

If you don’t have xUnit Plugin installed already you can save your settings now and edit them later (browse to your Job and click on Configure).

Now save the dialog and you will be presented with your Jobs webpage.

Hudson job page

Now click on Build Now to start the first build. This might take a while as the first checkout of symfony is quite big.

The successful build should look like this:

Hudson: Successful first build

Now you can of course start adding more features such as pDepend, Cloverage reports, Mess Detection, Copy and Paste Detection and many more but for this post we’re done.

As I’ve just started to look into phpUnderControl again I thought why not try to setup a project for testing sfImageTransformExtraPlugin running all the PHPUnit tests that I wrote about last week.

So the goal is to setup a phpuc project for a single plugin but with all dependencies (symfony itself) provided. This was a bit tricky so here we go!The first thing I have to say is that although I frequently link to Andries Seutens for installing phpUnderControl for the subject of this post it is not recommended to create a project by running $ phpuc project ..! With this post I will follow the phpuc getting started guide instead.

Creating the project

First we need a project directory.

$ mkdir /path/to/cruisecontrol/projects/yourProjectName

Next we need a very basic ant build.xml file.

1
2
3
4
5

// /path/to/cruisecontrol/projects/yourProjectName/build.xml
<?xml version="1.0" encoding="UTF-8"?>
<project name="yourProjectName" basedir=".">
 
</project>

Now CruiseControl must be told about this new project in its config.xml.

1
2
3
4
5
6
7

// /path/to/cruisecontrol/config.xml
<?xml version="1.0" encoding="UTF-8"?>
<cruisecontrol>
  <project name="yourProjectName" buildafterfailed="false">
 
  </project>
</cruisecontrol>

That the very basic project setup. Next we will need the subject.

Checkout the sources

First we need to create a sources folder for the new project.

$ mkdir /path/to/cruisecontrol/projects/yourProjectName/source

There we checkout the source of our symfony plugin.

// in /path/to/cruisecontrol/projects/yourProjectName/source/
$ svn co http://svn.symfony-project.org/plugins/sfImageTransformExtraPlugin/trunk sfImageTransformExtraPlugin

This alone will not do as there are dependencies that are not included in the plugin. The most important one being symfony itself so we have to check that out as well.

// in /path/to/cruisecontrol/projects/yourProjectName/source/
$ svn co http://svn.symfony-project.org/branches/1.4/ symfony

For sfImageTransformExtraPlugin there is an additional dependency towards sfImageTransformPlugin.

// in /path/to/cruisecontrol/projects/yourProjectName/source/
$ svn co http://svn.symfony-project.org/plugins/sfImageTransformPlugin/trunk sfImageTransformPlugin

So now we have three fresh copies of the two plugins and symfony.

Preparing the build

Before every build we want to update all the sources to test against the latest commits. For this we edit the build.xml again and add the following.

3
4
5
6
7
8
9
10
11
12
13
14
15
16

// /path/to/cruisecontrol/projects/yourProjectName/build.xml
...
  <target name="checkout">
    <exec executable="svn" dir="${basedir}/source/symfony">
      <arg line="up" />
    </exec>
    <exec executable="svn" dir="${basedir}/source/sfImageTransformPlugin">
      <arg line="up" />
    </exec>
    <exec executable="svn" dir="${basedir}/source/sfImageTransformExtraPlugin">
      <arg line="up" />
    </exec>
  </target>
...

The above will run svn up on all three working copies. You can varify this by running the following.

// in /path/to/cruisecontrol/projects/yourProjectName/
$ ../../apache-ant-1.7.0/bin/ant checkout
Buildfile: build.xml
 
<pre lang="sh">
checkout:
     [exec] ...
...
 
BUILD SUCCESSFUL
Total time: 17 seconds

Running the tests

Before we run the tests we have to create a logs folder to store the build results and we also want a coverage folder to store our coverage reports for the build.

// in /path/to/cruisecontrol/projects/yourProjectName/
$ mkdir -p build/{logs,coverage}

Now we can add a new target to the build.xml.

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

// in /path/to/cruisecontrol/projects/yourProjectName/build.xml
...
  <target name="phpunit">
    <exec executable="phpunit" dir="${basedir}/source/sfImageTransformExtraPlugin" failonerror="on">
      <env key="SYMFONY" value="${basedir}/source/symfony/lib"/>
      <arg line="--log-xml ${basedir}/build/logs/phpunit.xml
                 --log-pmd ${basedir}/build/logs/phpunit.pmd.xml
                 --log-metrics ${basedir}/build/logs/phpunit.metrics.xml
                 --coverage-clover ${basedir}/build/logs/phpunit.coverage.xml
                 --coverage-html ${basedir}/build/coverage
                 sfImageTransformExtraPluginTests
                 test/sfImageTransformExtraPluginTests.php"/>
    </exec>
  </target>
...

All this is more or less copied from the getting started page linked above except for the paths and this on line.

16
17
18
19

// in /path/to/cruisecontrol/projects/yourProjectName/build.xml
...
      <env key="SYMFONY" value="${basedir}/source/symfony/lib"/>
...

This line does the same as $ export SYMFONY=/path/to/cruisecontrol/projects/yourProjectName/source/symfony/lib and is needed for the testing bootstrap explained here to find the symfony library.

Of course all this can be tested again.

// in /path/to/cruisecontrol/projects/yourProjectName/
$ ../../apache-ant-1.7.0/bin/ant phpunit
Buildfile: build.xml
 
phpunit:
     [exec] PHPUnit 3.3.1 by Sebastian Bergmann.
     [exec]
     [exec] .........................................................S.. 60 / 72
     [exec] ............
     [exec]
     [exec] Time: 33 seconds
     [exec]
     [exec] OK, but incomplete or skipped tests!
     [exec] Tests: 72, Assertions: 92, Skipped: 1.
     [exec]
     [exec] Writing code coverage data to XML file, this may take a moment.
     [exec]
     [exec] Writing metrics report XML file, this may take a moment.
     [exec]
     [exec] Writing violations report XML file, this may take a moment.
     [exec]
     [exec] Generating code coverage report, this may take a moment.
 
BUILD SUCCESSFUL
Total time: 47 seconds

Activating the targets

The two targets above now need to be triggered. This is done by the build target which can be dependant on other targets.

3
4
5
6

// in /path/to/cruisecontrol/projects/yourProjectName/build.xml
...
  <target name="build" depends="checkout,phpunit" />
...

Configuring CruiseControl to build your project

Of course now cruisecontrol needs to be told about the details of the build so we edit the config.xml again.

First we need to include the required plugins in this case its for Subversion.

4
5

// in /path/to/cruisecontrol/config.xml
 <plugin name="svn" classname="net.sourceforge.cruisecontrol.sourcecontrols.SVN" />

Now we want to build the project whenever there was a change to any of the Subversion repositories testing every 60 seconds.

5
6
7
8
9
10
11
12

// in /path/to/cruisecontrol/config.xml
...
    <modificationset quietperiod="60">
      <svn localWorkingCopy="projects/${project.name}/source/symfony/"/>
      <svn localWorkingCopy="projects/${project.name}/source/sfImageTransformPlugin/"/>
      <svn localWorkingCopy="projects/${project.name}/source/sfImageTransformExtraPlugin/"/>
    </modificationset>
...

The following tells CruiseControl what to do (running the build.xml through ant).

10
11
12
13
14
15

// in /path/to/cruisecontrol/config.xml
...
    <schedule interval="300">
      <ant anthome="apache-ant-1.7.0" buildfile="projects/${project.name}/build.xml"/>
    </schedule>
...

The following listener adds the build status to the dashboard.

13
14
15
16
17
18

// in /path/to/cruisecontrol/config.xml
...
  <listeners>
    <currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
  </listeners>
...

Now let’s tell CruiseControl where to find the logs and reports.

16
17
18
19
20
21

// in /path/to/cruisecontrol/config.xml
...
  <log dir="logs/${project.name}">
    <merge dir="projects/${project.name}/build/logs/"/>
  </log>
...

Now publish the coverage report and some build metrics produced by phpuc.

19
20
21
22
23
24
25
26
27

// in /path/to/cruisecontrol/config.xml
...
    <publishers>
      <artifactspublisher dir="projects/${project.name}/build/coverage"
                          dest="artifacts/${project.name}"
                          subdirectory="coverage"/>
       <execute command="phpuc graph logs/${project.name} artifacts/${project.name}"/>
     </publishers>
...

Now you can fire up CruiseControl and look at your project.

Of course now you can start thinking about integrating PHP documentor, pdepend, PHP_Codesniffer and the lot.

Next steps for me will be to write the same post again for Hudson-CI and then compare the two.

The final files

Now here are the two XML files.

/path/to/cruisecontrol/projects/yourProjectName/build.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

<?xml version="1.0" encoding="UTF-8"?>
<project name="yourProjectName" basedir=".">
  <target name="build" depends="checkout,phpunit" />
  <target name="checkout">
    <exec executable="svn" dir="${basedir}/source/symfony">
      <arg line="up" />
    </exec>
    <exec executable="svn" dir="${basedir}/source/sfImageTransformPlugin">
      <arg line="up" />
    </exec>
    <exec executable="svn" dir="${basedir}/source/sfImageTransformExtraPlugin">
      <arg line="up" />
    </exec>
  </target>
  <target name="phpunit">
    <exec executable="phpunit" dir="${basedir}/source/sfImageTransformExtraPlugin" failonerror="on">
      <env key="SYMFONY" value="${basedir}/source/symfony/lib"/>
      <arg line="--log-xml ${basedir}/build/logs/phpunit.xml
                 --log-pmd ${basedir}/build/logs/phpunit.pmd.xml
                 --log-metrics ${basedir}/build/logs/phpunit.metrics.xml
                 --coverage-clover ${basedir}/build/logs/phpunit.coverage.xml
                 --coverage-html ${basedir}/build/coverage
                 sfImageTransformExtraPluginTests
                 test/sfImageTransformExtraPluginTests.php"/>
    </exec>
  </target>
</project>

/path/to/cruisecontrol/config.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

<?xml version="1.0" encoding="UTF-8"?>
<cruisecontrol>
  <plugin name="svn" classname="net.sourceforge.cruisecontrol.sourcecontrols.SVN" />
  <project name="yourProjectName" buildafterfailed="false">
    <modificationset quietperiod="60">
      <svn localWorkingCopy="projects/${project.name}/source/symfony/"/>
      <svn localWorkingCopy="projects/${project.name}/source/sfImageTransformPlugin/"/>
      <svn localWorkingCopy="projects/${project.name}/source/sfImageTransformExtraPlugin/"/>
    </modificationset>
    <schedule interval="300">
      <ant anthome="apache-ant-1.7.0" buildfile="projects/${project.name}/build.xml"/>
    </schedule>
    <listeners>
      <currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
    </listeners>
    <log dir="logs/${project.name}">
      <merge dir="projects/${project.name}/build/logs/"/>
    </log>
    <publishers>
      <artifactspublisher dir="projects/${project.name}/build/coverage"
                          dest="artifacts/${project.name}"
                          subdirectory="coverage"/>
       <execute command="phpuc graph logs/${project.name} artifacts/${project.name}"/>
     </publishers>
  </project>
</cruisecontrol>

As many of you may already have noticed, there will be a big change in my career as a professional software engineer and architect this summer. Together with Kore and Toby I am in the process of founding a company. The focus of this company will be on services all around the whole quality life cycle in PHP projects.

Under the hood of our company we will also offer support, trainings and consulting for several quality assurance tools, like pdepend, phpmd and phpUnderControl. For you, this opens a great opportunity. You can use the tools and the documentation, as well as participate in the community, as usual. But from now on you can also purchase professional support, if you get stuck or need general assitance. And when you miss a feature or need an individual extension for one of these tools, don't hesitate to contact us.

I am really excited what cool things will happen in the next couple of years and I am looking forward to cowork with professionals like Toby and Kore.

To finalize the little marketing for the company and its services I would appreciate to meet you, your colleagues and your compmany, to give presentations or into-depth trainings on one of the tools or on quality assurance in general.

This and some other posts have been summarised in the article Best practice of PHPUnit testing a symfony 1.4 plugin (in English).

Gestern sprach ich noch von Erkenntnissen. Heute muss ich eines davon bereits revidieren.

Mehr noch, ich habe bisher zwei Stellen entdeckt, die einem den Spass Am Testen so richtig verderben können: sfContext und so ziemlich jede Form von Konfigurations Klasse.

Also als erstes mal eine Richtigstellung.

Es ist keineswegs eine gute Idee die ProjectConfiguration Klasse aus dem Fixture Projekt in myPluginNameProjectConfiguration umzubenennen!

An irgendeiner Stelle benötigt man sicher auch mal eine sfApplicationConfiguration Instanz um bestimmte Settings zu laden.

Diese Klasse aber wiederum erbt von der Klasse ProjectConfiguration. Haben wir nun diese Klasse umbenannt wie ich gestern vorgeschlagen habe, wird sie nicht mehr gefunden und wir bekommen einen PHP Error.

• Wir bleiben also vorerst mit dem Problem zurück, dass TestSuites aus mehreren Plugins nicht in einer TestSuite zusdammen gefasst werden können, weil es dadurch zur Redeklaration von Klassen kommen kann (wird).

sfContext und symfony Konfigurations Klassen sind die Pest beim Testen!

Ernsthaft, das macht keinen Spass.

Ich wollte u.a. meine PluginConfiguration Klasse testen, da ich dort einige statische Callbacks implementiert habe, die z.B. auf routing.load_configuration Event reagieren.

Also Test Klasse anlegen und die öffentlichen Methoden (auch die statischen) mit jeweils einer Test Methode repräsentieren.

Nun kommt man immer wieder in die Situation, wo man die komplette Konfiguration laden muss oder schlimmer: sfContext instanzieren, um an bestimmte Objekte wie sfEventDispatcher oder sfViewCacheManager zu kommen. Und alles nur um einzelne Metthoden zu testen, die irgendwo eine kleine Abhängigkeit haben, die wiederum in einer Abhängigkeitskaskade in sfContext mündet.

Im Ergebnis muss man immer einen kompletten Bootstrap durchführen und das gesamte Framework “hochfahren”, um Unit Tests durchzuführen.

Einige meiner Tests dauern so bereits mehr als 30 Sekunden in der Ausführung. Das geht gar nicht!

Ich werde mich also jetzt daran machen, so viel Anhängigkeiten wie nur irgend möglich abzuschaffen. Aber ich fürchte, das wird mir nicht hundertprozentig gelingen.

symfony 1 ist einfach dann doch so sehr verkoppelt, dass ich damit leben oder auf einige Tests verzichten muss.

Mal schauen, ob ich da eine Lösung finde..

Users of PHPUnit frequently ask me questions such as "How do I use PHPUnit from a Git checkout?" or "How do I run PHPUnit's own test suite?" This article provides the answers to these questions.

Getting PHPUnit from Git

sb@thinkpad ~ % git clone git://github.com/sebastianbergmann/phpunit.git

You now have a phpunit directory in your current working directory that contains the branch for PHPUnit 3.5 (at the time of writing) because the 3.5 branch is currently configured as the default branch for the PHPUnit repository on GitHub. If you want to switch to the branch for PHPUnit 3.4, for instance, you can use

sb@thinkpad phpunit % git checkout -b 3.4 origin/3.4

This tells Git to create a new local branch of name 3.4 (-b 3.4) that is set up to track the remote branch 3.4 from origin (origin/3.4). Git automatically switches to the newly created local branch. Using

sb@thinkpad phpunit % git checkout 3.5

you can switch back to the branch for PHPUnit 3.5.

Running PHPUnit from a Git checkout

Inside the phpunit directory you will find the phpunit.php script. Using this script you can invoke the PHPUnit TextUI test runner:

sb@thinkpad ~ % phpunit/phpunit.php
PHPUnit @package_version@ by Sebastian Bergmann.
.
.
.

The @package_version@ placeholder string for the version information makes it clear that a non-release version of PHPUnit is being used. Upon installation, the PEAR Installer replaces this placeholder string with the relevant information.

Running PHPUnit's Own Test Suite

Running PHPUnit's own test suite is as easy as invoking phpunit in the checkout directory:

sb@thinkpad phpunit % phpunit                  
PHPUnit @package_version@ by Sebastian Bergmann.

............................................................  60 / 681
............................................................ 120 / 681
............................................................ 180 / 681
............................................................ 240 / 681
............................................................ 300 / 681
............................................................ 360 / 681
............................................................ 420 / 681
............................................................ 480 / 681
................................SSSSSSSSSSSSSSSSSSSSSSSSSSSS 540 / 681
SSSS.......................................S................ 600 / 681
............................................................ 660 / 681
.....................

Time: 27 seconds, Memory: 56.25Mb

OK, but incomplete or skipped tests!
Tests: 681, Assertions: 1459, Skipped: 33.

Writing code coverage data to XML file, this may take a moment.

Generating code coverage report, this may take a moment.

The above works because there is an XML configuration for PHPUnit (phpunit.xml.dist) in the directory that contains information about which tests to run and what logfiles to produce.

Almost a year ago I started to factor out all the code that deals with code coverage in PHPUnit and put it into a separate component. The name of this component is PHP_CodeCoverage.

PHP_CodeCoverage is a library that provides collection, processing, and rendering functionality for PHP code coverage information. Its architecture has support for multiple backends for collecting code coverage information (currently only support for Xdebug is implemented) and for reporting code coverage information (for instance using Clover XML or as an HTML report).

As mentioned earlier, the HTML report that is generated by PHP_CodeCoverage shows the CRAP Index software metric for each function or method. Today I built on this earlier development and added a "dashboard view" to the HTML report that shows the following software metrics:

• Class Coverage Distribution: This is a bar chart that shows how many classes have 0%, ..., 100% code coverage
• Class Complexity: This is a scatter chart that shows the coverage (X axis) and the complexity (Y axis) of the classes
• Top Project Risks: This is list of the classes with the highest CRAP Index
• Least Tested Methods: This is the list of the least tested methods

You can see an example of the dashboard view below:

Wie ihr gemerkt habe, brauchte ich heute morgen etwas länger für meinen Blogpost. Der Grund ist, dass ich endlich eine Lösung präsentieren wollte für die Anforderung, für die ich noch keine Idee hatte.

Rüdiger und Daniel haben mich korrekterweise darauf hingewiesen, dass keine Lösung schlechter ist, als eine unelegante.

Das wollte ich nicht auf mir sitzen lassen und habe nach einem Weg gesucht.

Da ich die Anforderungen ziemlich genau benennen konnte, war ich auch in der Lage, sie als Unit Test zu formulieren und somit konnte ich Testgetrieben entwickeln (Test Driven Development, TDD).

Witzigerweise bin ich mit dem Ergebnis sogar so zufrieden, dass ich es als (vorerst) elgante Lösung bezeichnen würde.

Die Anforderung lautet wie folgt:

Im sfImageTransformExtraPlugin hat der Anwender die Möglichkeit, die URL Struktur seiner Thumbnails selber zu bestimmen. Unter diesen URLs werden die generierten Thumbnails im Docroot abgelegt. Das Entfernen der generierten Thumbnails soll eingrenzbar sein, indem Parameter der URL vorgegeben werden können.

Die Schwierigkeit ist, dass ich nicht weiss, welche Parameter es gibt, oder in welcher Reihenfolge sie kommen. Ich brauche also eine Möglichkeit mir einen Pattern zu erstellen.

Zunächst habe ich mir einige Fixtures erstellt.

thumbnails/default/path/to/file/filename.gif
thumbnails/default.gif
thumbnails/original/path/to/file/filename.jpg
thumbnails/original.jpg
thumbnails/site/default/path/to/file/filename.gif
thumbnails/site/original/path/to/file/filename.jpg
thumbnails/testrecord/default/01/00/00/test-record-1.gif
thumbnails/testrecord/default/02/00/00/test-record-2.gif
thumbnails/testrecord/original/01/00/00/test-record-1.jpg
thumbnails/testrecord/original/02/00/00/test-record-2.jpg
thumbnails/testrecord2/default/01/00/00/test-record-1.gif
thumbnails/testrecord2/default/02/00/00/test-record-2.gif
thumbnails/testrecord2/original/01/00/00/test-record-1.jpg
thumbnails/testrecord2/original/02/00/00/test-record-2.jpg

Mit diesen Fixtures kann ich feststellen, ob das gezielte Entfernen erfolgreich war oder nicht.

Angefangen habe ich mit folgendem Test:

  public function testRemovePatternForFileSources()
  {
    $pattern = $this->cache->computePattern('sf_image_mock', array('filepath' => 'path/to/file/filename'));
    $finder = new sfFinder();
    $files = $finder->type('file')->in($this->cache->getOption('cache_dir'));
    $this->cache->removePattern($pattern);
    $this->assertNotContains($this->cache->getOption('cache_dir').'/thumbnails/default/path/to/file/filename.gif', $files);
    $this->assertNotContains($this->cache->getOption('cache_dir').'/thumbnails/original/path/to/file/filename.jpg', $files);
    $this->assertEquals(count($files) - 2, count($finder->type('file')->in($this->cache->getOption('cache_dir'))));
  }

Die Idee ist also, irgendwie einen Pattern zu erstellen (Zeile 93) und mit diesem die symfony Methode removePattern() aufzurufen.

Im Nachgang sollten nur einige bestimmte Dateien entfernt sein und der Rest eben nicht. Ich überprüfe also, ob die beiden Dateien nicht mehr vorhanden sind und ob die gesamt Anzahl der noch vorhandenen Dateien nur um genau diese beiden verringert worden ist.

Nach einigen Versuchen habe ich den Test grün machen können, war aber noch nicht hundertprozentig zufrieden. Der Pattern, den ich mir erstellt hatte war zu weich und konnte unter Umständen zu viel matchen.

Hierbei ist mir aufgefallen, dass das Pattern erstellen und überprüfen sehr eng nicht mit der sfRawFileCache Klasse, sondern viel mehr mit der sfImageTransformRoute Klasse zusammenhängt, denn diese hat bereits einen Pattern, der entsprechende URLs matcht. Ich musste eigentlich nur dafür sorgen, dass dieser sehr flexible Pattern einschränkbar ist und dann musste ich nur noch die symfony Methode sfRoute::matchesUrl() aufrufen.

Hier mal der entsprechende Test:

  public function testRemovePatternForFileSources()
  {
    $route = $this->getRoute('sf_image_file');
    $route->preassemblePattern(array('filepath' => 'path/to/file/filename'));
    $finder = new sfFinder();
    $starting_count = count($finder->type('file')->in($this->cache->getOption('cache_dir')));
    $this->cache->removePattern($route);
    $files = $finder->type('file')->in($this->cache->getOption('cache_dir'));
    $this->assertNotContains($this->cache->getOption('cache_dir').'/thumbnails/default/path/to/file/filename.gif', $files);
    $this->assertNotContains($this->cache->getOption('cache_dir').'/thumbnails/original/path/to/file/filename.jpg', $files);
    $this->assertEquals($starting_count - 2, count($files));
  }

Statt also einen Pattern generieren zu lassen, hole ich mir eine Route und schränke deren Pattern mit preassemblePattern() um ein paar Parameter ein.

Der sfRawFileCache übergebe ich dann statt einem Pattern dieses Route Objekt.

Hier die Änderungen dafür:

Index: /plugins/sfImageTransformExtraPlugin/trunk/lib/cache/sfRawFileCache.class.php
===================================================================
--- /plugins/sfImageTransformExtraPlugin/trunk/lib/cache/sfRawFileCache.class.php (revision 28779)
+++ /plugins/sfImageTransformExtraPlugin/trunk/lib/cache/sfRawFileCache.class.php (revision 28904)
@@ -99,5 +99,16 @@
public function removePattern($pattern)
{
-    if (false !== strpos($pattern, '**'))
+    if($pattern instanceof sfRoute)
+    {
+      $paths = array();
+      foreach (new RecursiveIteratorIterator(new RecursiveDirectoryIterator($this->getOption('cache_dir'))) as $path)
+      {
+        if(false !== $pattern->matchesUrl('/'.str_replace($this->getOption('cache_dir').DIRECTORY_SEPARATOR, '', $path), array('method' => 'get')))
+        {
+          $paths[] = $path;
+        }
+      }
+    }
+    else if (false !== strpos($pattern, '**'))
{
$pattern = str_replace(sfCache::SEPARATOR, DIRECTORY_SEPARATOR, $pattern);

und

Index: /plugins/sfImageTransformExtraPlugin/trunk/lib/routing/sfImageTransformRoute.class.php
===================================================================
--- /plugins/sfImageTransformExtraPlugin/trunk/lib/routing/sfImageTransformRoute.class.php (revision 28653)
+++ /plugins/sfImageTransformExtraPlugin/trunk/lib/routing/sfImageTransformRoute.class.php (revision 28904)
@@ -146,3 +146,21 @@
return call_user_func(array($this->getImageSourceStreamWrapper(), 'buildURIfromParameters'), $this->parameters);
}
+
+  /**
+   * Preassembles pattern with passed parameters
+   *
+   * This is used to limit matches when removing generated images
+   *
+   * @param  array $params Parameters to be encoded in the pattern
+   * @return void
+   */
+  public function preassemblePattern($params = array())
+  {
+    foreach($params as $key => $value)
+    {
+      $this->pattern = str_replace(':'.$key, $value, $this->pattern);
+    }
+
+    $this->compiled = false;
+  }
}

So, der Test ist grün, der Code ist eingecheckt und ich bin zufrieden!

Das ganze Changeset könnt ihr hier sehen:
http://trac.symfony-project.org/changeset/28904

Gestern hab ich mich etwas darüber ausgelassen, wie doof doch einige Code Koppelungen von symfony mir das Testen behinderten.

Vor allem drei Dinge haben mir das Leben schwer gemacht: die Konfiguration, sfContext und Datenbankabhängigkeiten. Und ich habe für alle drei auch Antworten finden können. Vielleicht nicht die endgültigen, aber schon sehr akzeptable, wie ich meine.

Vielleicht sag ich erstmal, was überhaupt meine Probleme waren.

Zum einen ist es nicht immer ganz leicht herauszufinden, was man nun alles initialisieren muss, um eine Klasse zu Testen. Und für die Dinge, die initialisiert werden sollen, ist es nicht immer ganz einfach herauszufinden, was man dazu alles hochfahren und welche Konfiguration geladen werden muss.

Das führte dazu, dass ein Test alleine durchlaufen konnte, aber es in einer Test Suite nicht mehr tat. Und wenn man ihn da zum Laufen bekommen hat, konnte er immer noch auf dem Continuous Integration Server fehlschlagen.

Ein anderes Problem war die Ausführungszeit der Tests vor allem auf dem CI Server. Spitzenzeiten von bis zu 5 Minuten finde ich mehr als grenzwertig, wenn man sich mal den Umfang meines Projektes vergegenwärtigt (ca. 13 überschaubare Klassen inklusive der Controller).

Unterm Strich waren meine Tests alle zusammen wenig stabil und zu langsam.

Als Grund für die lange Ausführungszeit konnte ich ganz klar den Bootstrap von symfony ausmachen. Das ständige Hochfahren des Frameworks ging extrem auf die Performance.

Bei dieser Testdauer Übersicht kann man schon ahnen, wo die Probleme liegen. Hier mal die teuren vier:

1. sfImageTransformExtraPluginConfigurationTest testet die Plugin Configuration, die bereits alle anderen Configs benötigt, um überhaupt richtig instanziiert werden zu können
2. sfImageSourceTemplateTest testet ein Doctrine Behaviour, welches ein Doctrine_Record benötigt, um die einzelnen Methoden aufrufen zu können
3. sfImageTransformManagerTest ist tatsächlich nicht ganz sauber entworfen.. (Das habe ich aber gestern bereits korrigiert!)
4. sfImageSourceDoctrineTest testet einen Stream Wrapper, der eine Resource zurückliefert, die er mit Hilfe der Datenbank bestimmt

Mittlerweile sieht die gleiche Übersicht wie folgt aus:

Das kann sich schon eher sehen lassen. Eine Ausführzeit von 6 Sekunden ist auf jeden Fall akzeptabel, was ja nicht nur für den CI Server sondern vor allem für die manuelle Ausführung während der Entwicklung kritisch ist.

Aber was habe ich gemacht?

Separation des Plugins

Unbedingt notwendig für eine höhere Stabilität der Tests unter verschiedenen Umgebungen (Entwicklungsumgebung und Continuous Integration Server) war es das zu testende Plugin aus dem Projekt zu lösen. Das heisst:

• Eine separate Entwicklungsumgebung. Um zu vermeiden, dass man doch Projekt-Abhängigkeiten einbaut und mittestet, sollte die Entwicklung des Plugins separat stattfinden oder zumindest das Testen.
• Ein separater PHPUnit Aufruf. Damit bekommt man auch Konflikte wie gleich benannte Klassen in den Griff. Überhaupt besteht im Grunde genommen kein Grund in einem Projekt auch alle darin genutzten Plugins zu testen, denn diese sollen ja vor allem Projekt-unabhängig funktionieren.
• Ein separater CI Job. Zu finden unter http://automat.ical.ly/job/sfImageTransformExtraPlugin/
• Ein separates Repository. Das macht es dem CI Server einfacher, einfach Subversion nach updates zu befragen, ohne dass Plugin und Projekt Updates sich gegenseitig ins Testing schiessen.

Als “Entwicklungsumgebung” reicht es mir aus, das Plugin irgendwo ohne symfony Projekt abzulegen und lediglich die symfony Bibliotheken bereitzustellen (export SYMFONY=…./symfony/lib). Sobald kein Projekt mehr vorhanden ist und die Tests trotzdem durchlaufen, kann man sicher sein, dass es keine Abhängigkeiten gibt.

Der separate PHPUnit Aufruf hat zwei Vorteile. Zum einen umgeht es den Namenskonflikt, über den ich vorgestern gestolpert bin. Zum anderen werde ich solche Aufrufe nur noch direkt aus dem Plugin Basisverzeichnis ausführen. Denn sobald ich eine phpunit.xml anlege, weiss ich, dass die darin enthaltenen Pfade relativ zum Plugin sind.

Ein separater CI Job macht ebenfalls total Sinn, denn dann kann ich dort separat konfigurieren und bringe nicht eventuelle Projekt-Abhängigkeiten in die Testumgebung.

Das separate Repository ist erstmal optional, aber sicher sinnig, wenn ich dieses Plugin auch in anderen Projekten nutzen will.

Doctrine Verbindung mocken

Die Doctrine Datenbank Verbindung kostet Zeit und ist bisher aber gar nicht wirklich nötig, weil ich gar keine Modelle im Projekt habe, sondern lediglich mit Modellen aus dem Projekt zusammenarbeiten will.

Eine DSN habe ich wie folgt angegeben, aber obwohl SQLite für Testzwecke sicher besser ist, als MySQL o.ä. ist die Auswirkung auf die Performance recht gross.

all:
  doctrine:
    class: sfDoctrineDatabase
    param:
      dsn: sqlite:////fixture.db

Deshalb habe ich in den setUp() Methoden der TestCases, die irgendwie mit Doctrine arbeiten folgenden Schnipsel eingebaut:

    $this->dbh = new Doctrine_Adapter_Mock('mysql');
    $this->conn = Doctrine_Manager::getInstance()->openConnection($this->dbh, 'mysql', true);

Das true ist wichig, denn es aktiviert auch gleich die so übergebene Verbindung.

Im Ergebnis wird die fixtures.db gar nicht mehr angelegt. Die DSN wird lediglich zum initialisieren benötigt, Doctrine danach aber wieder umkonfiguriert.

Im Sourcecode von Doctrine habe ich auch entdeckt, dass man eine DSN mit dem mock:// Protokoll angeben kann, aber das habe ich noch nicht zum Laufen bekommen. Sollte ich das noch hinbekommen, könnte ich mir aber sogar den Schnipsel sparen.

Ein Doctrine_Record mocken

Wenn ich keine eigenen Modelle habe und mich auch nicht Daten aus irgendeiner Datenbank bedienen möchte (und darf!), dann brauche ich ein Mock-Doctrine_Record und eine dazugehörige Mock-Doctrine_Table.

Das require() ich einfach überall, wo ich sowas benötige und implementiere dort alles, was ich von diesem Record in Tests benötige. Ich muss allerdings aufpassen, nichts Test-kritisches zu mocken, was eventuell zu einem allways-pass führt. Nur Sachen mocken, um Abhängigkeiten aufzulösen!

sfContext nicht öfter instanziieren als notwendig

Ich habe gelernt, dass die setUp() Methode in einem TestCase mit dessen Instanziierung ausgeführt wird. Damit hatte ich angenommen, dass sie einmal vor allen im TestCase enthaltenen Test-Methoden läuft. Das stimmt aber so nicht.

Ein TestCase wird für jede in ihm enthaltene Test Methode erneut instanziiert!

Wenn ich dann in der setUp() Methode ein sfContext Objekt erzeuge, dann mache ich das erneut pro Test Methode. Doof.

Folgender Schnipsel hilft da gewaltig:

if(!sfContext::hasInstance('frontend'))
    {
      sfContext::createInstance($this->projectConfiguration->getApplicationConfiguration('frontend', 'test', true));
    }

So wird die Instanz nur noch erzeugt, wenn es noch keine gibt.

Ich werde mir meine TestCases immer wieder ansehen. Sollte ich dabei feststellen, dass ich diesen Schnipsel mehr als nur wenige Male verwende, dann werde ich ihn wohl in die setUp() Methode der TestSuite umziehen, die kann dann für alle Tests was vorbereiten.

So, ich bin erstmal zufrieden mit mir. Der Release Tag kommt näher!

Ich bin am Wochenende wenigstens ein paar Stunden aktiv gewesen und habe meinem kleinen Projekt ein neues Feature hinzugefügt.

Mit dem sfImageTransformExtraPlugin erweitere ich das bereits existierende symfony Plugin sfImageTransformPlugin um einen Konfigurations Layer sowie die Möglichkeit, einen Webservice für Thumbnail Generierung zu betreiben. Ich werde das Ganze in Kürze auch auf der symfony Website als Open Source Plugin zur Verfügung stellen, bis dahin möchte ich aber eine bessere Testabdeckung erreichen.

Und genau da bin ich jetzt gerade bei und sammel mal wieder eine Menge kleine Erkenntnisse.

Bislang konnte ich vier kleinere Dinge erlernen. Ob man das dann schon als Best Practice ausloben soll, weiss ich noch nicht.

PHPUnit Pfade sind relational zum aktuellen Arbeitsverzeichnis

Das hatte ich schonmal erwähnt. Die Pfadangaben in der phpunit.xml können relativ angegeben werden. Das ist gut, denn so muss ich nicht wissen, wie die Umgebung aussieht, in der mein Code läuft.

Aber! Die Pfade sind nicht – wie ich erwartet hätte – relativ zur phpunit.xml Datei, sondern relativ zum aktuellen Arbeitsverzeichnis, also dem Verzeichnis, von dem aus ich den phpunit Befehl angestossen habe.

• Für die symfony Plugin Entwicklung heisst das:

PHPUnit muss immer im Wurzelverzeichnis des Plugins ausgeführt werden.

• Wenn das nicht geht, weil z.B. eine AllTests.php über mehrere Plugins Tests ausführt, dann muss ich via PHP Code das aktuelle Arbeitsverzeichnis vor den Plugin TestSuiten entsprechend wechseln.

Stream Wrapper muss man anders testen

In meinem Plugin habe ich drei Stream Wrapper geschrieben, die eine Datei aus jeweils anderen Quellen holen kann.

Ein Stream Wrapper ist eine einfache Klasse, allerdings mit einem Unterschied zu herkömmlichen Klassen. Nämlich werden seine öffentlichen Methoden nicht von PHP Code, sondern von der PHP Runtime ausgeführt. Vor allem die Instanzierung ist schwer nachzubilden.

Mein Fazit ist daher folgendes:

• Stream Wrapper sollten nicht instanziert werden, um dann ihre Methoden aufzurufen, sondern stattdessen sollten sie registriert und entsprechend benutzt werden.

Ich registriere sie also via stream_wrapper_register() und rufe dann auf dem registrierten Protokoll PHP Dateifunktionen wie fopen() und Co.

Einheiten mit Doctrine Abhängigkeit verlangen ein besonderes Setup

Bei dieser Thematik bin ich noch nicht ganz schlüssig. Einige meiner Klassen nutzen direkt oder indirekt Doctrine.

Wenn ich hier Datenverarbeitende Methoden teste, kann es sein, dass ich eine Datenbank benötige. Nicht schön, aber eventuell akzeptabel. Ich empfehle hier folgendes:

• Bei Test-Datenbanken ist SQLite zu bevorzugen, weil hierfür keine weiteren Services bereitgestellt werden müssen, wenn die Tests z.B. auf einem Continuous Integration Server laufen.

Aber ich muss so in jedem Fall eine Menge machen:

1. Eine Datenbank konfigurieren (SQLite)
2. Die Test-relevanten Tabellen anlegen und benutzbar machen (Doctrine::createTablesFromArray(array(’Model1′, ‘Model2′, ..)))
3. Fixtures laden (Doctrine::loadData())

Meistens allerdings will man gar nicht die Datenbank Aktivität testen sondern die Logik drumherum. Dafür ist dieser Ballast viel zu teuer, weil er die Ausführzeit der Tests verlängert.

Ein vorsichtiges Fazit ist daher folgendes:

• Alle zu testenden Methoden refaktorisieren, um ihre Abhängigkeiten zu injizieren oder soweit wegzukapseln, dass sie für die Tests möglichst optional, mindestens aber mockbar sind.

Das wird nicht einfach..

Das generierte Fixture Projekt in einem Plugin verursacht Schwierigkeiten in TestSuites

Wenn ich eine Plugin Struktur generiere, dann findet sich darin auch ein kleines symfony Projekt im fixtures Verzeichnis. Dieses wird benötigt, um ein Plugin auch unabhängig von aktuellen Projekt Settings testen zu können.

Eines, was sich mir aber so garnicht erschliesst ist, dass im config Verzeichnis eine ProjectConfiguration Klasse angelegt wird. Im richtigen Projekt heisst diese sfProjectConfiguration.

Das Problem ist folgendes:

Habe ich zwei Plugins, habe ich auch zwei gleichnamige Klassen. Möchte ich die TestSuites beider Plugins in einer Projekt weiten AllTests.php zusammenfassen, werden beide Klassen deklariert.

Das kracht natürlich (can not redeclare class..).

Ich umgehe das wie folgt:

• Nach dem Generieren einer Plugin Struktur ist die Datei ProjectConfiguration.class.php und die darin enthaltende Klasse umzubenennen in myPluginProjectConfiguration.class.php

Damit sind dann die Namenskonflikte gelöst.

Nachdem ich seit gestern nun einigermassen mit Tests ausgestattet bin, will ich mich heute ans Refactoring machen.

Eine Methode ist mir da besonders im Sinn, die ich als zu gross und zu prozedural empfinde. Sie macht einfach zu viel, was besser in mehreren kleinen Methoden erledigt werden sollte.

Ausserdem habe ich noch zwei Methoden in derselben Klasse, die da nun wirklich nicht hingehören.

Zunächst einmal will ich mich um diese angesprochene Methode kümmern, indem ich sie via Extract Method in kleinere Einheiten zerlege.

Hier ist sie vor dem Refaktorisieren.

Zugegeben mit 50 Zeilen stellt diese Methode sicher keine Rekorde in prozeduralen Spaghetti Code Wettbewerben auf.

Trotzdem macht sie zu viel unterschiedliche Sachen, wie:

• Parameter überprüfen
• eine Resource besorgen
• mit Konfiguration hantieren
• einen Mimetypen setzen
• über Transformationseinstellungen iterieren
• jeweils einen Adapter aktualisieren
• jeweils die Parameter vorbereiten
• jeweils die Transformationen ausführen
• die Qualität setzen
• die Bild Resource zurückgeben.

Und da die benutzten Klassen und Methoden jeweils dynamisch zusammengesetzt werden, gibt es in der Mitte einen riesen Berg an Magie, den ich nicht so gerne anfasse.

Das will ich aber, denn die Transformation-abhängigen prepareParametersFor..() Methoden sind aktuell Methoden an dieser Klasse, statt als Callback aufgerufen an den entsprechenden Transformationen zu hängen. Das wäre sinnvoller für die Entwicklung von Transformations, denn eine solche Methode wäre dann dort verortet, wo auch der Rest der Implementierung ist.

Aber dazu will ich erst alles in verdaubare Einheiten aufteilen. Also los.

Hier habe ich den Teil der über die Settings iteriert herausgelöst und in eine eigene Methode gepackt.

In der alten Methode habe ich diesen Teil durch einen entsprechenden Aufruf ersetzt.

Jetzt gleich meinen PHPUnit Test ausgeführt und sichergestellt, dass noch alles läuft. Tut es, weiter.

Den Teil der den Adapter setzt habe ich auch in eine eigene Methode verpackt und nur noch aufgerufen. Das hat mir auch gleich noch eine lokale Variable gespart.

Wieder den Test ausführen, läuft.

Die Parameter Vorbereitung habe ich dann ebenso separiert und lediglich aufgerufen.

Die erste extrahierte Methode transform() konnte ich dann noch um zwei weitere lokale Variablen entlasten und auf folgende Grösse verschlanken.

Natürlich habe ich auch jetzt wieder getestet!

Nach dieser Aktion habe ich drei Methoden extrahiert. Sicher noch nicht das Ende des Möglichen, aber genug für meine Zwecke, denn ich will ja die Vorbereitungs Methoden extrahieren.

Dazu habe ich alle prepareParametersFor..() Methoden, die hier private Member waren via Move Method in die entsprechenden Transformations Klassen geworfen und dort via Rename Method in prepareParameters() umbenannt. Statt private sind sie nun public static.

Nun musste ich natürlich noch die aufrufende prepareParameters() Methode so umschreiben, dass sie statt der lokalen Methoden die externen benutzt.

Noch einmal die Tests durchlaufen lassen und fertig sind wir!

Natürlich sind meine Tests nicht immer durchgelaufen. Ich habe genügend kleine Fehler gemacht. Aber durch ein häufiges Ausführen der Tests, lagen die zu den Fehlern führenden Änderungen nie allzu lange zurück, so dass ich sie leicht korrigieren konnte.

Wenn man sich dieses Beispiel so anschaut, sieht es nun wirklich nicht besonders schwierig aus. Aber ich habe es vor ein paar Wochen mal ohne Tests versucht und nach zwei Stunden alles wieder abgeblasen, weil ich mich ständig verfranst hatte. Jetzt hat es circa eine Stunde gedauert und alles war fertig inklusive DocComments.

Ich sehe das jetzt mal als meine erste wirkliche Testerfahrung unter Realbedingungen an und sie ist durchweg positiv!

Gerade erst gestern habe ich mich hier öffentlich gewundert, warum das Thema Testing auf der Symfony Live 2010 keine Rolle spielte als Symfony 2.0 enthüllt wurde. In den Sourcen von Symfony 2.0 war nur lime bzw. Lime2 zu entdecken.

Nun scheint die Antwort gefunden zu sein.

Benjamin Eberlei, seines Zeichens beteiligt am Zend Framework und Doctrine, machte mich gestern darauf aufmerksam, dass erst am vergangenen Wochenende eine Entscheidung zum Thema Testing gefallen sei.

Symfony 2.0 wird auf PHPUnit umsteigen!

Ich habe mir tatsächlich mal die Mühe gemacht, ein wenig auf GitHub herumzuschauen, ob denn da irgendein Commit zu finden ist, der diese Entscheidung bestätigt.

Nichts im Symfony Repository, nichts bei Fabien aber dann doch was!

Unter dieser URL gibt es die Symfony 2 Testing Conventions zu finden. Darin ist nur noch von PHPUnit die Rede!

Was mich auch freut ist, dass es sich dabei um Commits von Bernhard Schussek handelt und man wohl davon ausgehen kann, dass hier niemand vor den Kopf gestossen wurde. Immerhin war er die treibende Kraft hinter Lime2.

Ich finde es toll, dass sich Symfony mit der Verwendung zuerst einiger Zend Framework Komponenten und jetzt von PHPUnit immer mehr “fremden” PHP Standards öffnet.

Hier nochmal für alle zum Nachlesen. Was daraus zu lernen ist, werde ich mir ein anderes mal ansehen.

Symfony 2 Testing Conventions

Symfony 2 is programmatically tested using unit tests. You can read more about unit testing on Wikipedia.

Test Organization

Directory Structure

The src/ directory contains the three subdirectories Components/, Foundation/ and Framework/, which contains the core bundles. The components and core bundles should have a subdirectory Tests/ which contains all the tests for the component/bundle. The foundation tests belong in the subdirectory Tests/ of the Foundation/ directory.

Some examples:

• src/Symfony/Components/EventDispatcher/EventDispatcher.php should be tested insrc/Symfony/Components/EventDispatcher/Tests/EventDispatcherTest.php
• src/Symfony/Framework/WebBundle/User.php should be tested in src/Symfony/Framework/WebBundle/Tests/UserTest.php
• src/Symfony/Foundation/ClassLoader.php should be tested in src/Symfony/Foundation/Tests/ClassLoaderTest.php

The subdirectory structure in the Tests/ directories should match the directory structure of the classes.

Example:

• src/Symfony/Foundation/Bundle/Bundle.php should be tested in src/Symfony/Foundation/Tests/Bundle/BundleTest.php

Namespaces

The namespaces of the test classes should match their directory structure.

Example:

• The namespace of src/Symfony/Foundation/Tests/Bundle/BundleTest.php is Symfony\Foundation\Tests\Bundle

Test Suites

Every Tests/ directory should contain a configuration.xml file that tells PHPUnit where to find the test classes. You can use the following template for a new component or bundle.

Test Classes

Test classes should have the suffix Test and generally inherit PHPUnit_Framework_TestCase. The name of a test class should refer to a class or the state or aspect of a class.

Some examples:

• FormFieldTest is a good name because it refers to the FormField class
• FormFieldUnboundTest is a good name because it refers to the “unbound” state of the FormField class
• FormFieldCreate is a bad name, because it’s too generic

Test Methods

Methods should support agile documentation and should be named so that if it fails, it is obvious what failed. They should also give information of the system they test

For example the method test name testBindInvalidData() is a good name.

Test method names can be long, but the method content should not be. If you need several assert-calls, divide the method into smaller methods. There should never be assertions within any loops, and rarely within functions.

NOTE Commonly used testing method naming convention test[methodName] is not allowed in Symfony 2. So in this casetestBind() would not be allowed!

Test Fixtures

Shared test fixtures should be set up and deleted in the methods setUp() and tearDown().

Running Tests

Tests can be executed via the phpunit command. If you have not already, install PHPUnit via PEAR.

Then you can execute all tests of Symfony 2:

$ phpunit --configuration=src/Symfony/Tests/configuration.xml

You can also execute the tests of the foundation or of a specific component or bundle:

$ phpunit src/Symfony/Foundation/Tests/AllTests.php
$ phpunit src/Symfony/Components/EventDispatcher/Tests/AllTests.php
$ phpunit src/Symfony/Framework/WebBundle/Tests/AllTests.php

The last option is to execute the tests of all components or all core bundles:

$ phpunit src/Symfony/Components/Tests/AllTests.php
$ phpunit src/Symfony/Framework/Tests/AllTests.php

This article is part of a series on testing untestable code:

• Testing private methods
• Testing code that uses singletons
• Stubbing static methods
• Stubbing hard-coded dependencies

In a unit test, mock objects can simulate the behavior of complex, real (non-mock) objects and are therefore useful when a real object is difficult or impossible to incorporate into a unit test.

A mock object can be used anywhere in the program where the program expects an object of the mocked class. However, this only works as long as the object can be passed into the context where the original object is used.

Consider the following example:

<?php
require_once 'Bar.php';
 
class Foo
{
    public function doSomething()
    {
        // ...
 
        $bar = new Bar;
        $bar->doSomethingElse();
 
        // ...
 
        return TRUE;
    }
}
?>

<?php
class Bar
{
    public function doSomethingElse()
    {
        print '*';
    }
}
?>

With the code above, it is impossible to run a unit test for the Foo::doSomething() method without also creating an object of Bar. As the method creates the object of Bar itself, we cannot inject a mock object in its stead.

In a perfect world, code such as the above could be refactored using Dependency Injection:

<?php
require_once 'Bar.php';
 
class Foo
{
    public function doSomething(Bar $bar = NULL)
    {
        if ($bar === NULL) {
            $bar = new Bar;
        }
 
        // ...
 
        $bar->doSomethingElse();
 
        // ...
 
        return TRUE;
    }
}
?>

Unfortunately, this is not always possible (not because of technical reasons, though).

This is where the set_new_overload() function that is provided by the test_helpers extension for the PHP Interpreter comes into play. It can be used to register a callback that is automatically invoked when the new operator is executed:

<?php
require_once 'Foo.php';
 
class FooTest extends PHPUnit_Framework_TestCase
{
    protected function setUp()
    {
        $this->getMock(
          'Bar',                    /* name of class to mock     */
          array('doSomethingElse'), /* list of methods to mock   */
          array(),                  /* constructor arguments     */
          'BarMock'                 /* name for mocked class     */
        );
 
        set_new_overload(array($this, 'newCallback'));
    }
 
    protected function tearDown()
    {
        unset_new_overload();
    }
 
    protected function newCallback($className)
    {
        switch ($className) {
            case 'Bar': return 'BarMock';
            default:    return $className;
        }
    }
 
    public function testDoSomething()
    {
        $foo = new Foo;
        $this->assertTrue($foo->doSomething());
    }
}
?>

Lets run this unit test:

PHPUnit 3.4.10 by Sebastian Bergmann.

.

Time: 0 seconds, Memory: 7.50Mb

OK (1 test, 2 assertions)

Note that there is no * (printed from Bar::doSomethingElse()) in the output above.

There are few good reasons to share fixtures between tests, but in most cases the need to share a fixture between tests stems from an unresolved design problem.

A good example of a fixture that makes sense to share across several tests is a database connection: you log into the database once and reuse the database connection instead of creating a new connection for each test. This makes your tests run faster.

PHPUnit 3.5 removes the fixture sharing feature of the TestSuite class. It was tedious to use this feature as it required the usage of a custom TestSuite class in addition to the test case class. Furthermore, its implementation was a "hack".

PHPUnit 3.4 introduced the setUpBeforeClass() and tearDownAfterClass() template methods that can be used in a test case class. These methods allow a much cleaner and simpler implementation of fixture sharing between tests of the same test case class:

<?php
class DatabaseTest extends PHPUnit_Framework_TestCase
{
    protected static $dbh;
 
    public static function setUpBeforeClass()
    {
        self::$dbh = new PDO('sqlite::memory:');
    }
 
    public static function tearDownAfterClass()
    {
        self::$dbh = NULL;
    }
}
?>

The example above uses the setUpBeforeClass() and tearDownAfterClass() template methods to connect to the database before the test case class' first test and to disconnect from the database after the last test of the test case, respectively.

It cannot be emphasized enough that sharing fixtures between tests reduces the value of the tests. The underlying design problem is that objects are not loosely coupled. You will achieve better results solving the underlying design problem and then writing tests using test doubles, than by creating dependencies between tests at runtime and ignoring the opportunity to improve your design.

Gestern wurde ich von einem Kollegen (danke Nils) auf das Blog whitewashing von Benjamin Eberlei aufmerksam gemacht. Benjamin ist ein aktiver Contributer verschiedener Open Source Projekte und hat viele interessante Posts geschrieben.

An einem Beitrag bin ich etwas hängengeblieben. Er hat auf der IPCSpring im letzten Jahr einen Vortag über Frameworks und ihren Umgang mit Tests gehalten. Man muss ein wenig googlen, um seine Slides zu finden, aber zu finden sind sie schliesslich auf Slideshare.

Benjamin stellt das Zend Framwork, symfony, ezComponents und CakePHP gegenüber und beurteilt recht objektiv folgende Kriterien:

• Einstellung des jeweiligen Projektes zum Thema Tests
• Anzahl der vorhandenen Unit Tests
• Eingesetztes Testing Framework
• Code Coverage
• Code Duplication

Seine Ergebnisse sind nicht besonders überraschend. Oder doch?

Naja, vielleicht doch. Ich hätte durchaus erwartet, dass CakePHP das Schlusslicht bildet (ganz überheblich und ohne nennenswerte eigene Erfahrungen), auch erwartet hätte ich, dass symfony mit seinem eigenen Testing Framework lime und den (mitte letzten Jahres) nicht besonders zahlreichen Tests nicht viel besser da steht. Von den ezComponents hätte ich durchaus technische Brillianz erwartet, aber kein so gutes Abschneiden in Sachen Tests. Bisher hatte ich immer mal wieder mit ez Publish zu tun und da war zwar eine hohe Funktionalität aber nicht nicht besonders hohe Qualität zu erwarten (der ez Publish Code ist teilweise sehr Spaghetti-artig..). Zend hätte ich aber ganz klar ganz vorne erwartet.

Als ich vor ein paar Tagen in meinen Code Coverage Versuchen zunächst den symfony Code mit analysiert hatte, war ich erstaunt über die recht hohe Abdeckung. Ohne im Detail reingesehen zu haben, habe ich den Eindruck gehabt, als wenn mit der Entwicklung von symfony 1.4 deutlich mehr Wert auf Tests gelegt wurde. Vielleicht, aber das ist nur eine vorsichtige Hypothese, würde symfony heute, ein paar Monate später, besser da stehen.

Aber Benjamin hat sicher Recht, wenn er sagt, es lohnt sich die Tests eines Frameworks zu betrachten, das man einsetzen möchte. Es sollte zumindest den eigenen Ansprüchen genügen, besser aber noch mehr als das. Eine gute Testabdeckung gibt auch dem Anwender eines Frameworks die Chance die Stabilität von Releases zu beurteilen und gibt einen Anhaltspunkt, wie die eigenen Tests anzulegen sind. Im einfachsten Fall kann man immer etwas lernen.

This article is part of a series on testing untestable code:

• Testing private methods
• Testing code that uses singletons
• Stubbing static methods
• Stubbing hard-coded dependencies

With PHPUnit 3.5 it will be possible to stub and mock static methods.

Consider the class Foo:

<?php
class Foo
{
    public static function doSomething()
    {
        return static::helper();
    }
 
    public static function helper()
    {
        return 'foo';
    }
}
?>

When testing Foo::doSomething() we want to decouple it from its dependency Foo::helper(). With PHPUnit 3.5 and PHP 5.3 as well as consistent use of late static binding (using static:: instead of self::) the following is possible:

<?php
class FooTest extends PHPUnit_Framework_TestCase
{
    public function testDoSomething()
    {
        $class = $this->getMockClass(
          'Foo',          /* name of class to mock     */
          array('helper') /* list of methods to mock   */
        );
 
        $class::staticExpects($this->any())
              ->method('helper')
              ->will($this->returnValue('bar'));
 
        $this->assertEquals(
          'bar',
          $class::doSomething()
        );
    }
}
?>

The new staticExpects() method works similar to the non-static expects() variant.

This approach only works for the stubbing and mocking of static method calls where caller and callee are in the same class. This is because static methods are death to testability:

"Unit-Testing needs seams, seams is where we prevent the execution of normal code path and is how we achieve isolation of the class under test. Seams work through polymorphism, we override/implement class/interface and then wire the class under test differently in order to take control of the execution flow. With static methods there is nothing to override. Yes, static methods are easy to call, but if the static method calls another static method there is no way to override the called method dependency."

This article is part of a series on testing untestable code:

• Testing private methods
• Testing code that uses singletons
• Stubbing static methods
• Stubbing hard-coded dependencies

I frequently quote Miško Hevery with

"It is hard to test code that uses singletons."

And then my audience asks me ...

Why is it hard to test code that uses singletons?

Lets have a look at the default implementation of the Singleton design pattern in PHP:

<?php
class Singleton
{
    private static $uniqueInstance = NULL;
 
    protected function __construct() {}
    private final function __clone() {}
 
    public static function getInstance()
    {
        if (self::$uniqueInstance === NULL) {
            self::$uniqueInstance = new Singleton;
        }
 
        return self::$uniqueInstance;
    }
}
?>

The code above declares a class that cannot be instantiated (or cloned) by a client using the new (or clone) operator(s). To get a reference to the only instance of the class one has to use the static method getInstance(). Usually the code that uses the Singleton (which we will refer to as client) is strongly coupled to the getInstance() method:

<?php
class Client
{
    public function doSomething()
    {
        $singleton = Singleton::getInstance();
 
        // ...
    }
}
?>

It is impossible to write a test for the doSomething() method without also invoking the singleton's getInstance() method. This means that we cannot get a fresh instance of the Singleton class and thus have no guarantee that there are no side effects in multiple tests that interact with the singleton.

Dependency Injection

Dependency Injection can help with decoupling the client from the getInstance() method:

<?php
class Client
{
    public function doSomething(Singleton $singleton = NULL)
    {
        if ($singleton === NULL) {
            $singleton = Singleton::getInstance();
        }
 
        // ...
    }
}
?>

Instead of unconditionally invoking the getInstance() method inside the doSomething() we can now optionally pass in an instance of the Singleton class. This allows us to pass in a test-specific equivalent such as a mock object or stub:

<?php
class ClientTest extends PHPUnit_Framework_TestCase
{
    public function testSingleton()
    {
        $singleton = $this->getMock(
          'Singleton', /* name of class to mock     */
          array(),     /* list of methods to mock   */
          array(),     /* constructor arguments     */
          '',          /* name for mocked class     */
          FALSE        /* do not invoke constructor */
        );
 
        // ... configure $singleton ...
 
        $client = new Client;
        $client->doSomething($singleton);
 
        // ...
    }
}
?>

Alternative Singleton Implementations

Either as an alternative or in addition to rewriting the clients to optionally accept an instance of the Singleton class as an argument, we can also rewrite the Singleton class to make testing easier.

Resettable Singleton

The first approach is to add a reset() method to the Singleton class:

<?php
class Singleton
{
    private static $uniqueInstance = NULL;
 
    protected function __construct() {}
    private final function __clone() {}
 
    public static function getInstance()
    {
        if (self::$uniqueInstance === NULL) {
            self::$uniqueInstance = new Singleton;
        }
 
        return self::$uniqueInstance;
    }
 
    public static function reset() {
        self::$uniqueInstance = NULL;
    }
}
?>

Invoking the reset() method causes the getInstance() method to create a fresh object of the Singleton class the next time it is called.

Singleton with Test Context

The second approach is to add a testing context to the Singleton class:

<?php
class Singleton
{
    private static $uniqueInstance = NULL;
    public static $testing = FALSE;
 
    protected function __construct() {}
    private final function __clone() {}
 
    public static function getInstance()
    {
        if (self::$uniqueInstance === NULL ||
            self::$testing) {
            self::$uniqueInstance = new Singleton;
        }
 
        return self::$uniqueInstance;
    }
}
?>

Setting Singleton::$testing = TRUE; causes the getInstance() method to create a fresh object of the Singleton class each time it is called.

PHPUnit Can Help, Too

PHPUnit has a backup/restore mechanism for static attributes of classes.

This is yet another feature of PHPUnit that makes the testing of code that uses global state (which includes, but is not limited to, global and superglobal variables as well as static attributes of classes) easier.

Just Because You Can, Does Not Mean You Should

Yes, it is possible write testable code that uses singletons.
This does not mean, however, that you should use them without thinking twice.

Ich spiele immer noch mit meiner phpunit.xml Konfigurations Datei rum. Irgendwie bin ich immer noch nicht zufrieden, immer wieder google ich nach mehr Informationen. Aber ich will heute gar nicht gross darauf eingehen, was mir an meiner Konfiguration nicht passt, sondern auf zwei Details aufmerksam machen, die nicht dokumentiert sind (afaik) und die in Netz entweder so gut wie garnicht zu erfahren sind bzw. oftmals falsch beschrieben werden.

Hier nochmal der Link zur offiziellen Dokumentation dieser Datei.

Und hier die drei inoffiziellen Nachträge dazu:

Relative Pfadangaben in der phpunit.xml

In der PHPUnit XML Konfiguration kann man eine Menge Pfade angeben. Ich halte das oft für redundant, aber das soll nicht das Problem sein.

Wenn man aber versucht zu erfahren, worauf sich relative angegebene Pfade beziehen, wie z.B. ./test/AllTests.php dann findet man in der offiziellen Dokumentation nichts darüber. In vielen Foren und Blogs findet man wiederum viele Aussagen. Die meisten sind Fragen aber die wenigen Antworten sind zum größtenteil falsch, oder zumindest nicht richtig. Die Aussage findet sich häufig, dass relative Pfad Angaben in der PHPUnit Konfiguration relativ zur XML Datei sind. Führt man PHPUnit dort aus, wo sich diese Datei befindet stimmt das auch:

$ phpunit --configuration=phpunit.xml ..

Führt man aber PHPUnit an einem anderen Ort als dem der XML Datei aus, z.B.:

$ phpunit --configuration=pfad/zu/meiner/phpunit.xml ..

Dann stimmt das nicht mehr.

Der Grund: Pfadangaben in der PHPUnit Konfigurations XML Datei sind immer relativ zum Verzeichnis, in dem der phpunit Befehl abgegeben wird!

PHPUnit Coverage Reporte beachten einige Dateien nicht

Über diese Information bin ich eher zufällig und nur an einer Stelle gestolpert nämlich hier. Danke Axel!

Der normale Coverage Report Erstellung Mechanismus (tolles Wort!) von PHPUnit betrachtet alle Dateien, die von einem deiner Tests “angefasst” wurden.

Hast du eine Datei vergessen zu testen, taucht sie so nicht unbedingt im Report auf. Das kann ich dann mit addUncoveredFilesFromWhitelist umgehen.

...
<filter>
...
  <whitelist addUncoveredFilesFromWhitelist="true">
    <directory suffix=".php">../domain</directory>
    <directory suffix=".php">../system</directory>
      ...
  </whitelist>
...
</filter>
...

Prima Feature, nur leider undokumentiert.. Googled man nach dem Begriff findet man neben Axels Blog nicht viel, allerdings findet sich eine der Sourcen von PHPUnit und darin ein Beispiel XML in den DocBlocks.

In der PHPUnit Konfiguration den PHP Include Path setzen

Sich die Quellen von PHPUnit anzusehen, bringt weiteres zum Vorschein. So ist es neben den dokumentierten Möglichkeiten ini Settings, Konstanten und globale Variablen zu setzen auch möglich den Includepfad von PHP zu setzen.

...
<php>
  <includePath>.</includePath>
  <ini name="foo" value="bar"/>
  <const name="foo" value="bar"/>
  <var name="foo" value="bar"/>
  ...
</php>
...

Ich hab dieses Feature aber noch nicht selbst ausprobiert und kann nicht sagen, ob es den angegebenen Pfad hinzufügt (würde ich erwarten) oder ob es alle Includepfade überschreibt.

Versteht mich bitte nicht falsch, die Dokumentation von PHPUnit ist super umfangreich und informativ. Nur einige wenige Details fehlen, die vielleicht dem ein oder anderen weitrerhelfen, wenn er sie hier findet.

This article is part of a series on testing untestable code:

• Testing private methods
• Testing code that uses singletons
• Stubbing static methods
• Stubbing hard-coded dependencies

No, not those privates. If you need help with those, this book might help.

One question I get over and over again when talking about Unit Testing is this:

"How do I test the private attributes and methods of my objects?"

Lets assume we have a class Foo:

<?php
class Foo
{
    private $bar = 'baz';
 
    public function doSomething()
    {
        return $this->bar = $this->doSomethingPrivate();
    }
 
    private function doSomethingPrivate()
    {
        return 'blah';
    }
}
?>

Before we explore how protected and private attributes and methods can be tested directly, lets have a look at how they can be tested indirectly.

The following test calls the testDoSomething() method which in turn calls the doSomethingPrivate() method:

<?php
class FooTest extends PHPUnit_Framework_TestCase
{
    /**
     * @covers Foo::doSomething
     * @covers Foo::doSomethingPrivate
     */
    public function testDoSomething()
    {
        $foo = new Foo;
        $this->assertEquals('blah', $foo->doSomething());
    }
}
?>

The test above assumes that testDoSomething() only works correctly when testDoSomethingPrivate() works correctly. This means that we have indirectly tested testDoSomethingPrivate(). The problem with this approach is that when the test fails we do not know directly where the root cause for the failure is. It could be in either testDoSomething() or testDoSomethingPrivate(). This makes the test less valuable.

PHPUnit supports reading protected and private attributes through the PHPUnit_Framework_Assert::readAttribute() method. Convenience wrappers such as PHPUnit_Framework_TestCase::assertAttributeEquals() exist to express assertions on protected and private attributes:

<?php
class FooTest extends PHPUnit_Framework_TestCase
{
    public function testPrivateAttribute()
    {
        $this->assertAttributeEquals(
          'baz',  /* expected value */
          'bar',  /* attribute name */
          new Foo /* object         */
        );
    }
}
?>

PHP 5.3.2 introduces the ReflectionMethod::setAccessible() method to allow the invocation of protected and private methods through the Reflection API:

<?php
class FooTest extends PHPUnit_Framework_TestCase
{
    /**
     * @covers Foo::doSomethingPrivate
     */
    public function testPrivateMethod()
    {
        $method = new ReflectionMethod(
          'Foo', 'doSomethingPrivate'
        );
 
        $method->setAccessible(TRUE);
 
        $this->assertEquals(
          'blah', $method->invoke(new Foo)
        );
    }
}
?>

In the test above we directly test testDoSomethingPrivate(). When it fails we immediately know where to look for the root cause.

I agree with Dave Thomas and Andy Hunt, who write in their book "Pragmatic Unit Testing":

"In general, you don't want to break any encapsulation for the sake of testing (or as Mom used to say, "don't expose your privates!"). Most of the time, you should be able to test a class by exercising its public methods. If there is significant functionality that is hidden behind private or protected access, that might be a warning sign that there's another class in there struggling to get out."

So: Just because the testing of protected and private attributes and methods is possible does not mean that this is a "good thing".

Today I have released phpUnderControl version 0.5.1. It's a bug fix release that closes several issues open since a long time. First of all I would like to thank Sebastian Marek who was the main contributor to this releases, so a big thankyou to you.

• Now phpUnderControl should work with CruiseControl 2.8.3. Thanks to Mike van Riel who provided some hints on this issue in a blog comment.
• Fixed #983: Graph unitests throw fatal error when ezComponents not available.
• Fixed #966: phpcs-details.xsl not showing file name.
• Closed #863: Destination option is now deprecated.
• Fixed #862: Command line switches without parameter don't work.
• Fixed #861: Password is used as username in check outs. This patch was supplied by Thorsten Daners via e-mail.
• Fixed #734: Now the build dropdown redirects to the correct build uri.
• Implemented #703: PHPUnit test results are now the first entry on the project overview page.
• Fixed #700: Throw an exception when the specified project does not exist.
• Implemented #675: Use "php -l" for lint checking and not PHPUnit.
• Implemented #625: Integrate PHP_Depend results.

Beside the new release some more things have changed. From now on the phpUnderControl development is hosted on github. This means that from now on the latest version of phpUnderControl can be obtained with the following command:

mapi@arwen ~ $ git clone \
               git://github.com/manuelpichler/phpUnderControl.git

Additionally we have moved the phpUnderControl's PEAR Channel Server from pear.phpunit.de to its own server pear.phpundercontrol.org. At this point I would like to thank Sebastian for providing phpUnderControl's infrastructure under the PHPUnit umbrella for the last three years.

mapi@arwen ~ $ pear uninstall phpunit/phpUnderControl
mapi@arwen ~ $ pear channel-discover pear.phpundercontrol.org
mapi@arwen ~ $ pear install --alldeps phpuc/phpUnderControl-beta
Starting to download phpUnderControl-0.5.1.tgz (539,717 bytes)
..........................................done: 539,717 bytes
install ok: channel://pear.phpundercontrol.org/phpUnderControl-0.5.1

For the upcoming PHPUnit 3.5, I have factored out all code that is related to code coverage and put it into a separate component: PHP_CodeCoverage.

PHP_CodeCoverage is a component that provides collection, processing, and rendering functionality for PHP code coverage information. It makes PHPUnit's mature code coverage functionality available outside of PHPUnit.

Having all code that deals with code coverage in a separate component allows for easier development and better testing. The first result of these improved development conditions is a small new feature that I recently implemented, the support for the CRAP metric.

From crap4j.org:

The CRAP (Change Risk Analysis and Predictions) software metric [has] a mildly offensive metric name [and helps] to help protect you from truly offensive code.

The CRAP metric combines cyclomatic complexity and code coverage from automated tests (e.g. [PHPUnit] tests) to help you identify code that might be particularly difficult to understand, test, or maintain — the kind of code that makes developers say: “This is crap!” or, if they are stuck maintaining it, “Oh, crap!”.

The screenshot below shows how the CRAP metric is reported in the HTML code coverage report:

Over the Christmas holidays I took a break from translating and editing the book on quality assurance in PHP projects that I am working on to finally migrate the code repository of PHPUnit from a self-hosted Subversion repository to a Git repository that is hosted on GitHub.

How did I get here?

I was staying at Arjen's place in Brisbane in August 2008 and one day we talked about distributed version control. Although I knew the basic concepts behind it, I was not yet convinced enough to take the plunge and try it out. Our discussion changed that, but it was not until my second trip to Australia later that year that I took the initial code for PHP_ObjectFreezer (which I had on my laptop since I started to hack on it together with Stefan on the flight to Atlanta for php|works 2008) and put it into a bzr repository on Launchpad. I grew comfortable enough with bzr quickly, but never really warmed up to Launchpad.

Around that time, I was hearing more and more positive things about Git and GitHub from my peers. So I took the PHP_ObjectFreezer code base and migrated it to Git and hosted the repository and GitHub to experiment with both the tool and the platform. It was a step that I did not regret and all the projects that I started since then are hosted on GitHub: bytekit-cli, phpcpd, phpdcd, phploc, PHP_CodeCoverage, PHP_FileIterator, PHP_ObjectFreezer, Text_Template, PHP_TokenStream.

Why did I do this?

Let me start with a quote from GitHub.com:

• Git is a fast, efficient, distributed version control system ideal for the collaborative development of software.
• GitHub is the easiest (and prettiest) way to participate in that collaboration: fork projects, send pull requests, monitor development, all with ease.

Over the last year, I have come to appreciate Git and GitHub a lot and also wanted to use them for PHPUnit.

Sure, I could have used git-svn for bi-directional operation between the existing Subversion repository and Git, but this would have always felt like only going half the way.

The arguments in favour of distributed version control systems are discussed by others far better than I could discuss them here, so I will just quote Ian Clatworthy's paper on distributed version control systems, in which discusses not only the technical differences between traditional version control systems and distributed version control systems, but also the differences with regard to development workflows and developer interaction:

• Developers can collaborate directly without needing central authority or incurring central administration overhead
• Developers can still be productive when the umbilical cord to their central VCS repository is broken, e.g. when travelling.
• Creating and destroying branches are simple operations. This is particularly useful when experimenting with new ideas, e.g. a spike when using eXtreme Programming.
• Ad-hoc collaboration with peers [is facilitated by] intelligent merge tracking [because] merging early and merging often is both possible and surprisingly unpainful. It is difficult to explain just how much of an impact this can make on how co-developers can work together more easily, e.g. when Pair Programming.

What does this mean for you?

PHPUnit is now hosted on GitHub.

In a nutshell, this means that the development of PHPUnit is now more open than it was before. If you want to contribute, you just need to fork PHPUnit on GitHub and create a topic branch for your contribution. Then let me know about your topic branch and I may or may not merge it.

If you are currently using svn:externals to pull PHPUnit into your own Subversion repository, you need to think about an alternative as PHPUnit's Subversion repository is no longer updated and will eventually go away.

How did I do it?

As this might be of interest to someone who wants to migrate his/her repository from Subversion to Git, here is how I performed the migration.

I started by creating a local mirror of the Subversion repository:

# Fetch Subversion repository
svnadmin create phpunit.svn
 
cat 'EOF' > phpunit.svn/hooks/pre-revprop-change
#!/bin/sh
USER="$3"
 
if [ "$USER" = "svnsync" ]; then exit 0; fi
 
echo "Only the svnsync user can change revprops" >&2
exit 1
EOF
 
chmod +x phpunit.svn/hooks/pre-revprop-change
 
svnsync init \
  --username svnsync \
  file://`pwd`/phpunit.svn \
  svn://svn.phpunit.de/phpunit
 
svnsync sync \
  --username svnsync \
  file://`pwd`/phpunit.svn

Then I used svn2git for the actual migration:

# Create directory for Git repository
mkdir phpunit.git && cd phpunit.git
 
# Use svn2git to import repository
svn2git \
  --trunk trunk \
  --branches branches/release \
  --tags tags \
  --authors /home/sb/authors.txt \
  file:///home/sb/phpunit.svn/phpunit

After that I had to delete quite a few superfluous branches and tags, but that quickly dealt with. The final step was to create common ancestors for the master, 3.5, and 3.4 branches so that merging becomes easy and painless.

# Create 3.5 and 3.4 branches (in local repository)
git checkout remotes/origin/3.5 -b 3.5
git checkout remotes/origin/3.4 -b 3.4

# Create common ancestors for the master, 3.5, and 3.4 branches
git checkout 3.5 && git merge --strategy=ours master
git checkout 3.4 && git merge --strategy=ours 3.5

git checkout master && git merge --strategy=ours 3.5
git checkout 3.5 && git merge --strategy=ours 3.4

After that I had to delete quite a few superfluous branches and tags, but that quickly dealt with.

At this point I would like to thank David Soria Parra who helped me figure out some details here and there.

Update: GitHub now supports Subversion clients:

svn co https://svn.github.com/sebastianbergmann/phpunit

After quite some time of silence around phpUnderControl I have bundled the 0.5.0 release today. Beside minor changes, bugfixes and enhancements this release contains one new major feature, the PHP_CodeBrowser.

phpUnderControl integrating PHP_CodeBrowser

The PHP_CodeBrowser is a separate application that collects various XML log files with different project metrics/violations and presents them in a browseable source view with syntax highligting. This tool is a contribution by the Mayflower GmbH, where it was developed and now shared with the Open Source Community. At this point I would like to thank Mayflower as a whole and in particular at Elger and Thorsten, which were responsible for all technical aspects of this contribution.

To use the PHP_CodeBrowser you must add an additional execute publisher to your CruiseControl config.xml file that generates the PHP_CodeBrowser html report and an additional artifacts publisher to move the generated PHP_CodeBrowser report into the projects artifacts directory.

<?xml version="1.0" encoding="UTF-8" ?>
<cruisecontrol>
  <!-- ... -->
  <project name="PHP_Depend" buildafterfailed="false">
    <!-- ... -->
    <publishers>
      <!-- ... -->
      <execute command="phpcb 
        --log projects/${project.name}/build/logs 
        --source projects/${project.name}/source/PHP 
        --output projects/${project.name}/build/php-code-browser"/>
      <artifactspublisher 
        dir="projects/${project.name}/build/php-code-browser" 
        dest="artifacts/${project.name}" 
        subdirectory="php-code-browser"/>
      <!-- ... -->
    </publishers>
  </project>
</cruisecontrol>

But why do we use a CruiseControl publishers instead of a regular ant (Your build tool here) target? The answer is really simple. The PHP_CodeBrowser must be the very last artifact generated for a project, which means it must also run after PHPUnit, to collect the test result logs. But in most setups PHPUnit is configured with failonerror="on" to mark a build as failed, when an error occured during the test execution. But with this configuration a following PHP_CodeBrowser target would never be executed by ant, because the build failed already. This cannot happen with a CruiseControl publisher which is always executed in a separated process.

If you create a new project with phpUnderControl's project command, phpUnderControl will automatically search for an installed PHP_CodeBrowser and add the required publishers to your config.xml file when it is present.

mapi@arwen ~ $ phpuc project \
       --project-name PHP_Depend \
       --source-dir PHP \
       --test-case PHP_Depend_AllTests \
       --test-dir tests \
       --test-file PHP/Depend/AllTests.php \
       --version-control svn \
       --version-control-url http://svn.pdepend.org/branches/0.9.0 \
       /opt/cruisecontrol/cruisecontrol-bin-2.8.2

To get the latest version of phpUnderControl, you can use the PEAR-Channel-Server:

mapi@arwen ~ $ pear channel-discover pear.phpunit.de
mapi@arwen ~ $ pear install phpunit/phpUnderControl-beta

or you can check it out from the subversion repository:

mapi@arwen ~ $ svn co svn://phpunit.de/phpunit/phpUnderControl/trunk

• Best Practices
  • TYPO3: The Agile Future of a Ponderous Project
  • Unit Testing Bad Practices by Example
  • Quality Assurance at Digg
• Servers and Services
  • Testing Service Oriented APIs
  • Testing a WebDAV Server
  • Testing Database Interaction
• Architecture
  • Testing Symfony and Symfony Applications
  • Testing the ezcGraph Component
• QA in the Large
  • Large-Scale Selenium-Based Testing at studiVZ
  • Continuous Integration with phpUnderControl
  • swoodoo: A True Agile Story
• Non-Functional Aspects
  • Performance
  • Usability
  • Security

I have released PHPUnit 3.4.2 today.

Besides the usual bug fixes, this release disables the backup and restore operations for static attributes (that was introduced in PHPUnit 3.4.0) by default.

The previous default setting caused too many problems with existing test suites (problems that would have been recognized earlier had users actually tested the PHPUnit 3.4 release candidates).

To enable the backup and restore operations for static attributes, simply pass the --static-backup command-line switch, use the @backupStaticAttributes annotation, or the XML configuration file.

Below is the full list of changes:

• Fixed #889: --skeleton-class does not work with @depends annotation. [5270]
• Fixed #902: PHPUnit_Util_File::getClassesInFile() does not handle nested namespaces correctly. [5272]
• Fixed #905: Files with no methods or classes show incorrect code coverage with --coverage-clover. [5276] [5284]
• Fixed #909: Stubbing a web service with getMockFromWsdl() throws a fatal error. [5281]
• Fixed #918: Truncate operation throws an error. [5286]
• The backup and restore operations for static attributes has been disabled by default. [5288]

After my discussion with Arno about the right way to develop software I’ve decided that it would be a good chance to meet him in person while visiting PHP Con in Barcelona. And besides that, Barcelona and the PHP Folks there is always good for a visit.

Therefore there is - once again - another chance to enjoy my talks called ‘Refactor it! A practical journey into the test-driven world’ and ‘Agile Development with PHP in Practice’ . I did run both topics already quite often but obviously there is still a need for it. If you are new to agile development or interested in refactoring your old (shitty?) sourcecode - come in and join.

And if you are interested in the topic of the discussion with Arno - feel free to contact me. I will organize a meeting with where we will discuss the whole item over a few bottles of red wine. After receiving so many comments it would be nice if we get a few more opinions together.

After Barcelona Summer is definitly gone and it’s - as every year - time for the international PHP Conference in Karlsruhe. This time Thorsten and me will run a half-day Workshop called “Unittest for Dummies” which will address unit testing beginners. If you are not already into TDD you should definitly consider to come. Furthermore there will be a session about how swoodoo managed to withstand the massive amount of requests due to (*yeah*…) the big success. I’ve titled it “caching, sharding, distributing - Scaling best practices” - so you might get an idea about the contents.

That’s it - live from the “Oktoberfest” in munich

A year and a day has passed since the release of PHPUnit 3.3. Time to finally release PHPUnit 3.4.

Among the features introduced in this new version, the most notable are the support for test dependencies and fixture reuse as well as the possibility to run tests in separate PHP processes for increased test isolation. Please have a look at the ChangeLog for a complete list of changes.

Work on PHPUnit 3.5 has already started: the php-code-coverage project on GitHub is home to a refactoring of PHPUnit's code coverage functionality that will be one of the "hallmark features" of PHPUnit 3.5. It will also make PHPUnit's code coverage functionality available outside the scope of PHPUnit.

When I first saw the posting about ‘bad’ code in OXID over at phpterror I wondered if I could ignore this - but now I’ve realized that this article was published on Planet-PHP and even more - other people start to copy the content of the article.

As I am the guy who introduced the disliked functionality many years ago ( actually years before ZF popped up) I feel the need for a statement to put the things into the right order. Please note that I did work for OXID in the past (years ago) but I do not nowadays.

Arno criticised the way modules == classes are instantiated in OXID eShop. Actually I do believe that exactly this feature is the most coolest in OXID and should be implemented in more OSS.

Let’s start with the “why the fuck did the guy implement this”?

If you do run a website out there you know how important it is to keep it up-to-date with the latest patches due to security reasons. Now imagine - we are in eCommerce area. You deal with payments, credit card data and sensitive information about what people ordered from your shop. Not only Creditcard data is sensitive - imagine it would leak that you ordered the extra-big-boobs doll? You see - especially in this area you need to make absolutly sure that your servers and the software is safe.

Unfortunately you can’t start an eCommerce business out-of-the-box. You need to adopt the software to your needs and processes. Think of payment, ERP, delivery notifications, link to your stock, uploads to price-comparison sites etc. Some of the functionality you might need to develop yourself - for other stuff you might find already existing modules out there.

Therefore you need to change code, and/or install external modules which modify the functionality. In former days you simply edited the source code (as we are talking about OSS) and made manually sure that the modules you install are compatible. Don’t forget - each of them will work with the out-of-the-box shop - but they still should work after you added your changes and - also need to work after you installed some other external modules which might overwrite/change the same classes/functionality which your new module also want to change.

You end up with a highly customized shop, many changes, a lot of work to dig through the sourcecode of the installed modules to make sure that they don’t harm themself and… you lost the possibility to automatically appply patches/releases. Upon each patch or new release you need to manually redo/check your changes.

This sucks. It sucks a lot. And this is exactly why I’ve introduced the criticised module functionality.

So what did I do?

As Arno already copy&pasted the source I won’t repeat it here. But let me explain the main idea behind the concept.

As OXID is fully OOP you can change all the functionality by inheriting your class from any base class you might want to change. You add your changes and - et voila - you still can overwrite the bases classes with new releases/patches and your changes will work. This is called object oriented programming.

Let’s assume you want to change the method “getPrice” in the class “oxarticle”. You simply overwrite oxArticle::getPrice and… it would work if the system would know that your class exists. Therefore you need to register your class and let the OXID Framework know about it so that it automatically will instantiate your class instead of oxArticle each time the object is needed.

So far, so good, so what?

What happens if you install some other module ( e.g. from here) which will change the same method in same class? It would screw your changes and you would end up with manual changes again. To avoid this, the OXID Framework supports “chaining” of inheritance… and the order is set in the config. So you can define that the modules oxArticle implementation is executed before your class or after.

I do believe that by overriding the classes you do get the most flexible option to change everything without loosing the functionality to update the shop whenever you want. For sure this could be solved differently. Events or Hooks would have been a way to go - but this would involve a lot of additional coding == lines of code and therefore introduce new vectors for bugs. And - it is by far not as flexible as overriding the class. Nowadays it could be solved also with Reflection. But this is a bit too “magic” in my opinion. Therefore I still believe that the way I’ve chosen is the best way to solve the problem and - most likely - I would do it again today.

Last but not least - I would like to comment the show me your 94% Unit Test coverage! posting.  I’ve contacted OXID already one year ago about this and I fully agree with Arno here. It sucks to advertise with some test-driven-development features and then keep the tests for yourself. OXID has to change this. Now.

Today I had a good skype discussion about the issue with Arno - I really like his style even though we both do not agree - and the result for me was that  I’ve decided to submit a few proposals to the PHP Conference in Barcelona to get the chance to meet Arno and discuss the whole issue over a bottle of spanish Rioja. Or two. Or…

The first release candidate of PHPUnit 3.4 is now available.

Among the many new features introduced in this new version, the most notable are the support for test dependencies and fixture reuse as well as the possibility to run tests in separate PHP processes for increased test isolation. Please have a look at the ChangeLog for a complete list of changes.

You can help make PHPUnit 3.4 a "good release" by running your test suites with it and report any problems or regressions that you may encounter. You help is much appreciated!

Folks, it’s Paaaaaaaaaaarty Time!

We like to invite you to the PHP 5.3 release party which is an event to
celebrate the 5.3 release, happening Friday, the 17th of July in Munich.
The release party offers a chance to come together with other php
enthusiasts and enjoy that php is alive and kicking. And of course people
in favour of a decent barbecue, together with some beer and other drinks
are invited.

The happening will take place at Waldwirtschaft beer garden, at any weather. We will meet at 19:00 o’clock - open end. The location is famous for its huge beer garden (2500 available seats, a children’s playground) and its typical Bavarian but also international food. On sunny weather you even may enjoy live-music and listen to the sounds of Jazz, Blues, Swing or Dixi.

Catering will be provided and as a special delicacy you may enjoy a suckling pig!

If you like to join the event please register at PHPUG-Munich Wiki and follow it for updates.
Alternatively you may register at Facebook as well and follow this for updates.

For any questions please visit IRC channel: #phprp on irc.uni-erlangen.de.

The PHP 5.3. BBQ release party is sponsored by:

* Microsoft
* Mayflower GmbH
* Swoodoo AG
* Zend Technologies GmbH

Supporters for the PHP 5.3 BBQ release party are:

• Sun Microsystems
• PHPUG.de

I hope to see you there!

Update: edited sponsor list

Software metrics are currently on everyone's lips and a frequently discussed topic. There are many conference talks, blog posts and other presentations that talk about software metrics. But to me it seems as if this subject is a closed book for many developers, so I decided to write this little post about a special category of software metrics, the complexity metrics.

Complexity metrics are a theoretical approach to measure the subjective complexity of a software fragment, where the words software fragment stand for a paraphrase for functions, methods, classes and nearly every logical unit that can be found in a software system. The most prevalent procedure to calculate complexity values is static code analysis, where an application parses the raw source code of a project, counts different statements and expressions and packs up the determined results in simple classification numbers. And with this information you already know the main concepts behind most software metrics, classification numbers and counting. As you can see there is no magic behind the scene, the only thing required is a good background knowledge to interpret those values.

The Cyclomatic Complexity Number or short CCN is the oldest complexity metrics. The first time this software metric was mentioned was 1976 by Thomas J. McCabe. This metric counts the available decision paths in a software fragment to determine its complexity. Each decision path starts with one of the conditional statements from the following list, so that it is fairly easy to detect them in existing source code.

• ?
• case
• elseif
• for
• foreach
• if
• while

A look at this list of statements may result in the questions: Is this list wrong, it doesn't list else and default? But it is correct. The assumption is that both statements will contain the defaut execution path of a program which also exists when there are no special cases to capture.

Each decision path gets the value 1 and the sum of all these values represents the Cyclomatic Complexity of the analyzed software fragment. Note that each function and method also counts with a value of 1 With this knowlegde we can now calculate the complexity of the following example code:

Based on the previous definition the Cyclomatic Complexity Number of the example code example is 5. But you may have noticed that this approach does not capture all decision paths that exist. We haven't catched those paths that came from the by the boolean expression || line 6 and && line 8, and the logical or expression in line 25. A variation of the Cyclomatic Complexity Number that also captures those paths is the so called CCN2. The CCN2 is the most widely used variation of this software metrics. Tools like PHPUnit, PMD and Checkstyle report it as Cyclomatic Complexity of an analyzed software fragment.

Now we get a complexity value of 8 when we apply the CCN2 to the previous example, what is a growt of the software's complexity of 60%.

Due to the fact that Cyclomatic Complexity Number was originally invented for procedural programming languages, this definition for the Cyclomatic Complexity Number still misses one element to measure the complexity of an object oriented software system. With the concept of exceptions a software gets additional decision paths for each catch statement used in the source code. While try contains the code for the regular execution code without special cases, similar to else and default statements.

• ?
• &&
• ||
• or
• and
• xor
• case
• catch
• elseif
• for
• foreach
• if
• while

Now that we know what the Cyclomatic Complexity Number is, what can we do with the measured information? We can find the complexity hotspots in a system, for example the top ten artifacts with the highest complexity, but this is only important during an initial analyses phase to get the big picture of an application. For a continuous inspection this information is not so important. A continuous analyses requires thresholds that help to categories calculated values. During the time four values have emerged as good thresholds for the Cyclomatic Complexity Number of a software system.

• A software fragment with a CCN value between 1-4 has low complexity.
• A complexity value between 5-7 is moderate and still easy to understand.
• Everything between 6-10 has a high complexity, while everything greater 10 is very complex and hard to understand.

You may ask, why should I care about the complexity of a software system, where is the value of benefit in this metric?

Mostly the complex parts of an application contain business critical logic. But this complexity has negative impacts on the readability and understandability of source code. Those parts will normally become a maintainence and bug fixing nightmare, because no one knows all the constraints, side effects and what's exactly going on in that part of the software. This situation results in the well known saying "Never touch a running system" which in turn mostly ends in copy&paste programming. The situation can even become more critical when the original author leaves the development team or the company.

Finally a small example how to apply the new knowledge about the Cyclomatic Complexity Number, thresholds and the negative impacts of complex software to an existing development process. The following source listing shows a complex method taken from PHP_Depend's source. This method has a Cyclomatic Complexity Number of 16 and I must admit that the original author needed some time to understand what was going on in this method.

The first thing to do is to make sure that the test suite is good enough to ensure that the required refactorings will not change the public behavior of the component or class. When this is donw and we are sure our that api breaks will be detected by the test suitewe can start to extract logic into separate methods.

The following example shows the result of the refactoring:

The subjective feeling of readability heavily depends on the complexity of control structures, as we can see by a comparison of the original and the refactored version of the method example. The new version with its Cyclomatic Complexity Number of 5 is much easier to read and understand.

This text is the first of two blog posts. The second article will give a short introduction into the NPath Complexity You liked this article and you are interested in this and other quality assurence related topics? - Then you should now order your copy of the Book Quality Assurance in PHP Projects. The book talks about nearly all aspect of quality assurence, with practical tips and expert knowledge contributed by certain PHP professionals.

For the full article visit Introduction to PHPUnit

This one reminds me on my past - after seeing it, I feel very happy that I do not do any contracting work any more. I still don’t know if I should laugh or cry. It might be old - but nevertheless worth to watch it.

(English - with German subtitles)

via Guido Muehlwitz

• Fixed #386: Parse error in eval() while mocking SoapClient::__soapCall. [4897]
• Fixed #782: Missing include in DefaultDatabaseConnection.php. [4876]
• Fixed #783: Tests getting executed twice when using multiple groups. [4907]
• Fixed #801: PHPUnit_Util_Fileloader::load() fails silently when file does not exist. [4928] [4929]

This is an abstract for a chapter from a book on Quality Assurance in PHP Projects.

Although Unit Testing is a recommended practice for any software project, care has to be exercised such that testing yields the desired benefits. Bad programming practices in both test code and production code can make Unit Testing a nightmare. Maintenance of an overly complex test suite can easily become a burden to the project team.

Situations of hard to maintain test code can have extremely negative outcomes for project quality. Programmers start to ignore tests, because of their inconclusive description of what is going wrong. New features might not be tested at all, because writing tests for the current architecture considerably extends development time. In the end the project manager might decide to stop Unit Testing alltogether, because the costs outweight the benefits.

Bad practices of Unit Testing manifest in so called "Test Smells". These are an early indicator of problems for long-run maintainability and utility of a project test suite. To constantly derive a benefit from Unit Tests, a commitment for high quality tests has to be made.

This case study will discuss Unit Testing bad practices and well as Test Smells and gives hints on how to avoid them. For each test smell, examples are shown from well-known PHP Open Source projects. As a result, the reader should be aware of the possible pitfalls of Unit Testing and that any test code requires the same care that is put into production code.

Benjamin Eberlei is a Project Lead with direkt effekt GmbH and contributes to the Zend Framework and other Open Source projects.