From Walter Cedric Wiki
Jump to: navigation, search



This solution is not the official one from This work is a work in progress! I hope to be able to support Joomla! core contiuous build and also extensions development.

Contact ME  if you want to help:

Continuous Integration

is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage. Martin Fowler about continuous integration

Things get clearer in my mind...I will use a set of Open Sources software to reach this ambitious goal.


Continuous build is all about delivering software faster and with more confidence

This solution use Apache Maven

By using Apache Maven ( and Maven for PHP (, you will be able to have a frame for the lifecycle phase


See and

There is a huge number of reporting and build plugins available for CSS, HTML, Code convention check (chekstyle ala PHPCheckstyle)
Documentation (PHPDocumentor, Doxygen), is generated along with a customizable site (part of lifecycle, one like this done without any effort!), tar.gz, zip is generated in maven phase

Apache Maven force Conventions over configuration: Joomla! code will be in src/main/php and test cases in same project in /src/test/php and run if your maven phase is = or bigger than test. Developer have a proven frame, all plugins expect to find code there (you can have a custom mapping but it is not recommended)

PHPUnit, Selenium, Accunetix and any java tool for functionnal testing cab be plugged in build lifecycle.

Freedom to choose any continuous build server

will be able to run the build: no boat anchor software:

  • Jetbrains Teamcity, support up to 20 build plan without license, they offer free licenses with unlimited functionnalities for Open Source Project
  • Atlassian Bamboo, they offer free licenses with unlimited functionnalities for Open Source Project
  • Apache Hudson,
  • Cruise Control

and so on will work just if you configure them (you will find numerous tutorials to build a maven project in these continuous build server)

You can even AVOID using a continuous build server!

  1. make a checkout of your code (cvs, svn, git) in a directory
  2. run maven in a command line: mvn test
  3. put all the above steps 1,2 in a crontab and you are done



Anybody will be able to run your project or use any build server

  • Buildserver have all now a build in assistant for configuring Apache maven projects
  • Apache Maven is an universal software lifecycle

Every commit can/will trigger a build

Commit can be rejected if build is breaking test cases with delayed commit (

IDE support

Build grid Agents

A grid to... Support multiple OS, PHP version, Browser version testing!

With Bamboo or TeamCity you can start Java agent that are waiting for a task from the controller (Teamcity or Bamboo).

You can have an unlimited number of agent distributed worldwide, ONE for EACH configuration you want to

Each remote agent

  • checkout code,
  • compile php, run quality checks, 
  • run test cases (PhpUnit)
  • deploy into a running environment the php code in phase "pre-integration-test"
  • run selenium testcase in phase "integration-test"
  • report to controller their results

Agent run in parallel

You can then design selenium test against a local selenium RC that is start in maven phase pre-integration test (each agent has its own) .

Added Value for Joomla!

You can have one agent per system you would like to support

We can imagine multiple agents:

  • agent1 Linux server / php 5 / mysql
  • agent2 Linux server / php 5 / sql server
  • agent 3 windows server / iis server / mysql
  • agent 4 ...

Today I use 4 Teamcity Agent at

  • agent1 to agent3 is local and headless (without UI)
  • kubuntuagent is a kubuntu 10.4 + php5 + chromium + firefox

Agent architecture

Build agent

  • Is a JAVA process daemon that connect to Teamcity or Bamboo using TCP socket
  • Can run on any Operating system: Redhat, Debian, Windows, Mac thanks to java
  • Has a Maven engine (java)
  • Has a Maven local repository /home/agent/.m2/repository
  • Database for Unit Test/App runtime (MySQL or any database we would like to support)
  • Application runtime: any PHP version supported by the operating system or we would like to support
  • Has a Selenium RC server started as part of the Maven build lifecycle

NOTE: due to the nature of selenium GRID and all selenium RC using TCP protocol, it is not mandatory to run them in the same agent. 

  • A server named can expose a selenium grid to the continuous build grid
  • Individual people can register their selenium RC and their custom browser

The continuous build grid can then request a browser to the grid

It is just nicer to have them in agent at the cost of a bit more memory consumption (hey memory is cheap)

What an Agent is doing

Zoom on distributed agent "kubuntuagent"


Why Kubuntu?

  • I can not provide a free windows operating system image due to licensing issues, so it has to be a linux
  • Ubuntu is what most people described as user friendly
  • I prefer KDE over Gnome, see no offense

Download it at

An actual version of the agent can be found at

(link to appear soon)

Run the agent

  1. Unpack kubuntuagent.7z, 8GB required
  2. use vmplayer to open the image,

it will auto register to under the agent name "kubuntuagent" and be ready to accept job tasks

If not open a terminal

su - agent
cd ./buildagent/bin
./ start

Force a build on that agent

In teamcity choose the build 


And force a new build to run on this newly registered agent


How to create a pool of identical agent

Every developer that commit a change to SVN will trigger a Private Build:

Developer changes will land in the SVN tree onyl if it pass all JUNIT testcases and all Selenium tests. A private build is blocking an agent, and can run a long time, So it may be useful to run a lot of them. Agent may be sometimes identical (PHP, MYSQL version) sometimes not!

  1. Copy the VM multiple times, as many as you want the pool size to be
  2. You can run all VM locally or relocate them anywhere in the world as long as they have an internet access and can reach

As root, in a terminal, edit the file 

# vi /home/agent/buildagent/conf/

And change the name value, it has to be unique

## The unique name of the agent used to identify this agent on the TeamCity server
## Use blank name to let server generate it

Restart the build agent

# /home/agent/buildagent/bin/ start

Wait some seconds to see the agent registering in Teamcity and appear in the list at

How KUbuntuAgent was prepared

(Technical notes to recreate you own agent with any other linux os)

I did download a VMWARE appliance at  (999 MB)

but you

  • can use any virtualization runtime: XEN / Virtual desktop / Vmware
  • can use any LInux operating system

The Kubuntu team is proud to introduce our latest release - 10.04 LTS, the 'Lucid Lynx'. This is our first long-term support release featuring the KDE Plasma desktop. Our selection of tools and applications will provide you with all that you need for most of your tasks, with many more available just a few clicks away. Whether browsing the web, playing your music, composing an email or connecting with your friends on social networks, Kubuntu 10.04 LTS brings you a stable, innovative and attractive platform for all your desktop needs.

Login: vmplanet

Start the Virtual machine, log in as root in a console

Install some browser

chromium, firefox

Install PHP

# sudo apt-get install php5-cli
# sudo apt-get install php-pear

Prepare user, group

Open a terminal, as root

Add a new group and user for more security (one user for every daemon with limited or no rights to do something else is a good Linux guideline!)

# groupadd agent
# useradd -g agent -c 'agent user for agent' -m agent

Install Teamcity agent

as root

# su - agent
# wget
# apt-get install unzip
# unzip
# cd buildagent
# cd conf

rename file buildAgent.dist.propertiesto have a file

# mv
# vi

change at least the following line

## The address of the TeamCity server. The same as is used to open TeamCity web interface in the browser.

## The unique name of the agent used to identify this agent on the TeamCity server
## Use blank name to let server generate it

Install Maven

Every teamcity agent has it own version of Maven, for more control, we install our own. I use a symbolic link because the path to Maven will be in Teamcity web gui admin configuration, so I want to keep it stable

As root

# su - agent
# wget
# apt-get install unzip
# unzip
# ln -s apache-maven-3.0-beta-2-bin maven

Configure the settings.xml

As root

# su - agent
# vi settings.xml

put this content inside

<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns=""


<snapshots />
<snapshots />

install java JDK

as root

# sudo apt-get install default-jdk
# update-alternatives --config java

Check that java -versio return a real SUN jdk and not an openJDK

vmplanet@ubuntu:~$ java -version
java version "1.6.0_18"
OpenJDK Runtime Environment (IcedTea6 1.8.1) (6b18-1.8.1-0ubuntu1)
OpenJDK Client VM (build 16.0-b13, mixed mode, sharing)

install PHP/PhpUnit

I follow the very good doc at

sudo pear channel-discover
sudo pear channel-discover
sudo pear install phpunit/PHPUnit

install selenium grid


install as much selenium RC as browser

as root

# sudo - agent
# wget
# unzip
# vi

and put inside

nohup java -jar ./selenium-server-1.0.3/selenium-server.jar -browserSessionReuse

Make this script runnable and start it

# chmod +x
# ./

See it live

I eat my own dog food, you can see this solution live running at

Joomla! project file structure

Following Maven best practices: conventions over configurations test cases belong to the same root tree as Joomla! source code. This is because a change in the code of Joomla! may require to add/remove/alter test cases, so a unit of work (down to a commit) may also contains those test cases changes

/joomla is the name of the project, it is a Maven project / eclipse project / netbeans projects (a root in your ide)

     /src/main/php/ this is now the root of Joomla source code

     /src/test/php   this contains pure junit test, run all test in maven phase "test"

     /src/test/php  this contains selenium test, the selenium plugin will locate them based on path regular expression **/selenium/**/*

Mapping to new structure

I did made a checkout of

Move files around following a Maven for PHP project file structure, and commit the result in my own svn at

Normally this is wrong but since Joomla! official team is not using this solution, nor creating a branch for me, this is a temporary workaround.

Continuous build

Now in this SVN

  1. Every normal commit  will trigger a "public build" (everybody in eclipse/netwbeans will be notified of files committed, that the build is starting) after 60 seconds (default grace build delay)
  2. A private build or delayed commit,  will trigger a "private build", only if all tests are successful the commit will be done, a "public build" will follow to alert registered user of that build in 

Teamcity builds 


Build are located at   a guest login is enough to see the status

But You will need to create a registered user and be granted by me to start a build manually (run button)

Behind the scene

Maven phase "compile"

Nothing special is done, the PHP compiler compile all PHP files, checking for consistency

Maven phase "test"

Some PHPUnit tests require to have a valid database to run test against it

  1. This database is created by a Maven plugin in Maven phase "generate-test-resources"
  2. PHPunit test are then run in Maven phase "test"
  3. The test database is then deleted in Maven phase "prepare-package"

Normaly PHPunit test thta require a database are call integration tests and SHOULD run in Maven phase "integration-test"

NOTE from Joomla WIKI

Set Up Configuration File

Many unit tests can run independent of an actual Joomla! instance or database. Other tests require a connection to a Joomla! database. For these tests to run correctly, you need to create a test config.php file.
If you don't have a correct configuration file, you will see a message similar to the one below when you try to run certain unit tests:
PHP Fatal error: Class 'JElement' not found in
/local/www/joomla_trunk/libraries/joomla/html/parameter/element/list.php on line 22
To create a configuration file:
Copy the file tests/unit/config.php-dist to the name tests/unit/config.php.
Edit the config.php file to match your test configuration. The configuration must point to a valid Joomla! 1.6 database. You can use any installed Joomla! 1.6 database.
The values you oneed to edit are as follows:
$user: database user id (for example, 'root' or whatever login needed to connect to the db)
$password: database user password
$db: database name
$tmp_path: <Joomla! root>/tmp
$log_path: <Joomla! root>/log
You can use your configuration.php file as a guide for these entries, if needed.
When you run unit tests that use the database, the database will become unusable for a normal Joomla! instance. There are two ways to fix this.
Re-install Joomla! after the unit tests have run. This recreates the database and fixes any problems.
Use a different database for the unit tests. For example, you can run the installation of Joomla! and give it a different database name and use this database name in the tests/unit/config.php file.

Developing Joomla! extensions

File structures

Not in any way mandatory but this reflect my current developer environment based on eclipse PDT: I will deliver this package name PhpStart as download


                 \xampp\htdocs\DEV                <--- My developer Joomla! instance, where Maven auto deploy all my extensions projects
                               \TEST              <---  instance where I deploy extensions manually
                               \PROD              <--- another Joomla! instance, similar to my production
                               \com_myguestbook   <--- one component project that is versionned in SVN and contains only my extensions code
                               \mod_related_thumb_items  <--- one module PHP and Maven for PHP project
                               \plg_related_thumb_items    <--- one content plugin  PHP and maven for PHP project
                 \mysql                                    <--- the database comming with XAMPP
                 \maven                                    <--- external Maven runtime
                 \maven\repository                         <--- Maven local repository
                 \maven\settings.xml                       <--- Global Maven user settings


Ready to use package

I will provide a ready to use package based on Eclipse PDT soon. Unzip and Run!

Preparing your environment

Not needed if you use the ready to use download package "PhpStart"

Apache Maven



is required, so install a java runtime environment (JRE) or java development toolkit (JDK)


You'll have to use this settings.xml, copy it to your home directory. Your Maven home is depending on your operating system

  • Windows XP: c:\document and settings\{yourlogin}\.m2\settings.xml
  • Windows Vista: c:\users\{yourlogin}\.m2\settings.xml
  • Windows 7: c:\users\{yourlogin}\.m2\settings.xml
  • /home/{yourlogin}/.m2/settings.xml
Example of settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns=""
					<snapshots />
					<snapshots />

Environment variables

These variables can be found in pom.xml

  • They act as default values (in a bloc <properties></properties>)
  • Can be ovveridden in pom hierarchy
  • Can be provided to the Java Virtual Machine as -D variables

-D variables

Possible values Example Description
Defined in pom.xml
dev dev You need to tell Apache Maven that you you are currently developing. By defining the value "dev" This will at any resources changes copy the content of your project to a running Joomla! instance.
valid OS path c:\xampp\htdocs\joomla15 The full path to a running Joomla! instance
system.phpDocFilePath /usr/bin/phpdoc
path to executable phpDocumentor, sometimes referred to as phpdoc or phpdocu, is the current standard auto-documentation tool for the php language. Similar to Javadoc, and written in php, phpDocumentor can be used from the command line or a web interface to create professional documentation from php source code. phpDocumentor has support for linking between documentation, incorporating user level documents like tutorials and creation of highlighted source code with cross referencing to php general documentation.
system.doxyGenExe /usr/bin/doxygen path to executable Source code documentation generator tool

Doxygen is a documentation system for C++, C, Java, Objective-C, Python, IDL (Corba and Microsoft flavors), Fortran, VHDL, PHP, C#

It can help you in three ways:

It can generate an on-line documentation browser (in HTML) and/or an off-line reference manual (in ) from a set of documented source files. There is also support for generating output in RTF (MS-Word), PostScript, hyperlinked PDF, compressed HTML, and Unix man pages. The documentation is extracted directly from the sources, which makes it much easier to keep the documentation consistent with the source code.
You can configure doxygen to extract the code structure from undocumented source files. This is very useful to quickly find your way in large source distributions. You can also visualize the relations between the various elements by means of include dependency graphs, inheritance diagrams, and collaboration diagrams, which are all generated automatically.
You can even `abuse' doxygen for creating normal documentation (as I did for this manual).
Doxygen is developed under Linux and Mac OS X, but is set-up to be highly portable. As a result, it runs on most other Unix flavors as well. Furthermore, executables for Windows are available.
see more


Integrated development environment (IDE)

Netbeans IDE

maven is supported right out of the box, you have nothing to do

Eclipse IDE

Install Eclipse PDT

Install M2Eclipse

Create a new project, add the php nature (right click - configure - add PHP nature)

IntelliJ IDE


Notepad, Ultraedit, Jedit, command line

Download Maven - Download the latest version of Maven, unpack anywhere on filesystem

Set the following environment variables

SET MAVEN_HOME={DirectoryWhereYouUnpackMaven}\bin

PHP runtime

I recommend you to use XAMPP,

Joomla! runtime

When you develop Joomla extensions, you need a runtime, aka a working Joomla! instance where

  • You can test and debug your new code.
  • Apache Maven will be able to deploy your code when you make any changes to PHP or resources.

For this reason, you'll have to install a standard Joomla!, preferably the latest version or the targeted release you would like to develop against.

Download Joomla! from and unpack it anywhere on file system.

Note the path where you did install Joomla!, you will have to tell Apache Maven where this directory is (same value as JPATH_SITE), NOTE that if you use XAMPP, you must unpack it into /xampp/htdocs/xxxx/

Developing a Joomla! extension

5 minutes to be ready

Copy settings.xml under c:\users\{yourlogin}\.m2\settings.xml or /home/{youruser}/.m2/settings.xml

Download an unpack MAVEN in c:\

Add MAVEN_HOME in your environment variable or open a command prompt: type set MAVEN_HOME=c:\maven\bin

Create a new directory for your extension {extension.path}, depending on the extension type (plugin, component, module), create a new file pom.xml with this content:

you need to have at least these folder, create them if they do not exist {extension.path}/src/main/php, {extension.path}/src/test/php, {extension.path}/src/main/assembly

go to that directory and type mvn package, code is compile, testcases are run and your empty component is ready to use in /target

Real case with an  PHP IDE

Fire your favorite IDE and create a standard PHP project.

I provide up to date example of pom.xml for your different kind of project in subversion at

  • com_example
  • mod_example
  • plg_example

Can be checkout or viewed at

I recommend you to use them as templates, make a checkout, disconnect them from SVN, and rename to your liking

Joomla! Plugins (or rename plg_example )

At the Root of this project, create a new file named pom.xml with that content

You must change

  • <groupId>com.waltercedric</groupId>      this is just something that is used to categorize your extension
  • <artifactId>related_thumb_items</artifactId>  this is the name of your extension, as default also the name of your deliverable artifacts

Plugin Type

use one of the following values for <plugins-type>content</plugins-type>

  • authentication
  • content
  • editors
  • editors-xtd
  • search
  • system
  • user
  • xmlrpc


Use SNAPSHOT if you are developing in Trunk (SVN) or Head (CVS). Ex:


<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi=""





Joomla! Modules (or rename mod_example)

At the Root of this project, create a new file named pom.xml with that content

  • Put your module code into mod_example/src/main/php
  • Put your module PHPUnit code into mod_example/src/test/php

Joomla! Components (or example com_example)

At the Root of this project, create a new file named pom.xml with that content

  • Put your module code into com_example/src/main/php
  • Put your module PHPUnit code into com_example/src/test/php


How it is working

Maven Project structure

Here I presentYou the Maven pom hierarchy, all these pom.xml are versionned at and deployed into with their eclipse launcher.

You can browse my Maven repository online at

Thanks to this inheritance structure of pom.xml, we are able to distribute nicely functionnalities/build/reporting pluginsinto this hierarchy. All these nodes are identified by an ArtifactID, GroupID and version number: It is the combinaison of these 3 values that will allow us to build a Joomla! extension .


The correct hierarchy is not represented here if Joomla! team ever accept this solution for continuous integration of joomla! extensions, the pom named "company" should not exist and be replaced by the pom named "joomla"


Top/down artifactid


This is my pom that contains nothing related to Joomla! nor Java, ideally this pom do not exist.


This pom contains some specifics about Joomla!,

  • Maven for PHP settings for how to compile PHP code. 
  • Dependencies to PHPUnit
  • How Unit Test are run: from src/test/php
  • How SeleniumTest are run: from src/test/selenium


This contains Maven plugins:

  • How to package a Joomla! module
  • How to deploy a Joomla! module to a Joomla! runtime (see ${joomla.runtime.home})

This contains Maven plugins:

  • How to package a Joomla! plugin
  • How to deploy a Joomla! plugin to a Joomla! runtime

This contains Maven plugins:

  • How to package a Joomla! component
  • How to deploy a Joomla! comp to a Joomla! runtime

All your extensions: modules, plugins or components are child of parent pom joomla-modules, joomla-plugins or joomla-components respectively

Creation of deliverables

Users or developer in your team expect to download zip or tar.gz package ready to be installed using Joomla! administrator installer

If you move to the directory where the project pom.xml and run:

mvn package

in the directory where your pom.xml extension is:

  • Apache Maven will go through all Lifecycle build phase (compile, test being the most important), and
  • Create a ready to be installed Joomla! packages in /target/


This is achieved by using the Maven Assembly plugin in the following parent pom: joomla-modules, joomla-plugins and joomla-component

Create a homepage (site)

Maven automatize the creation of a HTML site, based on pom.xml content. It contains a lot of informations

If you move to the directory where the project pom.xml and run:

mvn site

in the directory where your pom.xml extension is:
Apache Maven will go through all Lifecycle build phase (compile, test being the most important), and
Create a ready to be installed Joomla! packages in /target/


If you dont have PHPDocumentator at the default location, you can start your maven build with



[INFO] Error during page generation, Embedded error: Error rendering Maven report: PHPDocumentor not found

You need to add the path to phpdoc to the include_path in php.ini in linux, type

which phpdoc 


add /usr/bin to php include_path

Regression testing

Unit test

  • Save your PHPUnit test into src/test/php
  • They will be run in Apache Maven phase 'test'
  • The dependency to PHPUnit is defined in parent pom: joomla

PHPUnit compiler settings

All these settings

can be put into <configuration></configuration> under <compileArgs></compileArgs> in mavenphp plugin (in joomla/pom.xml or in your own project pom.xml)

Example of a PHPUnit test



Functionnal testing with Selenium

The pom.xml of parent pom joomla contains a plugin that will start automatically selenium in phase 'integration-test'

Your integration test have to be saved in src/test/selenium



Automatic site and documentation generation

Documentation generation

This is done in maven phase 'site', you can create a fully default Maven site looking like this one by just either

  • Opening a command line and running mvn site in your extension project
  • Having a continuous build in your continuous build server that exexute the goals site

This will create an index.html in /target/site/index.htm.


  • You can add an unlimited number of plugin in the pom.xml in section <reporting></reporting> to add new interesting figures.
  • In super pom (or parent pom), there is a default configuration that use phpdoc and doxygen
  • While the site is a good start, there is some additional efforts to develop you own skin (css) to make it look good

Maven generates a standard documentation. Through the doxygen goal, an api doc is created. The doxygen configuration file must be located under src/site/doxygen/doxygen.conf.

Maven site lifecycle

pre-site executes processes needed prior to the actual project site generation
site generates the project's site documentation
post-site executes processes needed to finalize the site generation, and to prepare for site deployment
site-deploy deploys the generated site documentation to the specified web server

Site deployment

If you run your maven phase till site-deploy, you'll have to tell in pom.xml of your project where and how, this is really simple:

Using SCP


Using File copy


see also

See it running live

You can see the build


And its browsable result at