Building Complex Multi-Platform Projects with Hudson
When building multi-platform projects where C++ is involved, developer teams are confronted with the task to compile and run the project on different platforms with different compilers. Besides, maybe just a part of the project is written in C++, other parts are written in Java and an automated build process has to be established including both languages. In our project we have found a solution based on Hudson.
The project we are working on is a code generator for a rule defining language: rules are written in a domain specific language (DSL) and our code generator creates source code in Java and C++ from it. The idea is that these generated sources are used as libraries in other projects (some of them written in C++, some of them in Java). Since the C++-based software, which our generated code is a part of, should be running on different platforms, we have to guarantee that our code is platform-independent, too.
Integrating CMake using CppTask
We use CMake to use central, compiler-independent configuration files for the C++ build process on the different platforms. So, before the actual C++ compile starts, it is necessary to run CMake for the designated compiler to get the correct configuration files. An alternative for defining compiler-independent configuration files would be to use the Cpptask Add-on for Ant. But since our C++ developers are more experienced with CMake, we preferred this tool.
To ensure that our generated code runs correctly, we have established an automatic build which performs tests on several platforms and for both languages.
Anatomy of our Build process
The whole build is divided into several Hudson jobs, where some jobs are dependent on others but can be performed in parallel. The following diagram gives an overview:
The first job CodeGeneration scans every two hours if something new was checked in into our Subversion source control system.
If this is the case, a new build is triggered. All our sub-projects are checked out into the Hudson workspace and an Ant target is called which performs the following tasks:
- It builds all the basic projects necessary for the code generation. These projects are all written in Java.
- It runs a Java application which generates the C++ and Java source code from a central database where the rules are defined in a DSL
If this job fails, emails are sent to the build manager and to the people who have checked in into subversion in this period of time and so may have broken the build:
If the job succeeds the following three jobs run parallel:
The job Test_Java performs an Ant-Task which compiles the generated Java code and runs a set of defined JUnit test cases. The results of the tests are then published in Hudson. If the build fails, E-Mails are sent to the members of the team which are responsible for the code generation in Java (and the build manager).
The job Test_Linux_Cplus has the following build steps: Via an Ant-Task the cplus subproject and the test cases are zipped and sent to the Linux virtual machine (VM) via secure copy. Via the Ant-Task sshexec a shell script is started on this VM which performs the following tasks
- It unzips the projects on the VM
- It runs CMake to generate the configuration files for the gcc compiler
- It runs make to compile the generated sources and the test program
- It performs the CPPUnit tests
The test results are sent back to Hudson via secure copy. To publish CppUnit test results, it is necessary to install the CppUnit Plugin. When done, the reports can be published by the following entry:
If the job fails, emails are sent to the members of the C++ code generation team.
The job Test_Visual_Cplus runs an Ant target which is analogous to the Test_Linux_Cplus job except that it performs the tasks for the Visual C++ compiler on a Windows XP VM. Instead of a Linux based shell script, a Windows based batch file is executed.
Another option to perform the tests on the different platforms is for each platform to completely check-out the project from the version control and start the whole build process on each machine independently.
We did not follow this approach because of the following drawback for our project: Only a small part of the project is written in C++. Before compiling the C++ sources, all the basic Java packages for the code-generation have to be compiled. Therefore, we would perform a lot of overhead (the complete Subversion checkout alone takes quite a long time) where no new insight into the code quality is gained. In other projects, where the C++ part is larger, this approach may also be a good strategy.
With the described process we are able to test our C++ code on different platforms. If a build fails, only the relevant developers are notified. If we want to perform the tests for C++ on a new platform (e.g. we have planned to run them on Windows 7, 64 bit), we just have to write a new Ant task similar to the already existing ones. When compared to the great effort to set up a new virtual machine (installing the necessary programs, etc.) itself, the effort for setting up the new Hudson job is really negligible.