For the last 5 years, I have tried several Continuous Integration servers: Team City, Jenkins, AnthillPro, and ThoughtWorks Go. I have to admit, even if it may sound bias, that I really like the latter. There are several reasons why I like it: its CCTray XML feed, its UI and model designed around pipelines (i.e. builds) and dependencies, and its security model. But above all, the entire Go configuration can easily be managed as source code: version controlled, built, tested, and automatically deployed. In this post, I will explain what I mean by that.
Go Config Overview
Go is entirely configured using a single XML file, cruise-config.xml (Cruise Control is the old name for Go, prior to version 2). This file is located in /etc/go on Linux and by default in C:\Program Files\Go Server\config on Windows. It is composed of 5 main sections:
<server>Global server settings such as license and users<pipelines>Your build pipelines<templates>Your build pipeline templates (to create re-usable pipeline blueprints)<environments>The definition of the hardware/software resources the pipelines will use and act upon<agents>The build agents
Although Go, like the others CI tools, has an Admin UI to configure the general features (e.g. LDAP integration) or pipelines, I much prefer to manipulate the cruise-config.xml file directly. Indeed, if you change it, then the changes are automatically applied to your Go server (if they pass Go validation). So adding a new pipeline is a simple as adding a few lines of XML!
First Pipeline
Let’s for instance write a dumb pipeline that will just output “Hello World” to the console. To do so, simply add the following lines of XML to your Go server cruise-config.xml file (note that I use a personal Github repository as <material>, and you will gave to change it to something else):
<cruise>
<server>
[...]
</server>
<pipelines group="Demo">
<pipeline name="HelloWorld">
<materials>
<git url="http://github.com/jdamore/cruise-config" dest="cruise-config" />
<materials>
<stage name="HelloWorld">
<jobs>
<job name="SayHelloWorld">
<tasks>
<exec command="echo">
<arg>Hello World</arg>
</exec>
</tasks>
</job>
</jobs>
</stage>
</pipeline>
</pipelines>
</cruise>
For me being able to edit cruise-config.xml, add a few lines of code, save the changes, and see the UI updated with my new pipeline, stage or job is really really cool. But of course why stop there?
Cruise Config Pipeline
Modifying cruise-config.xml in an uncontrolled manner is dangerous. True, Go will backup any version that is rejected because syntactically or logically incorrect, so that you do not loose your latest changes, but what if you have to come back to the version before last? What if you want to revert the last two changes? Of course what I am getting at is: cruise-config.xml must be version controlled. So first thing you must do is stick it in the VCS repo of your choice. At least you would be able to push new changes or rollback previous changes if you wanted to.
But what if…. what if anytime you commit a change to the cruise-config.xml in your source control repository, it gets automatically applied to your Go Server, instead of having to manually copy the latest file to Go? Is it not what Go is good at, automatic deployment and all that? Sure. So let’s create a CruiseConfig pipeline that will take the cruise-config.xml file from your repo and copy it into the Go Server configuration directory. In the example below, I use a GitHub repo to control my cruise-config.xml. The pipeline below will download the content of the repo and execute a cp command (my Go Server is on Linux) to copy the cruise-config.xml file to /etc/go. Of course, in my case, the Go Agent running this job will have to be a local agent installed on the Go Server machine. If you want to use a remote agent, then you could copy over SSH (scp command) on Unix/Linux or over a shared drive on Windows.
<pipelines group="Config">
<pipeline name="CruiseConfig">
<materials>
<git url="http://github.com/jdamore/cruise-config"/>
</materials>
<stage name="Deployment">
<jobs>
<job name="DeployConfig">
<tasks>
<exec command="cp">
<arg>cruise-config.xml</arg>
<arg>/etc/go/.</arg>>
</exec>
</tasks>
</job>
</jobs>
</stage>
</pipeline>
</pipelines>
So now I have two pipelines: CruiseConfig and HelloWorld.
It means that if I want to change my HelloWorld pipeline I can simply edit cruise-config.xml an check it in. As a test, I will add another stage HelloWorldAgain as follows:
<pipeline name="HelloWorld">
<materials>
<git url="http://github.com/jdamore/cruise-config" dest="cruise-config" />
</materials>
<stage name="HelloWorld">
<jobs>
<job name="SayHelloWorld">
<tasks>
<exec command="echo">
<arg>Hello World</arg>
</exec>
</tasks>
</job>
</jobs>
</stage>
<stage name="HelloWorldAgain">
<jobs>
<job name="SayHelloWorldAgain">
<tasks>
<exec command="echo">
<arg>Hello World Again</arg>
</exec>
</tasks>
</job>
</jobs>
</stage>
</pipeline>
Then I commit the changes.
Then the CruiseConfig pipeline will automatically be triggered and deploy the changes to Go.
The CruiseConfig pipeline runs
The HelloWorld pipeline now has a second stage
Et voila! The HelloWorld pipeline changed instantly and automatically, with the addition of the new stage.
Cruise Config split
I just want to finish this post with what I strongly recommend people do in order to make the cruise-config.xml more manageable. Because the number of pipelines and pipeline groups will grow, particularly if more than one team uses Go, you need to be able to split and isolate the various pipelines and composing elements of the configuration file. I use a Ruby template (erb) to do so as follows:
<?xml version="1.0" encoding="utf-8"?>
<cruise xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="cruise-config.xsd" schemaVersion="62">
<server ...>
<license ...>
[...]
</license>
<security>
[...]
</security>
</server>
<%= pipeline_xml 'oloservices' %>
<%= pipeline_xml 'olohtml5working' %>
<%= pipeline_xml 'olohtml5release' %>
<%= pipeline_xml 'spdev' %>
<%= pipeline_xml 'goenvironment' %>
<%= section_xml 'templates' %>
<%= section_xml 'environments' %>
<agents>
[...]
</agents>
</cruise>
I just have one line for various sections, the first 5 being <pipelines>, and the last two being the <templates> and <environments> sections. I have implemented two functions, pipeline_xml and section_xml, whose job is to output the XML for the specified pipeline group or section. For instance, pipeline_xml 'spdev' will output the content of the file called pipelines.xml that is located in a directory called spdev. The idea here is to reconstruct the full cruise-config.xml before deploying it to Go. Besides, by doing so, one can also execute acceptance tests before deploying the config changes to Go, to make sure they are harmless. I then ended-up with a GoConfig pipeline that has 3 Stages:
- Build – Reconstruct the file
- Test – Validate the config
- Deploy – Copy the file to the Go config directory
Corsamore 