The Build Process Using CruiseControl.Net & NAnt

Published: 15 April, 2009Posted in: Programming, WorkTags: ,

CruiseControl.Net

What Is CruiseControl.Net?

  • CruiseControl.Net is a free framework for continuous build integration.
  • It runs as a Windows service on our build server, watching for changes to our source control system. When it detects a change (ie a developer commits some code) it triggers an action.
  • It also comprised of a Web Dashboard, so that build reports (ie the outcome of an action) can be viewed in a browser.
  • It’s configured using XML.
  • CruiseControl.Net lives on our “Kermit” build server.

Configuring CruiseControl.Net

  • As mentioned, CruiseControl.Net is configured using XML.
  • At current a single xml file for all build configs, although it is possible to separate them.
  • The config is separated into projects. A “project” defines a build process, such as, compile the source code, or deploy some source code, or run integration tests.
<project name="trunk build" queue="trunk" queuePriority="2">
<webURL>http://kermit/server/local/project/trunk+build/ViewLatestBuildReport.aspx</webURL>
<workingDirectory>e:\build\huddle\working\trunk</workingDirectory>
<artifactDirectory>e:\build\huddle\artifact\trunk</artifactDirectory>

<triggers>
<intervalTrigger seconds="60" />
</triggers>

<labeller type="iterationlabeller">
<prefix>2</prefix>
<duration>2</duration>
<releaseStartDate>2009/2/17</releaseStartDate>
<separator>.</separator>
</labeller>

<sourcecontrol type="filtered">
<sourceControlProvider type="svn">
<trunkUrl>http://sourcecontrol</trunkUrl>
<username>build</username>
<password>build</password>
</sourceControlProvider>
<inclusionFilters>
<pathFilter>
<pattern>/trunk/src/myhuddle/**/*.*</pattern>
</pathFilter>
<pathFilter>
<pattern>/trunk/src/shared/**/*.*</pattern>
</pathFilter>
<pathFilter>
<pattern>/trunk/src/services/**/*.*</pattern>
</pathFilter>
<pathFilter>
<pattern>/trunk/build/**/*.build</pattern>
</pathFilter>
<pathFilter>
<pattern>/trunk/lib/**/*.dll</pattern>
</pathFilter>
<pathFilter>
<pattern>/trunk/tools/**/*.*</pattern>
</pathFilter>
</inclusionFilters>
</sourcecontrol>

<tasks>
<nant>
<executable>tools\nant\bin\nant.exe</executable>
<nologo>true</nologo>
<buildFile>"build\nant\variables.build"</buildFile>
<buildArgs>-D:root.directory=\\192.168.1.2\e$ -D:environment=dev</buildArgs>
<logger>NAnt.Core.XmlLogger</logger>
<targetList>
<target>compile</target>
</targetList>
<buildTimeoutSeconds>600</buildTimeoutSeconds>
</nant>
</tasks>

<publishers>
<merge>
<files>
<file>e:\build\huddle\artifact\trunk\reports\nunit\*.xml</file>
</files>
</merge>
<xmllogger logDir="e:\build\huddle\artifact\trunk\buildlogs\build\" />
<modificationHistory onlyLogWhenChangesFound="true" />
<artifactcleanup cleanUpMethod="KeepLastXBuilds" cleanUpValue="100" />
</publishers>
</project>
  • workingDirectory – where the project works from and checks out files to. For example, when building the trunk, CruiseControl.Net checks out the trunk to the working directory, and uses those files to compile it.
  • artifactDirectory – where the projects output is placed. For example, the compiled output of the trunk, or any reports/logs that are generated.
  • triggers – this defines how often CruiseControl.Net triggers this project. You can set interval triggers (ie every ‘x’ minutes, or specific triggers for nightly builds etc…).
  • sourcecontrol – this defines the source control system in use, and the location of it. We have filters setup as we don’t want the build to trigger when certain parts of source control are commited to.
  • tasks – these are the tasks that are run as part of the project. We typically only use the NAnt task so that responsibility is shifted to NAnt.
  • publishers – these are used to merge all the reports/logs together. For example when building the trunk, we merge the NAnt report and Unit Tests report.
  • To find out about more of the config elements, it’s best to checkout the CruiseControl.Net Documentation.

CCTray

  • CCTray is a client for CruiseControl.Net.
  • It connects to a specified build server, and shows the status of all the build projects (ie green, red, amber).
  • You can use CCTray to trigger a build manually.
  • It’s possible to set build sounds when a project passes or fails. We do this by hooking up a speaker to Ed’s computer, and have “woohoo” and “d’oh” homer simpson sounds for pass and fail.

Web Dashboard

  • I’ve theme’d our dashboard to look like Huddle.
  • There are HTML templates and CSS that can be rewritten.
  • For reports, there are XSLT’s. For example the “Build Report”, tests passed, errors etc… are all generated using XSLT.
  • NAnt XML output (for unit tests, nCover etc…) are read on-the-fly, by the web dashboard, and converted to HTML using the above XSLT’s.
  • You can write your own XSLT for any XML that is output via NAnt.

Our Build Projects

trunk build

  • Triggers every 60 seconds
  • Creates a deployable version of the code.
  • Compiles the code, runs the unit tests, outputs the code to a “ready” directory.

trunk deploy

  • Triggers on a nightly schedule. For deployments during the day it a “Force Build” needs to be done manually via CCTray.
  • Takes the code in the “ready” directory and deploys it to the .dev server

release build, release deploy

  • These are both the same as “trunk build” and “trunk deploy” except that the code gets deployed to .test instead of .dev.

# standards

  • Triggered by “trunk build” on success.
  • Runs tasks relating to coding standards such as StyleCop and NDepend etc…
  • Was created to make “trunk build” lighter and faster.

# reports

  • Triggers on a nightly schedule.
  • Runs various metrics against the code such as NCover, Simian, FxCop, and NDepend.
  • Uses the last successful build of the trunk to run reports against

# externals

  • Triggers every 60 seconds.
  • Used purely to keep the externals directory on the build server up to date. For example, if we add a new app to externals, this will trigger and check it out onto the build server.

# cruise control

  • Triggers every 4 minutes.
  • Used to update CruiseControl.Net
  • We keep many of the CruiseControl.Net configs in source control. Upon one being changed, and commited to source control, this project triggers, and CruiseControl.Net deploys the new code to itself. This makes changing the configs easy (rather than having to log onto the server).

CruiseControl.Net Guidelines

Here are some guidelines I’ve learnt along the way.

  • Build Responsibility – I prefer to move responsibility of the build to NAnt. Developers can’t run CruiseControl.Net on their machines as part of the build, but with NAnt they can. Using NAnt a developer should be able to mimic the build and be virtually certain that if the code passes on their machine, it will pass on the build server. My Go.bat is Your Friend article gives more detail.
  • Deploy CruiseControl.Net to Itself - as mentioned, put all your CruiseControl.Net configs into source control, and setup a build project so that it deploys to itself. This means making changes is much easier. A word of caution though, if you invalidate the ccnet.config, check the Event Viewer on the build server and it’ll tell you what’s wrong.
  • Merging Logs – Cruise Control merges all the logs you tell it into one XML file. So be careful:
    • Firstly on the verbosity and size of your logs. Obviously you want your logs to be readable and make sense but cutting out any noise keeps the size of them down.
    • Don’t merge logs that don’t relate to a build project. For example merging StyleCop logs as part of the Trunk Deploy.
    • Keeping your logs leaner, will make the Web Dashboard load faster. There’s nothing more annoying that having to wait ages for the dashboard to load.

NAnt

What Is NAnt?

  • NAnt is a tool for performing tasks, commonly used as part of a build process.
  • It is configured by XML (stored as .build files)
  • It has a number of inbuilt tasks (and contributed tasks), but can typically run anything that can run via command line. You can also write your own Custom Tasks.

NAnt Tasks

  • There are dozens of inbuilt NAnt Tasks.
  • For using them it’s best to use the documentation (which is very good)
  • Here’s an example:
    • <copy file="${source.directory}\fileA" tofile="${source.directory}\fileB"; overwrite="true" />
  • Simple and easy to do. The only downside is that it’s not strongly typed (hmmm Fluent NAnt…)

Custom NAnt Tasks

  • Custom tasks are very easy to write for NAnt. It’s as simple as a) Create A New Project, b) Import the NAnt dll’s, c) Write Your Code, d) Compile it, e) Grab the dll that is created and use it as part of your scripts.
  • Command Pattern – NAnt uses the command pattern for writing tasks. It’s simple and easy to follow.
  • Here’s an example, in the form of our Robocopy custom task.
  • There are plenty more tasks we could write, these include:
    • TinyGet – for build integration
    • CSS – merging & compressing
    • Regex – checking a file for a string
    • JavaScript / CSS Validation

Our NAnt Processes

  • We have a separate file for each part of the build.
  • For more complex parts of the build, we have a sub folder for sub tasks. For example “compilation.build” has a folder “compilation” with other tasks such as “unit-tests.build”.
  • variables.build – maybe not the best name (start.build?). This is the entry point for all builds (ie build, deploy, standards etc…). Variables relating to the type of build are passed to this script:
    • Target – ie, trunk build or trunk deploy etc…
    • Environment – local, dev, test
    • Build Type – release or debug
    • Compile Type – publish or compile
  • compilation.build
    • Pre compile tasks
    • MSBuild compile of the code
    • Post compile tasks
    • aspnet_compiler of the code
    • Unit tests
    • Publishes the code (if necessary) to the “ready” directory
  • package.build – takes all the compiled apps, services, configs, scripts etc… and packages it into a nice set of folders (or zip) ready for deployment.
  • deployment.build – deploys the packaged code to the specified server using the Robocopy custom task.
  • metrics.build – runs any code metrics against the last successfully built version of the trunk. Runs NCover, Simian, FxCop, ToDo’s etc…
  • standards.build – runs StyleCop against the trunk, and fails the build if more styling violations have been added.
  • live.build - a script for converting the last successfully packaged build, into a package that is ready for live. It basically goes through the configs and changes them so that they are configured for live.

NAnt Guidelines

  • Variable Naming – there’s no official standard, but I like to lowercase all elements (if writing a custom task) and variables. Dots or dashes can be used for variable names (ie, tools-directory, or tools.directory)
  • Dependencies – try to reduce dependencies between scripts. For example, when writing “compilation.build” you should only reference tasks and variables etc… related to compilation.
  • Required Dependencies – just to make things clearer, I like to list the required variables at the top of the script. Ideally if statements are best used, to check the required variables exist, and if not the fail the build.

Putting It All Together

Tips & Tricks

  • Visual Studio Integration – grab the NAnt XSD (it’s included in the download) and put it in your “C:\Program Files\Microsoft Visual Studio 9.0\Xml\Schemas” folder. If you’re worried about breaking NAnt, you can always validate your build script against the XSD.
  • Build Sound – ensure you have a build sound set. You’ll be surprised how helpful it is. As mentioned in the CCTray part of this article, you can use CCTray and a speaker to do this. It’s far easier for developers to hear and react to a sound, than constantly having to check CCTray.
  • Colour NAnt – the output of NAnt is typically just grey font on a white background. There is a way of making the output color coded (ie red = fail, yellow = warn etc…). This is a lot more beneficial as it’s more visual and easier to see errors and warnings amongst the noise. Eric Lui has an article to get started on making NAnt Color. Or if you’re lazy, you can download my pre-done modified NAnt.Core.dll.
  • Verbosity – NAnt has a few bugs regarding output, and even if you specify otherwise, it still outputs a lot of information. This makes it harder to found out what is wrong as you have to look through all the noise. There is a solution though. Rory Primrose has a good solution using a custom task.

The Future

  • Recovery Scenarios & Rollbacks – should a build make it past compilation, unit tests and various other checks, once deployed you may still get a runtime error. If QA is being blocked it can be handy to rollback to the previous release. This can be done using NAnt and CruiseControl.Net.
  • Fluent NAnt – is it worth it is the main question. I’ve looked at trying to create a wrapper around NAnt, but the way it’s written and structured makes it very difficult to do. After spending some time doing so I questioned if fluent loading was worth it.
    • For things like IoC fluent configuration is great as everything is strongly typed. However with NAnt you generally reference paths and variables, something you can’t strongly type.
    • Some may argue it stops grammatical errors, but this can be corrected by using the XSD’s that come with NAnt and checking the XML is valid.
    • It will typically be more to code fluently, especially if you nest a lot of targets and tasks.