Alt.Net, Openspaces, whatever its coming...

Google groups and Twitter has been busy today after I seemed to have started some hype about an Alt.Net styled openspaces event (#ozopenspace)

PLEASE spread the word. We need ideas when people can come and where they would go to (Australia is a big place, we are all very spread out)

The wiki looks like its going to be based here: http://ozopenspace.pbworks.com

Register you interest here. Seriously please do this otherwise we can not estimate or cater for your needs/wants etc. This will also show sponsors if we are serious or not and may help convince overseas guest to attend too, of which there is already interest :)


The discussion is on the oz alt.net google groups here

I cant wait!

Powershell and the web

Lately I have been playing with PowerShell to see if it can help with some uptime/availability checks for our web sites and services at work; Yes i know what you are thinking but I'm not a sys-admin and I need to know if my stuff is working with out having access to the server... long story I'm sure you will appreciate.

In my reading I have seen lots of flaky posts about PowerShell and web services. I think people need to remember that web services still conform to the http protocol, there is no magic, you do not need to access any visual studio dll or do any weird crap like that, you just need to hit the url and receive a payload.

    $wc = new-object system.net.WebClient
    $webpage = $wc.DownloadData($url)     

That's it. No weird proxies no importing custom dll's.

Here's an example calling the Google Maps API (which is a nice API BTW) to get a latitude and longitude of an address, specifically Perth's craziest bar Devilles Pad (think Satan living in a trailer park that hangs out at a Go-Go club that used to be a James Bond-villians liar, yeah its that cool:

    $url = "http://maps.google.com/maps/api/geocode/xml?address=3+Aberdeen+St,+Perth,+WA+6000,+Australia&sensor=false"
    $wc = new-object system.net.WebClient
    $webpage = $wc.DownloadString($url)#Edit -Thanks Ken
    $xmldata = [xml]$webpage
    $lat = $xmldata.GeocodeResponse.result.geometry.location.lat
    $lng = $xmldata.GeocodeResponse.result.geometry.location.lng
    write-host "Latitude = $lat - Longitude = $lng" 

Like that? Go play: Google Maps API Doco

MSBuild 101

Lately for some reason I have seen a fair bit of tension around MSBuild. I guess this is good thing, more people are using build tools like PSake, Rake etc and most of the time if you are building .Net stuff sooner or later you will have to call into MSBuild, unless your feel bold enough to uses csc... um... not for me.

MSBuild is, to be honest, very basic. 

There are typical 2 things that I use MSBuild for

-Defining a build script

-Running a build

The term MSBuild is a little confusing to some as its covers a few things: a file type and an executable.

Firstly we need to be aware that all .Net solution files (*.sln) and project files (*.*proj) files in .Net are MSBuild files; that should hopefully ease some tension as it is now pretty obvious that you are already using MSBuild! The MSBuild files are XML and therefore human readable and modifiable. The files typically define properties and tasks. The properties can reference other properties and they can be collections too. The tasks can have dependencies on other task (i.e. Test requires Build which requires Clean) and the task can make use of our properties we have defined in our script as well as define items that are basically task level variables. This is starting to sound just like procedural programming, which I hope we are all au fait with. It is probably also a good time to note that pretty much all build tools use the notion of dependencies. This basically means if you call a task and it has a dependant task then that task will be called first. You can have chained dependencies like the example we mentioned before (i.e. Test requires Build which requires Clean) and you can also have multiple dependencies for one given task (Deploy Requires Package, Document, Test).

After reading that it is quite clear that MSBuild is actually very simple. It is the syntax and general XML noise that scares most people, including myself.

Running MSBuild

To achieve the most trivial usage out of MSBuild we need to know how the executable works so we can actually call MSBuild to do something. MSBuild comes with Visual studio, I don’t think it actually comes with the .Net framework (someone can correct me here) and it is also bound to a framework version.

To start with I will show the most trivial example of how to use MSBuild and that is to just build a solution.

@C:\Windows\Microsoft.NET\Framework\v3.5\MSbuild.exe MySolution.sln

Assuming that this command is run in the directory that MySolution.sln resides this will build that solution. It can’t get much easier than that. Note this will not do anything clever, it will just build to your default location ie bin/debug for each project in the solution. Personally for me this offers little value other than it is a bit faster than building in visual studio.

Typically if I am building using the command line it is because I am building from a build tool. Build tools like PSake allow me a lot more flexibility as they are not constrained by the bounds of XML and have powerful functions I can use that may be associated to builds and deployments that might not otherwise exist in MSBuild. If you are using a tool like PSake or Rake as your build script then it is more likely that you will use the following syntax:

&$msbuild "$solution_file" /verbosity:minimal /p:Configuration="Release" /p:Platform="Any CPU" /p:OutDir="$build_directory"\\  /logger:"FileLogger,Microsoft.Build.Engine;logfile=Compile.log"

This is from a PSake script so anything with a $ in front of it is a PowerShell variable (i.e. not MSBuild syntax). Walking through this I have defined the location of the MSBuild exe ($msbuild) and the sln file. Note that the sln file is note associated to a switch. It is the first argument and this indicates that this is the file we are building. Other arguments are prefixed with a switch. These switches begin with a forward slash and end in a colon and contain either full descriptive words (e.g. /verbosity:minimal) or short hand syntax for a word (e.g. /p:Platform="Any CPU" which is short for property and in this example defines the platform property)

FYI : The & at the start of the line is a PowerShell construct to say "run this command, don’t just print it to the screen"

Defining Build Scripts with MSBuild

First up we will walk through a very basic build script that justs cleans and builds a solutions. It will be pretty obvious that this scripts really offers little value but we will get to some of the nitty gritty stuff soon.

<Project DefaultTargets="Clean" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">

  <Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>

 

  <PropertyGroup>

    <ApplicationName>MySolutionApplicationName>

    <SolutionFile>..\$(ApplicationName).slnSolutionFile>

  PropertyGroup>

  <Target Name="Clean">

    <MSBuild Targets="Clean" Projects="$(SolutionFile)"/>

  Target>

  <Target Name="Build" DependsOnTargets="Clean;" >

    <MSBuild Targets="Rebuild" Projects="$(SolutionFile)" Properties="Configuration=Release;" />

  Target>

Project>

Let’s go over this line by significant line:

The xml documents root is the project where we define the xml namespace, the MSBuild Tools version and the default target. Make sure you tools version work with your .Net framework version. This first line is pretty standard for me. I like having a default target and im typically working with .net 3.5 at the moment so this is all good.

Next up we are importing an external extension. This is an example of importing the very useful MSBuild Community Extension toolset. If you are making use of MSBuild a lot then you will want this toolset. Another Import I often make is for Gallio a very handy test runner. Please note that this import demonstrated is not being used in the script so could safely be removed, it is just here to show the syntax

Next we define our property group section. The nodes in here are user defined. MSBuild does not have “ApplicationName in its schema, i made that name up. These properties are in effect you global variables. You will also note that you can reference other Properties from with properties as shown in the solution file property. This is often very handy when dealing with file paths. Then Syntax for referencing a single properties is $(MyProperty)

Next up we define our first target. Targets are just like methods in that they define functionality. The Clean target just calls an MSBuild task that cleans the solution. Not very impressive, I know. The next target shows how we can specify dependencies. This means any time we call the Build target the Clean target must run first.

Having a build file is all very nice but we need to be able to run it. As it is not a solution of project file I tend to treat them a little different to how I would call those files directly. I typically prefer to be explicit in calling target, even though it is a DRY violation.

I have an example extracted from a bat file that calls into an MSBuild script below to show you the syntax (this is the same syntax as if you were to run from the cmd line):

@C:\Windows\Microsoft.NET\Framework\v3.5\MSbuild.exe MyMSBuildFile.build /t:UnitTests /l:FileLogger,Microsoft.Build.Engine;logfile="UnitTests.log"

From this you can see:

-I am calling MSBuild 3.5; MSBuild is slight different for each .Net version, be sure you are not calling the .Net 2 version for your new shiny .Net 4 solutions!

-MyMSbuildFile.build is in the location I am running this command. If it was not I would need a relative or full path.

-The extension of an MSBuild script is not important. I personally prefer to call my MSBuild files *.build to make it clear that they are in fact build scripts

-The "/t:" switch defines the target I am calling: UnitTest. You do not need to specify this if you have defined the Initial Target in the project tag in you build file. I recommend defining a default target and make sure it is a safe one... you don’t want to accidently deploy something to production "by default"

-By using the "/l:" switch I can define a logger so I don’t lose the info once the console window disappears. I pretty much only use this syntax and change the name of the output file.

 For more help you can just call the MSBuild exe and use the /? for a detail example of what the exe can do. I recommend doing this :)

Stepping Up

So everything I have shown you so far is all well and good but it is of little value in the real world. So here is a quick fire list of problem and MSBuild solutions, a cheat sheet of sorts:

Note the command line syntax used below is from PowerShell and to get the MSBuild variable assigned I have called:

[System.Reflection.Assembly]::Load('Microsoft.Build.Utilities.v3.5, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a') | Out-Null

 $msbuild = [Microsoft.Build.Utilities.ToolLocationHelper]::GetPathToDotNetFrameworkFile("msbuild.exe", "VersionLatest")

Change a property from outside the script

You have a property, say application name defined in a script, you want to reuse the script but not have to edit the script for everything that uses it.

Solution : use the property switch

&$msbuild "$build_file" /p:ApplicationName="RhysCNewApp"

Directories with spaces are failing when passing them to MSBuild

If you are passing in paths from a build tool that requires you to use quote (e.g. the path has spaces in it) then MSBuild may throw its toys and blow up on you. To dodge this use double backslashes (\\) at the end of the argument to escape the nastiness that is MSBuild

&$msbuild "$solution_file"  /p:OutDir="c:\Rhys Hates Spaces\In His\ Directory Paths\"\\ 

 

Inside your Scripts

Display a message to the console

I want to display the paths of my Visual Studio tools folder for ‘05 and ‘08. Note VS80COMNTOOLS and VS90COMNTOOLS are environment variable, so yeah MSBuild can pick those up too J

<Target Name="VS">

  <Message Text="***********$(VS80COMNTOOLS)***********" />

  <Message Text="***********$(VS90COMNTOOLS)***********" />

Target>

I want to reference a whole Item group

Sometimes we have collections of related items and want to reference all of those items:

 <ItemGroup>

    <Items Include="Foo" />

    <Items Include="Bar" />

    <Items Include="Baz" />

  ItemGroup>

  <Target Name="Message">

    <Message Text="@(Items)" />  

  Target>

This will produce:

Foo;Bar;Baz

I want a conditional execution

I want to delete a folder, but only if it exists

<Target Name="Clean">

  <RemoveDir Directories="$(OutputDirectory)"

       Condition="Exists($(OutputDirectory))">

  RemoveDir>

Target>

This show the use of Exists() and the MSBuild command RemoveDir.

Error Messages

I want to display a meaningful error when an exception should be raised in my task. Solution use the Error command within you task

<Target Name="UnitTests" DependsOnTargets="Build">

  <Gallio

      Assemblies="@(UnitTestProject)"

      IgnoreFailures="true"

      EchoResults="true"

      ReportDirectory="$(ReportOutputDirectory)"

      ReportTypes="$(ReportTypes)"

      RunnerType="$(TestRunnerType)"

      ShowReports="$(ShowReports)">

   

    <Output TaskParameter="ExitCode" PropertyName="ExitCode"/>

  Gallio>

  <Error Text="Tests execution failed" Condition="'$(ExitCode)' != 0" />

Target>

This examples show the use of the Gallio MSBuild task and how we can raises an error if the exit code is not 0 was the command is run

Conditional Properties

I want properties to have value dependant on conditions:

  <PropertyGroup>

    <ReportTypes Condition="'$(ReportTypes)'==''">

      Html-Condensed

    ReportTypes>

  PropertyGroup>

Swap out Configuration files

This is last as it is a personal preference on how to manage environmental differences. I basically have all my configuration files in one directory and have N+1 files per file type where N is the number of environments. I may have a ConnectionString.Config and right next to it will be ConnectionString.Config.DEV, ConnectionString.Config.TEST, ConnectionString.Config.UAT, ConnectionString.Config.PROD with each environments values in the respective file. My build only acknowledges the ConnectionString.Config file and it copies that one, the rest are not included in the build. So when i build for an environment I swap out the config files as part of the build. A contrived example of how to do this is below:

<Target Name ="SetDevConfigFiles">

  <ItemGroup>

    <SourceFiles Include="$(ApplicationConfigFolder)\*.Dev"/>

  ItemGroup>

  <Copy   SourceFiles="@(SourceFiles)"

          DestinationFiles="@(SourceFiles->'$(ApplicationConfigFolder)\%(Filename)')" />

 

Target>

This grabs all of the files from the Application Config Folder that end in .DEV and copies them to the same location without the DEV.

Explaination: The %(Filename) means I get the filename with out the extension. MSBuild basially just removes anything after the last “.” in the file name, this means I over write ConnectionString.Config with the contents from ConnectionString.Config.DEV. Its handy, it works for me, take it or leave it.

Creating your own MSBuild Task

This was way easier than I thought it was going to be. To create a task you just need to inherit from Microsoft.Build.Utilities.Task, override the execute method and add some properties. Here is a SQL migration build task I built (please excuse the logging and the IoC Cruft). The code to do all the leg work was already done and it had a console to fire it up. At the time I was using MSBuild to call it and i was using the exec functionality of MSBuild when I thought: I could probably just make a plug in for this! Below is the result

using System;

using Microsoft.Build.Framework;

using Microsoft.Build.Utilities;

using Microsoft.Practices.ServiceLocation;

namespace SqlVersionMigration.MsBuildTask

{

    public class RunAllScripts : Task

    {

        [Required]

        public string ScriptLocation { get; set; }

        [Required]

        public string ConnectionString { get; set; }

        public override bool Execute()

        {

            try

            {

                ContainerRegistration.Init();

                var versionMigrationService = ServiceLocator.Current.GetInstance<VersionMigrationService>();

                TaskHelper.LogEntry(Log, ScriptLocation, ConnectionString);

                var result = versionMigrationService.RunAllScriptsInOrder(ConnectionString, ScriptLocation);

                TaskHelper.LogResult(Log, result);

                return !result.Isfailed;

            }

            catch (Exception ex)

            {

                TaskHelper.LogError(Log, ex);

                return false;

            }

        }

    }

}

Some hints to get the most out of MSBuild: 

Download the MSBuild Community extensions, there is lots of good stuff here. 

Use plugins where it is a logical idea as opposed to calling out to exes. This can be done by using the import tag at the start of your file:

<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>

Know when to use MSBuild. It is a good starting point if you are learning about automated builds, especially as you have to use it every day (if you are a .Net dev) whether you like it or not (remember all sln and proj file are MSBuild files). Be warned though that it is not really the best option for stuff out side of a pure build. Once you are comfortable with MSBuild you will soon hit a very low ceiling. At that point investigate other tools like PSake and Rake. Due to the XML nature of MSBuild certain task are just messy or impossible to perform; Loops is a good example, just don’t do it.

I hope that is enough to get you up and running, if not let me know and I will try to fill in any of the gaps