;

Enabling WebDeploy in an Auto-Deployed Azure Cloud Service

Posted : Monday, 31 March 2014 21:49:00

I’ve just started a fantastic new job at truRating – it’s a really cool startup and 5 weeks in I am loving it. Its a really great team, great product and loads and loads of cool Tech. A large number of the components which comprise out application ecosystem are C# projects of one type or another and for the most part they’re hosted in Windows Azure. After establishing the environments we will initially support and an effective naming strategy for the various components (essentially based around a <brand>-<environment>-<product> pattern), we’re currently finalising our deployment/release pipeline.

My colleague Toby Reid did an excellent job of setting up TeamCity to build and deploy each of the components that comprise an environment and while this is very effective and well managed, there is a significant lead time (>10minutes) between the build being triggered and the code being available for any of the team to test/review/enjoy etc. This is principally down to the Azure resource commissioning system and the Blue-Green deployment process. While this is exactly what we want for most environments, one of the things we really need is a CD (Continuous Delivery) environment where, on every commit, the code is automatically compiled, unit tested and deployed to the CD environment as soon as possible.

One of the things I’ve become a massive fan of over the last couple of year is the WebDeploy deployment tool, there are tons of resources out there and I have blogged about how great it is before. Its a free install and easy to add via the Web Platform Installer. The principle attraction of the tool to me is just how fast it is – I previously worked in a team where we needed to deploy a platform consisting of 12 separate components and by moving from using MSBuild to build and deploy each step to a single initial step which built all the packages and then subsequent WebDeploy steps to actually install them the build time was cut by 60%.

NB this should not be implemented in a production environment for numerous obvious reasons. Also using WebDeploy in Azure requires the following (all reasonable) conditions to be met:

For development and testing purposes only

Only web roles can be updated

Can only support a single instance of a web role

You must enable remote desktop connections

I began a quest to see if we could use WebDeploy in our Azure cloud services...

After a few days struggling I have managed to discover, digest and harness the beauty that is WebDeploy in Microsoft Azure cloud services. This can be thought of as two distinct challenges. The first challenge is to create a cloud service capable of hosting a  WebDeploy service, the second (and far more simple) challenge is to WebDeploy into the cloud service. Incidentally publishing from Visual Studio gives different results than publishing from command-line/TeamCity and given that what we want is fully automated deployment (why would you ever want anything but?) – the goal of this exercise was to support WebDeploy on a newly created Azure cloud service with no manual intervention.

I have covered the important parts of each step below:

1) Create the Microsoft Azure cloud service

The Visual Studio publish wizard gives a lovely helpful dialogue which is pretty clear about how to go about this:

image

Indeed hitting publish after completing the wizard creates a new cloud service with three endpoints (browsing port 80, Remote Desktop port 3343 and WebDeploy port 8172). Lovely. Save the publish settings and we’re good to go right? Wrong!

I run publish from Visual Studio and can WebDeploy and Remote Desktop to the cloud service fine...

image

...but when I executed the build from the command-line the WebDeploy endpoint disappeared...

image

This drove me mad for about a day and a half!

So, Googling gave me nothing, Bing gave me nothing and DuckDuckGo DuckDuckWent! There were some helpful posts from guys a few years back but nothing since – I had to figure this one out myself. The heart of the problem was that Visual Studio MUST be doing something extra/different than plain old MSBuild from the command line. I ran the Visual Studio publish, verified the Cloud service was deployed with WebDeploy enabled and saved the contents of the Azure cloud project bin directory. Then I ran the build from the command line, verified that WebDeploy was not enabled and compared the service definition (.csdef) files from the two deployments.

The contents of the non-WebDeployable (non-working) ServiceDefinition.csdef file are shown below:

<ServiceDefinition name="ContinuousDelivery" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2013-10.2.2">
  <WebRole name="YOUR_SITE_NAME" vmsize="Small">
    <Sites>
      <Site name="Web">
        <Bindings>
          <Binding name="Endpoint1" endpointName="Endpoint1" />
        </Bindings>
      </Site>
    </Sites>
    <Endpoints>
      <InputEndpoint name="Endpoint1" protocol="http" port="80" />
    </Endpoints>
    <Imports>
      <Import moduleName="Diagnostics" />
      <Import moduleName="RemoteAccess" />
      <Import moduleName="RemoteForwarder" />
    </Imports>
  </WebRole>
</ServiceDefinition>

The contents of the WebDeployable (working) ServiceDefinition.csdef file are shown below – note the extra XML nodes shown in bold:

<ServiceDefinition name="ContinuousDelivery" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2013-10.2.2">
  <WebRole name="YOUR_SITE_NAME" vmsize="Small">
    <Sites>
      <Site name="Web">
        <Bindings>
          <Binding name="Endpoint1" endpointName="Endpoint1" />
        </Bindings>
      </Site>
    </Sites>
    <Endpoints>
      <InputEndpoint name="Endpoint1" protocol="http" port="80" />
      <InputEndpoint name="Microsoft.WindowsAzure.Plugins.WebDeploy.InputEndpoint" protocol="tcp" port="8172" localPort="8172" />
    </Endpoints>
    <Imports>
      <Import moduleName="Diagnostics" />
      <Import moduleName="RemoteAccess" />
      <Import moduleName="RemoteForwarder" />
      <Import moduleName="WebDeploy" />
    </Imports>
  </WebRole>
</ServiceDefinition>

So Visual Studio publish does “something” that causes the generated ServiceDefinition.csdef file to contain extra configuration data – slightly weird given that the source ServiceDefintion.csdef file used to generate these files was the same in both cases!

So to work out what extra magic Visual Studio was doing I set the Visual Studio output level as verbose as possible, published the project and examined the output - I found these lines in the log window…

2>    Task "Message"
2>        Task Parameter:Text=Adding Web Deploy component...
2>        Adding Web Deploy component...
2>    Task "Message"
2>        Task Parameter:Text=EnableWebDeploy is true
2>        EnableWebDeploy is true
2>    Task "Message"
2>        Task Parameter:Text=WebDeployPorts = YOUR_SITE_NAME:8172|
2>        WebDeployPorts = YOUR_SITE_NAME:8172|
2>    Task "Message"
2>        Task Parameter:Text=WebDeploy roles = YOUR_SITE_NAME
2>        WebDeploy roles = YOUR_SITE_NAME
2>    Task "Message"
2>        Task Parameter:Text=TargetServiceDefinition is bin\Release\ServiceDefinition.csdef
2>        TargetServiceDefinition is bin\Release\ServiceDefinition.csdef
2>    Using "EnableWebDeploy" task from assembly "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\Windows Azure Tools\2.2\Microsoft.VisualStudio.WindowsAzure.Tasks.2.2.dll".
2>    Task "EnableWebDeploy"
2>        Task Parameter:ServiceConfigurationFile=bin\Release\ServiceConfiguration.cscfg
2>        Task Parameter:ServiceDefinitionFile=bin\Release\ServiceDefinition.csdef
2>        Task Parameter:RolesAndPorts= YOUR_SITE_NAME:8172|

...which then led me to the Azure SDK build targets file, this section in particular...

<!--
  ===============ConfigureWebDeploy=============================================

    Enables Web Deploy on web roles if the user chooses to do so.
 
    [IN]
 
    TargetServiceDefinition: (item) The service definition file that is actually published.
    TargetServiceConfiguration: (item) The service configuration file that is actually published.
    WebRoleReferences: (item) The web roles being published.
   
  ==============================================================================
  -->
<Target Name="ConfigureWebDeploy" Condition="'$(EnableWebDeploy)'=='true'">

  <ItemGroup>
    <RolesToConfigure Include="@(WebRoleReferences->'%(RoleName)')" />
  </ItemGroup>

  <Message Text="Adding Web Deploy component..." />
  <Message Text="EnableWebDeploy is $(EnableWebDeploy)" />
  <Message Text="WebDeployPorts = $(WebDeployPorts)" />
  <Message Text="WebDeploy roles = @(RolesToConfigure)" />
  <Message Text="TargetServiceDefinition is @(TargetServiceDefinition)" />

  <!-- Add Web Deploy Plugin -->
  <EnableWebDeploy
    ServiceConfigurationFile="@(TargetServiceConfiguration)"
    ServiceDefinitionFile="@(TargetServiceDefinition)"
    RolesAndPorts="$(WebDeployPorts)" />
</Target>

I fiddled about with the settings listed and sure enough, adding the argument below to the MSBuild command line...

/p:EnableWebDeploy=true,WebDeployPorts=<YOUR_SITE_NAME>:8172,RolesToConfigure=<YOUR_SITE_NAME>

…caused the extra configuration info to be included in the resultant ServiceDefinition file. Hooking this up to actually publish the service was a simple matter of adding the extra parameter to the TeamCity build step. TeamCity will give a warning if you set system parameters this way (ie via the command line) but I cant see that this one has any reuse potential so am happy for it to remain a command line arg.

2) WebDeploy the new code

This is super-simple compare to challenge 1, and involves getting either the IP address or DNS name from the Azure management console and plugging them into the command line build (or WebDeploy powershell Snap-In that comes with WebDeploy)!

 

Also bare in mind that any time the role is restarted the WebDeploy-able customisations will be lost and you’ll need to redeploy the instance – this is due the the fact that WebDeploy goodness is added after the package is uploaded via startup tasks created dynamically and these are not reapplied when the role is recycled – we overcame this by adding logic to TeamCity to redeploy the whole instance to Azure in the event of any errors.

All this was very blackbox but its all good now – as stated previously there are limitations to using webdeploy in an Azure cloud service but IMHO its worth it – we can get instant (<1minute) feedback from other members of the team without needing to peer over a developer’s shoulder – as I’m finding out things move fast in the StartUp world so every second counts. It’s also pretty cool to know that despite all the great stuff that MSBuild and .Net give us, its not magic and if you can be bothered to poke around enough – the answers are there.

  • (This will not appear on the site)

SOLVED Toshiba Satellite laptop will not boot from CD

Posted : Thursday, 30 January 2014 21:45:00

So fellow geeks, we know how we are a lot of peoples “IT guy” right? Well a friend of mine just got a new Windows 8.1 laptop and asked me to install some software on it etc. Generally when I do this for someone – or get a new machine of my own – after I’ve installed the millions of OS and application updates since the laptop came off the production line in ChengDu, I run the excellent CloneZilla. In the event of some critical crash or virus its so simple and quick to reinitialize the device from a clone that is worth the minor hassle of cloning it every so often.

*DISCLAIMER* IF YOU’RE GOING TO MESS AROUND WITH THE BIOS AND THE INTERNALS OF YOUR COMPUTER MAKE SURE YOU TAKE A COPY OF ANY IMPORTANT STUFF FIRST – COMPUTERS101 – I HAVE LEARNED THIS THE HARD WAY

Well I finished installing everything, set up user accounts etc so it was time to clone before handing it back. Generally laptops are set to boot from the hard drive first these days so I needed to go into the BIOS and check this. For this particular laptop I found (thanks to Google) that you need to hold the delete key while the laptop powers on and then press F2 at an indeterminate time after the lights come on Smile! I powered down, entered the BIOS and altered it to boot from CD first...

BootOrder

 

Next I popped the CloneZilla CD into the drive, hit F10 to Save and Exit and waited…I was confronted by a "Checking Media...fail" message before the laptop booted into Windows.

This was slightly irritating – particularly given that after a few tries I looked at my CloneZilla DVD and decided it was probably borked so I burned another! I was confronted by the same message Sad smile

I rebooted and went back into the BIOS, I then went through every level of every available setting in the BIOS and eventually found this promising sounding option...

image

I have no idea what UEFI-capable OS’s are and why they are the preferred option but given my CloneZilla image is so old it has its own slippers I thought I’d try going back to the good old days of CSM booting...

WP_20140126_002

 

So now again I hit F10 to Save and Exit and waited...this time I booted straight into CloneZilla and within 40 minutes I had a full clone of the machine on my 2TB USB hard drive. Once I’d done that I switched back to UEFI boot (still wondering what it was) and altered the boot order back to hard drive first. I wished Tosh#23544 all the best and sent the little fella on his way!

Result!

  • (This will not appear on the site)

Disabling specific protocols for WCF services

Posted : Tuesday, 31 December 2013 22:14:00

I’m currently working for a client who host a large number of online integration solutions. For the most part these are .Net IIS applications exposing both intranet and internet endpoints. One such application hosts an intranet-only WCF service and an internet-available SignalR hub – in order to lock down security we needed to lock down access to the WCF service to only those machines within the corporate network so using the netTcp binding was the obvious choice. The problem we needed to solve was how to remove HTTP support from the WCF service but retain support for use by clients accessing the SignalR Hub.

I love WCF and make no secret about it – not everyone shares my opinion I know but the fact that a service can be developed with ease using any appropriate developer environment and all subsequent deployment-time tweaks performed with zero-code change is fantastic – okay there is a bewildering array of configuration options but that's the case in any service orientated architecture so for me its a no brainer to keep that all that noise out of the code. By the way for anyone wanting to learn WCF I would recommend these two titles - Learning WCF: A Hands-on Guide and Programming WCF Services: Mastering WCF and the Azure AppFabric Service Bus - I have them both and have read both numerous times!

Given the degree of flexibility that WCF offers I was fairly sure what we wanted to do was achievable but no amount of Googling,Binging or even old-school book reading gave me the answer. The obvious choice would have been to simple remove the http binding from the IIS application node but doing so would have meant the Hub would become unavailable. After a bit of fiddling around I managed to discover how to do what we needed – it was as simple as adding the following to the web config file of the host application...

<system.serviceModel>
  <behaviors>
    <serviceBehaviors>
      <behavior>
        <serviceMetadata httpGetEnabled="false" httpsGetEnabled="false" />
        <serviceDebug includeExceptionDetailInFaults="false" />
      </behavior>
    </serviceBehaviors>
  </behaviors>
  <protocolMapping>
    <remove scheme="https"/>
    <remove scheme="http"/>
  </protocolMapping>

</system.serviceModel>

The key section is <protocolMapping> – by default any services hosted in an IIS application will support all protocols – adding this configuration step will remove support for HTTP and HTTPS while maintaining support for all other protocols. We have this running in production now where not only is the application more secure but we are getting around 30% performance gain in WCF call times. Okay so it took a while to find but then the obvious way of doing this would be to remove the binding in IIS but in this case that wasn’t an option however WCF still came through for us, you gotta love WCF right?

  • (This will not appear on the site)

Recently read, highly rated...

previous next