;

hosting a minecraft server in windows azure using resource manager templates

Posted : Saturday, 19 March 2016 21:10:00

My eldest child is 5 and like so many other parents of children of that age, Minecraft has become part of my life (by proxy at least). While up until this point it was purely a conversational presence with minimal technical input required, my son recently asked if he could play online with his school friends. Reading up on this there are plenty of service providers out there as well as plenty of guides on how to self host….I figured that it couldn't be that hard so I’d give self hosting a go.

Originally I planned to run it on my home server but after a bit of thought I decided that a cloud based approach would make the most sense (reliability, availability, support etc.) especially as I get regular monthly Azure credits by virtue of my employer being part of the BizSpark program. So how to go about this...

I found a VM Image on the Marketplace:

https://azure.microsoft.com/en-us/marketplace/partners/microsoft/minecraftserver/

Going through the setup process I noted that this image relies on the Classic deployment model so might possibly be a bit old...

image

 

I ran through the setup process, clicked create and after about 10 minutes the VM was up and running, I fired up the Minecraft client and tried to connect to the server via the DNS name on the settings page of the VM in the Azure portal...

image

Rats! The Minecraft client on my PC is 1.9 and the VM server in the image is 1.8 - based on that message 1.9 is not backwards compatible. I considered setting up a VNC client and trying to remote in or using SSH to upgrade the server both of which sounded a bit fiddly...after a bit of Googling I found this post:

https://msftstack.wordpress.com/2015/09/05/creating-a-minecraft-server-using-an-azure-resource-manager-template/

This post uses a different approach, namely scripting the Minecraft server via an Azure Resource Manager template. This approach is preferable for a number of reasons including the fact that it uses the currently favoured Azure Resource Management API. I followed the blog post which said to go to this GitHub project:

https://github.com/Azure/azure-quickstart-templates/tree/master/minecraft-on-ubuntu

and hit this button:

image

This button took me to the Azure portal where I filled in a few details as per the post, accepted some Ts+Cs and bosch! I was done – RIDICULOUSLY easy!

10 or so minutes later and my server was up and running, as before I grabbed the DNS name for the VM and connected my Minecraft client to it:

image

Slightly disappointing but I figured this would be easier to upgrade than the VM image I had used before. I went back to the GitHub repo and of the five files listed:

image

I figured that “install_minecraft.sh” would probably be the place to look...

image

Sure enough lines 50, 52 and 74 looked good candidates to update, I made sure that the 1.9 server jar was located where I guessed it should be, https://s3.amazonaws.com/Minecraft.Download/versions/1.9/minecraft_server.1.9.jar, which it was. I forked the repo, edited the file via the GitHub user interface and committed. I had a quick review of the other files and noted a couple of additional required changes:

1) README.md – this is the page that contains the magical “Deploy to Azure” button, I had to update this use the template from my fork

image

2) azuredeploy.json – I had to alter this file to use the updated “install_minecraft.sh” script from my fork

image

I committed both these changes and the clicked “Deploy to Azure” button on the home page of my fork (the one I updated in 1) above. I filled out the fields as before, hit create and 10 minutes later my updated server was up and running. I grabbed the DNS name for the VM from the Azure portal:

image

Connected to it from my Minecraft client (fingers crossed)…

image

 

TA DA!!! ace! – my son, his friends and my nephews are all able to connect and play Smile.

For the record I knew nothing about Azure Resource Manager templates before this post and without any prior reading was able to jump straight into modifying them (thanks to GitHub) and seeing the results pretty much instantly. Azure just keeps getting better!

The only other thing I did was add the Minecraft version as a parameter to the script which allows the choice of Minecraft version via the template so future upgrades should now be easier to support. The whole thing was really easy, really satisfying and as a bonus I have submitted my first pull request too – sweet!

  • (This will not appear on the site)

Configuring newrelic In Azure CloudServices

Posted : Wednesday, 20 August 2014 21:58:00

With truRating so close to launch we have just put in place our monitoring service – we looked at the options and newrelic was the stand-out choice. One problem we needed to solve that wasn’t supported natively by newrelic was the ability to dynamically configure application name at deploy time! – the basic portal new relic screen looks as follows showing a list of all Applications.

 

image

 

I have none :-(

IIS Application names (CloudService web roles) are registered via an appsetting:

  <appSettings>

    <add key="NewRelic.AppName" value="LOCAL_APP" />

  </appSettings>

if you want to “promote” this to a cloud setting its bit fiddly as you need to get the cloud package which contains the transforms to set the web.config value. To do this you need to alter any ServiceName.*.cscfg files (the service configuration file for each env) and the ServiceName.csdef (the service definition file) files manually. It looks like Visual Studio doesn’t reliably support much beyond vanilla configuration with these projects but these customisations is fairly niche so I wouldn’t expect it to.

 

To configure this application name to be set from cloud configuration  setting is a three step process:

First the Service Definition file (.csdef).

After installing the new relic nuget package the service definition file should include a startup task for each role in the cloud project:

    5     <Startup>

    6       <Task commandLine="newrelic.cmd" executionContext="elevated" taskType="simple">

    7         <Environment>

    8           <Variable name="EMULATED">

    9             <RoleInstanceValue xpath="/RoleEnvironment/Deployment/@emulated" />

   10           </Variable>

   11           <Variable name="IsWorkerRole" value="false" />

   12           <Variable name="LICENSE_KEY">

   13             <RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/ConfigurationSettings/ConfigurationSetting[@name='NewRelic.LicenseKey']/@value" />

   14           </Variable>

   15         </Environment>

   16       </Task>

   17     </Startup>

This startup task will execute as the role initializes and will execute the specified newrelic.cmd file using supplied environment variables. (more on that later) .

Define another environment variable that will be declared based on a cloud configuration value:

    5     <Startup>

    6       <Task commandLine="newrelic.cmd" executionContext="elevated" taskType="simple">

    7         <Environment>

    8           <Variable name="EMULATED">

    9             <RoleInstanceValue xpath="/RoleEnvironment/Deployment/@emulated" />

   10           </Variable>

   11           <Variable name="IsWorkerRole" value="false" />

   12           <Variable name="LICENSE_KEY">

   13             <RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/ConfigurationSettings/ConfigurationSetting[@name='NewRelic.LicenseKey']/@value" />

   14           </Variable>

   15           <!-- environment variable indicating name of new relic application as it will apppear on the newrelic Dashboard -->

   16           <Variable name="newrelic_APPLICATION" >

   17             <RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/ConfigurationSettings/ConfigurationSetting[@name='NewRelic_AppName']/@value" />

   18           </Variable>

   19         </Environment>

   20       </Task>

   21     </Startup>

NB: Ideally for consistency I would have used NewRelic.AppName but the xpath query was failing, I think, due to the “.” character.

Next scroll to the ConfigurationSettings section and add the following

   37     <ConfigurationSettings>

   38       <Setting name="TableStorageConnectionString" />

   39       <Setting name="DbConnection" />

   40       <Setting name="RandomSettingOne" />

   41       <Setting name="NewRelic.LicenseKey" />

   42       <Setting name="newrelic_APPLICATION" />

   43       <Setting name="RandomSettingTwo" />

   44     </ConfigurationSettings>

This is all the configuration thats required for the service definition file.

Second the service configuration files. For each environment add/update the ServiceName.<env>.cscfg configuration file

    1 <?xml version="1.0" encoding="utf-8"?>

    2 <ServiceConfiguration serviceName="MyRoleService" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="3" osVersion="*" schemaVersion="2014-01.2.3">

    3   <Role name="MyWebRole">

    4     <Instances count="1" />

    5     <ConfigurationSettings>

    6       <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="UseDevelopmentStorage=false" />

    7       <Setting name="TableStorageConnectionString" value="UseDevelopmentStorage=false" />

    8       <Setting name="DbConnection" value="Server=DBSERVER; Database=Database; Integrated Security=SSPI" />

    9       <Setting name="RandomSettingOne" value="1" />

   10       <Setting name="NewRelic.LicenseKey" value="xxxxxxxxxxxxxxxxxxxxxxxxxx" />

   11       <Setting name="newrelic_APPLICATION" value="LOCAL_APP" />

   12       <Setting name="RandomSettingTwo" value="2" />

   13     </ConfigurationSettings>

   14   </Role>

   15 </ServiceConfiguration>

Create/Update the configuration for each environment and you’re done.

Last the newrelic.cmd file.

This step is optional if you dont mind having a newrelic related appsetting in your application configuration file, anything in developer code that can get changed accidentally will (Sod’s Law) so putting this in the cloud service projects only (and not the target project itself) is worth the effort for the potential error it prevents.

This following files gets added to the .csproj (presumably .vbproj also) as a “copy always” resource.

image

NB versions will differ based on local environment

At role startup the batch file runs and installs all the necessary monitoring software that new relic uses to capture performance metrics.

The top of the file should be something like...

----------

SETLOCAL EnableExtensions

 

for /F "usebackq tokens=1,2 delims==" %%i in (`wmic os get LocalDateTime /VALUE 2^>NUL`) do if '.%%i.'=='.LocalDateTime.' set ldt=%%j

set ldt=%ldt:~0,4%-%ldt:~4,2%-%ldt:~6,2% %ldt:~8,2%:%ldt:~10,2%:%ldt:~12,6%

 

SET NR_ERROR_LEVEL=0

 

:: Comment out the line below if you do not want to install the New Relic Agent

CALL:INSTALL_NEWRELIC_AGENT

 

:: Comment out the line below if you do not want to install the New Relic Windows Server Monitor

CALL:INSTALL_NEWRELIC_SERVER_MONITOR

----------

To dynamically set the appSetting value I opted for a task to execute appcmd.exe from the command file and set the value as a machine level appsetting – changes in red!

----------

SETLOCAL EnableExtensions

 

for /F "usebackq tokens=1,2 delims==" %%i in (`wmic os get LocalDateTime /VALUE 2^>NUL`) do if '.%%i.'=='.LocalDateTime.' set ldt=%%j

set ldt=%ldt:~0,4%-%ldt:~4,2%-%ldt:~6,2% %ldt:~8,2%:%ldt:~10,2%:%ldt:~12,6%

 

SET NR_ERROR_LEVEL=0

 

:: Custom cmd function - execute first

CALL:SET_APP_NAME

 

:: Comment out the line below if you do not want to install the New Relic Agent

CALL:INSTALL_NEWRELIC_AGENT

 

:: Comment out the line below if you do not want to install the New Relic Windows Server Monitor

CALL:INSTALL_NEWRELIC_SERVER_MONITOR

 

IF %NR_ERROR_LEVEL% EQU 0 (

    EXIT /B 0

) ELSE (

    EXIT %NR_ERROR_LEVEL%

)

 

:: --------------

:: Functions

:: --------------

:SET_APP_NAME

 

IF [%newrelic_APPLICATION%] == [] (

    ECHO  no value for APP_MONITOR_NAME -skipping step. >> "d:\tr.log" 2>&1

    GOTO:EOF

)

----------

The custom function first checks if that environment variable newrelic_APPLICATION is set, if not the function is skipped. If the variable is declared then appSetting will be added to the machine config file.  Whizz a few requests through, wait a few mins and bada-bing…

image

N.B. machine.config is used as the structure of the local filesystem in the cloud package has a predicable (but i suspect) changeable path to the virtual root of each application. Using appcmd to set application configuration files requires prior knowledge of the local filesystem. Applying the settings at machine.config level means the value will consistently be available wherever ever the site root is on located on the filesystem.

In summary its a bit of a convoluted setup but what happens is as follows: When the startup task runs, it and passes variables configured for each environment into the newrelic.cmd process. The supplied appsetting value is written to machine.config in the Cloud Service node. This is then read by the NewRelic Agent on first web request. It is also compatible with the loveliness of Azure autoscale – because the configuration is baked into the service package, any new nodes will automatically pull in the correct settings

NB this does not work in the emulator so you need to deploy to “real” Azure to test it out – or run locally in IIS having first installed the .NET Agent and restarted IIS as per the documentation.

newRelic is great – their support isn’t the quickest but its a really good, low maintainence, easy-to-install, monitoring option I would recommend for anyone wanting an affordable enterprise monitoring solution. I set up a billing account but due to a clerical error its the account managed through the Azure portal so we’re not 100% sure we’ll go with it for launch – we are on the trial stage though so provided its resolved soon I have no doubt we’ll use newrelic!

Update 2014-08-20: The problem previously mentioned is, IMHO, down to shortcomings in the Azure Portal AddOn system– we dont have Azure Portal integration but we but we have newrelic for launch, and thats a good place to be :-D

  • (This will not appear on the site)

Enabling WebDeploy in an Auto-Deployed Azure Cloud Service

Posted : Monday, 31 March 2014 21:49:00

I’ve just started a fantastic new job at truRating – it’s a really cool startup and 5 weeks in I am loving it. Its a really great team, great product and loads and loads of cool Tech. A large number of the components which comprise out application ecosystem are C# projects of one type or another and for the most part they’re hosted in Windows Azure. After establishing the environments we will initially support and an effective naming strategy for the various components (essentially based around a <brand>-<environment>-<product> pattern), we’re currently finalising our deployment/release pipeline.

My colleague Toby Reid did an excellent job of setting up TeamCity to build and deploy each of the components that comprise an environment and while this is very effective and well managed, there is a significant lead time (>10minutes) between the build being triggered and the code being available for any of the team to test/review/enjoy etc. This is principally down to the Azure resource commissioning system and the Blue-Green deployment process. While this is exactly what we want for most environments, one of the things we really need is a CD (Continuous Delivery) environment where, on every commit, the code is automatically compiled, unit tested and deployed to the CD environment as soon as possible.

One of the things I’ve become a massive fan of over the last couple of year is the WebDeploy deployment tool, there are tons of resources out there and I have blogged about how great it is before. Its a free install and easy to add via the Web Platform Installer. The principle attraction of the tool to me is just how fast it is – I previously worked in a team where we needed to deploy a platform consisting of 12 separate components and by moving from using MSBuild to build and deploy each step to a single initial step which built all the packages and then subsequent WebDeploy steps to actually install them the build time was cut by 60%.

NB this should not be implemented in a production environment for numerous obvious reasons. Also using WebDeploy in Azure requires the following (all reasonable) conditions to be met:

For development and testing purposes only

Only web roles can be updated

Can only support a single instance of a web role

You must enable remote desktop connections

I began a quest to see if we could use WebDeploy in our Azure cloud services...

After a few days struggling I have managed to discover, digest and harness the beauty that is WebDeploy in Microsoft Azure cloud services. This can be thought of as two distinct challenges. The first challenge is to create a cloud service capable of hosting a  WebDeploy service, the second (and far more simple) challenge is to WebDeploy into the cloud service. Incidentally publishing from Visual Studio gives different results than publishing from command-line/TeamCity and given that what we want is fully automated deployment (why would you ever want anything but?) – the goal of this exercise was to support WebDeploy on a newly created Azure cloud service with no manual intervention.

I have covered the important parts of each step below:

1) Create the Microsoft Azure cloud service

The Visual Studio publish wizard gives a lovely helpful dialogue which is pretty clear about how to go about this:

image

Indeed hitting publish after completing the wizard creates a new cloud service with three endpoints (browsing port 80, Remote Desktop port 3343 and WebDeploy port 8172). Lovely. Save the publish settings and we’re good to go right? Wrong!

I run publish from Visual Studio and can WebDeploy and Remote Desktop to the cloud service fine...

image

...but when I executed the build from the command-line the WebDeploy endpoint disappeared...

image

This drove me mad for about a day and a half!

So, Googling gave me nothing, Bing gave me nothing and DuckDuckGo DuckDuckWent! There were some helpful posts from guys a few years back but nothing since – I had to figure this one out myself. The heart of the problem was that Visual Studio MUST be doing something extra/different than plain old MSBuild from the command line. I ran the Visual Studio publish, verified the Cloud service was deployed with WebDeploy enabled and saved the contents of the Azure cloud project bin directory. Then I ran the build from the command line, verified that WebDeploy was not enabled and compared the service definition (.csdef) files from the two deployments.

The contents of the non-WebDeployable (non-working) ServiceDefinition.csdef file are shown below:

<ServiceDefinition name="ContinuousDelivery" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2013-10.2.2">
  <WebRole name="YOUR_SITE_NAME" vmsize="Small">
    <Sites>
      <Site name="Web">
        <Bindings>
          <Binding name="Endpoint1" endpointName="Endpoint1" />
        </Bindings>
      </Site>
    </Sites>
    <Endpoints>
      <InputEndpoint name="Endpoint1" protocol="http" port="80" />
    </Endpoints>
    <Imports>
      <Import moduleName="Diagnostics" />
      <Import moduleName="RemoteAccess" />
      <Import moduleName="RemoteForwarder" />
    </Imports>
  </WebRole>
</ServiceDefinition>

The contents of the WebDeployable (working) ServiceDefinition.csdef file are shown below – note the extra XML nodes shown in bold:

<ServiceDefinition name="ContinuousDelivery" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2013-10.2.2">
  <WebRole name="YOUR_SITE_NAME" vmsize="Small">
    <Sites>
      <Site name="Web">
        <Bindings>
          <Binding name="Endpoint1" endpointName="Endpoint1" />
        </Bindings>
      </Site>
    </Sites>
    <Endpoints>
      <InputEndpoint name="Endpoint1" protocol="http" port="80" />
      <InputEndpoint name="Microsoft.WindowsAzure.Plugins.WebDeploy.InputEndpoint" protocol="tcp" port="8172" localPort="8172" />
    </Endpoints>
    <Imports>
      <Import moduleName="Diagnostics" />
      <Import moduleName="RemoteAccess" />
      <Import moduleName="RemoteForwarder" />
      <Import moduleName="WebDeploy" />
    </Imports>
  </WebRole>
</ServiceDefinition>

So Visual Studio publish does “something” that causes the generated ServiceDefinition.csdef file to contain extra configuration data – slightly weird given that the source ServiceDefintion.csdef file used to generate these files was the same in both cases!

So to work out what extra magic Visual Studio was doing I set the Visual Studio output level as verbose as possible, published the project and examined the output - I found these lines in the log window…

2>    Task "Message"
2>        Task Parameter:Text=Adding Web Deploy component...
2>        Adding Web Deploy component...
2>    Task "Message"
2>        Task Parameter:Text=EnableWebDeploy is true
2>        EnableWebDeploy is true
2>    Task "Message"
2>        Task Parameter:Text=WebDeployPorts = YOUR_SITE_NAME:8172|
2>        WebDeployPorts = YOUR_SITE_NAME:8172|
2>    Task "Message"
2>        Task Parameter:Text=WebDeploy roles = YOUR_SITE_NAME
2>        WebDeploy roles = YOUR_SITE_NAME
2>    Task "Message"
2>        Task Parameter:Text=TargetServiceDefinition is bin\Release\ServiceDefinition.csdef
2>        TargetServiceDefinition is bin\Release\ServiceDefinition.csdef
2>    Using "EnableWebDeploy" task from assembly "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\Windows Azure Tools\2.2\Microsoft.VisualStudio.WindowsAzure.Tasks.2.2.dll".
2>    Task "EnableWebDeploy"
2>        Task Parameter:ServiceConfigurationFile=bin\Release\ServiceConfiguration.cscfg
2>        Task Parameter:ServiceDefinitionFile=bin\Release\ServiceDefinition.csdef
2>        Task Parameter:RolesAndPorts= YOUR_SITE_NAME:8172|

...which then led me to the Azure SDK build targets file, this section in particular...

<!--
  ===============ConfigureWebDeploy=============================================

    Enables Web Deploy on web roles if the user chooses to do so.
 
    [IN]
 
    TargetServiceDefinition: (item) The service definition file that is actually published.
    TargetServiceConfiguration: (item) The service configuration file that is actually published.
    WebRoleReferences: (item) The web roles being published.
   
  ==============================================================================
  -->
<Target Name="ConfigureWebDeploy" Condition="'$(EnableWebDeploy)'=='true'">

  <ItemGroup>
    <RolesToConfigure Include="@(WebRoleReferences->'%(RoleName)')" />
  </ItemGroup>

  <Message Text="Adding Web Deploy component..." />
  <Message Text="EnableWebDeploy is $(EnableWebDeploy)" />
  <Message Text="WebDeployPorts = $(WebDeployPorts)" />
  <Message Text="WebDeploy roles = @(RolesToConfigure)" />
  <Message Text="TargetServiceDefinition is @(TargetServiceDefinition)" />

  <!-- Add Web Deploy Plugin -->
  <EnableWebDeploy
    ServiceConfigurationFile="@(TargetServiceConfiguration)"
    ServiceDefinitionFile="@(TargetServiceDefinition)"
    RolesAndPorts="$(WebDeployPorts)" />
</Target>

I fiddled about with the settings listed and sure enough, adding the argument below to the MSBuild command line...

/p:EnableWebDeploy=true,WebDeployPorts=<YOUR_SITE_NAME>:8172,RolesToConfigure=<YOUR_SITE_NAME>

…caused the extra configuration info to be included in the resultant ServiceDefinition file. Hooking this up to actually publish the service was a simple matter of adding the extra parameter to the TeamCity build step. TeamCity will give a warning if you set system parameters this way (ie via the command line) but I cant see that this one has any reuse potential so am happy for it to remain a command line arg.

2) WebDeploy the new code

This is super-simple compare to challenge 1, and involves getting either the IP address or DNS name from the Azure management console and plugging them into the command line build (or WebDeploy powershell Snap-In that comes with WebDeploy)!

 

Also bare in mind that any time the role is restarted the WebDeploy-able customisations will be lost and you’ll need to redeploy the instance – this is due the the fact that WebDeploy goodness is added after the package is uploaded via startup tasks created dynamically and these are not reapplied when the role is recycled – we overcame this by adding logic to TeamCity to redeploy the whole instance to Azure in the event of any errors.

All this was very blackbox but its all good now – as stated previously there are limitations to using webdeploy in an Azure cloud service but IMHO its worth it – we can get instant (<1minute) feedback from other members of the team without needing to peer over a developer’s shoulder – as I’m finding out things move fast in the StartUp world so every second counts. It’s also pretty cool to know that despite all the great stuff that MSBuild and .Net give us, its not magic and if you can be bothered to poke around enough – the answers are there.

  • (This will not appear on the site)

Recently read, highly rated...

previous next

Although Dali seems already to have settled into his mature style by the '30s, the omega replica still witnessed unexpected flashes of innovation that pointed ahead to the postwar developments of a younger generation. For instance, the shaped canvases of 1936, their frames conforming to anatomical contours, might be perceived as unexpected foreshadowings of louis vuitton replica; and the Mae West Lips Sofa of 1938 would look quite at home in a Claes Oldenburg retrospective. But the major revelation of the Philadelphia exhibition was Dali's postwar work, which brims with totally fresh inventions, rolex replica, yet very different from his classic icons. This material also constantly surprises us by intersecting with the art of the later twentieth century, a time when few paid heed to what Dali was up to, as was the case with the omega replica late work of Picabia and de Chirico. Of these unexpected discoveries, perhaps the most precocious are The Sistine Madonna, 1958, and Portrait of My Dead Brother, 1963, in which painted screens of benday dots (some made of cartier replica!) veil phantom images, including a rolex replica of Raphael's religious icon afloat in a giant ear and the ghostly head of Dali's brother, also named Salvador, who died nine months before the artist's own birth. These ambitious paintings seem simultaneously to belong to the Surrealist past and to the young art of the '60s, acting as replica watches uk of Roy Lichtenstein, Sigmar Polke, and Chuck Close, not to mention Op art.