Node.js, Ruby, and Python in Windows Azure: A Look at What's Possible

Steve Marx
blog.smarx.com
@smarx

Hello, my name is smarx!

Agenda

What: "Smarx Role"

What: Examples

Why: Why Windows Azure?

Cool example

app.rb

require 'sinatra'
require 'sinatra/reloader' if development?
require 'RedCloth'
require 'titleize'

set :public, File.dirname(__FILE__) + '/static'

$content_path = File.absolute_path File.dirname(__FILE__)

class String
  def starts_with?(prefix)
    prefix = prefix.to_s
    self[0, prefix.length] == prefix
  end
end

def safe_path(path)
  filename = File.absolute_path path, $content_path
  if filename.starts_with? $content_path + '/' and File.file? filename
    return filename
  end
end

module HighlightTag
  def highlight(opts)
    filename = safe_path opts[:text]
    if filename
      `pygmentize -f html #{filename}`
    else
      'ERROR: code file not found'
    end
  end
end

def which(cmd)
    exts = ENV['PATHEXT'] ? ENV['PATHEXT'].split(';') : ['']
    ENV['PATH'].split(File::PATH_SEPARATOR).each do |path|
        exts.each { |ext|
            exe = "#{path}/#{cmd}#{ext}"
            return exe if File.executable? exe
        }
    end
    return nil
end

def highlight(name)
  filename = safe_path name
  pygmentize = which('pygmentize') || File.dirname(which('python')) + '/scripts/pygmentize'
  if filename
    `#{pygmentize} -f html #{filename}`
  else
    'ERROR: code file not found'
  end
end

get '/*' do
  name = unless params[:splat].first.empty? then
    params[:splat].first
  else
    'index'
  end
  filename = safe_path name + '.textile'
  if filename
    r = RedCloth.new File.read filename
    r.extend HighlightTag
    haml :article, :locals => {
        :content => r.to_html.gsub(/\[\[([^\]]+)\]\]/) { |m| highlight($1) },
        :title => name.gsub('-', ' ').titleize
    }
  else
    filename = safe_path params[:splat].first
    if filename
      send_file filename
    else
      404
    end
  end
end

How: Interoperability

How: Windows Azure roles

How: The Big Picture

big picture

How: Components

How: Reverse Proxy

installARR.cmd

cd /d "%~dp0"
msiexec /i webfarm_amd64_en-US.msi /qn /log installWebfarm.log
msiexec /i requestRouter_amd64_en-US.msi /qn /log installARR.log

exit /b 0

setupReverseProxy.cmd

setlocal enableextensions enabledelayedexpansion
set appcmd="%windir%\system32\inetsrv\appcmd"
set hosts="%windir%\system32\drivers\etc\hosts"

%appcmd% set config -section:webFarms
    /-"[name='localfarm']" /commit:apphost
%appcmd% set config -section:webFarms
    /+"[name='localfarm']" /commit:apphost
%appcmd% set config -section:webFarms
    /[name='localfarm'].applicationRequestRouting
     .protocol.cache.enabled:false
%appcmd% set config -section:webFarms
    /[name='localfarm'].applicationRequestRouting
     .healthCheck.url:"http://localhost"
%appcmd% set config -section:webFarms
    /[name='localfarm'].applicationRequestRouting
     .healthCheck.timeout:"00:00:02"
%appcmd% set config -section:webFarms
    /[name='localfarm'].applicationRequestRouting
     .loadBalancing.algorithm:WeightedRoundRobin

del %hosts%
for /l %%n in (0, 1, 7) do (
    echo 127.0.0.1 localhost%%n >> %hosts%
    set /a port=9000+%%n
    %appcmd% set config -section:webFarms
        /+"[name='localfarm'].[address='localhost%%n']"
        /commit:apphost
    %appcmd% set config -section:webFarms
        /[name='localfarm'].[address='localhost%%n']
         .applicationRequestRouting.httpPort:!port!
)

How: Reverse Proxy (Cont'd)

applicationHost.config

<webFarms>
    <webFarm name="localfarm">
        <server address="localhost0">
            <applicationRequestRouting httpPort="9000" />
        </server>
        <server address="localhost1">
            <applicationRequestRouting httpPort="9001" />
        </server>
        <!-- ... -->
        <applicationRequestRouting>
            <protocol>
                <cache enabled="false" />
            </protocol>
            <healthCheck url="http://localhost" timeout="00:00:02" />
            <loadBalancing algorithm="WeightedRoundRobin" />
        </applicationRequestRouting>
    </webFarm>
    <applicationRequestRouting>
        <hostAffinityProviderList>
            <add name="Microsoft.Web.Arr.HostNameRoundRobin" />
            <add name="Microsoft.Web.Arr.HostNameMemory" />
        </hostAffinityProviderList>
    </applicationRequestRouting>
</webFarms>

Web.config

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <system.webServer>
    <handlers>
      <clear />
    </handlers>
    <rewrite>
      <rules>
        <rule name="Reverse proxy" patternSyntax="Wildcard" stopProcessing="true">
          <match url="*" />
          <action type="Rewrite" url="http://localfarm/{R:0}" />
        </rule>
      </rules>
    </rewrite>
  </system.webServer>
</configuration>

How: Ruby

installRuby.cmd

cd /d "%~dp0"
powershell -c "set-executionpolicy unrestricted"
for /f %%p in ('powershell .\getLocalResource.ps1 Ruby') do set RUBYPATH=%%p
for /f %%p in ('powershell .\getLocalResource.ps1 Git') do set GITPATH=%%p

7za x ruby-1.9.2-p180-i386-mingw32.7z -y -o"%RUBYPATH%"
7za x DevKit-tdm-32-4.5.1-20101214-1400-sfx.exe -y -o"%RUBYPATH%\devkit"

md "%RUBYPATH%\home"

set PATH=%PATH%;%RUBYPATH%\ruby-1.9.2-p180-i386-mingw32\bin;%GITPATH%\bin

cd /d "%RUBYPATH%\devkit"
ruby dk.rb init
echo - %RUBYPATH%\ruby-1.9.2-p180-i386-mingw32 >> "%RUBYPATH%\devkit\config.yml"
ruby dk.rb install

call gem install specific_install --no-ri --no-rdoc
call gem specific_install -l http://github.com/eventmachine/eventmachine.git
call gem install bundler thin --no-ri --no-rdoc

echo y| cacls "%RUBYPATH%" /grant everyone:f /t

exit /b 0

Ruby.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace smarx.BlobSync
{
    public class Ruby : IRunner
    {
        public IEnumerable<string> DetectionFiles
        {
            get
            {
                return new [] {
                    "app.rb"
                };
            }
        }
        public IEnumerable<CommandWithArguments> DependencyCommands
        {
            get
            {
                return new [] {
                    new CommandWithArguments(Environment.GetEnvironmentVariable("ComSpec"), "/c bundle install")
                };
            }
        }

        public CommandWithArguments RunCommand
        {
            get
            {
                return new CommandWithArguments("ruby", "app.rb");
            }
        }
    }
}

How: Python

installPython.cmd

cd /d "%~dp0"
setlocal enableextensions enabledelayedexpansion
if exist %SYSTEMDRIVE%\python27 (
    set PYTHONPATH=%SYSTEMDRIVE%\python27
) else (
    powershell -c "set-executionpolicy unrestricted"
    for /f %%p in ('powershell .\getLocalResource.ps1 Python') do set PYTHONPATH=%%p

    msiexec /i python-2.7.1.msi /qn TARGETDIR="!PYTHONPATH!" /log installPython.log
)
%PYTHONPATH%\python -c "import sys, os; sys.path.insert(0, os.path.abspath('setuptools-0.6c11-py2.7.egg')); from setuptools.command.easy_install import bootstrap; sys.exit(bootstrap())"
%PYTHONPATH%\scripts\easy_install pip

echo y| cacls "%PYTHONPATH%" /grant everyone:f /t

exit /b 0

Python.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace smarx.BlobSync
{
    public class Python : IRunner
    {
        public IEnumerable<string> DetectionFiles
        {
            get
            {
                return new[] {
                    "app.py"
                };
            }
        }
        public IEnumerable<CommandWithArguments> DependencyCommands
        {
            get
            {
                return new[] {
                    new CommandWithArguments("pip", "install -r Pipfile")
                };
            }
        }

        public CommandWithArguments RunCommand
        {
            get
            {
                return new CommandWithArguments("python", "app.py");
            }
        }
    }
}

How: Node.js

installNode.cmd

cd /d "%~dp0"
powershell -c "set-executionpolicy unrestricted"
for /f %%p in ('powershell .\getLocalResource.ps1 Node') do set NODEPATH=%%p

7za x node-0.4.3-i686-pc-cygwin-complete.7z -y -o"%NODEPATH%"

powershell .\makeResolvConf.ps1 > "%NODEPATH%\etc\resolv.conf"

for /f %%p in ('powershell .\getLocalResource.ps1 Git') do set GITPATH=%%p

REM Bleeding edge! Grab the latest NPM  from GIT (scary)
"%GITPATH%\bin\git" clone http://github.com/isaacs/npm.git
cd npm
"%GITPATH%\bin\git" submodule update --init
call "%NODEPATH%\bin\setenv"
node cli.js install -g --force

REM Not strictly necessary, but I'm going to always compile *.coffee from the src/ directory
node /usr/bin/npm install coffee-script -g --verbose

echo y| cacls "%NODEPATH%" /grant everyone:f /t

exit /b 0

makeResolvConf.ps1

[System.Net.NetworkInformation.NetworkInterface]::GetAllNetworkInterfaces() `
    | where { $_.Supports([System.Net.NetworkInformation.NetworkInterfaceComponent]::IPv4) `
      -and (-not $_.IsReceiveOnly) `
      -and ($_.OperationalStatus -eq "Up") `
    } `
    | foreach { $_.GetIPProperties() } `
    | foreach { $_.DnsAddresses } `
    | where { $_.AddressFamily -eq "InterNetwork" } `
    | foreach { "nameserver $_" }

Node.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace smarx.BlobSync
{
    public class Node : IRunner
    {
        public IEnumerable<string> DetectionFiles
        {
            get
            {
                return new[] {
                    "app.js",
                    @"src\app.coffee"
                };
            }
        }
        public IEnumerable<CommandWithArguments> DependencyCommands
        {
            get
            {
                return new[] {
                    new CommandWithArguments("node", "/usr/bin/coffee -o . src/*.coffee"),
                    new CommandWithArguments("node", "/usr/bin/npm install --verbose")
                };
            }
        }

        public CommandWithArguments RunCommand
        {
            get
            {
                return new CommandWithArguments("node", "app.js");
            }
        }
    }
}

How: Code Syncing

How: Git

installGit.cmd

cd /d "%~dp0"
powershell -c "set-executionpolicy unrestricted"
for /f %%p in ('powershell .\getLocalResource.ps1 Git') do set GITPATH=%%p

7za x PortableGit-1.7.4-preview20110204.7z -y -o"%GITPATH%"

echo y| cacls "%GITPATH%" /grant everyone:f /t

exit /b 0

OneWayGitSync.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.WindowsAzure.ServiceRuntime;
using System.IO;
using System.Diagnostics;

namespace smarx.BlobSync
{
    public class OneWayGitSync : IOneWaySync
    {
        private string gitUrl;
        private string localPath;

        public event SyncCompletedHandler SyncCompleted;

        private bool firstTime = true;
        public void SyncAll()
        {
            string output = null;
            if (firstTime)
            {
                output = Git(localPath, "clone {0} .", gitUrl);
                firstTime = false;
            }
            else
            {
                output = Git(localPath, "pull");
            }
            if (!output.Contains("Already up-to-date.") && SyncCompleted != null)
            {
                SyncCompleted(this);
            }
        }

        public static string Git(string workingDirectory, string formatString, params object[] p)
        {
            var gitExecutable = Path.Combine(RoleEnvironment.GetLocalResource("Git").RootPath, @"bin\git.exe");
            var startInfo = new ProcessStartInfo(gitExecutable, string.Format(formatString, p))
            {
                UseShellExecute = false,
                CreateNoWindow = true,
                WorkingDirectory = workingDirectory,
                RedirectStandardOutput = true,
                RedirectStandardError = true,
                StandardOutputEncoding = Encoding.UTF8
            };

            startInfo.EnvironmentVariables["LOGONSERVER"] = @"\\" + Environment.MachineName;
            startInfo.EnvironmentVariables["PATH"] += Path.GetDirectoryName(gitExecutable);
            var proc = new Process { StartInfo = startInfo };

            var sb = new StringBuilder();
            DataReceivedEventHandler recv = (_, e) =>
            {
                lock (sb)
                {
                    sb.AppendLine(e.Data);
                }
            };
            proc.OutputDataReceived += recv;
            proc.ErrorDataReceived += recv;
            proc.Start();
            proc.BeginOutputReadLine();
            proc.BeginErrorReadLine();
            proc.WaitForExit(120000);

            return sb.ToString();
        }

        public OneWayGitSync(string gitUrl, string localPath)
        {
            this.gitUrl = gitUrl;
            this.localPath = localPath;
        }
    }
}

How: Accessing Storage

Future Work

Questions?