Earlier this year Microsoft released the Azure VM Agent and Extensions as part of the Windows Azure Infrastructure Services. VM Extensions are software components that extend the VM functionality and simplify various VM management operations; for example, the VMAccess extension can be used to reset a VM’s password, or the Custom Script extension can be used to execute a script on the VM.
Today, we are introducing the PowerShell Desired State Configuration (DSC) Extension for Azure VMs as part of the Azure PowerShell SDK. You can use new cmdlets to upload and apply a PowerShell DSC configuration on an Azure VM enabled with the PowerShell DSC extension. PowerShell DSC extension will call into PowerShell DSC to enact the received DSC configuration on the VM.
If you already have the Azure PowerShell SDK installed, you will need to update to version 0.8.6 or later.
Once you have installed and configured Azure PowerShell and authenticated to Azure, you can use the Get-AzureVMAvailableExtension cmdlet to see the PowerShell DSC extension.
PS C:\> Get-AzureVMAvailableExtension-Publisher Microsoft.PowerShell Publisher : Microsoft.Powershell ExtensionName : DSC Version : 1.0 PublicConfigurationSchema : PrivateConfigurationSchema : SampleConfig : ReplicationCompleted : True Eula : http://azure.microsoft.com/en-us/support/legal/ PrivacyUri : http://www.microsoft.com/ HomepageUri : http://blogs.msdn.com/b/powershell/ IsJsonExtension : True
Executing a simple scenario
One scenario in which this new extension can be used is the automation of software installation and configuration upon a machine’s initial boot-up.
As a simple example, let’s say you need to create a new VM and install IIS on it. For this, you would first create a PowerShell script that defines the configuration (NOTE: I saved this script as C:\examples\IISInstall.ps1):
001 002 003 004 005 006 007 008 009 010 011 012 | configuration IISInstall { node ("localhost") { WindowsFeature IIS { Ensure = "Present" Name = "Web-Server" } } } |
Then you would use Publish-AzureVMDscConfiguration to upload your configuration to Azure storage. Publish-AzureVMDscConfiguration is one of the new cmdlets in the Azure PowerShell SDK. The example below uses all the default values, but later in this post we’ll go over more details of how this works.
PS C:\> Publish-AzureVMDscConfiguration-ConfigurationPath C:\examples\IISInstall.ps1
This cmdlet creates a ZIP package that follows a predefined format that the PowerShell Desired State Configuration Extension can understand and then uploads it as a blob to Azure storage. The ZIP package in the above example was uploaded to
https://examples.blob.core.windows.net/windows-powershell-dsc/IISInstall.ps1.zip
“examples” in this URI is the name of my default Azure storage account, “windows-powershell-dsc” is the default storage container used by the cmdlet, and “IISInstall.ps1.zip” is the name of the blob for the file I just published.
Now my sample configuration is available for VMs to use, so let’s write a script that creates a VM that uses our sample configuration (NOTE: I saved this script as C:\examples\example-1.ps1):
001 002 003 004 005 006 007 008 | $vm = New-AzureVMConfig -Name "example-1" -InstanceSize Small -ImageName "a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-201407.01-en.us-127GB.vhd" $vm = Add-AzureProvisioningConfig -VM $vm -Windows -AdminUsername "admin_account" -Password "Bull_dog1" $vm = Set-AzureVMDSCExtension -VM $vm -ConfigurationArchive "IISInstall.ps1.zip" -ConfigurationName "IISInstall" New-AzureVM -VM $vm -Location "West US" -ServiceName "example-1-svc" -WaitForBoot |
New-AzureVMConfig, Add-AzureProvisioningConfig, and New-AzureVM are the existing Azure cmdlets used to create a VM. The new kid on the block is Set-AzureVMDscExtension:
005 | $vm = Set-AzureVMDSCExtension -VM $vm -ConfigurationArchive "IISInstall.ps1.zip" -ConfigurationName "IISInstall" |
This cmdlet injects a DSC configuration into the VM configuration object ($vm in the example). When the VM machine boots, the Azure VM agent will install the PowerShell DSC Extension, which in turn will download the ZIP package that we published previously (IISInstall.ps1.zip), will execute the “IISInstall” configuration that we included as part of IISInstall.ps1, and then will invoke PowerShell DSC by calling the Start-DscConfiguration cmdlet.
Now, let’s go ahead and execute the sample script (NOTE: if you get an error telling you that the VM vhd is not available or you don’t have access to it, that likely means that the image referenced on line 1 of the script has been updated, and you will need to find the new image name. You can do so by enumerating the available images with Get-AzureVMImage and picking the image that you wish to use. See Azure SDK documentation for more details on this. In my case, I will use a 2012-R2 machine).
PS C:\> C:\examples\example-1.ps1
OperationDescription OperationId OperationStatus
-------------------- ----------- ---------------
New-AzureVM 9cfb922d-db5b-cdd0-9c74-1a4e34b91e28 Succeeded
New-AzureVM 17acca22-c6ff-cb5a-8116-a41ff9764d35 Succeeded Our sample configuration was very simple: it just installed IIS. As a quick verification that it executed properly, we can logon to the VM and verify that IIS is installed by visiting the default web site (http://localhost):
That is the PowerShell DSC Extension in a nutshell.
And now for the gory details…
Publish-AzureVMDscConfiguration
As the previous example illustrated, the first step in using the PowerShell Desired State Configuration Extension is publishing. In this context, publishing is the process of creating a ZIP package that the extension can understand and uploading that package to Azure blob storage. This is accomplished using the Publish-AzureVMDscConfiguration cmdlet.
Why use a ZIP package for publishing? Publish-AzureVMDscConfiguration will parse your configuration looking for Import-DSCResource statements and will include a copy of the corresponding modules along with the script that contains your configuration. For example, let’s take a look at the ZIP package produced by a configuration that creates an actual website instead of just installing IIS. This new example is the FourthCoffee website, which you may have already seen in other DSC blog posts or demos). The FourthCoffee demo has a dependency on the DSC resource xWebAdministration, which is included in the DSC Resource Kit Wave 5.
(NOTE: I saved this script as C:\examples\FourthCoffee.ps1)
001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 | configuration FourthCoffee { Import-DscResource -Module xWebAdministration # Install the IIS role WindowsFeature IIS { Ensure = "Present" Name = "Web-Server" } # Install the ASP .NET 4.5 role WindowsFeature AspNet45 { Ensure = "Present" Name = "Web-Asp-Net45" } # Stop the default website xWebsite DefaultSite { Ensure = "Present" Name = "Default Web Site" State = "Stopped" PhysicalPath = "C:\inetpub\wwwroot" DependsOn = "[WindowsFeature]IIS" } # Copy the website content File WebContent { Ensure = "Present" SourcePath = "C:\Program Files\WindowsPowerShell\Modules\xWebAdministration\BakeryWebsite" DestinationPath = "C:\inetpub\FourthCoffee" Recurse = $true Type = "Directory" DependsOn = "[WindowsFeature]AspNet45" } # Create a new website xWebsite BakeryWebSite { Ensure = "Present" Name = "FourthCoffee" State = "Started" PhysicalPath = "C:\inetpub\FourthCoffee" DependsOn = "[File]WebContent" } } |
To inspect the ZIP package created by the publish cmdlet I used the -ConfigurationArchivePath parameter, which saves the package to a local file instead of uploading it to Azure storage (NOTE: I typed the command below in two separate lines using the ` character; the >>> characters are PowerShell’s prompt):
PS C:\> Publish-AzureVMDscConfiguration C:\examples\FourthCoffee.ps1 ` >>> -ConfigurationArchivePath C:\examples\FourthCoffee.ps1.zip
When I look at the ZIP package using the File Explorer I can see that it contains my configuration script and a copy of the xWebAdministration module:
That copy comes from the xWebAdministration module that I already installed on my machine under “C:\Program Files\WindowsPowerShell\Modules”. The publish cmdlet requires that the imported modules are installed on your machine, and that they are located somewhere in $PSModulePath.
(NOTE: To simplify the example, I slightly altered the xWebAdministration module so it included the files needed for the website as part of the xWebAdministration module, in the “BakeryWebsite” directory)
The two previous examples use a PowerShell Script file (.ps1) to define the configuration that will be published. You can also do this in a PowerShell Module file (.psm1), or if the configuration you want to publish is part of a larger module, you can create the ZIP package manually and simply copy the directories for the module that defines your configuration and any modules referenced by your configuration. For example, if the configuration of our example was defined within a PowerShell module named FourthCoffee the ZIP package would include these two directories: the FourthCoffee module folder, and the dependent DSC resource module folder for xWebAdministration
Once you have a local ZIP package (either created manually, or using the publish cmdlet), you can upload it to Azure storage with the publish cmdlet:
PS C:\> Publish-AzureVMDscConfiguration C:\examples\FourthCoffee.ps1.zip ContainerName and StorageContext parameters
By default Publish-AzureVMDscConfiguration will upload the ZIP package to Azure blob storage using “windows-powershell-dsc” as the container and picking up the default storage account from the settings of your Azure subscription.
You can change the container using the –ContainerName parameter:
PS C:\> Publish-AzureVMDscConfiguration C:\examples\FourthCoffee.ps1.zip ` >>> -ContainerName mycontainer
And you can change the storage account (and authentication settings) using the –StorageContext parameter (you can use the New-AzureStorageContext cmdlet to create the storage context).
Set-AzureVMDSCExtension
Once a configuration has been published, you can apply it to any Azure virtual machine using the Set-AzureVMDSCExtension cmdlet. This cmdlet injects the settings needed by the PowerShell DSC extension into a VM configuration object, which can then be applied to a new VM, as in our first example, or to an existing VM. Let’s use this cmdlet again to update the VM we created previously (NOTE: the first example used the configuration defined in C:\examples\IISInstall.ps1; now we will update this machine with the configuration defined in C:\examples\FourthCoffee.ps1; the script that we will use was saved as C:\examples\example-2.ps1)
001 002 003 004 005 006 | $vm = Get-AzureVM -Name "example-1" -ServiceName "example-1-svc" $vm = Set-AzureVMDSCExtension -VM $vm -ConfigurationArchive "FourthCoffee.ps1.zip" -ConfigurationName "FourthCoffee" $vm | Update-AzureVM |
PS C:\> C:\examples\example-2.ps1
OperationDescription OperationId OperationStatus
-------------------- ----------- ---------------
Update-AzureVM afa38e1a-5717-cac6-a6e7-6f72d0af51d2 Succeeded In our first example we were working with a new VM, so the Azure VM agent first installed the PowerShell DSC Extension and then it invoked it using the information provided by the Set-AzureVMDSCExtension cmdlet. In this second example we are working on an existing VM on which the extension is already installed so the Azure VM agent will skip the installation part and just invoke the PowerShell DSC Extension with the new information provided by the set cmdlet.
The extension will then
- download the ZIP package specified by the –ConfigurationArchive parameter and expand it to a temporary directory
- remove the .zip extension from the value given by –ConfigurationArchive and look for a PowerShell script or module with that name and execute it (in our second example, it will look for FourthCoffee.ps1)
- look for and execute the configuration named by the -ConfigurationName parameter (in this case "WebSite")
- invoke the Start-DscConfiguration with the output produced by that configuration
To verify that our second configuration was applied successfully we can again check the default website:
Configuration Arguments
DSC configurations are very similar to PowerShell advanced functions and can be parameterized for greater flexibility. The PowerShell DSC extension provides support for configuration arguments via the –ConfigurationArgument parameter of Set-AzureVMDSCExtension.
As a very simple example, let’s change our last script in such a way that the name of the website is a parameter to the FourthCoffee configuration. The updated configuration has been saved as C:\examples\FourthCoffeeWithArguments.ps1; notice that we have added the $WebSiteName parameter (lines 4-7), which is used as the Name property of the BakeryWebSite resource (line 51).
001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 | configuration FourthCoffee { [CmdletBinding()] param( [Parameter(Mandatory=$true, Position=0)] [string] $WebSiteName ) Import-DscResource -Module xWebAdministration # Install the IIS role WindowsFeature IIS { Ensure = "Present" Name = "Web-Server" } # Install the ASP .NET 4.5 role WindowsFeature AspNet45 { Ensure = "Present" Name = "Web-Asp-Net45" } # Stop the default website xWebsite DefaultSite { Ensure = "Present" Name = "Default Web Site" State = "Stopped" PhysicalPath = "C:\inetpub\wwwroot" DependsOn = "[WindowsFeature]IIS" } # Copy the website content File WebContent { Ensure = "Present" SourcePath = "C:\Program Files\WindowsPowerShell\Modules\xWebAdministration\BakeryWebsite" DestinationPath = "C:\inetpub\FourthCoffee" Recurse = $true Type = "Directory" DependsOn = "[WindowsFeature]AspNet45" } # Create a new website xWebsite BakeryWebSite { Ensure = "Present" Name = $WebSiteName State = "Started" PhysicalPath = "C:\inetpub\FourthCoffee" DependsOn = "[File]WebContent" } } |
Our third example publishes the new configuration script and updates the VM that we created previously (I saved this script as C:\examples\example-3.ps1:
001 002 003 004 005 006 007 008 009 010 011 | Publish-AzureVMDscConfiguration C:\examples\FourthCoffeeWithArguments.ps1 $vm = Get-AzureVM -Name "example-1" -ServiceName "example-1-svc" $vm = Set-AzureVMDscExtension -VM $vm ` -ConfigurationArchive "FourthCoffeeWithArguments.ps1.zip" ` -ConfigurationName "FourthCoffee" ` -ConfigurationArgument @{ WebSiteName = "FourthCoffee" } $vm | Update-AzureVM |
PS C:\> C:\examples\example-3.ps1
OperationDescription OperationId OperationStatus
-------------------- ----------- ---------------
Update-AzureVM 2b6f18e7-42f2-c216-8199-edfa06b52e33 Succeeded The value of the –ConfigurationArgument parameter on line 8 of C:\examples\example-3.ps1 is a hashtable that specifies the arguments to the WebSite configuration, i.e. a string specifying the name of the website (this corresponds to parameter $WebSiteName, on line 7 of C:\examples\FourthCoffeeWithArguments.ps1)
Configuration Data
Configuration data can be used to separate structural configuration from environmental configuration (see this blog post for an introduction to those concepts). The PowerShell DSC extension provides support for configuration data via the –ConfigurationDataPath parameters of Set-AzureVMDSCExtension.
Let’s create another variation of the FourthCoffee configuration: IIS and ASP.NET will always be installed by the configuration, but the FourthCoffee website will be installed only if the role of the VM is “WebServer”. The updated configuration has been saved as C:\examples\FourthCoffeeWithData.ps1; the check for the VM’s role is on line 20:
001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 | configuration FourthCoffee { Import-DscResource -Module xWebAdministration # Install the IIS role WindowsFeature IIS { Ensure = "Present" Name = "Web-Server" } # Install the ASP .NET 4.5 role WindowsFeature AspNet45 { Ensure = "Present" Name = "Web-Asp-Net45" } # Setup the website only if the role is "WebServer" Node $AllNodes.Where{$_.Role -eq "WebServer"}.NodeName { # Stop the default website xWebsite DefaultSite { Ensure = "Present" Name = "Default Web Site" State = "Stopped" PhysicalPath = "C:\inetpub\wwwroot" DependsOn = "[WindowsFeature]IIS" } # Copy the website content File WebContent { Ensure = "Present" SourcePath = "C:\Program Files\WindowsPowerShell\Modules\xWebAdministration\BakeryWebsite" DestinationPath = "C:\inetpub\FourthCoffee" Recurse = $true Type = "Directory" DependsOn = "[WindowsFeature]AspNet45" } # Create a new website xWebsite BakeryWebSite { Ensure = "Present" Name = $WebSiteName State = "Started" PhysicalPath = "C:\inetpub\FourthCoffee" DependsOn = "[File]WebContent" } } } |
The configuration data has been saved as C:\examples\FourthCoffeeData.psd1:
001 002 003 004 005 006 007 008 009 | @{ AllNodes = @( @{ NodeName = "localhost"; Role = "WebServer" } ); } |
And the script that publishes and applies this new configuration is C:\examples\example-4.ps1:
001 002 003 004 005 006 007 008 009 010 011 | Publish-AzureVMDscConfiguration C:\examples\FourthCoffeeWithData.ps1 $vm = Get-AzureVM -Name "example-1" -ServiceName "example-1-svc" $vm = Set-AzureVMDscExtension -VM $vm ` -ConfigurationArchive "FourthCoffeeWithData.ps1.zip" ` -ConfigurationName "FourthCoffee" ` -ConfigurationDataPath C:\examples\FourthCoffeeData.psd1 $vm | Update-AzureVM |
C:\ PS> C:\examples\example-4.ps1
OperationDescription OperationId OperationStatus
-------------------- ----------- ---------------
Update-AzureVM fa6a525b-c411-c213-8f57-69dc2a09df1c Succeeded The value of the –ConfigurationDataPath parameter on line 8 of C:\examples\example-4.ps1 is the path to a local .psd1 file containing the configuration data. A copy of this file will be uploaded to Azure blob storage and then downloaded to the VM by the PowerShell DSC Extension and passed along to the FourthCoffee configuration. This file is uploaded to the default container (“windows-powershell-dsc”) and storage account; similarly to the Publish-AzureVmDscConfiguration cmdlet, the Set-AzureVMDscExtension cmdlet includes parameters –ContainerName and –StorageContext that can be used to override those defaults.
ACQUIRING REMOTE ACCESS TO OUR VM
Since we know the name of your VM we can simply use Azure SDK cmdlets to get RDP file and kick off a remote access to it. See Azure Powershell SDK documentation for more information.
001 002 003 004 005 006 | $vm = Get-AzureVM–ServiceName "example-1-svc" –Name "example-1" $rdp = Get-AzureEndpoint -Name "RDP" -VM $vm $hostdns = (New-Object "System.Uri" $vm.DNSName).Authority $port = $rdp.Port Start-Process "mstsc" -ArgumentList "/V:$hostdns`:$port /w:1024 /h:768" |
READING LOGS
Let us say that I wish to check in detail that everything went well on my VM. How would I do that? I can log in to my VM from Azure and check the local logs. The files of interest to us will be the following two locations on VM hard drive:
C:\Packages\Plugins\Microsoft.Powershell.DSC\1.0.0.0
C:\WindowsAzure\Logs\Plugins\Microsoft.Powershell.DSC\1.0.0.0
You may find that your VM has a newer version of Powershell DSC extension,in which case the version number at the end of the path might be slightly different.
“C:\Packages\Plugins\Microsoft.Powershell.DSC\1.0.0.0” contains the actual extension files. You generally don’t need to worry about this location. However, if an extension failed to install for some reason and this folder isn’t present, that is a critical issue.
Now let’s start digging into the logs: C:\WindowsAzure\Logs. This folder contains general Azure logs that were captured for us. If for some reason DSC extension failed to deploy or there was some general infrastructure error, it would appear here under log files “WaAppAgent.*.log”
The lines of interest in these files are as follows. Note that your log may look slightly different.
- [00000003] [07/28/2014 23:57:33.02] [INFO] Beginning installation of plugin Microsoft.Powershell.DSC.
- [00000003] [07/28/2014 23:59:47.25] [INFO] Successfully installed plugin Microsoft.Powershell.DSC.
- [00000009] [07/29/2014 00:02:51.02] [INFO] Successfully enabled plugin Microsoft.Powershell.DSC.
- [00000009] [07/29/2014 00:02:51.03] [INFO] Setting the install state of the handler Microsoft.Powershell.DSC_1.0.0.0 to Enabled
Now we know that DSC extension was successfully installed and enabled. We continue our analysis by going to DSC extension logs. “C:\WindowsAzure\Logs\Plugins\Microsoft.Powershell.DSC\1.0.0.0” contains various logs from DSC extension itself.
PS C:\>PS C:\WindowsAzure\Logs\Plugins\Microsoft.Powershell.DSC\1.0.0.0> dir Directory: C:\WindowsAzure\Logs\Plugins\Microsoft.Powershell.DSC\1.0.0.0 Mode LastWriteTime Length Name ----------------------------a---7/29/201412:28 AM 1613 CommandExecution.log -a--- 7/28/2014 11:59 PM 1429 CommandExecution_0.log -a--- 7/29/2014 12:01 AM 2113 CommandExecution_1.log -a--- 7/29/2014 12:02 AM 1613 CommandExecution_2.log -a--- 7/29/2014 12:28 AM 13744 DSCBOOT_script_20140729-002759.log -a--- 7/29/2014 12:03 AM 473528 DSCLOG_metaconf__20140729-000322.json -a--- 7/29/2014 12:28 AM 713196 DSCLOG_metaconf__20140729-002823.json -a--- 7/29/2014 12:03 AM 608050 DSCLOG__20140729-000311.json -a--- 7/29/2014 12:28 AM 713196 DSCLOG__20140729-002826.json
As you can see there are a number of various logs present.
“CommandExecution*.log” are logs written by Azure infrastructure as it enabled the DSC extension.
“DSCBOOT_script*.log” is a high level log that applied our configuration that we mentioned previously. It is fairly concise. If everything went well towards the end of the log you should be able to see a line such as this:
VERBOSE: [EXAMPLE-1] Configuration application complete.
If we wish to dig deeper into DSC logs, then the rest of logs can tell us much deeper story. “DSCLOG_*.json” logs are ETL DSC logs converted to JSON format. If configuration has completed successfully you should be able to see an event like this one:
{
"EventType": 4,
"TimeCreated": "\/Date(1406593703182)\/",
"Message": "[EXAMPLE-1]: LCM: [ End Set ] in 14.1745 seconds.",
“DSCLOG_metacong*.json” are the logs for your configuration if your PowerShell DSC configuration had a meta-config that modified PowerShell DSC properties, such as this:
LocalConfigurationManager
{
ConfigurationID = "646e48cb-3082-4a12-9fd9-f71b9a562d4e"
RefreshFrequencyMins = 23
}
You would see a similar event if meta configuration was applied successfully.
{
"EventType": 4,
"TimeCreated": "\/Date(1406593703182)\/",
"Message": "[EXAMPLE-1]: LCM: [ End Set ] in 14.1745 seconds.",
More info
Here are some additional resources about PowerShell DSC, Azure VM agent and extensions:
Desired State Configuration Blog Series – Part 1, Information about DSC (by Michael Green, Senior Program Manager, Microsoft)
VM Agent and Extensions - part 1 (by Kundana Palagiri, Senior Program Manager, Windows Azure)
VM Agent and Extensions - Part 2 (by Kundana Palagiri, Senior Program Manager, Windows Azure)
Automating VM Customization tasks using Custom Script Extension (by Kundana Palagiri, Senior Program Manager, Windows Azure)