1 Summary
In March we introduced Windows PowerShell Workflow (PSWF), which explained why we integrated workflows with PowerShell, and what were our major investment areas for our first release.
In a recent post, Jeffrey Snover summarizes this decision:
“We integrated the Windows Workflow Foundation engine into PowerShell to make it simple and easy to automate things that take a long time, that operate against a very large scale, or that require the coordination of multiple steps across multiple machines.”
While these posts talked about workflows as robust multi-machine commands, it didn’t address all the work we’ve done under the covers to make that happen. For the developers, architects, and highly-technical among you, here’s your chance to look under the hood. This is the first part of a two-blog series about the high-level architecture of the Windows PowerShell Workflow. The first one is a general description of the architecture; the second one will go into more details.
After these initial two blogs regarding the architecture, we will have a series of blogs that will go deeper into each area of functionality.
2 Goals
The main high-level goals and requirements that influenced the Windows PowerShell Workflow architecture are:
a. Provide a platform to minimize the complexity of managing multi-machine environments. For example, to simplify the execution of a large number of concurrent operations performed on many computers.
b. Provide ability for robust execution of tasks despite failed network connections, reboots, and system crashes.
c. Make workflows available to existing Windows PowerShell users:
- Provide support to author workflows as scripts and host them in a Windows PowerShell environment.
- Provide a built in library of management tasks.
- Provide workflow management through job cmdlets.
d. Provide a runtime that achieves good performance and a reasonable level of scalability without imposing a lot of complexity on the user.
e. Support delegated administration.
f. Provide a “one true PowerShell workflow implementation” that everyone could take advantage of, allowing people to share workflows in the same way that they currently share scripts, etc.
g. Provide extensibility points that others can use to provide workflow hosting and execution functionality.
3 Architectural Overview
This section covers the high-level architecture for Windows PowerShell Workflow and its components. We will start with the Windows PowerShell Workflow architecture when workflows are executed in-process in the client application. The client application can be the Windows PowerShell console host, Windows PowerShell Integrated Scripting Environment (ISE) process or other client executable, including 3rd party host.
Once we finished with in-process execution, we’ll move to describing the Windows PowerShell Workflow high level architecture when workflows are executed in the dedicated Windows PowerShell Workflow Configuration. Windows PowerShell Workflow Configuration is the default Windows Remote Management (WinRM) configuration for Windows PowerShell Workflow. (Look for why we created a new configuration in a future blog). WinRM is the Microsoft implementation of the WS-Management protocol. WS-Management is a general Web services protocol based on SOAP for managing systems such as PCs, servers, devices, Web services and other applications, and other manageable entities. The WinRM Configuration is a node you connect to, identified by a ConnectionURI and ConfigurationName. Sometimes, a WinRM Configuration is called WinRM Endpoint.
At the end of this section, we will describe the components of the Windows PowerShell Workflow Executive.
3.1 In-process execution of workflows
The following diagram outlines the high-level architecture of the Windows PowerShell Workflow when the workflow is executed in-process within Powershell.exe, Powershell_ise.exe or another client application executable. In the diagram, the outer boxes represent machine boundaries, dotted red lines represent process boundaries, and the smallest boxes represent conceptual components.
A workflow comprises a series of programming steps called activities. Workflows are integrated into Windows PowerShell thorough a set of extensions to the Windows PowerShell scripting language. One of these extensions is the workflow keyword. A workflow is defined by the workflow keyword followed by the name and the body of the workflow. In addition, Windows PowerShell provides a built-in library of activities.
A script workflow is a workflows written in the Windows PowerShell language.
Example of a script workflow in Windows PowerShell ISE:
#using parallel foreach
workflow Invoke-ParallelForEach
{
foreach -parallel ($i in 1..10)
{
InlineScript
{
"foo: $using:i"
}
Get-Process -Name PowerShell*
}
}
Invoke-ParallelForEach -PSComputerName localhost -AsJob
Let’s see how this script is transformed into a working workflow.
When you run a script workflow, Windows PowerShell parses the script into an abstract syntax tree (AST). The presence of the “workflow” keyword causes the script-to-workflow compiler to use this AST to generate XAML, the format required by the Windows Workflow Foundation runtime. To create the user experience for interacting with this workflow, we then create a wrapper function that has the same parameters – but instead coordinates execution of the workflow within the PowerShell Workflow executive. You can see both the wrapper function and the generated XAML by executing:
Get-Command Invoke-ParallelForEach |Format-List *
To create the wrapper function from the XAML workflow, Windows PowerShell uses .NET 4.0 ActivityXamlServices class to compile the XAML into an activity tree, which is the basis for all workflows in Windows Workflow Foundation 4.0. Then, it adds the activity tree to an internal cache so it can be re-used in subsequent workflow executions. (Our performance and scalability blog will explain why this is important.)
The activity tree is then executed by the Windows PowerShell Workflow Executive. Each workflow instance wraps a .NET 4 WorkflowApplication, which is instantiated by using the activity tree as the workflow definition.
If the workflow is executed in-process, such as is done in Windows PowerShell ISE, the Windows PowerShell Workflow Executive is hosted by the client application. The Executive is responsible for running the workflow and passivating and reactivating the workflow, as necessary. Passivation is the process of removing the workflow state from memory and saving it in a database. Reactivation is the process of loading the workflow’s state into memory and executing the workflow.
By default, most activities run in the same process as the Windows PowerShell Workflow Executive. If the Windows PowerShell Workflow Executive determines that an activity has the potential to be unreliable (based on its configuration), it will run the activity out of process in a pool of available Windows PowerShell Workflow Activity Host processes. The InlineScript activity is, by default, executed out of process in one of the Windows PowerShell Workflow Activity Host processes.
The activity can target different managed nodes and can take advantage of different remote management technologies, such as Windows PowerShell Remoting (PSRP), Common Information Model (CIM) or Windows Management Instrumentation (WMI) technologies. CIM is a remotable object model defined and published by the Distributed Management Task Force (DMTF). WMI is the Microsoft implementation of the CIM standard.
3.2 Execution of workflows within Windows PowerShell Workflow Configuration
The following diagram outlines the high-level architecture of the Windows PowerShell Workflow while the workflow is executed in the Windows PowerShell Workflow Configuration. The short version of why you would want to do this is that a process can invoke a workflow and then go away and the workflow continues to execute.
Example of a workflow executed in the Windows PowerShell Workflow Configuration:
$s = New-PSWorkflowSession
Invoke-Command $s {
workflow Invoke-ParallelForEach
{
foreach -parallel ($i in 1..10)
{
InlineScript
{
"foo: $using:i"
}
Get-Process -Name PowerShell*
}
}
}
Invoke-Command $s { Invoke-ParallelForEach -PSComputerName localhost }
In this case, the client connects to the Windows PowerShell Workflow Configuration using Windows PowerShell Remoting. Typically, workflows are installed on the machine that contains the Windows PowerShell Workflow Executive.
In the example above, the Invoke-ParallelForEach workflow is sent to the Windows PowerShell Workflow Executive by using Windows PowerShell remoting. The Windows PowerShell Workflow Executive, which is responsible for the execution of the workflow, is hosted by the WinRM provider host (wsmprovhost.exe). The WinRM service, which is the server-side implementation of the WS-Management protocol, implements a provider subsystem.
Each WinRM provider, including Windows PowerShell Workflow Executive, is hosted by the wsmprovhost.exe. Wsmprovhost.exe (the Windows PowerShell Workflow Executive) runs under the security context of the remote client. By default, all workflows in a given security context run under the same wsmprovhost.exe and are orchestrated by the same Windows PowerShell Workflow Executive. Different users run workflows in different wsmprovhost.exe instances.
If the admin creates a Windows PowerShell Workflow “RunAs” configuration, the WinRM service translates all incoming, configured user identities into one trusted security context. “RunAs” configuration is a WinRM Configuration configured for delegated administration, which runs with the permissions of a pre-defined user identity, instead of the remote user. Therefore, the workflows for all configured users for the particular “RunAs” configuration are routed to the same Windows PowerShell Workflow host and run under the same “RunAs” configured identity.
3.3 Windows PowerShell Workflow components
Now let’s break the Windows PowerShell Workflow Executive into its high level sub components:
All workflows are executed as jobs. This allows us to manage more operations at a time, which is critical for multi-machine management. They are also long-running – you don’t want to be sitting at your console for 2 days waiting for the task to complete. All operations related to jobs are handled by the Windows PowerShell job infrastructure component (“Job Infra” in the diagram above). The job infrastructure creates the job and returns a local proxy job that can be used to manage the workflow execution. The client can use the standard Windows PowerShell Job cmdlets to start, stop, suspend, resume, and remove the job. This will be illustrated in a separate blog.
The Job infrastructure interacts with “Workflow Compilation, Caching and Validation” component which is responsible for compiling the XAML into the activity tree, validating the activity tree by using the .NET 4.0 ActivityValidationServices class based on the Windows PowerShell Workflow configuration, and caching the XAML and the activity tree. More about these later.
For each job, “Workflow Execution Engine” component instantiates a WorkflowApplication using the activity tree as the workflow definition. WorkflowApplication is a component introduced in Windows Workflow Foundation (WF) 4.0 that provides a host for a single workflow instance. WorkflowApplication is basically a thread-safe proxy for the actual workflow instance managed by the WF runtime. For more details, see the WF 4.0 documentation.
Since a workflow is a series of activities, the “Workflow Execution Engine” interacts with the “Activities” component. Windows PowerShell Workflow provides a default set of PowerShell activities that any host can adopt. These activities are based on Windows PowerShell Remoting, CIM, or WMI, or are special activities, such as Pipeline, PowerShellValue, GetPSWorkflowData and SetPSWorkflowData activities. Most of these built-in activities are derived from the PSActivity base class, with the exception of the special activities specified above. When you look at the ecosystem of previous PowerShell + Workflow integrations, there’s not much of a “network effect”. If somebody makes a great PowerShell-based activity, they make their own decisions around serialization, remoting behavior, and user experience. The activities that work in one product rarely work in anyone else’s. By providing a base activity implementation and many immediately-useful PowerShell activities, we tame that inconsistency so that every new activity adds broader value.
These activities can run:
- In-process with the Workflow Execution Engine. For example, CIM activities (GetCimInstanceActivity, InvokeCimMethodActivity etc) or WMI activities (GetWmiObject, InvokeWmiMethod)
- Out-of-Process in the ActivityHost process. For example, the InlineScript activity. Activities are executed out of process to provide isolation for reliability.
- Remotely, if the PSComputerName parameter is specified
Windows PowerShell remoting activities and, to some extent, CIM activities that are executed remotely, interact with the “ConnectionManager” subcomponent. ConnectionManager is responsible for connection pooling and throttling. After all, if you’re invoking high number of workflows in parallel – the last thing you need is to be killing your own servers. The connections pool is indexed based on the machine and session configuration, and uniquely identified by the rest of remoting parameters (authentication, credentials, certificate thumbprint, proxy settings, etc.), in addition to the machine and session configuration.
If executed out of process, the activity interacts with the “Activity Controller” subcomponent. The Activity Controller is responsible for queuing all out-of-process activities and executing them in the pool of Activity Host processes. The out-of-process activities are specified by the configuration.
By default, on both Windows client and server operating system, the WorkflowApplication in the Workflow Execution Engine uses a file-based store to store the workflow definition, metadata, streams, job state. This is handled by the “Persistence” subcomponent.
Windows PowerShell Workflow is also a library that others can use to provide workflow hosting and execution functionality. The hosting SDK will be presented in a separate blog.
4 More info
While you wait for our next post, which will go deeper into the Windows PowerShell Workflow architecture, here are some additional resources about Windows PowerShell 3.0 and Windows PowerShell Workflow:
When Windows PowerShell Met Workflow (by Mir Rosenberg [MSFT], Senior Program Manager, Windows PowerShell)
Getting started with Windows PowerShell Workflow (Download WMF3 CTP2 Windows PowerShell Workflow.pdf)
PowerShell v3: Workflow is the Flagship Feature (by Don Jones – PowerShell MVP)
PowerShell Workflow, Defined (by Hal Rottenberg – PowerShell MVP)
Rocking the Windows Server 8 Administrative Experience
TechNet docs on the Workflow module
Writing a Windows PowerShell Workflow in the Visual Studio Designer
Sorin Oprea [MSFT]
Senior SDE Lead
Windows PowerShell