Summary: A new version of PSScriptAnalyzer is now available with many new features, rules, fixes and improvements.

You might remember me from my previous cross-platform remoting blog post, but just to introduce myself: I am Christoph Bergmeister, a full stack .Net developer in the London area and since the start of this year I am now also an official PSScriptAnalyzer maintainer although I do not work at Microsoft.
On GitHub, you can find me as @bergmeister.

After half a year, a new version of PSScriptAnalyzer (also known as PSSA) has been published and is now available on the PSGallery.
Some of you might have been wondering what has happened.
First, the former maintainer has switched projects, therefore it took some time for finding and arranging a hand over.
PSScriptAnalyzer is now mainly being maintained by @JamesWTruher from the Microsoft side and myself as a community maintainer.
After having already contributed to the PowerShell Core project, I started doing development on PSScriptAnalyzer last autumn and since then have added a lot of new features.

New Parameters

Invoke-ScriptAnalyzer now has 3 new switch parameters:

• -Fix (only on the -Path parameter set)
• -ReportSummary
• -EnableExit

The -Fix switch was the first and probably most challenging feature that I added.
Similar to how one can already get fixes for a subset of warnings (e.g. for AvoidUsingCmdletAlias) in VSCode, this feature allows to auto-fix the analysed files, which can be useful to tidy up a big code base.
When using this switch, one must still inspect the result and possibly make adaptions.
The AvoidUsingConvertToSecureStringWithPlainText rule for example will change a String to a SecureString, which means that you must create or get it in the first place.
A small warning should be given about encoding: Due to the way how the engine works, it was not possible to always conserve the encoding, therefore before checking in the changes, it is also recommended to check for a change in that in case scripts are sensitive to that.

The -ReportSummary switch was implemented first by the community member @StingyJack, thanks for that.
The idea is to see a summary, like Pester but since it writes to host, we decided to not enable it by default but rather have a switch for it to start with.
It got refined a bit later to use the same colouring for warnings/errors as currently configured in the PowerShell host.

The -EnableExit was an idea being proposed by the community member @BatmanAMA as well and the idea is to have a simpler, faster to write CI integration.
The switch will return an exit code equivalent to the number of rule violations to signal success/failure to the CI system.
Of course, it is still best practice to have a Pester test (for each file and/or rule) for it due Pester’s ability to produce result files that can be interpreted by CI systems for more detailed analysis.

New Rules

AvoidAssignmentToAutomaticVariable

PowerShell has built-in variables, also known as automatic variables.
Some of them are read-only and PowerShell would throw an error at runtime.
Therefore, the rule warns against assignment of those variables.
Some of them, like e.g. $error are very easy to assign to by mistake, especially for new users who are not aware.
In the future more automatic variables will be added to the ‘naughty’ list but since some automatic variables can be assigned to (by design), the process of determining the ones to warn against is still in process and subject to future improvement.

PossibleIncorrectUsageOfRedirectionOperator and PossibleIncorrectUsageOfAssignmentOperator

I have written those rules mainly for myself because as a C# programmer, I have to switch between different languages quite often and it happened to me and my colleagues quite often that we forgot simple syntax and were using e.g. if ($a > $b) when in fact what we meant was if ($a -gt $b) and similar for the ambiguity of the assignment operator = that can easily be used by accident instead of the equality operator that was probably intended.
Since this only applies to if/elseif/while/do-while statements, I could limit the search scope for it.
To avoid false positives, a lot of intelligent logic went into it.
For example, the rule is clever enough to know that if ($a = Get-Something) is assignment by design as this is a common coding pattern and therefore excluded from this rule.
I received some interesting feedback from the community and because PSSA does not support suppression on a per line basis at the moment, the rule offers implicit suppression in CLANG style whereby wrapping the expression in extra parenthesis tells the rule that the assignment is by design.
Thanks for this idea, which came from the community by @imfrancisd

AvoidTrailingWhiteSpace

This rule was implemented by the well known community member @dlwyatt and really does what it says on the tin.
The idea behind this was especially to prevent problems that can be caused by whitespace after the backtick.
Personally, I have the following setting in my settings.json for VSCode file that trims trailing whitespace automatically upon saving the file.

    "": {
        "files.trimTrailingWhitespace": true
    },

AvoidUsingCmdletAliases

This rule is not new but a new feature has been added:
If one types a command like e.g. ‘verb’ and PowerShell cannot find it, it will try to add a ‘Get-‘ to the beginning of it and search again.
This feature was already present in PowerShell v1 by the way.
However, although ‘service’ might work on Windows, but on Linux ‘service’ is a native binary that PowerShell would call.
Therefore it is not only the implicit aliasing that makes it dangerous to omit ‘Get-‘, but also the ambiguity on different operating systems that can cause undesired behavior.
The rule is intelligent enough to check if the native binary is present on the given OS and therefore warns when using ‘service’ on Windows only.

Miscellaneous engine improvements and fixes

A lot of fixes for thrown exception, false positives, false negatives, etc. are part of this release as well.
Some are notable:

• The PowerShell extension of VSCode uses PowerShellEditorServices, which in turn calls into PSScriptAnalyzer for displaying the warnings using squiggles and also uses its formatting capabilities (shortcut: Ctrl+K+F on the selection).
  There was one bug whereby if a comment was at the end of e.g. an if statement and the statement got changed to have the brace on the same line, the formatter placed the comment before the brace, which resulted in invalid syntax.
  This is fixed now.
  The PSUseConsistentWhiteSpace was also tweaked to take unary operators into account to have formatting that naturally looks better to humans rather than having a strict rule.
• The engine is now being built using the .Net Core SDK version 2 and targets .Net Standard 2.0 for PowerShell Core builds.
  The used referenced for the PowerShell Parser also got updated to the latest version or the corresponding reference assemblies for Windows PowerShell, which highly improved the behaviour of PSScriptAnalyzer on PowerShell 3.
• Various parsing issues existed with the -Settings parameter when it was not a string that was already resolved.
  This got fixed and should now work in any scenario.
• PSSA has a UseCompatibleCmdlet rule and command data files are now present for all versions of Windows PowerShell and even OS specific for PowerShell Core 6.0.2.
  In effect the rule allows you to get warnings when calling cmdlets that are not present in the chosen PowerShell versions.
  More improvements to analyse type usage as well is planned.
• The PSUseDeclaredVarsMoreThanAssignments rule has been a pet peeve for many in the past due to its many false positves.
  The rule received a few improvements.
  Some of its limitations (it is e.g. not aware of the scriptblock scope) are still present but overall, there should be less false positives.
• Lots of internal build and packaging improvements were made and PSScriptAnalyzer pushed the envelope as far as using AppVeyor’s Ubuntu builds, which are currently in private Beta.
  Many thanks to @IlyaFinkelshteyn for allowing us to use it and the great support.
  We are now testing against PowerShell 4, 5.1 and 6.0 (Windows and Ubuntu) in CI.
• Many community members added documentation fixes, thank you all for that!
• Parser errors are now returned as diagnostic messages
• Using ScriptAnalyzer with PowerShell Core requires at least version 6.0.2

Enjoy the new release and let us know how you find it.
PSScriptAnalyzer is also open to PRs if you want to add features or fix something.
Let me know if there are other PSScriptAnalyzer topics that you would like me to write about, such as e.g. custom rules or PSScriptAnalyzer setting files and VSCode integration.

Christopher Bergmeister
PSScriptAnalyzer Maintainer

During the May 2018 Community Call and a tweet a few weeks later, we mentioned that PowerShell team was spending significant time in the Windows codebase. We even demoed using the Active Directory PowerShell Module from PowerShell Core 6 during the PowerShell Community Call.

We started investigating some of the top requested modules that ship with Windows to see how to get them working in PowerShell Core 6. Our eventual goal is to have near 100% parity with Windows PowerShell for the in-box modules, but we expect this effort to span across multiple releases of Windows as there are many modules and PowerShell team is working with each feature team on an individual basis to port and validate their modules. The good news is that you can pick up newly compatible modules with frequent updates to Windows 10, and Windows Insiders will see them even sooner.

TL;DR; (Too Long; Didn’t Read)

Starting with Windows 10 Insiders build 17711 or newer, you can use many Windows PowerShell modules with PowerShell Core 6.1-Preview.4 and if you see an issue or difference compared to Windows PowerShell 5.1, please open an issue in the PowerShellModuleCoverage repository.

The Journey to Porting Windows PowerShell modules

Prioritizing Windows PowerShell Modules

Windows ships with many PowerShell modules. We created a prioritized list of Windows PowerShell modules to investigate compatibility and portability based on customer and partner requests.
We ended up with a list of what we thought would have maximum customer impact. For example, the top requested module was the Active Directory PowerShell Module, so we made sure to partner with the AD team and investigate their module first. After investigation, we also deferred work on certain modules that were infeasible to port to PowerShell Core given our time frame and expertise.

ApiPort

First, we looked at was using ApiPort to identify APIs used by Windows PowerShell modules that were not available in .NET Core. Depending on the API, we were able to include additional .NET Core compatible assemblies, and in others, we rewrote the code to use a different API that is functionally equivalent. In some cases, the API was simply unavailable, so we could only mark the module as incompatible by setting CompatiblePSEditions to just Desktop in the module manifest. The initial investigation showed that >95% of the APIs used by Windows PowerShell modules were available in .NET Standard (or compatible assemblies).

.NET Standard 2.0

.NET Standard is a formal specification of .NET APIs intended to be available on all .NET implementations. For example, .NET Framework 4.6.1+ used by Windows PowerShell and .NET Core 2.0+ used by PowerShell Core 6 both implement APIs specified by .NET Standard 2.0. By rebuilding existing source code for Windows PowerShell modules to target .NET Standard instead of .NET Framework, we would get compile-time errors if an API wasn’t available and shouldn’t be used.

The Windows build environment is self-contained and did not support building code targeting .NET Standard. The next work item was to work with the folks on the Windows Engineering System team to enable building managed assemblies and target .NET Standard 2.0. They were excellent partners, and together, we built a simple PowerShell module against .NET Standard 2.0 that worked in both Windows PowerShell 5.1 and PowerShell Core 6.

Windows Compatibility Pack

Although the proof of concept to use the Windows build environment to produce a .NET Standard assembly worked, actual Windows PowerShell modules used many APIs beyond just what is in the formal .NET Standard 2.0 specification (and associated reference assembly). To help developers move from .NET Framework to .NET Core, the .NET team published a set of assemblies called the Windows Compatibility Pack as an extension to .NET Standard. The Windows Compatibility Pack brought back important namespaces like System.DirectoryServices, that Windows PowerShell modules required.

To make this work with PowerShell Core 6, we added the Windows Compatibility Pack 2.0.0 assemblies into PowerShell Core 6.1. This also means that any module author who wants to leverage the Windows Compatibility Pack will not need to redistribute the Windows Compatibility Pack assemblies with their modules, as they are now built into PowerShell Core 6.1 (.NET Framework would already have its own implementations of the same APIs).

Trouble in Paradise

Things were moving along getting the infrastructure ready for the porting work. However, we were also up against a strict timeline. We made a tactical decision to minimize risk of not getting our changes into Windows by moving away from targeting .NET Standard 2.0 and Windows Compatibility Pack explicitly. Building against .NET Standard 2.0 and relying on Windows Compatibility Pack meant we had to ship the Windows Compatibility Pack facade assemblies in Windows so that .NET Framework knew how to find the actual implementation assembly to work correctly. Adding new assemblies in the Windows build environment is easy compared to adding new assemblies into Windows itself. Adding new binaries to Windows require making changes to specific files in the code base and you can only validate it worked successfully when you have a complete build of Windows. Windows code base is huge and takes a long time to build on a developer machine. A complete build may not be successful due to other changes happening in the overall code base which is what happened to us.

In addition, we also hit issues where other assemblies in the Windows code base had a dependency on the Windows PowerShell module assembly (like a graphical user interface sitting on top of PowerShell) and would break the build since the GUI code built with .NET Framework didn’t know how to reference a .NET Standard built assembly.

Since we could not successfully get a working build of Windows with our assembly additions, we decided to abandon that work and go without .NET Standard compilation.

Moving Forward

Because .NET Standard is an API contract (like a C/C++ header file), we did not need to formally rebuild Windows PowerShell modules with .NET Standard to have them work in .NET Core and thus PowerShell Core 6. What .NET Standard gives us is some assurance that future additions to Windows PowerShell modules would catch new API usage that is not part of .NET Standard or Windows Compatibility Pack. However, although .NET Standard indicates that an API is available, we found out during testing that even if the API exists, it may throw PlatformNotSupportedException at runtime which prevents the Windows PowerShell module from working correctly in PowerShell Core 6. As we found these, we’ve had to make code changes in the modules to handle these cases appropriately during runtime whether it’s running within Windows PowerShell or PowerShell Core 6.

Changes in PowerShell Core 6.1

In anticipation of the changes we were making in Windows PowerShell modules, we also needed changes in PowerShell Core 6.1 to make the experience seamless. In mid-June we published an RFC to get community feedback on the user experience we would enable for using compatible Windows PowerShell modules from PowerShell Core 6.

Basically, with PowerShell Core 6.1-Preview.4 the PowerShell engine will look in $env:WinDir\System32\WindowsPowerShell\v1.0\Modules as part of enumerating available modules from $env:PSModulePath. However, by default, only Windows PowerShell module manifests that explicitly indicate Core as a supported in the CompatiblePSEditions property. Windows PowerShell module manifests that don’t have the CompatiblePSEditions property or only has Desktop will not show up as an available module by default. The -SkipEditionCheck switch on Import-Module can be used to override this behavior. It is important to reiterate that this logic only applies to modules in $env:WinDir\System32\WindowsPowerShell\v1.0\Modules path. If you used Install-Module to install a module from the PowerShell Gallery and its module manifest either doesn’t contain CompatiblePSEditions nor includes Core, the PSEdition check logic doesn’t apply and that module will show up as available since it was explicitly installed from PowerShell Core 6.

Availability of changes in Windows

The Windows PowerShell module changes we’ve made will start showing up in Windows 10 Insiders Build 17711. This means that with that build of Windows 10 and PowerShell Core 6.1-Preview.4 you can start using some of the Windows PowerShell modules we’ve identified as Core compatible. A reminder that as we continue to make changes in Windows, they will show up in newer builds of Windows 10 Insiders. Our target is to get >65% of the Windows PowerShell in-box modules compatible with PowerShell Core 6.1 within the next Windows 10 release. We’ll continue to work with Windows partner teams to get the number of compatible modules closer to 100%.

Community Feedback

We work with individual Windows teams who are the owners of their modules to validate our changes as much as possible with PowerShell Core 6. However, due to how they wrote their tests, it’s not as simple as rerunning their existing tests within PowerShell Core 6 and in many cases, we opted to do some limited manual validation. This means that although we’ve made some effort to validate Windows PowerShell modules marked as compatible with Core, we cannot guarantee they are fully compatible (see the note about runtime issues above) and we would treat them as bugs.

Here the community can help start using Windows PowerShell modules in PowerShell Core 6.1-Preview.4 and if you see an issue or difference compared to Windows PowerShell 5.1, please open an issue in the PowerShellModuleCoverage repository. We are only using that repository for issue tracking. If someone else has opened an issue on a module (or identified a module they need but isn’t ported yet), please give it a ?? (thumbs up) to up-vote the issue which helps us prioritize the work.

Note that if there are other Windows PowerShell modules by Microsoft that aren’t in-box in Windows that you think is a priority to port, please open an issue in that repo! PowerShell Core compatibility will help us with engagements with those partner teams if we can show them the demand for it.

Windows Compatibility Module

As we continue this journey porting Windows PowerShell modules which I expect to take multiple Windows releases, you can use any Windows PowerShell module from PowerShell Core 6 without waiting for a port. A significant majority of the Windows codebase is written in C/C++ (95+%!). A number of teams who are primarily native code developers wrote their PowerShell modules in Managed C++. However, there are currently no plans to support Managed C++ in .NET Core. For these sets of modules and those waiting to be ported, we published the Windows Compatibility module which leverages implicit remoting from PowerShell Core 6 to Windows PowerShell, so it makes the cmdlets from that module available in PowerShell Core 6 but executes it within Windows PowerShell.

You can use this today with the stable release of PowerShell Core 6.0.3 and doesn’t require using PowerShell Core 6.1-Preview builds. If you find issues with this module, please open issues in the Windows Compatibility repo.

Summary

The initial goal of the PowerShell Core 6.0 release was to provide a cross-platform automation platform. I expect by end of July, we should hit a new high of 3 million startups of PowerShell Core 6!
Approximately 80% of the usage is on Linux, so I consider us working with the community to have achieved our first goal.

The goal of PowerShell Core 6.1 release (targeting end of August for General Availability) is to help move Windows PowerShell users forward to PowerShell Core 6 by providing compatibility with in-box Windows PowerShell modules that they depend upon. It is important to understand that the Windows PowerShell module porting work won’t be complete by the time PowerShell Core 6.1 GA nor when the next version of Windows 10 is released, and we expect to continue this work as needed to eventually get complete coverage.

Thanks to all the feedback and contributions from the community helping PowerShell move forward!

Steve Lee
Principal Software Engineering Manager
PowerShell Team
Azure Compute
https://twitter.com/Steve_MSFT

At the DEFCON security conference last year, we presented the session: “Get $pwnd: Attacking Battle Hardened Windows Server“.

In this talk, we went through some of the incredibly powerful ways that administrators can secure their high-value systems (for example, Just Enough Administration) and also dove into some of the mistakes that administrators sometimes make when exposing their PowerShell code to an attacker. The most common form of mistake is script injection, where a script author takes a parameter value (supplied by an attacker) and runs it in a trusted context (such as a function exposed in a Just Enough Administration endpoint). Here’s an example:

There are many coding patterns that can introduce security flaws like this, all of which have secure alternatives. The presentation goes into these in great detail, and what we also promised to release is a tool to help you detect them as you are writing the scripts. We’ve now released this tool, and you can download it from the PowerShell Gallery:

Using it this way from the command line is an excellent way to automate security analysis during builds, continuous integration processes, deployments, and more.

Wouldn’t it be REALLY cool if you could detect these dangers while writing your scripts? I’m glad you asked!

PowerShell’s Visual Studio Code plugin already does live script analysis to help you discover issues like unassigned variables, and we can customize that rule set to include InjectionHunter capabilities. Here’s what Visual Studio Code looks like with this running:

Here’s how to get this on your system:

First, find out the location of the InjectionHunter module. You can do this by typing:

Get-Module InjectionHunter -List | Foreach-Object Path

On my system, this returns:

D:\Lee\WindowsPowerShell\Modules\InjectionHunter\1.0.0\InjectionHunter.psd1

Next, create a file – ‘PSScriptAnalyzerSettings.psd1’ in a location of your choice. Use the following for the content – replacing the path to InjectionHunter with the one on your system.

This is the first of a series of blog posts that will help you take advantage of a new NuGet package PowerShellStandard Library 5.1.0. This package allows developers to create modules that are portable between Windows PowerShell 5.1 and PowerShell Core 6.0. This means that you can create PowerShell modules that run on Windows, Linux, and macOS with a single binary!

The version of PowerShell Standard Library indicates the lowest version of PowerShell that it is compatible with. The community promise is that it is always forward compatible. So a module built against PowerShell Standard Library v3 is compatible with Windows PowerShell v3, v4, v5.1, PowerShell Core 6, and the upcoming PowerShell Core 6.1. Compatibility is achieved by providing a subset of the APIs common across all those versions of PowerShell. This reference assembly is the equivalent to a header file for C/C++ where it has the APIs defined, but no implementation. During runtime, the module would use the version of System.Management.Automation.dll that is used by the PowerShell host.

Creating a PowerShell Module

In this post, I’ll walk through the steps for creating a simple C# module with a single cmdlet. I will also be using the DotNet CLI tools for creating everything I need.

Installing the PowerShell Standard Module Template

First, we can leverage a new template that we published for DotNet CLI, but we need to install it first:

PS> dotnet new -i Microsoft.PowerShell.Standard.Module.Template
  Restoring packages for C:\Users\James\.templateengine\dotnetcli\v2.1.302\scratch\restore.csproj...
  Installing Microsoft.PowerShell.Standard.Module.Template 0.1.3.
  Generating MSBuild file C:\Users\James\.templateengine\dotnetcli\v2.1.302\scratch\obj\restore.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\James\.templateengine\dotnetcli\v2.1.302\scratch\obj\restore.csproj.nuget.g.targets.
  Restore completed in 1.66 sec for C:\Users\James\.templateengine\dotnetcli\v2.1.302\scratch\restore.csproj.

Usage: new [options]

Options:
  -h, --help          Displays help for this command.
  -l, --list          Lists templates containing the specified name. If no name is specified, lists all templates.
  -n, --name          The name for the output being created. If no name is specified, the name of the current directory is used.
  -o, --output        Location to place the generated output.
  -i, --install       Installs a source or a template pack.
  -u, --uninstall     Uninstalls a source or a template pack.
  --nuget-source      Specifies a NuGet source to use during install.
  --type              Filters templates based on available types. Predefined values are "project", "item" or "other".
  --force             Forces content to be generated even if it would change existing files.
  -lang, --language   Filters templates based on language and specifies the language of the template to create.


Templates                                         Short Name         Language          Tags                             
----------------------------------------------------------------------------------------------------------------------------
Console Application                               console            [C#], F#, VB      Common/Console                   
Class library                                     classlib           [C#], F#, VB      Common/Library                   
PowerShell Standard Module                        psmodule           [C#]              Library/PowerShell/Module    
...

A new template called psmodule is now available making it easy to start a new C# based PowerShell module. Any issues, feedback, or suggestions for this template should be opened in the PowerShell Standard repo.

Creating a new project

We need to create a location for our new project and then use the template to create the project:

PS> mkdir myModule
Directory: C:\Users\James
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 8/3/2018 2:41 PM myModule
PS> cd myModule
PS C:\Users\James\myModule> dotnet new psmodule
The template "PowerShell Standard Module" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on C:\Users\James\myModule\myModule.csproj...
  Restoring packages for C:\Users\James\myModule\myModule.csproj...
  Installing PowerShellStandard.Library 5.1.0-preview-06.
  Generating MSBuild file C:\Users\James\myModule\obj\myModule.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\James\myModule\obj\myModule.csproj.nuget.g.targets.
  Restore completed in 1.76 sec for C:\Users\James\myModule\myModule.csproj.

Restore succeeded.

You can see that the dotnet cli has created a source file and .csproj file for my project:

PS C:\Users\James\myModule> dir


    Directory: C:\Users\James\myModule


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----         8/3/2018   1:48 PM                obj
-a----         8/3/2018   1:48 PM            376 myModule.csproj
-a----         8/3/2018   1:48 PM           1698 TestSampleCmdletCommand.cs

The sample from the template demonstrates a simple cmdlet with two parameters that outputs results with a custom class.

Building the module

Building the sample is code is easy with DotNet CLI:

PS C:\Users\James\myModule> dotnet build
Microsoft (R) Build Engine version 15.7.179.6572 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 76.85 ms for C:\Users\James\myModule\myModule.csproj.
  myModule -> C:\Users\James\myModule\bin\Debug\netstandard2.0\myModule.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:05.40

Testing the built module

To test this sample module, we just need to import it. We can check to see what it supports and try running it:

PS C:\Users\James\myModule> ipmo .\bin\Debug\netstandard2.0\myModule.dll
PS C:\Users\James\myModule> Test-SampleCmdlet -?

NAME
    Test-SampleCmdlet

SYNTAX
    Test-SampleCmdlet [-FavoriteNumber] <int> [[-FavoritePet] {Cat | Dog | Horse}] [<CommonParameters>]


ALIASES
    None


REMARKS
    None



PS C:\Users\James\myModule> Test-SampleCmdlet -FavoriteNumber 7 -FavoritePet Cat

FavoriteNumber FavoritePet
-------------- -----------
             7 Cat

This sample is pretty simple as it’s intended to just show how to get started on writing a PowerShell module from scratch. The important point is that using PowerShell Standard Library, this assembly can be used in both PowerShell Core 6 as well as Windows PowerShell. This sample will even work on Windows, Linux, or macOS without any changes.

In the next part of this series, I’ll cover other aspects of PowerShell module authoring such as module manifests and writing Pester tests.

James Truher
Senior Software Engineer
PowerShell Team

The PowerShell Gallery and PowerShellGet have just been updated to provide new features, performance improvements, and a new modern design.  

NOTE: This post has important information for publishers in the “Accounts and publishing” section. 

PowerShell Gallery Home Page

The PowerShell Gallery is the place to find PowerShell code that is shared by the community, Microsoft, and other companies. The site has averaged over 21 million downloads per month for the past 6 months, and has more than 3,800 unique packages available for use. It’s amazing when we consider we were handling just under 4 million downloads in July 2017. We clearly needed to invest in the PowerShell Gallery to support that kind of growth.

We have been working for some time to improve the performance of the PowerShell Gallery. The result is now available to everyone, and includes new features, performance enhancements, security improvements to accounts and publishing keys, and better alignment with the NuGet.org codebase that we rely on for our service and cmdlets.

New features and performance enhancements

Most users should see an improvement in package download speeds from the PowerShell Gallery. The new release takes advantage of CDN to provide faster downloads, particularly for those outside the United States. This should be most noticeable when installing a module with many dependencies.  

The new updates include things users have requested for a long time, including:

• A manual download option from the PowerShell Gallery. It cannot replace install-module / install-script, but does solve some specific issues for those with private repositories or older versions of PowerShell.
• A change to Install-Module and Install-Script to simply install to the current user scope when not running in an elevated PowerShell session.

The new user experience is more than just a face-lift, as providing a modern UI also improves the performance. The PowerShell Gallery pages now display only the most critical information initially, and move the details to expanding sections in the UI. This makes the pages faster and easier for users to find the content they want to see.

Accounts and publishing improvements

The changes with the most immediate impact in this release are for publishers and users with PowerShell Gallery accounts.   

Most important: Publishers must update to PowerShellGet module version 1.6.8 or higher to publish to the PowerShell Gallery. Older versions of PowerShellGet are fine for find, save, install, and update functions, but not for publishing.    

The PowerShell Gallery implemented several security best practices:

• New API keys you create will have an expiration that ranges from 1 to 365 days.
• We will not show the value of an API key in the UI, and the value must be copied immediately after creating or regenerating it.
• Multiple API keys can be created, and defined for specific uses – such as only being available to publish packages with specific names.
• Your existing API key will still work, and will be listed as a “Full access API key”. However, you will not be able to view the current API key value or refresh it. If you lose the key value, you will need to create a new key that has an expiration date.

These changes are explained in more detail in the PowerShell Gallery documentation, and are the most significant changes included in this release.

Account management in the Powershell Gallery is also improved, and adds support for

• Two factor authentication for accessing the PowerShell Gallery account. This is a security best practice and is highly recommended.
• Changing the email address or login account associated with their PowerShell Gallery ID

You can find out more about the new Account settings features here.

Aligning with NuGet

The previous versions of PowerShell Gallery and PowerShellGet were based on older versions of NuGet. With this change we are aligning much more closely with the current state of the NuGet server and client. Many of the changes listed above – including the account and API key management – came directly from the NuGet updates. Another feature NuGet implemented is the ability to delete a package they have published accidentally, within the first hour after publishing.

 As we move closer to alignment with how NuGet.org works, we expect to provide new features that are available from the NuGet team.  Other changes we are considering that are available today at NuGet.org include support for namespaces and organizational accounts.

Let us know what you think

If you have any feedback on the changes we have made, or future changes we should consider, please do let us know. Visit https://aka.ms/PowerShellGalleryIssues to review what others are saying, or to let us know of other things we should be looking into.

2018 has been the most active year ever for the DSC community. The DSC team is taking on major new areas of work in Azure, and we have made significant progress in development of the new DSC platform.

In this Planning Update for DSC, I want to cover these topics in detail and share major changes to the timing of our Open Source plans for the DSC platform.

Increase in DSC Community contributions

Since January 1st, across modules that are linked from the DscResources repo, over 475 Pull Requests have been merged to accept community contributions and over 300 issues have been closed. The number of DSC Resources in the PowerShell Gallery almost doubled over the course of the summer. We now have over 1300! You can verify this by running the following command.

(find-dscresource).count

This represents a new level of activity from the DSC community. Thank you to all of the community contributors and maintainers who made this possible. I would encourage you to visit the Maintainers list and pass on gratitude via Twitter if their work is making you successful.

I would especially like to thank Johan Ljunggren who has been assisting the PowerShell team with this effort and providing us with thoughts and guidance on how we can continue supporting the community effort.

If you are interested in getting involved but not sure where to start, you are welcome to joining the monthly Community Call for DSC.

Expanding the use of DSC in Azure

As part of joining the Azure Compute organization, the DSC engineering team has taken on significant new responsibilities. This is a really positive and healthy journey for the DSC platform. Much of this work is happening “behind the scenes” but I am optimistic that these investments in DSC at cloud scale will result in a more optimized platform in the future.

New Azure State Configuration interface generally available

Earlier this year we received feedback that when managing many thousands VM’s from Azure Automation, the interface was not optimized for finding and using information. As a result we have completely revised the experience in the Azure portal. The previous items in the Table of Contents including Nodes, Configurations, and Node Configurations, have been consolidated in to a single tabbed view that pages. The information is interactive and can be navigated by right-clicking rows to filter for correlation such as other nodes using the same configuration. The new interface is now generally available and enabled by default.

You will also find a new authoring experience from the Configurations tab. From here you can select among composite resources, either built-in or any that you have added as modules in Azure Automation. This is designed to help new users of DSC understand configuration authoring without the need to start from a ‘blank page’ in an editor. As always, configurations generated by this experience can be saved locally and added to source control.

Starting this week, we now have a preview that extends this experience to the Virtual Machine page in Azure.  You will find a new item, “State configuration (Preview)” listed under “Operations”. This makes it much easier to onboard to the DSC solution. You will find features like the new reporting and authoring experiences are also available from this single-VM view. The intent of this effort is to make it easier for anyone new to get started with configuration as code.  Note that based on feedback we will soon be renaming this item on the VM table of contents to “Configuration management”.

You can learn more in the following video:

Azure Fridays – Azure State Configuration experience

Thank you to everyone that has provided feedback and helped us to improve. If you have thoughts and opinions to share please create a new item in on our feedback page.  You can also mention @powershell_team on Twitter or post to the DSC forum on PowerShell.org.

Improving integration between Azure Automation and Azure DevOps

Very soon you will see tasks available in the Visual Studio marketplace to provide integration with Azure Automation. This includes tasks for Build and Release Management phases to publish runbooks, modules, and configurations to an Azure Automation account, and to assign configurations to registered nodes.

This is the first of multiple steps we are planning to reduce the complexity of getting started with release pipeline (CI/CD) scenarios.

Guest Configuration feature coming to Azure Policy

Very soon we will be releasing a new feature for Azure Policy that will help people to audit settings inside Linux and Windows servers. Much like the services for Inventory, Change Tracking, Update Management, and Security, this new capability includes features built on the DSC v2 platform.

I have heard the following question times:

I am using Group Policy to manage Windows Server in my private datacenter. Now I have application owners deploying to servers in the cloud. If the servers are not domain members and they are re-deployed frequently, what does Microsoft recommend I use to verify the servers meet our security requirements?”

If you are facing this challenge, Azure Policy Guest Configuration is the path to success.

The new feature will focus on auditing servers after they are deployed. To deploy servers with the correct operational and security settings, customers can use their existing configuration management tools such as DSC, Chef, Puppet, Ansible, Salt, etc.

Over the next couple of months we will be providing more information. I look forward to hearing feedback to help shape future work in this area.

New features for DSC coming in Server 2019

Some of the top requests for DSC will ship as new features in Server 2019.

• Servers registered to Azure Automation State Configuration will automatically rotate certificates without requiring re-registration
• To further increase security, during the process of applying configurations no temporary files will be written to disk -A new mode will be available to monitor DSC configurations without applying settings (the full list of modes will be MonitorOnly, ApplyOnly, ApplyandMonitor, ApplyandAutoCorrect)
• ESENT database settings in Windows Pull Server will be configurable including max length of download file
• To further increase security when Device Guard is enabled in Windows, DSC will no longer allow the built-in Script resource (MSFT_ScriptResource)
• Windows Pull Server will support using SQL as a database

Unfortunately these will not be back-ported to previous versions of DSC that shipped with Windows. We are taking this in to consideration for future releases of the DSC platform.

Security update for DSC in Windows

As part of the September 2018 updates for Windows (server and workstation) we have released an update that applies to all existing versions of DSC in Windows.  This addresses a concern where in rare situations the configuration would remain in the system temp folder after it is applied. The risk is mitigated by multiple factors.

Once this update has been applied, DSC will no longer write any temp files during the process of applying a configuration.  The update is included in KB4457128.

Changes to timing for next DSC platform to be Open Source

At the PowerShell and DevOps Global Summit and PowerShell Conference EU, I presented on future plans for the DSC platform including our intention to publish a new version of DSC as an Open Source project. At the time I hoped that the project would be published by late summer. I also said that we would not begin the project until we could be responsible project maintainers, and that I am not interested in simply “sharing code”. I insist that when we make DSC a public project we must be ready to manage incoming contributions from the community to show respect for their time and effort in helping us to make the solution the best it can be.

Soon after those events our team took on additional responsibility. We determined it would not be feasible for engineering to meet these new commitments if, during the same time, we delivered on our plans to maintain a new Open Source project. So, we had to change our plans.

We still expect to make the new DSC platform an Open Source project. The codebase is already in use across services in Azure and will be part of new services we introduce this year. Our long term goal is to have a single DSC platform in use everywhere. Our plans for features of the next generation of the DSC platform were given in the last planning update post.

Just to make a clear distinction, this new DSC platform is where we are making future investments and we want it to be available for configuring both Windows and Linux in the future. We will also continue to support the existing version of DSC following the Windows Server support lifecycle.

Unfortunately I am not ready to set a new date on when the project can become public. To set a date now would be premature and would risk needing to make changes again in the future. In the meantime, I hope to publish RFC (Request for Comments) and continue our commitment to the DSC open source community for Resources, Configurations, and Tooling. As soon as our engineering schedule allows us to maintain the project as a public repo, I will follow up here with more information.

Thank you!
Michael Greene
Principal Program Manager
Azure Security & Management Services
@migreene

The Windows Compatibility module (WindowsCompatibility) is a PowerShell module that lets PowerShell Core 6 scripts access Windows PowerShell modules that are not yet natively available on PowerShell Core. (Note: the list of unavailable commands is getting smaller with each new release of PowerShell Core. This module is just for things aren’t natively supported yet.)

You can install the module from the PowerShell Gallery using the command

Install-Module WindowsCompatibility

and the source code is available on GitHub. (This is where you should open issues or make suggestions.)

Once you have WindowsCompatibility installed, you can start using it. The first thing you might want to run is Get-WinModule which will show you the list of available modules. From that list, choose a module, say PKI and and load it. To do this, run the following command:

Import-WinModule PKI

and you’ll have the commands exported by the PKI module in your local session. You can run them just like any other command. For example:

New-SelfSignedCertificate -DnsName localhost

As always, you can see what a module exported by doing:

Get-Command -module PKI

just like any other module.

These are the most important commands but the WindowsCompatibility module provides some others:

• Invoke-WinCommand allows you to invokes a one-time command in the compatibility session.
• Add-WinFunction allows you to define new functions that operate implicitly in the compatibility session.
• Compare-WinModule lets you compare what you have against what’s available.
• Copy-WinModule will let you copy Window PowerShell modules that are known to work in PowerShell 6 to the PowerShell 6 command path.
• Initialize-WinSession gives you more control on where and how the compatibility session is created. For example. it will allow you to place the compatibility session on another machine.

(See the module’s command help for more details and examples on how to use the WindowsCompatibility functions.)

How It Works

The WindowsCompatibility module takes advantage of the ‘Implicit Remoting‘ feature that has been available in PowerShell since version 2. Implicit remoting works by retrieving command metadata from a remote session and synthesizing proxy functions in the local session. When you call one of these proxy function, it takes all of the parameters passed to it and forwards them to the real command in the “remote” session. Wait a minute you may be thinking – what does remoting have to do with the WindowsCompatibility module? WindowsCompatibility automatically creates and manages a ‘local remote’ session, called the ‘compatibility session’ that runs with Windows PowerShell on the local machine. It imports the specified module and then creates local proxy functions for all of commands defined in that module.

OK – what about modules that exist in both Windows PowerShell and PowerShell core? Yes – you can import them. After all, there are still a fair number of base cmdlets that aren’t available in PowerShell core yet.

So how does this work? WindowsCompatibility is very careful to not overwrite native PowerShell core commands. It only imports the ones that are available with Windows PowerShell but not with PowerShell Core. For example, the following will import the PowerShell default management module

 Import-WinModule  Microsoft.PowerShell.Management

which contains, among others, the Get-EventLog cmdlet. None of the native PowerShell Core cmdlets get overwritten but now you have Get-EventLog available in your session.

At this point, if you call Get-Module, you will see something a bit strange:

Get-Module | ForEach-Object Name

results in output that looks like:

Microsoft.PowerShell.Management
Microsoft.PowerShell.Management.WinModule
Microsoft.PowerShell.Utility
NetTCPIP

Import-WinModule renames the compatibility module at load time to prevent collisions with identically named modules. This is so the module qualified commands will resolve against the current module. In fact, if you want to see what additional commands were imported, you can run:

Get-Command -Module  Microsoft.PowerShell.Management.WinModule

Limitations

Because WindowsCompatibility is based on implicit remoting, there are a number of significant limitations on the cmdlets imported by the module. First, because everything is done using the remoting protocol, the imported cmdlets will return deserialized objects that only contain properties. Much of the time, this won’t matter because the parameter binder binds by property name rather than by object type. As long as the required properties are present on the object, it doesn’t matter what type the object actually is. There are, however, cases where the cmdlet actually requires that the object be of a specific type or that it have methods. WindowsCompatibility won’t work for these cmdlets.

Windows Forms and other graphical tools

The remoting session is considered non-interactive so graphical tools such as notepad or Winforms scripts will either fail, or worse hang.

Linux and Mac support

This module depends on WinRM and the client libraries on these platforms are known to be unstable and limited. So for this release, only PowerShell Core running on Windows is supported. (This may change in the future. But you’ll still need a Windows machine with Windows PowerShell to host the compatibility session.)

PowerShell 6.1 Dependency

WindowsCompatibility depends on a feature introduced in PowerShell Core 6.1 for keeping the current working directory in both the local and compatibility sessions synchronized. Earlier versions of PowerShell will work with WindowsCompatibility but won’t have this directory synchronization feature. So if you’re running PowerShell Core 6.0, import a command that writes to files, do Set-Location to a new directory, then use that command to write to a file with an unqualified path; it will use the original path from when the module was imported rather than your sessions current working directory. On PowerShell Core 6.1, it will correctly use the current working directory.

Summary

To sum it all up, the WindowsCompatibility module provides a set of commands that allow you to access Window PowerShell modules from PowerShell Core 6. There are however, some limitations that make it unsuitable for all scenarios. Over time, as more and more modules are ported to .NET Core/PowerShell 6 natively there will be less need for this module.

Cheers!
Bruce Payette,
PowerShell Team.

On January 16, 2019 at 5:00PM PDT, the PowerShell-Docs repositories are moving from the PowerShell
organization to the MicrosoftDocs organization in GitHub.

The tools we use to build the documentation are designed to work in the MicrosoftDocs org. Moving
the repository lets us build the foundation for future improvements in our documentation experience.

Impact of the move

During the move there may be some downtime. The affected repositories will be inaccessible during
the move process. Also, the documentation processes will be paused. After the move, we need to test
access permissions and automation scripts.

After these tasks are complete, access and operations will return to normal. GitHub automatically
redirects requests to the old repo URL to the new location.

For more information about transferring repositories in GitHub,
see About repository transfers.

• If the transferred repository has any forks, then those forks will remain associated with the
  repository after the transfer is complete.
• All Git information about commits, including contributions, are preserved.
• All of the issues and pull requests remain intact when transferring a repository.
• All links to the previous repository location are automatically redirected to the new location.

When you use git clone, git fetch, or git push on a transferred repository, these commands will
redirect to the new repository location or URL.

However, to avoid confusion, we strongly recommend updating any existing local clones to point to
the new repository URL. You can do this by using git remote on the command line:

git remote set-url origin new_url

For more information, see Changing a remote’s URL.

Which repositories are being moved?

The following repositories are being transferred:

• PowerShell/PowerShell-Docs
• PowerShell/powerShell-Docs.cs-cz
• PowerShell/powerShell-Docs.de-de
• PowerShell/powerShell-Docs.es-es
• PowerShell/powerShell-Docs.fr-fr
• PowerShell/powerShell-Docs.hu-hu
• PowerShell/powerShell-Docs.it-it
• PowerShell/powerShell-Docs.ja-jp
• PowerShell/powerShell-Docs.ko-kr
• PowerShell/powerShell-Docs.nl-nl
• PowerShell/powerShell-Docs.pl-pl
• PowerShell/powerShell-Docs.pt-br
• PowerShell/powerShell-Docs.pt-pt
• PowerShell/powerShell-Docs.ru-ru
• PowerShell/powerShell-Docs.sv-se
• PowerShell/powerShell-Docs.tr-tr
• PowerShell/powerShell-Docs.zh-cn
• PowerShell/powerShell-Docs.zh-tw

Call to action

If you have a fork that you cloned, change your remote configuration to point to the new upstream URL.

Help us make the documentation better.

• Submit issues when you find a problem in the docs.
• Suggest fixes to documentation by submitting changes through the PR process.

Sean Wheeler
Senior Content Developer for PowerShell
https://github.com/sdwheeler