Powershell - Webcast Speeding Up PowerShell: Multithreading

Interesting Web cast next Monday about Powershell Multi threading.

Monday, February 20th, 2012 12:00 PM to 1:00 PM Central Time

Here is the Event home page

PowerShell is single-threaded by design which is why PowerShell can execute tasks only sequentially. To speed up scripts, PowerShell v2 offers background jobs which are somewhat heavyweight. In this webcast, PowerShell MVP Dr. Weltner illustrates how you can spawn lightweight threads to execute multiple things in parallel (at the same time). Unlike background jobs, this technique uses less resources and allows threads to communicate with each other. This will not only enable you to considerably speed up scripts. It will also enable you to open multiple GUI windows and keep them open while other PowerShell threads update the window content. To round things up, Weltner will quickly highlight some of the future parallel processing capabilities built into the upcoming PowerShell version 3.0 which is currently publicly available as CTP (community technology preview).

VMware Virtual Machine Template - Windows Server 2008 R2

Update: 2013/03/29

Virtual Machine Template
Virtual Machine Templates come in very handy when you want to have a repository of Virtual Machine base or images that you can easily create a virtual machine from. This saves you the time of installing/configuring the operating system, installing regular common applications and patching the system.

Here are a few references I found to create my Windows Server 2008 R2 Template on a VMware environment

Create, Install and Config

• Jason Samuel - How to Build a VMware vSphere VM Template for Windows Server 2008 R2
• VMPros Blog - VMware: My favorite Windows 2008R2 template configuration
• Jeremy Waldrop's Blog - How to Build a Windows 2008 VMware ESX VM Template
• Shawn Ross IT Blog - Creating a Windows Server 2008 R2 (x64) vSphere 4.1 VM Template
• Effin Blog - Creating an Activated Windows Server 2008 R2 VMware Template using Microsoft Sysprep
• Solori - In-the-Lab: Windows Server 2008 R2 Template for VMware
• Abiquo - VM Template Guide
• Abiquo - Generating VM Templates

Performance and Tweak

• Citrix  - CPU and Power Tweak
• Fix Slow Mouse on Windows Server 2008 Guest Os

Active Directory - How to remove SID History with Powershell ?

Source

In the United States we celebrate Thanksgiving tomorrow.  No matter where you are in the world let us all give thanks for PowerShell.  If Windows were the pumpkin pie, then PowerShell would be the whipped cream on top.  If Active Directory were the turkey, then PowerShell would be the stuffing.  If Exchange were the mashed potatoes, then PowerShell would be the gravy.  Eating and scripting are complementary passions, so we give thanks for both.
This post is part four in the "PowerShell: SID Walker, Texas Ranger" series on documenting and remediating SID history in your AD forest. Go back and read over the previous articles in the series if you haven't had a chance yet. In today's post we will look at the final step of remediating SID history:  removing the SID history data from our migrated AD objects.  Cleaning up this stale data will greatly reduce the chance of token size issues for your users.

Background

In an earlier post I explained the process for cleaning up SID history, and the last step is actually removing the data from your environment.  This is the last step.  It is crucial that you have already remediated your ACLs, swapping old SIDs for new SIDs.  The preferred method is the ADMT, and then for NAS file shares you can use the function I provided in part 2.  Please note that removing SID history may cause significant impact to your users if you have not taken the time to migrate resource ACLs.  I highly recommend reviewing the ADMT Guide to make sure you have completed all migration steps up to this point.  You have been warned.  Proceed carefully.  As a backout plan make sure you have a good system state backup from two DCs per domain.  Using VBScript to clear SID history was painful. PowerShell achieves the same end with far fewer lines of code.

Get-SIDHistory | Remove-SIDHistory

I decided to divide this function into two parts.  I wanted a dynamic AD query to retrieve users and groups by specific criteria as a way to surgically target the SID history removal.  Then I wanted to pipe the entries returned into a dedicated function for doing the actual removal.  This allows us to craft the right AD query in a safe environment before pulling the trigger on SID history.

Get-SIDHistory

When removing SID history it is best to proceed in phases by department or business unit.  You know the drill.  Start with a few users in IT, then all of IT, then company departments beginning with the least critical business impact (ie. Grounds Keeping before Purchasing).  You must send notification to your users asking them to verify that all of their resource access and drive letters are still good after the change.
Using Get-ADObject to query by name or OU is pretty straightforward, so I won't go into much detail there.  The magic of this function, though, is the -DomainName parameter.  You can specify the FQDN of the former domain for which you wish to remove SID history.  Now that is cool!  We can easily target the SID history by domain, because the SID object has a property called AccountDomainSid. We don't even have to parse it out. When we get back the SID history entries, we can filter on this attribute to target a specific domain in the SID history.
In order for this to work you have to first use Export-DomainSIDs to create the DomainSIDs.CSV file that maps the domain names to the domain SIDs.  If you don't know the old domain name, then you can use the -DomainSID switch to remove SID history for any unnamed domain.  You can copy the old domain SID from the SIDHistoryReport.CSV file produced by the function Export-SIDMapping (included in the attached code).  You could also modify the DomainSIDs.CSV file by manually adding any other domain names and SIDs from your own documentation.
Another feature of the Get-SIDHistory query is that it handles the multi-value nature of the sIDHistory attribute.  We do this using the ExpandProperty parameter of Select-Object which I demonstrated here.  This allows us to identify and target specific SID history entries when a user has been migrated more than once.  The ExpandProperty parameter ensures that we get a result row for every SID history entry regardless of the count.
Here is an example without ExpandProperty:
Get-ADUser user1 -Property sIDHistory | Select-Object name, sIDHistory | Format-Table name, sIDHistory –AutoSize
name   sidhistory
----   ----------
user1  {S-1-5-21-3013500491-1380372588-2491230877-1122, S-1-5-21-179552...
Here is an example with ExpandProperty:
Get-ADUser user1 -Property sIDHistory | Select-Object name, sIDHistory -ExpandProperty sidHistory | Format-Table name, sIDHistory –AutoSize
name   Value
----   -----
user1  S-1-5-21-3013500491-1380372588-2491230877-1122
user1  S-1-5-21-1795525639-2355993942-847153066-1695
Notice that ExpandProperty modifies the property name to "value".  We can rename that at the end of the pipe using Select-Object:
Select-Object DistinguishedName, @{name="SID";expression={$_.Value}}
Get-SIDHistory takes the parameters you specify, shapes them into an AD query, and then uses Invoke-Expression to run it.  In other words, this is code that writes code.  I love that feature of PowerShell.  The query parameters are cumulative.  Therefore you can pile on two or three criteria for laser accuracy.
Here are a few examples of how you can use Get-SIDHistory:

• Get-SIDHistory –SamAccountName ashleym
• Get-SIDHistory –DomainName wingtiptoys.com
• Get-SIDHistory –DomainName wingtiptoys.com –SamAccountName ashleym
• Get-SIDHistory –DomainSID S-1-5-21-2371126157-4032412735-3953120161
• Get-SIDHistory –ObjectClass group
• Get-SIDHistory –SearchBase "OU=Sales,DC=contoso,DC=com"
• Get-SIDHistory –MemberOf MigratedUsers
• Get-SIDHistory | Measure-Object

Experiment with different combinations of parameters until you isolate exactly the targets you need.  Then you can script out multiple statements for each phase of the remediation.

Remove-SIDHistory

This short TechNet article provides syntax for removing SID history:
Get-ADUser –filter 'sidhistory –like "*"' –searchbase "dc=name,dc=name" –searchscope subtree –properties sidhistory | foreach {Set-ADUser $_ -remove @{sidhistory=$_.sidhistory.value}}
That single line of PowerShell is a "resumé-generating-event".  DO NOT RUN THIS LINE IN YOUR ENVIRONMENT IF YOU WANT TO KEEP YOUR JOB.  Wiping SID history for every user all at once is not a best practice (in case you were wondering).
You can call Remove-SIDHistory by specifying the distinguishedName of the object and the SID entry to remove.  However, that is a lot of complicated typing.  The intention is to use Get-SIDHistory to tailor your query to exactly the results you want, and then simply pipe it to Remove-SIDHistory.  This works through a feature of PowerShell advanced functions called cmdlet binding.  The ValueFromPipelineByPropertyName argument allows us to pipe the output from one function into the next as long as the first output properties match the second input parameters. Features like this make PowerShell truly remarkable.
Get-SIDHistory –SamAccountName ashleym | Remove-SIDHistory
I recommend piping the output of the Remove-SIDHistory to a CSV file for change documentation.  Like this:
Get-SIDHistory –SamAccountName ashleym | Remove-SIDHistory | Export-CSV removed.csv
Your CSV will include a date/time stamp for each entry's removal:

This documentation will come in handy if users start to call and report issues after the removal.  But that shouldn't happen if you are thorough in scrubbing all SID history dependencies prior to removing the data.

Conclusion

If you've been putting off SID history remediation due to the mystery of how to remove it, then your puzzle is solved.  Using the functions in today's post will make the removal process swift and painless.  Let me reiterate that this is the LAST step in the migration process.  Do not proceed until all of your resource translation is complete and verified.  Phase the remediation by user groups to limit any unforeseen impact.  Following these recommendations will make the transition as smooth as possible.
So if you are celebrating Thanksgiving with family tomorrow and everyone around the table shares something they are thankful for, you can be the geek that is thankful for PowerShell.  Most likely no one else at the table will know what you're talking about.  Just smile and nod.

Coming Up Next…

In the next installment of our SID history series we'll provide a handy summary of each post in the series.  We will also take the function library we've been using and upgrade it to a PowerShell module.  Then we'll walk through the entire SID history remediation process using the provided cmdlets in this module.

SIDHistoryV1.2.zip

New Version : LazyWinAdmin 0.3c - 2011/10/02

LazyWinAdmin 0.3c - 2011/10/02

-Fixed/Improved Query/Stop/Start Service buttons
-Add Descriptions in logs RichTextBox for Query/Stop/Start Service buttons
-Changed the button :80 to HTTP
-Add FTP, TELNET, HTTPS buttons

Available on Download Page.

Figuring out which VMNIC is assigned to what physical NIC port

esxcfg-nics -l

Scaling and Queuing PowerShell Background Jobs

Great article from Travis Jones [Source]

 A couple of months ago I had asked the PowerShell MVPs for suggestions on blog topics. Karl Prosser, one of our awesome MVPs, brought up the topic of scaling and queuing background jobs.
The scenario is familiar: You have a file containing a bunch of input that you want to process and you don’t want to overburden your computer by starting up hundreds of instances of PowerShell at once to process them.
After playing around for about an hour on Friday afternoon, here is what I came up with… This example assumes you have a text file containing the names of many event logs and you want to get the content of each log.

# How many jobs we should run simultaneously

$maxConcurrentJobs = 3;

# Read the input and queue it up

$jobInput = get-content .\input.txt

$queue = [System.Collections.Queue]::Synchronized( (New-Object System.Collections.Queue) )

foreach($item in $jobInput)

{

    $queue.Enqueue($item)

}

# Function that pops input off the queue and starts a job with it

function RunJobFromQueue

{

    if( $queue.Count -gt 0)

    {

        $j = Start-Job -ScriptBlock {param($x); Get-WinEvent -LogName $x} -ArgumentList $queue.Dequeue()

        Register-ObjectEvent -InputObject $j -EventName StateChanged -Action { RunJobFromQueue; Unregister-Event $eventsubscriber.SourceIdentifier; Remove-Job $eventsubscriber.SourceIdentifier } | Out-Null

    }

}

# Start up to the max number of concurrent jobs

# Each job will take care of running the rest

for( $i = 0; $i -lt $maxConcurrentJobs; $i++ )

{

    RunJobFromQueue

}

The English version of this script is:

• Given a file input.txt containing the name of many event logs, queue up each line of input
• Kick off a small number of jobs to process one line of input each. Each job just gets the content of a particular log.
• When a job finishes (determined by the StateChanged Event), start a new job with the next piece of input from the queue
• Clean up the jobs corresponding to the event subscription so at the end we only have jobs containing event data

The “Synchronized” code you see when defining the queue is just for good measure to make sure that only one job can access it at a time.
Have something you want to see on the PowerShell blog? Leave a comment… Can’t promise we’ll get to everything but it’s nice to see what everyone is interested in.

Travis Jones
Windows PowerShell PM
Microsoft Corporation

Google+ Invitations

Here is my link for some 150 invitations to Google+

Google+ Invitations

Powershell - Recommended coding style

source
Here are some Recommended coding style for powershell by Richard Berg

<#
.SYNOPSIS
Cmdlet help is awesome.  Autogenerate via template so I never forget.

.DESCRIPTION
.PARAMETER
.PARAMETER
.INPUTS
.OUTPUTS
.EXAMPLE
.EXAMPLE
.LINK
#>
function Get-Widget
{
    [CmdletBinding()]
    param (
        # Think about which params users might loop over.  If there is a clear
        # favorite (80/20 rule), make it ValueFromPipeline and name it InputObject.
        [parameter(ValueFromPipeline=$True)]
        [alias("Server")]
        [string]$InputObject,

        # All other loop candidates are marked pipeline-able by property name.  Use Aliases to ensure the most 
        # common objects users want to feed in will "just work".
        [parameter(Mandatory=$true, Position=0, ValueFromPipelineByPropertyName=$True)]
        [alias("FullName")]
        [alias("Path")]
        [string[]]$Name,

        # Provide & document defaults for optional params whenever possible.
        [parameter(Position=1)]
        [int]$Minimum = 0,

        [parameter(Position=2)]
        [int]$ComputerName = "localhost",

        # Stick to standardized parameter names when possible.  *Especially* with switches.  Use Aliases to support 
        # domain-specific terminology and/or when you want to expose the parameter name of the .Net API you're wrapping.
        [parameter()]
        [Alias("IncludeFlibbles")]
        [switch]$All,
    )

    # The three main function blocks use this format if & only if they are short one-liners    
    begin { $buf = new-list string }

    # Otherwise they use spacing comparable to a C# method
    process    
    {
        # Likewise, control flow statements have a special style for one-liners
        try
        {
            # Side Note: internal variables (which may be inherited from a parent scope)  
            # are lowerCamelCase.  Direct parameters are UpperCamelCase.
            if ($All)
                { $flibbles = $Name | Get-Flibble }   
            elseif ($Minimum -eq 0)          
                { $flibbles = @() }
            else
                { return }                       

            $path = $Name |
                ? { $_.Length -gt $Minimum } |
                % { $InputObject.InvokeGetAPI($_, $flibbles) } |
                ConvertTo-FullPath
        }
        finally { Cleanup }

        # In general, though, control flow statements also stick to the C# style guidelines
        while($true)
        {
            Do-Something
            if ($true)
            {
                try
                {
                    Do-Something
                    Do-Something
                    $buf.Add("abc")
                }
                catch
                {
                    Do-Something
                    Do-Something
                }
            }            
        }    
    }    
}

<# 
Pipelines are a form of control flow, of course, and in my opinion the most important.  Let's go 
into more detail.

I find my code looks more consistent when I use the pipeline to nudge all of Powershell's supported 
language constructs (within reason) toward an "infix" style, regardless of their legacy origin.  At the 
same time, I get really strict about avoiding complexity within each line.  My style encourages a long,
consistent "flow" of command-to-command-to-command, so we can ensure ample whitespace while remaining
quite compact for a .Net language. 

Note - from here on out I use aliases for the most common pipeline-aware cmdlets in my stable of 
tools.  Quick extract from my "meta-script" module definition:
sal ?? Invoke-Coalescing
sal ?: Invoke-Ternary
sal im Invoke-Method
sal gpv Get-PropertyValue
sal spv Set-PropertyValue
sal tp Test-Path2
sal so Select-Object2        
sal eo Expand-Object        

% and ? are your familiar friends.
Anything else that begins with a ? is a pseudo-infix operator autogenerated from the Posh syntax reference.
#>        
function PipelineExamples
{
    # Only the very simplest pipes get to be one-liners:
    $profileInfo = dir $profile | so @{Path="fullname"; KBs={$_.length/1kb}}
    $notNull = $someString | ?? ""        
    $type = $InputObject -is [Type] | ?: $InputObject $InputObject.GetType()        
    $ComObject | spv Enabled $true
    $foo | im PrivateAPI($param1, $param2)
    if ($path | tp -Unc)
        { Do-Something }

    # Any time the LHS is a collection (i.e. we're going to loop), the pipe character ends the line, even 
    # when the expression looks simple.
    $verySlowConcat = ""            
    $buf |
        % { $verySlowConcat += $_ }
    # Always put a comment on pipelines that have uncaptured output [destined for the caller's pipeline]
    $buf |
        ? { $_ -like "*a*" }


    # Multi-line blocks inside a pipeline:
    $orders |
        ? { 
            $_.SaleDate -gt $thisQuarter -and
            ($_ | Get-Customer | Test-Profitable) -and
            $_.TastesGreat -and
            $_.LessFilling
        } |
        so Widgets |        
        % {                
            if ($ReviewCompetition)
            {
                $otherFirms |
                    Get-Factory |
                    Get-ManufactureHistory -Filter $_ |
                    so HistoryEntry.Items.Widgets                     
            }
            else
            {
                $_
            }
        } |            
        Publish-WidgetReport -Format HTML


    # Mix COM, reflection, native commands, etc seamlessly
    $flibble = Get-WmiObject SomethingReallyOpaque |
        spv AuthFlags 0xf -PassThru |
        im Put() -PassThru |
        gpv Flibbles |
        select -first 1

    # The coalescing operator is particularly well suited to this sort of thing
    $initializeMe = $OptionalParam |
        ?? $MandatoryParam.PropertyThatMightBeNullOrEmpty |
        ?? { pwd | Get-Something -Mode Expensive } |
        ?? { throw "Unable to determine your blahblah" }           
    $uncFolderPath = $someInput |
        Convert-Path -ea 0 |
        ?? $fallback { tp -Unc -Folder }

    # String manipulation        
    $myName = "First{0}   Last{1}   " |
        ?+ "Suffix{2}" |
        ?replace "{", ": {" |
        ?f {eo richard berg jr | im ToUpper}            

    # Math algorithms written in this style start to approach the elegance of functional languages
    $weightedAvg = $values |
        Linq-Zip $weights {$args[0] * $args[1]} |
        Linq-Sum |
        ?/ ($weights | Linq-Sum)
}

# Don't be afraid to define helper functions.  Thanks to the script:Name syntax, you don't have to cram them into 
# the begin{} block or anything like that.  Name, params, etc don't always need to follow the cmdlet guidelines.
# Note that variables from outer scopes are automatically available.  (even if we're in another file!)
function script:Cleanup { $buf.Clear() }

# In these small helpers where the logic is straightforward and the correct behavior well known, I occasionally 
# condense the indentation to something in between the "one liner" and "Microsoft C# guideline" styles
filter script:FixComputerName
{
    if ($ComputerName -and $_) {            
        # handle UNC paths 
        if ($_[1] -eq "\") {   
            $uncHost = ($_ -split "\\")[2]
            $_.Replace($uncHost, $ComputerName)
        } else {
            $drive = $_[0]
            $pathUnderDrive = $_.Remove(0,3)            
            "\\$ComputerName\$drive`$\$pathUnderDrive"
        }
    } else {
        $_
    }
}