2014/08/17

PowerShell - Parse this: NetStat.exe

In the last couple of months I had to parse different types of output using PowerShell and I thought it would be a cool idea to explain how this can be done. Command line tool: Netstat.exe

This small tool allows you to display active TCP connections, ports on which the computer is listening, Ethernet statistics, the IP routing table, IPv4 and IPv6 statistics.

In this example I will use the parameter '-n' which displays active TCP connections (without name resolution).








Parse this


Before we start to write code, we need to :

  1. Getting the Data: The actual information starts from the line 4, after the properties line.
  2. Define what is the separator between each properties. In this case we see that the properties are separated by one or more spaces (those spaces are also called whitespaces)
  3. Define each of the Properties. First column represents the Protocol, second column the Local Address, third...etc...



Getting the Data

In this example we see that all the data we need is below the first 4 lines... until the last line.
How to remove the first 4 lines ? How to get the total count of lines ? How to get a range of lines ?

# Store the output in the variable $data
$data = netstat -n

# Get a count of the lines in the output
$data.count

# Get the third line in the output
$data[3]

# Get a range of lines
$data[4..8]

# Get the data from the line 4 to the end
$data[4..$data.count]





Defining the separator

We have the data ready to be parse, next we need to separate the different properties.
Those properties are separated by spaces (also called whitespace characters). Regular expression (regex) can help us for this task. I'm using regexpal to show the matches.

\s
Using the parameter \s we can find any whitespace inside a string. But as you can see the regex find multiple matches and will split on each individual whitespace.

\s+
The solution to avoid the previous example is to add a + to the parameter \s. This way we can find any substring that contains more that one whitespace.

^\s+
Finally we need to get rid of the first whitespaces at the the beginning of the string, before the protocol property. If we don't do this step, the first property of our split will be empty.
The ^ sign means that we are looking at the very beginning of the line.



Let's test the split in PowerShell:

Defining your properties


Now we can loop through each lines and output our properties using the new-object Cmdlet.

FOREACH ($line in $data)
{
    
    # Remove the whitespace at the beginning on the line
    $line = $line -replace '^\s+', ''
    
    # Split on whitespaces characteres
    $line = $line -split '\s+'
    
    # Define Properties
    $properties = @{
        Protocole = $line[0]
        LocalAddress = $line[1]
        ForeignAddress = $line[2]
        State = $line[3]
    }
    
    # Output object
    New-Object -TypeName PSObject -Property $properties
}
}


And here is the Output! Magnifique! :-)






Extra step: Splitting IPAddress and Port number inside the LocalAddress and the ForeignAddress properties


If you want the ports information in a separated fields, we can simply split on the ":", this does not require a lot of additional work.



FOREACH ($line in $data)
{
    
    # Remove the whitespace at the beginning on the line
    $line = $line -replace '^\s+', ''
    
    # Split on whitespaces characteres
    $line = $line -split '\s+'
    
    # Define Properties
    $properties = @{
        Protocole = $line[0]
        LocalAddressIP = ($line[1] -split ":")[0]
        LocalAddressPort = ($line[1] -split ":")[1]
        ForeignAddressIP = ($line[2] -split ":")[0]
        ForeignAddressPort = ($line[2] -split ":")[1]
        State = $line[3]
    }
    
    # Output object
    New-Object -TypeName PSObject -Property $properties
}

Output:

Extra step: Building a function


Finally we can wrap this powershell code inside a function to create a re-usable tool.

function Get-NetStat
{
<#
.SYNOPSIS
    This function will get the output of netstat -n and parse the output
.DESCRIPTION
    This function will get the output of netstat -n and parse the output
#>
    PROCESS
    {
        # Get the output of netstat
        $data = netstat -n
        
        # Keep only the line with the data (we remove the first lines)
        $data = $data[4..$data.count]
        
        # Each line need to be splitted and get rid of unnecessary spaces
        foreach ($line in $data)
        {
            # Get rid of the first whitespaces, at the beginning of the line
            $line = $line -replace '^\s+', ''
            
            # Split each property on whitespaces block
            $line = $line -split '\s+'
            
            # Define the properties
            $properties = @{
                Protocole = $line[0]
                LocalAddressIP = ($line[1] -split ":")[0]
                LocalAddressPort = ($line[1] -split ":")[1]
                ForeignAddressIP = ($line[2] -split ":")[0]
                ForeignAddressPort = ($line[2] -split ":")[1]
                State = $line[3]
            }
            
            # Output the current line
            New-Object -TypeName PSObject -Property $properties
        }
    }
}

Download on Technet Gallery


Now...Let's say you want to know the most frequent ForeignAddress IP, the count for each IP and ... don't want to show the header....


Get-NetStat |
    Group-Object -Property ForeignAddressIP |
    Select-Object -Property Count, Name |
    Sort-Object count -Descending |
    Format-Table -AutoSize -HideTableHeaders


Thanks for reading! If you have any questions, leave a comment or send me an email at fxcat@lazywinadmin.com. I invite you to follow me on Twitter @lazywinadm / Google+ / LinkedIn. You can also follow the LazyWinAdmin Blog on Facebook Page and Google+ Page.

12 comments:

  1. Well written! One of my favorite features is the ability to enhance existing built in tools in all of the Windows environments with Windows PowerShell. This is one of the best and often forgotten features :)
    Good write up man!
    Sean

    ReplyDelete
  2. Thanks Sean! Very appreciated :-)


    Indeed powershell does a pretty good job handling those kind of tools and I thought it was a good idea to remind people.


    See you in Seattle in november!

    ReplyDelete
  3. Awesome post! Great material to get started with log parsing. What do you think is the best way to handle IPv6 addresses and ports once you loop into breaking out local and remote ports? IPv6 address typically use double colons in place of contiguous zeros; ex: IPv6 localhost = [::1]. You can probably assume the weird output once splitting on ':', lol.

    ReplyDelete
  4. Hope you don't mind the reference; I expanded and added remoting capability. https://gist.github.com/vN3rd/add54f2af5c16c1f1d56

    ReplyDelete
  5. Hey Kevin! Thanks for your comments very appreciated! Will like into it :-)
    Should not be too hard to add this condition

    ReplyDelete
  6. Excellent post and even better explaination!

    ReplyDelete
  7. I've added a bit to it to enable parsing many files gathered over time.

    https://github.com/RowdyVinson/Powershell-Tools/

    ReplyDelete
  8. Better question is when is Powershell going to add a netstat that can be Piped to Select Column1,Column2, etc... Were on version 5 now right? Should I expect it version 8 or 9?

    ReplyDelete
  9. You can use Get-NetTCPConnection

    But it's not perfect yet.

    https://technet.microsoft.com/en-us/library/hh826153.aspx

    ReplyDelete
  10. I want my cake and I want to eat it too. One command that does PID and supports Select Column. One day, one day. Very Good article. It took me 8 more months to noticed what you did last year.

    ReplyDelete