Download Files With PowerShell Dynamically!

Knowing PowerShell can come in handy when you need to download files. Invoke-WebRequest is the command to get to know when working with web parsing, and obtaining downloads.

I've noticed, however, that different files show up as different content types, and parsing out the file name requires all sorts of voodoo. What if there was a way to use one tool that could utilize the power of PowerShell, and make downloading files a modular experience?

This tool, and blog post, are is inspired by folks asking me for help downloading files via PowerShell. I always appreciate feedback and questions, and this is exactly why!

Prerequisites

  • PowerShell 3.0+ 
  • Access to the internet

Ninja Downloader Overview

Ninja Downloader works by executing the main script (download.ps1), which takes the following parameters:

  • DownloadName
    • This is the name of the script you'd like to execute (use 'all' to execute all scripts)
    • Scripts are located in the .\scripts directory
    • Argument must omit the .ps1 extension of the script in the .\scripts directory
  • OutputType
    • This parameter let's you specify the output type, the default is none.
      • XML -> Export results as clixml
      • CSV -> Export results as a CSV file (default)
      • HTML -> Export results as an HMTL file
      • All -> Export results as all of the above
    • Output is exported to the .\output directory
  • DownloadFolder
    • This parameter allows you to specify a location to place the downloaded files
      • Folder will be created if it does not exist
    • If left empty the folder .\downloads will be used
  • UnZip
    • This parameter will look for zip archives after files are downloaded and attempt to extract them
      • Files extracted to .\downloads\fileName_HHmm-MMddyy\
  • ListOnly
    • This parameter (a switch) will give you a list of all possible names to use for DownloadName, as well as the paths to the scripts.

Downloading a File

There are several scripts included by default with tool. 

  • Ccleaner.ps1
  • Chrome.ps1
  • FireFox.ps1
  • Java.ps1
  • Skype.ps1
  • template.ps1 (never executed, this is a template for creating your own download script)

So how do we use them, then?

  1. Open PowerShell, and navigate to the root directory of the project/script.
  2. Run the following code:
$downloadResults = .\download.ps1 -DownloadName ccleaner

This will give us access to the results in $downloadResults.

downloadresults.PNG

You can see that the results we get include the name of the script executed, if it was executed successfully, any errors, and another object inside of this object named FileInfo.

FileInfo contains the file name, path to the file (full), any errors, and if we could verify it exists.

This attempt was successful, and our results echoed that! Let's take a look in the downloads folder just to be sure...

Awesome!

Downloading All Files

What if we wanted to download all the files via every script in the .\scripts folder?

  1. Open PowerShell, and navigate to the root directory.
  2. Run the following code:
$downloadResults = .\download.ps1 -DownloadName all -Verbose

This time we can see some of the output as it happens via the -Verbose switch.

Now let's take a look at $downloadResults:

For good measure, we'll also look at the downloads folder:

Alright! It worked.

Output Types

This script allows you output the results in various ways. All of the results will be time-stamped with the date and time.

CSV

To output results as a CSV, run:

$downloadResults = .\download.ps1 -DownloadName all -OutputType csv

After it runs, the results will be in the .\output folder.

csv.PNG

HTML

To output results as HTML, run:

$downloadResults = .\download.ps1 -DownloadName all -OutputType html

After it runs, the results will be in the .\output folder.

XML

To output results as XML, run:

$downloadResults = .\download.ps1 -DownloadName all -OutputType xml

After it runs, the results will be in the .\output folder.

All

To output results in all three formats, run:

$downloadResults = .\download.ps1 -DownloadName all -OutputType all

After it runs, the results will be in the .\output folder.

Creating Your Own Download Script

You can create your own script to use with the Ninja Downloader tool. The template provided is a working example of how Firefox is downloaded. The template is located in the .\scripts folder.

Template code:

#Template example (works for Firefox, adjust as needed for your download)

#Set this to the URL you'll be navigating to first
$navUrl    = 'https://www.mozilla.org/en-US/firefox/all/' 

#Text to match on, if applicable
$matchText = '.+windows.+64.+English.+\(US\)'

# IMPORTANT: This is the format of the object needed to be returned to the description
# Whichever way you get the information, you need to return an object with the following properties:
# DownloadName (string, file name)
# Content (byte array, file contents)
# Success (boolean)
# Error (string, any error received)
$downloadInfo = [PSCustomObject]@{

    DownloadName = ''
    Content      = ''
    Success      = $false
    Error        = ''  

} 

#Go to first page
Try {

    $downloadRequest = Invoke-WebRequest -Uri $navURL -MaximumRedirection 0 -UserAgent [Microsoft.PowerShell.Commands.PSUserAgent]::FireFox -ErrorAction SilentlyContinue

}
Catch {

    $errorMessage = $_.Exception.Message

    $downloadInfo.Error = $errorMessage

    return $downloadInfo

}

#Look for urls that match
$downloadURL = $downloadRequest.Links | Where-Object {$_.Title -Match $matchText} | Select-Object -ExpandProperty href

#Go to matching URL, look for download file (keeping redirects at 0)
try {

    $downloadRequest = Invoke-WebRequest -Uri $downloadURL -MaximumRedirection 0 -UserAgent [Microsoft.PowerShell.Commands.PSUserAgent]::FireFox -ErrorAction SilentlyContinue

}
catch {
    
    $errorMessage = $_.Exception.Message

    $downloadInfo.Error = $errorMessage

    return $downloadInfo

}

#Get file info
$downloadFile = $downloadRequest.Headers.Location

#Parse file name, whichever way needed
if ($downloadRequest.Headers.Location) {
            
    $downloadInfo.DownloadName = $downloadFile.SubString($downloadFile.LastIndexOf('/')+1).Replace('%20',' ')

}  

#Switch out the StatusDescription, as applicable
Switch ($downloadRequest.StatusDescription) {

    'Found' {
                
        $downloadRequest = Invoke-WebRequest -Uri $downloadRequest.Headers.Location -UserAgent [Microsoft.PowerShell.Commands.PSUserAgent]::FireFox 

    }

    default {

        $downloadInfo.Error = "Status description [$($downloadRequest.StatusDescription)] not handled!"
        
        return $downloadInfo

    }

}

#Switch out the proper content type for the file download
Switch ($downloadRequest.BaseResponse.ContentType) {

    'application/x-msdos-program' {
                
        $downloadInfo.Content = $downloadRequest.Content
        $downloadInfo.Success = $true

        return $downloadInfo

    }
   
    Default {

        $downloadInfo.Error = "Content type [$($downloadRequest.BaseResponse.ContentType)] not handled!"
        
        return $downloadInfo

    }

}

What matters the most is that you return an object that has the following properties:

  • DownloadName
    • String value of the downloaded file name
  • Content
    • Byte array containing the actual file
  • Success
    • Boolean (true if the file was able to be downloaded)
  • Error
    • String representing any errors encountered

Example script creation:

For this example, let's download ElvUI (as we learned how to in detail, here).

First, I'll save the template as elvui.ps1

Then, based on our knowledge of web parsing (more here if you would like to learn more about Invoke-WebRequest), we can edit the template to reflect the correct web parsing needs and content type.

Full code for elvui.ps1

#Template example (works for Firefox, adjust as needed for your download)

#Set this to the URL you'll be navigating to first
$navUrl    = 'http://www.tukui.org/dl.php' 

# IMPORTANT: This is the format of the object needed to be returned to the description
# Whichever way you get the information, you need to return an object with the following properties:
# DownloadName (string, file name)
# Content (byte array, file contents)
# Success (boolean)
# Error (string, any error received)
$downloadInfo = [PSCustomObject]@{

    DownloadName = ''
    Content      = ''
    Success      = $false
    Error        = ''  

} 

#Go to first page
Try {

    $downloadRequest = Invoke-WebRequest -Uri $navUrl -UserAgent [Microsoft.PowerShell.Commands.PSUserAgent]::FireFox -ErrorAction SilentlyContinue

}
Catch {

    $errorMessage = $_.Exception.Message

    $downloadInfo.Error = $errorMessage

    return $downloadInfo

}

#Look for urls that match
$downloadURL = ($downloadRequest.Links | Where-Object {$_ -like '*elv*' -and $_ -like '*download*'}).href
$downloadInfo.DownloadName = $downloadURL.Substring($downloadURL.LastIndexOf('/')+1)

#Go to matching URL, look for download file (keeping redirects at 0)
try {
     
    $downloadRequest = Invoke-WebRequest -Uri $downloadURL 

}
catch {
    
    $errorMessage = $_.Exception.Message

    $downloadInfo.Error = $errorMessage

    return $downloadInfo

}

#Switch out the proper content type for the file download
Switch ($downloadRequest.BaseResponse.ContentType) {

    'application/zip' {
                
        $downloadInfo.Content = $downloadRequest.Content
        $downloadInfo.Success = $true

        return $downloadInfo

    }
   
    Default {

        $downloadInfo.Error = "Content type [$($downloadRequest.BaseResponse.ContentType)] not handled!"
        
        return $downloadInfo

    }

}

Let's test it out!

.\download.ps1 -downloadName elvui

And there you have it, modular file downloading via PowerShell.

Github Repository

This project is available on Github!

Click here to go to the psNinjaDownloader repository.

You can download the contents as a ZIP file by going to 'Clone or download', and then selecting 'Download ZIP'.

If you download the code, and would like to run it, you'll need to unblock download.ps1 first.

To do this, right click download.ps1, and go to Properties, and check 'unblock', then click Apply/OK.

unblock.PNG

You'll also need to do this for all the scripts in the .\scripts folder

You can repeat the above step for every script in the .\scripts folder, or we can use PowerShell to do it!

  1. In PowerShell, navigate to the .\scripts folder of psNinjaDownloader
  2. Run the following command:
Get-ChildItem | Unblock-File

Voila, you now have a working copy of the script!

What's Next?

  • Create your own download scripts, and test downloading things auto-magically.
  • Use this to download the latest version of tools as a schedule task
  • See the full help for the script by running: 
Get-Help .\download.ps1 -Full

If you have any feedback, questions, or issues, leave a comment here or contact me!

[top]