The value of the BitTitan SDK to your Enterprise migration project

For those of you with consulting experience, specially in Enterprise projects, currently working in the migration business as a consultant, or in other roles, you’ll know that there’s a huge difference between small/medium size projects and enterprise projects, specially in the way the project is executed.

It’s usually very easy to use a user interface of a tool, to perform actions for 10, 20 or 50 users, but when you have dozens or hundreds of thousands of users to migrate, that quickly becomes a challenge. In this post, we’ll discuss some of the key areas where, leveraging the BitTitan PowerShell module as a key element for task execution in your project, will bring a huge value.

Automate reoccurring tasks

In enterprise migration projects you’ll have hourly, daily or weekly tasks that you have to perform, such as:

  • start/restart migrations
  • create reports
  • move users between migration stages
  • retry failed migrations
  • add new batch of users to migration project
  • schedule the Outlook profile reconfiguration for a batch of users

During your project, most if not all of the tasks described above will have to be executed multiple times a week or even multiple times a day, depending on the task.

So why should you automate those tasks? The answer depends on the task, but reasons should include saving man hours, making sure tasks get executed on time, link task executions (i.e move a user to a different project and start another migration pass), etc

Create your own reports or integrate with an external reporting system

The BitTitan user interface provides you a whole variety of “out of the box” reports, such as user migration statistics, user devices details, etc. But from my experience there’s always one or multiple reports, that you or your customer needs and there’s no easy way of extracting that out of any user interface. With PowerShell you can extract any information and combine them in custom reports, that you can build to cover the exact needs you have.

As an example, if you want the total number of users across all project with migrations completed, running, failed, completed with errors or queued, the best way to get that is via the BitTitan PowerShell. You can filter in the user interface and you can also send to your email the project statistics CSV, but that is done per project and it doesn’t scale, specially if you have a large number of projects.

Another example is to get item counts and sizes of successfully migrated items for all users in all projects. Again in the case you can use the project statistics CSV sent via email, but that does not scale if you have 50 projects and need that report twice a day, does it?

You can also leverage the information provided by PowerShell to feed a reporting portal, from where you can monitor all major aspects of the migration (like the ones mentioned in the above examples), and that PowerShell can update in short intervals of time (i.e every 5 minutes). That gives you the ability to synthesize the information that you consider most relevant and make it available to the entire team executing the project, as well as of course keeping it up to date, without human interaction, instead of for example having to compile it and send it via email multiple times a day.

Multi platform task execution

In the scope of an Enterprise migration project, the “BitTitan tasks” are just part of the equation. There’s much more to do than just move the data and reconfigure the Outlook clients. That being said, it’s very common for consultants to build and use complex scripting for Enterprise level migrations, that leverage multiple SDKs and PowerShell modules. Some examples:

  • Leverage the MSOnline PowerShell module to create users and assign licenses in Office 365 and after the task is checked and being completed, leverage the BitTitan PowerShell to start the prestage migration for those users.
  • Stamp forwarding addresses, either via Exchange online or Exchange on premises PowerShell modules, before triggering the full migration pass with the BitTitan PowerShell.
  • Verify that the BitTitan migration is completed, via the BitTitan PowerShell, and convert mailboxes to mail users or remote mailboxes, once the completion status is marked as successful.

The three examples above, highlight some of the most common tasks that require multi-platform execution, but it can get much more complex than that. You can automate task precedence and execute much more than just two tasks, leveraging all available PowerShell modules

Trying to link those tasks via the user interfaces, with human interaction, is very challenging and time consuming. They will of course require access to multiple user interfaces and during the execution you can’t automate the task dependencies, which will make it prone to error.

Optimize execution of complex or lengthy tasks

We talked about the benefits of automating reoccurring tasks, about multi platform task execution, but it’s also very relevant to mention task optimization for very complex or lengthy tasks. This doesn’t need to be a reoccurring task nor it needs to be cross platform, all you need to consider when you plan to code a task to be done via PowerShell is, “Can I remove a lot of complexity and execution time, if I run this task via PowerShell?”. If the answer is yes then, code it.

Removing the complexity will allow you to have literally anyone executing that task, and not just a senior resource in your project.

Removing execution time is self explanatory: It saves you money and helps keep you within the estimated project timelines, since it’s easier to predict a task execution time when it’s scripted vs when it’s executed by a human.

Now to put this into context, let me give you some examples of complex or lengthy BitTitan tasks:

  • Retry errors to all illegible users in a project with 10k users
  • Schedule DeploymentPro for users with 5 different vanity domains
  • Start a migration for a list of users provided via CSV
  • Create 25 MigrationWiz projects, one per each user batch
  • Create 10 MigrationWiz endpoints for multiple admin accounts
  • Add 5k recipient mappings into a MigrationWiz project

To be honest all of the tasks described above are more lengthy than complex, since in my opinion there’s not a lot of complexity in the BitTitan tools, nevertheless you might not want to have some lower level resources executing them.

On the other hand, all of the tasks above take a significant amount of time to complete and scripting them will save you a lot of precious hours.

Proactive monitoring and task execution

The BitTitan user interface already has some monitoring tasks, such as send an email to the administrator when a migration fails, but with PowerShell you can take monitoring and proactive task execution to the next level.

Instead of just notifying the project executor that a migration failed, you can trigger another migration retry, but you can go to the detail of only doing it if the failure reason is not something like bad admin credentials. You can make it as clever as you want.

You can also chose not to have to check your email, to see if migrations failed, and just create a script that every 30 minutes checks for failed migrations, checks the reason for failure and if it’s worth retrying, starts another migration pass.

The sentence above explains why I think proactive monitoring and proactive task execution are tied together. Let that sync in, imagine a project with 75 thousand users and with around 5 thousand concurrent migrations being executed at any given point in time, and think about how proactive monitoring and task execution can not only save you hundreds of work hours, but also keep your project timelines within schedule. The last thing that you want, in a large and complex migration project, is to lose hours of migration time, where nothing is being migrated for anyone, because an error occurred and the resolution time is high.

Summary

So how do you, based on everything you read above, plan an execute an Enterprise migration project, with automation?

Use a tool that you can automate with:

When you’re choosing a tool for your Enterprise project, you should heavily consider one that has a powerful PowerShell module, like the BitTitan tools do. You can check the BitTitan PowerShell documentation here.

Prepare and test all of your automation:

Make sure you have all your scripts ready and tested. If you need helped coding with the BitTitan SDK please reach out to me directly or to the BitTitan support, that will forward you to the appropriate department that will provide the help that you need.

And that’s it.. I hope this post has been helpful and please reach out if you have any questions!

 

 

Advertisements

BitTitan SDK: Color code your MigrationWiz project users

The BitTitan SDK is a key feature for all Enterprise migration projects. Some tasks, in large migration projects, are better being automated. It will save you hundreds of hours of repetitive work.

Just recently I got asked by a partner for a way to easily execute actions in batches of users, within the same MigrationWiz project. Some times the best option is to divide those users into separate projects and just execute those actions for all the users, but that’s not always the best option.

Now imagine this scenario: You have a project with 10000 users and you need to start a migration to just 800 of them. What should you do? Color code those users, filter by that color and execute the action.

(More information about color coding here on the BitTitan help center)

So how can you categorize 800 users in a project with 10000? Using the BitTitan SDK, of course.

The script below, that you can also find here, can be used to automatically color code your MigrationWiz users, based on a CSV file.

Below there’s a sample of the CSV file. It’s needs two columns:

  • Source Email – the MigrationWiz source email address
  • Flags – A number between 1 and 6. Each has its individual color.

CSVCategories

The execution is as follows:

  1. Prompt to authenticate with BitTitan credentials
  2. Prompt you to select the BitTitan workgroup where the MigrationWiz project is
  3. Prompt you to select the BitTitan Customer where the MigrationWiz project is
  4. Prompt you to select the MigrationWiz project
  5. Enter the full path of the CSV file (i.e C:\scripts\MyUsers.csv)
<#

.DESCRIPTION

This script will move mailboxes from a mailbox project to a target project

    

.NOTES

    Author          Antonio Vargas

    Date         Jan/2019

    Disclaimer:     This script is provided 'AS IS'. No warrantee is provided either expressed or implied.

Version: 1.1

#>

### Function to create the working and log directories

Function Create-Working-Directory {

param

(

[CmdletBinding()]

[parameter(Mandatory=$true)] [string]$workingDir,

[parameter(Mandatory=$true)] [string]$logDir

)

if ( !(Test-Path-Path $workingDir)) {

        try {

            $suppressOutput = New-Item -ItemType Directory -Path $workingDir -Force -ErrorAction Stop

$msg="SUCCESS: Folder '$($workingDir)' for CSV files has been created."

Write-Host-ForegroundColor Green $msg

        }

        catch {

$msg="ERROR: Failed to create '$workingDir'. Script will abort."

Write-Host-ForegroundColor Red $msg

Exit

        }

}

if ( !(Test-Path-Path $logDir)) {

try {

$suppressOutput=New-Item-ItemType Directory -Path $logDir-Force -ErrorAction Stop

$msg="SUCCESS: Folder '$($logDir)' for log files has been created."

Write-Host-ForegroundColor Green $msg

}

catch {

$msg="ERROR: Failed to create log directory '$($logDir)'. Script will abort."

Write-Host-ForegroundColor Red $msg

Exit

}

}

}

### Function to write information to the Log File

Function Log-Write

{

param

(

[Parameter(Mandatory=$true)] [string]$Message

)

$lineItem="[$(Get-Date-Format "dd-MMM-yyyy HH:mm:ss") | PID:$($pid) | $($env:username) ] "+$Message

    Add-Content -Path $logFile -Value $lineItem

}

### Function to display the workgroups created by the user

Function Select-MSPC_Workgroup {

#######################################

# Display all mailbox workgroups

#######################################

$workgroupPageSize=100

  $workgroupOffSet = 0

    $workgroups = $null

Write-Host

Write-Host-Object "INFO: Retrieving MSPC workgroups ..."

do

{

$workgroupsPage=@(Get-BT_Workgroup-PageOffset $workgroupOffSet-PageSize $workgroupPageSize)




if($workgroupsPage) {

$workgroups+=@($workgroupsPage)

foreach($Workgroupin$workgroupsPage) {

Write-Progress-Activity ("Retrieving workgroups ("+$workgroups.Length+")") -Status $Workgroup.Id

}

$workgroupOffset+=$workgroupPageSize

}

} while($workgroupsPage)

if($workgroups-ne$null-and$workgroups.Length-ge1) {

Write-Host-ForegroundColor Green -Object ("SUCCESS: "+$workgroups.Length.ToString() +" Workgroup(s) found.")

}

else {

Write-Host-ForegroundColor Red -Object "INFO: No workgroups found."

Exit

}

#######################################

# Prompt for the mailbox Workgroup

#######################################

if($workgroups-ne$null)

{

Write-Host-ForegroundColor Yellow -Object "ACTION: Select a Workgroup:"

Write-Host-ForegroundColor Gray -Object "INFO: your default workgroup has no name, only Id."

for ($i=0; $i-lt$workgroups.Length; $i++)

{

$Workgroup=$workgroups[$i]

if($Workgroup.Name-eq$null) {

Write-Host-Object $i,"-",$Workgroup.Id

}

else {

Write-Host-Object $i,"-",$Workgroup.Name

}

}

Write-Host-Object "x - Exit"

Write-Host

do

{

if($workgroups.count-eq1) {

$result=Read-Host-Prompt ("Select 0 or x")

}

else {

$result=Read-Host-Prompt ("Select 0-"+ ($workgroups.Length-1) +", or x")

}




if($result-eq"x")

{

Exit

}

if(($result-match"^\d+$") -and ([int]$result-ge0) -and ([int]$result-lt$workgroups.Length))

{

$Workgroup=$workgroups[$result]

Return$Workgroup.Id

}

}

while($true)

}

}

### Function to display all customers

Function Select-MSPC_Customer {

param

(

[parameter(Mandatory=$true)] [String]$WorkgroupId

)

#######################################

# Display all mailbox customers

#######################################

$customerPageSize=100

  $customerOffSet = 0

    $customers = $null

Write-Host

Write-Host-Object "INFO: Retrieving MSPC customers ..."

do

{

$customersPage=@(Get-BT_Customer-WorkgroupId $WorkgroupId-IsDeleted False -IsArchived False -PageOffset $customerOffSet-PageSize $customerPageSize)




if($customersPage) {

$customers+=@($customersPage)

foreach($customerin$customersPage) {

Write-Progress-Activity ("Retrieving customers ("+$customers.Length+")") -Status $customer.CompanyName

}

$customerOffset+=$customerPageSize

}

} while($customersPage)

if($customers-ne$null-and$customers.Length-ge1) {

Write-Host-ForegroundColor Green -Object ("SUCCESS: "+$customers.Length.ToString() +" customer(s) found.")

}

else {

Write-Host-ForegroundColor Red -Object "INFO: No customers found."

Exit

}

#######################################

# {Prompt for the mailbox customer

#######################################

if($customers-ne$null)

{

Write-Host-ForegroundColor Yellow -Object "ACTION: Select a customer:"

for ($i=0; $i-lt$customers.Length; $i++)

{

$customer=$customers[$i]

Write-Host-Object $i,"-",$customer.CompanyName

}

Write-Host-Object "x - Exit"

Write-Host

do

{

if($customers.count-eq1) {

$result=Read-Host-Prompt ("Select 0 or x")

}

else {

$result=Read-Host-Prompt ("Select 0-"+ ($customers.Length-1) +", or x")

}

if($result-eq"x")

{

Exit

}

if(($result-match"^\d+$") -and ([int]$result-ge0) -and ([int]$result-lt$customers.Length))

{

$customer=$customers[$result]

Return$Customer.OrganizationId

}

}

while($true)

}

}

### Function to display all mailbox connectors

Function Select-MW_Connector {

param

(

[parameter(Mandatory=$true)] [guid]$customerId

)

#######################################

# Display all mailbox connectors

#######################################




$connectorPageSize=100

  $connectorOffSet = 0

    $connectors = $null

Write-Host

Write-Host-Object "INFO: Retrieving mailbox connectors ..."




do

{

$connectorsPage=@(Get-MW_MailboxConnector-ticket $global:mwTicket-OrganizationId $customerId-PageOffset $connectorOffSet-PageSize $connectorPageSize)




if($connectorsPage) {

$connectors+=@($connectorsPage)

foreach($connectorin$connectorsPage) {

Write-Progress-Activity ("Retrieving connectors ("+$connectors.Length+")") -Status $connector.Name

}

$connectorOffset+=$connectorPageSize

}

} while($connectorsPage)

if($connectors-ne$null-and$connectors.Length-ge1) {

Write-Host-ForegroundColor Green -Object ("SUCCESS: "+$connectors.Length.ToString() +" mailbox connector(s) found.")

}

else {

Write-Host-ForegroundColor Red -Object "INFO: No mailbox connectors found."

Exit

}

#######################################

# {Prompt for the mailbox connector

#######################################

if($connectors-ne$null)

{




for ($i=0; $i-lt$connectors.Length; $i++)

{

$connector=$connectors[$i]

Write-Host-Object $i,"-",$connector.Name

}

Write-Host-Object "x - Exit"

Write-Host

Write-Host-ForegroundColor Yellow -Object "ACTION: Select the source mailbox connector:"

do

{

$result=Read-Host-Prompt ("Select 0-"+ ($connectors.Length-1) +" o x")

if($result-eq"x")

{

Exit

}

if(($result-match"^\d+$") -and ([int]$result-ge0) -and ([int]$result-lt$connectors.Length))

{

$global:connector=$connectors[$result]

Break

}

}

while($true)

}

}

Function Add-MW_Category {

param

(

[parameter(Mandatory=$true)] [Object]$Connector

)

# add items to a MigrationWiz project

$count=0

Write-Host

Write-Host-Object ("Aplying categories to migration item(s) in the MigrationWiz project "+$connector.Name)

    $importFilename = (Read-Host -prompt "Enter the full path to CSV import file")

    # read csv file

    $users = Import-Csv -Path $importFilename

    foreach($user in $users)

    {

     $sourceEmail = $user.'Source Email'

$flags=$user.'Flags'

        if($sourceEmail -ne $null -and $sourceEmail -ne "" -and $flags -in 1..6)

        {

$count++

Write-Progress-Activity ("Applying category to migration item ("+$count+")") -Status $sourceEmail

$mbx=get-mw_mailbox-ticket $mwTicket-ExportEmailAddress $sourceEmail

if ($mbx)

{

$Category=";tag-"+$flags+";"

$result=Set-MW_Mailbox-Ticket $mwTicket-ConnectorId $connector.Id-mailbox $mbx-Categories $Category

}

else

{

Write-Host"Cannot find MigrationWiz line item with source address: '$($sourceEmail)'"-ForegroundColor Yellow

}

}

else {

Write-Host"The line item with the address '$($sourceEmail)' and the flag '$($flags)' is not valid."-ForegroundColor Yellow

}

    }




if($count-eq1)

{

Write-Host-Object "1 mailbox has been categorized in",$connector.Name-ForegroundColor Green

}

if($count-ge2)

{

Write-Host-Object $count," mailboxes have been categorized in",$connector.Name-ForegroundColor Green

}

}

#######################################################################################################################

# MAIN PROGRAM

#######################################################################################################################

#Working Directory

$workingDir = "C:\scripts"

#Logs directory

$logDirName = "LOGS"

$logDir = "$workingDir\$logDirName"

#Log file

$logFileName = "$(Get-Date -Format yyyyMMdd)_Move-MW_Mailboxes.log"

$logFile = "$logDir\$logFileName"

Create-Working-Directory -workingDir $workingDir -logDir $logDir

$msg = "++++++++++++++++++++++++++++++++++++++++ SCRIPT STARTED ++++++++++++++++++++++++++++++++++++++++"

Log-Write -Message $msg

# Authenticate

$creds = Get-Credential -Message "Enter BitTitan credentials"

try {

# Get a ticket and set it as default

$ticket=Get-BT_Ticket-Credentials $creds-ServiceType BitTitan -SetDefault

# Get a MW ticket

$global:mwTicket=Get-MW_Ticket-Credentials $creds

} catch {

$msg="ERROR: Failed to create ticket."

Write-Host-ForegroundColor Red $msg

Log-Write -Message $msg

Write-Host-ForegroundColor Red $_.Exception.Message

Log-Write -Message $_.Exception.Message

Exit

}

#Select workgroup

$WorkgroupId = Select-MSPC_WorkGroup

#Select customer

$customerId = Select-MSPC_Customer -Workgroup $WorkgroupId

#Select connector

Select-MW_Connector-customerId $customerId

$result = Add-MW_Category -Connector $connector

$msg = "++++++++++++++++++++++++++++++++++++++++ SCRIPT FINISHED ++++++++++++++++++++++++++++++++++++++++`n"

Log-Write -Message $msg

##END SCRIPT
This is the link for my GitHub Gist, where all comments are welcomed regarding the code. Use the comment section as well if you want something changed.
You can find all my BitTitan SDK scripts in my GitHub repository.

BitTitan SDK: Retry individual errors for all users in your MigrationWiz document migration project

The BitTitan SDK is a key feature for all Enterprise migration projects. Some tasks, in large migration projects, are better being automated. It will save you hundreds of hours of repetitive work.

The script below, that you can also find in here, can be used to automatically retry errors in all users of your MigrationWiz project.

To retry errors in a user, he needs to:

  • Be in a “Completed” state
  • have at least one item error

The execution is as follows:

  1. Prompt to authenticate with BitTitan credentials
  2. Prompt you to select your MigrationWiz document project
  3. Identifies number of users eligible for a retry errors pass
  4. Exports to a CSV, created in the same folder from where the script was executed, a list of all successfully initiated retry errors passes
<#



.DESCRIPTION

This script needs to be run on the BitTitan Command Shell

    

.NOTES

.Version        1.0

    Author          Antonio Vargas

    Date            Feb/13/2019

Disclaimer: This script is provided ‘AS IS’. No warrantee is provided either expresses or implied.

    Change Log

#>

######################################################################################################################################################

# Main Program

######################################################################################################################################################

$connectors = $null

#Working Directory

$global:workingDir = [environment]::getfolderpath("desktop")

#######################################

# Authenticate to MigrationWiz

#######################################

$creds = $host.ui.PromptForCredential("BitTitan Credentials", "Enter your BitTitan user name and password", "", "")

try {

$mwTicket=Get-MW_Ticket-Credentials $creds

} catch {

write-host"Error: Cannot create MigrationWiz Ticket. Error details: $($Error[0].Exception.Message)"-ForegroundColor Red

}

#######################################

# Display all document connectors

#######################################

Write-Host

Write-Host -Object "Retrieving Document connectors ..."

Try{

$connectors=get-mw_mailboxconnector-Ticket $mwTicket-RetrieveAll -ProjectType Storage -ErrorAction Stop

}

Catch{

Write-Host-ForegroundColor Red -Object "ERROR: Cannot retrieve document projects."

Exit

}

if($connectors -ne $null -and $connectors.Length -ge 1) {

Write-Host-ForegroundColor Green -Object ("SUCCESS: "+$connectors.Length.ToString() +" document project(s) found.")

}

else {

Write-Host-ForegroundColor Red -Object "ERROR: No document projects found."

Exit

}

#######################################

# {Prompt for the document connector

#######################################

if($connectors -ne $null)

{

Write-Host-ForegroundColor Yellow -Object "Select a document project:"

for ($i=0; $i-lt$connectors.Length; $i++)

{

$connector=$connectors[$i]

Write-Host-Object $i,"-",$connector.Name,"-",$connector.ProjectType

}

Write-Host-Object "x - Exit"

Write-Host

do

{

$result=Read-Host-Prompt ("Select 0-"+ ($connectors.Length-1) +" or x")

if($result-eq"x")

{

Exit

}

if(($result-match"^\d+$") -and ([int]$result-ge0) -and ([int]$result-lt$connectors.Length))

{

$connector=$connectors[$result]

Break

}

}

while($true)

#######################################

# Get mailboxes

#######################################

$mailboxes=$null

$MailboxesWithErrors=@()

$MailboxErrorCount=0

$ExportMailboxList=@()

Write-Host

Write-Host-Object ("Retrieving mailboxes for '$($connector.Name)':")

Try{

$mailboxes=@(Get-MW_Mailbox-Ticket $mwTicket-ConnectorId $connector.Id-RetrieveAll -ErrorAction Stop)

}

Catch{

Write-Host-ForegroundColor Red "ERROR: Failed to query users in project '$($connector.Name)'"

Exit

}

Foreach ($mailboxin$mailboxes){

$LastMigration=get-MW_MailboxMigration-ticket $mwTicket-MailboxID $mailbox.id|? {$_.Type-ne"Verification"} |Sort-Object-Property Startdate -Descending |select-object-First 1

if ($LastMigration.Status-eq"Completed"){

try{

$MailboxErrors=get-mw_mailboxerror-ticket $mwTicket-mailboxid $mailbox.id-severity Error -erroraction Stop

}

Catch{

Write-Host-ForegroundColor Yellow "WARNING: Cannot find errors for mailbox '$($mailbox.ExportEmailAddress)'"

}

if (-not ([string]::IsNullOrEmpty($MailboxErrors))){

$MailboxesWithErrors+=$mailbox

$MailboxErrorCount=$MailboxErrorCount+$MailboxErrors.count

}

}

}

if($MailboxesWithErrors-ne$null-and$MailboxesWithErrors.Length-ge1)

{

Write-Host-ForegroundColor Green -Object ("SUCCESS: "+$MailboxesWithErrors.Length.ToString() +" mailbox(es) elegible to retry errors found")

Write-Host-ForegroundColor Green -Object ("SUCCESS: '$($MailboxErrorCount)' individual errors found that will be retried")

$RetryMigrationsSuccess=0

Foreach ($mailboxwitherrorsin$MailboxesWithErrors){

try{

$RecountErrors=get-mw_mailboxerror-ticket $mwTicket-mailboxid $mailboxwitherrors.id-severity Error -erroraction Stop

$result=Add-MW_MailboxMigration-ticket $mwTicket-mailboxid $mailboxwitherrors.id-type Repair -ConnectorId $connector.id-userid $mwTicket.userid-ErrorAction Stop

write-host-ForegroundColor Green "INFO: Processing $($mailboxwitherrors.ExportEmailAddress) with $($RecountErrors.count) errors"

$ErrorLine=New-Object PSCustomObject

$ErrorLine|Add-Member-Type NoteProperty -Name MailboxID -Value $mailboxwitherrors.id

$ErrorLine|Add-Member-Type NoteProperty -Name "Source Address"-Value $mailboxwitherrors.ExportEmailAddress

$ErrorLine|Add-Member-Type NoteProperty -Name "Destination Address"-Value $mailboxwitherrors.ImportEmailAddress

$ErrorLine|Add-Member-Type NoteProperty -Name "Error Count"-Value $RecountErrors.count

$ExportMailboxList+=$ErrorLine

$RetryMigrationsSuccess=$RetryMigrationsSuccess+1

}

Catch{

Write-Host-ForegroundColor Red "ERROR: Failed to process $($mailboxwitherrors.ExportEmailAddress). Error details: $($Error[0].Exception.Message)"

}

}

if ($RetryMigrationsSuccess-ge1){

Write-Host-ForegroundColor Yellow "INFO: $($RetryMigrationsSuccess) retry migrations executed. Exporting List to CSV."

$ExportMailboxList|Export-CSV .\List-UsersWithErrors.csv -NoTypeInformation

}

Else{

Write-Host-ForegroundColor Yellow "INFO: No retry migration passes were executed with success."

}

}

else

{

Write-Host-ForegroundColor Yellow "INFO: no users in project '$($connector.Name)' qualify for a retry errors pass. Make sure the users are in a completed state and have individual item errors logged."

Exit

}

}
This is the link for my GitHub Gist, where all comments are welcomed regarding the code. Use the comment section as well if you want something changed.
You can find all my BitTitan SDK scripts in my GitHub repository.

Azure Tip: Use PowerShell to check all blob spaced used in a Storage Account

Just recently, I had the need to be able to know the exact volume of all blob container data, within a specific Azure Storage Account.

This was part of a migration project, which in this case meant that I needed to report that data amount multiple times per day. Data was constantly being copied to and deleted from that Storage account, and the same applies to Blob containers being created, filled with data and deleted afterwards. So my only constant was the Storage Account, and I needed to know, every 2 hours, what was the volume of blob container data in that account.

After a quick of research I found this outstanding Microsoft article on how to leverage the Azure PowerShell module (yes, PowerShell to save the day again!!) to calculate the size of a Blob Storage Container.

The only limitation with the script in the article above was that it’s calculating the size of a single blob container, and I needed the combined size of all blob containers in my Storage Account.

So I had to adapt that script to my scenario, and I turned it into the following script:

# Connect to Azure
Connect-AzureRmAccount

# Static Values for Resource Group and Storage Account Names
$resourceGroup = "ChangeToYourResourceGroupName"
$storageAccountName = "changetoyourstorageaccountname"

# Get a reference to the storage account and the context
$storageAccount = Get-AzureRmStorageAccount `
-ResourceGroupName $resourceGroup `
-Name $storageAccountName
$ctx = $storageAccount.Context

# Get All Blob Containers
$AllContainers = Get-AzureStorageContainer -Context $ctx
$AllContainersCount = $AllContainers.Count
Write-Host "We found '$($AllContainersCount)' containers. Processing size for each one"

# Zero counters
$TotalLength = 0
$TotalContainers = 0

# Loop to go over each container and calculate size
Foreach ($Container in $AllContainers){
$TotalContainers = $TotalContainers + 1
Write-Host "Processing Container '$($TotalContainers)'/'$($AllContainersCount)'"
$listOfBLobs = Get-AzureStorageBlob -Container $Container.Name -Context $ctx

# zero out our total
$length = 0

# this loops through the list of blobs and retrieves the length for each blob and adds it to the total
$listOfBlobs | ForEach-Object {$length = $length + $_.Length}
$TotalLength = $TotalLength + $length
}
# end container loop

#Convert length to GB
$TotalLengthGB = $TotalLength /1024 /1024 /1024

# Result output
Write-Host "Total Length = " $TotallengthGB "GB"

 

The script above will provide you an output into the console of the total volume, in GB, that you have on a specific storage account.

To execute the script, follow the steps below:

  • Copy the entire code above to a notepad
  • Change the values of line 2 and 3, to the correct names of your Azure Resource group and your Azure Storage Account
  • Save the file as .ps1
  • Open a PowerShell window and execute the “script.ps1” file you just saved (see screenshot below)
  • Authenticate with your Azure username and password, when prompted

ScriptAllBlobs1

Execute the script as shown above.

AzureAuth

When prompted, authenticate.

endresult

And this is how the end result should look like.

Before I end this blog post I’d just like to point out that this script was written in a very simplistic way, and to address an urgent need that I had. With a couple more hours of work, you can make this script even easier to use and add all sorts of different features to it, such as:

  • Error handling
  • remove the hard coded values and list for selection all available storage accounts and resource groups
  • change the output format (i.e to CSV) and list sizes per blob container
  • allow you to select between multiple Azure subscriptions under the same account

The above are just some ideas on how to improve the script. I haven’t done it because I had no need for it, but by all means please let me know if you want/need an improved version. This one works just fine, if all you want is the total volume of blob data in a specific storage account.

Happy New Year!!!

Azure Resource Manager PowerShell: How to change between subscriptions

Today’s post is a very simple one. For those of you that like me, have multiple subscriptions on your Azure account and automate a lot of your Azure work via PowerShell, you might need to change between subscriptions, in the same PowerShell session, to execute multiple tasks.

This can be done with one of the two following cmdlets:

And here is where the confusion comes. What’s the difference between the two cmdlets and which one should you use?

Well the answer is the cmdlets do the exact same thing, and you should use the “Set-AzureRMContext” cmdlet, specially if you put it into scripts, since it seems to be the replacement for the “Select-AzureRMSubscription” cmdlet.

In fact, this is what you get when you do a “Get-Help Select-AzureRMContext”:

CAS

As you can see above all references point to the new cmdlet.

Now a quick note on how the cmdlet works.

To list all of your subscriptions:

Get-AzureRMSubscription

To change the context to a different subscription:

Set-AzureRMContext -subscription <SubscriptionID or SubscriptionName>

I hope the above is helpful. Happy scripting!

BitTitan PowerShell Tips: How to connect the BitTitan SDK to the German instance

As many of you might know, the BitTitan SaaS has multiple instances across the globe, such as:

If you’re using the BitTitan SDK, to automate most, if not all of your project tasks, keep in mind the following:

  • By default, the BitTitan SDK connects to the global instance
  • There is no specific version of the SDK to connect to a different instance
  • There are 2 cmdlets that allow you to connect to the German instance:
    • Set-BT_Environment -Environment Germany
    • Set-MW_Environment -Environment Germany
  • You need to run the cmdlets above before you create the BitTitan PowerShell ticket

The BitTitan SDK interacts with both our MSPC platform – for tasks such as managing customers, endpoints, DeploymentPro, etc – and with our MigrationWiz platform, to manage your migration projects among many other things. Depending on the platform you want to interact with, you’ll need to run one of the commands above, or both. The “BT” commands applies to MSPC and the “MW” command applies to MigrationWiz.

See below the sample code screenshot.

BTSDK1

On a side note, if you’re wondering what a BitTitan PowerShell ticket is (mentioned multiple times in this blog post), all of the BitTitan cmdlets are backed by a ticket, that determines which environment we’re connecting into (MSPC or MigrationWiz), which workgroup/account and has the credentials to access it.

For much more information on how to code with the BitTitan SDK, go here or ping me an email via the blog contact form.

Exchange Public Folders: Export item count, per item type, of your public folder structure

Just recently, I was asked to help investigate which Exchange cmdlets would help a partner the I work with, do an item count in an on premises Exchange Public folder structure. Their specific ask was to get, per folder, the number of contact items.

So starting with the best command to do this, it’s easy to get to the conclusion that it will be the Get-PublicFolderItemStatistics, and the first thing that you need to know about that cmdlet is that it’s only available in Exchange 2010+.

The second thing you need to focus on is, in which folders do you want to run the count on? All of them? And if not all, do you want to run the count based on folder type? i.e do you want to just count calendar items on folders of type calendar? How can we achieve this?

Lets break this down:

  • To be able to select the folders you want to count the items for, you need of course to start with the Get-PublicFolder cmdlet
  • If you want to filter just one or multiple folder type (i.e Calendar, Contacts, etc) you need to do it using the “FolderClass” attribute.

Note: The “FolderClass” attribute doesn’t exist in all versions of Exchange. I haven’t checked in detail but at least apparently in Exchange 2010 you won’t be able to leverage this attribute to filter just the folders you want. Worst case scenario you can always run a count against all folders. Also note that as you can see below, not all folders have a “FolderClass”.

PFCount1

And finally the code to grab all the folders you want.

With the FolderClass attribute filtering:

#Get all folders
$folders = get-publicfolder \ -recurse -resultsize unlimited | ? {$_.FolderClass -like “IPF.Contact”}
And without:
#Get all folders
$folders = get-publicfolder \ -recurse -resultsize unlimited

 

Note: The Where-Object filtering (? sign in the command above) in PowerShell caches all its results into memory, so if you have a very large public folder structure you might want to have that in mind and run the commands in a machine with enough resources.

Now that we know how to grab all the folders we need, lets look at how to do the folder count:

  • The command used to do the folder count is, as mentioned above in this post, the Get-PublicFolderItemStatistics
  • Because all you want to do is count items of a certain type, you will leverage the “ItemType” attribute in your filtering
  • Don’t forget that the Get-PublicFolderItemStatistics is an Exchange 2010+ cmdlet

Below see the output of an item count of a specific folder.

PFCount2

Now, finally, the entire script (in bold the item count):

PFCount4

(and the copy/paste version)
#Get all folders
$folders = get-publicfolder \ -recurse -resultsize unlimited | ? {$_.FolderClass -like “IPF.Contact”}
#Process All folders
Foreach ($folder in $folders){
$ContactCount = 0
$Contacts = get-publicfolderitemstatistics $Folder.Identity|? {$_.ItemType -like “IPM.Contact”}
If($Contacts -eq $null){
Write-Host”The folder ‘$($Folder.Identity)’ has 0 Contacts”
}
Else{
foreach($Contact in $Contacts){
$ContactCount++
}
Write-Host”The folder ‘$($Folder.Identity)’ has $($ContactCount) Contacts”
}
}
Lets break down the script above:
  • we start by getting all folders of class contact. Again you can do this filtering or not, depending on the Exchange version and what you need exactly.
  • we then enter a loop where, for each folder, we will grab all items of type contact and count them
  • once that is done we write the output into the console

This script is very simple and doesn’t have error handling, logging and output to CSV. If you want those features feel free to contact me via the blog and I can build you a very complete version of the script.

Running the simple version of the script in a large environment can make the results difficult or impossible to analyse, however, with the code above gives you an insight in how to filter and count Public folders, by type and class.

As always I hope this is helpful.

Office 365: Run a script connected to 2 Exchange online sessions

Have you ever wondered how you can connect to 2 Exchange Online sessions, in the same PowerShell window?

For example, if you want to run a script that connects to 2 tenants, exports all mailbox permissions from one tenant and imports them into the other. Same thing applies to Distribution groups and memberships.

With the Microsoft Tenant 2 Tenant Migrations in high demand, and because there are so much that you might want to bring from one Exchange Online to the other, I thought I should write a quick blog article on how to connect and manage 2 Exchange Online tenants in one PowerShell window, ideal for scripting.

Before you look at the code below, let me outline two key parameters, of the Import-PSSession cmdlet to achieve your goal:

  • Prefix – Specifies a prefix to the nouns in the names of imported commands.
    Use this parameter to avoid name conflicts that might occur when different commands in the session have the same name.
    For instance, if you specify the prefix Remote and then import a Get-Date cmdlet, the cmdlet is known in the session as Get-RemoteDate, and it is not confused with the original Get-Date cmdlet.
  • AllowClobber – Indicates that this cmdlet imports the specified commands, even if they have the same names as commands in the current session.
    If you import a command with the same name as a command in the current session, the imported command hides or replaces the original commands. For more information, see about_Command_Precedence.
    By default, Import-PSSession does not import commands that have the same name as commands in the current session.

Note: Both the definitions above were taken from the Import-PSSession cmdlet official Microsoft article, that you can see by clicking here.

So how does this work actually? Have a look at the code below:

<#
.NOTES
 Author: antonio.vargas@myexchangeltd.co,uk

Date: October 4th 2017
 Version: 1

.SYNOPSIS
 This lines of code will connect 2 PowerShell Exchange Online sessions to 2 different tenants. 
.DESCRIPTION
 By opening 2 PowerShell sessions, using the PREFIX parameter for each one of those sessions it will allow you to manage both tenants at the same time (ideal for tasks where you want to migrate configurations from one tenant to the other)
#>

### Input source and destination credentials

$SourceCred = Get-credential -message "Please Enter your SOURCE tenant credentials"

$DestCred = Get-credential -message "Please Enter your DESTINATION tenant credentials"

### Create Source EXO Session

$SourceSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.outlook.com/powershell -AllowRedirection -Authentication Basic -Credential $SourceCred

$result = Import-PSSession $SourceSession -prefix SRC -AllowClobber

### Create Destination EXO Session

$DestSession = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.outlook.com/powershell -AllowRedirection -Authentication Basic -Credential $DestCred

$result = Import-PSSession $DestSession -prefix DST -AllowClobber

### Run a get-mailbox to validate connection to both tenants

Write-Host "Listing mailboxes in the source tenant" -ForegroundColor Green

Start-Sleep -s 5

Get-SRCMailbox -resultsize unlimited |ft alias, *smtp*

Write-Host "Listing mailboxes in the destination tenant" -ForegroundColor Green

Start-Sleep -s 5

Get-DSTMailbox -resultsize unlimited |ft alias, *smtp*

### Showing a count of mailboxes in source and destination

Write-Host "Counting mailboxes in the source tenant" -ForegroundColor Green

Start-Sleep -s 5

(Get-SRCMailbox -resultsize unlimited).count

Write-Host "Counting mailboxes in the destination tenant" -ForegroundColor Green

Start-Sleep -s 5

(Get-DSTMailbox -resultsize unlimited).count

### LISTING PS SESSIONS

Write-Host "Your list of active PS Sessions" -ForegroundColor Green

Get-PSSession |fl

Some considerations of the code above:

  • There’s no logging or error handling in the code. The purpose of the code above is to provide you the insight on how to connect to 2 sessions with the same cmdlets.
  • The code is provided as is and you should test it before you run it in production.
  • The code includes blocks to perform the following:
    • Request input for source and destination credentials
    • Create the source Exchange Online session
    • Create the destination Exchange Online session
    • Code to demonstrate how to run cmdlets in the source and destination tenant (example with get-mailbox)
    • Code to list both PS Sessions created

Now lets see the output of the code:

2sessions

Simple, right? Again this can be very useful for tenant to tenant migrations.

Azure: “CurrentStorageAccountName is not accessible” error when creating a VM via the SDK

I just recently faced an error when doing an Azure lab, that I thought I should blog about, since the resolution is very simple.

Here’s what happened, I was creating a VM via the SDK using the Service Module (Classic – see note below) and the following cmdlet:

New-AzureQuickVM –Windows –ServiceName “AV-AutoSVC” –name “AV-AutoVM” –ImageName $image –Password $password -Location “East US” -InstanceSize “Basic_A0” -AdminUsername avargasadmin

Note: You can create virtual machines both with the service model (classic deployment) or with the Resource manager (new portal). Both methods are available via SDK but the way to connect is different. You use Add-AzureAccount to login to the service model and Add-AzureRMAccount to login to the resource manager. See differences here.

And I got the following error:

Azure01

New-AzureQuickVM: CurrentStorageAccountName is not accessible. Ensure the current storage account is accessible and in the same location or affinity group as the cloud service.

Now after digging a little bit more in my Azure tenant and what might be the cause of the problem I confirmed that I did had the storage account, and even with the -Location parameter in the cmdlet above forcing it to be “East US” (where my storage account is) I was still getting the same error.

Then I decided to run a the following cmdlet:

Get-Azuresubscription

azure02

I then realized that I have no valid storage account associated with my subscription, and the solution to my problem was to run:

Set-AzureSubscription –SubscriptionName <YourSubscriptionName> –CurrentStorageAccount <YourStorageAccountName>

If you don’t know your storage account name run:

Get-Azurestorageaccount |fl storageaccountname, location

Once you do this, re run your New-AzureQuickVM cmdlet (note: this error should happen also when you’re running the New-AzureVM cmdlet) and the error should be gone.

 

Understand and script the Get-MailboxFolderStatistics cmdlet

This blog post is going to give you some insight on how you can script the Get-MailboxFolderStatistics cmdlet, but also how to understand it’s output, as well as how is it important to plan a mailbox migration project.

Why are the mailbox folder statistics important to plan a migration project?

That is indeed the first thing you need to consider: Why do I need those statistics? To better answer that let me start by showing you an output of the command.

1

What you see above is the output of a mailbox, where I filtered the Folder Path, the number of items in the folder and sub folders, as well as the size of each folder. So let me bullet point why is this important for a migration project planning:

  • It will give you item counts per user and per item type (Mail, calendar, contacts). Those numbers are important, specially if extremely large, to estimate migration timelines when using the Bittitan MigrationWiz mailbox migration.
  • It will show you the size of every folder as well as the entire mailbox size. This is important to identify which folders are larger on a very big mailbox. I’ve seen examples of some of those folder being considered irrelevant for a migration (i.e deleted items, sent items) and with a flexible tool like the Bittitan MigrationWiz you can filter them out.
  • It will show you not only the folders visible to the end user, but also the recoverable deleted items folder. This might be very important in scenarios where you have in place hold active. It will give you the exact estimate of how many items are under in place hold on the recoverable items, that you will eventually need to move to the same folder on the destination mailbox.

So in summary, and giving a quick example, if you’re moving 1000 mailbox from Exchange Online tenantA to tenantB, you should use the Bittitan MigrationWiz tool, that will be not only fast but also flexible on what you can include and exclude in the migration, leveraging the Exchange Web Services API. To do that the insight of what’s in the mailbox is fundamental to plan the entire migration. You can plan the timelines and how long it will take, what type of licensing you will need at the destination tenant and many other important variables for the project.

How can I filter the output per item type?

When running the cmdlet you need to use the “-FolderScope” parameter to filter the output per item type. That parameter allows you to enter many different folder scope types, the most common being:

  • All
  • Calendar
  • Contacts
  • RSSSubscriptions
  • RecoverableItems

See the entire list in the official Technet article of the Get-MailboxFolderStatistics cmdlet.

Some cmdlet examples

See below some examples on how to obtain specific data:

Get total number of items in the mailbox (does not include recoverable deleted items):

Get-MailboxFolderStatistics user@domain.com | Where-Object {$_.foldertype -eq ‘root’} |ft folderpath, itemsinfolderandsubfolders, folderandsubfoldersize

2

Get total number of contacts in the mailbox:

Get-MailboxFolderStatistics user@domain.com -FolderScope Contacts |ft folderpath, items*, folderandsubfoldersize

3

Note: the above will work for calendar items if you change the folder scope from “Contacts” to “Calendar”

Get total number of items in the Recoverable deleted items folders:

Get-MailboxFolderStatistics user@domain.com| where {$_.FolderType -eq ‘RecoverableItemsRoot’}|ft folderpath, itemsinfolderandsubfolders, folderandsubfoldersize

4

And finally… the script that exports all of this to a csv:

Now that we discussed how important it is to get the statistics, and I gave you same examples on how to run some one line commands to get some insight on the mailbox, I will share with you a script that was done by me and my colleague and friend Alberto Nunes.

Below are the details of what the script does:

  • The script will run the statistics against a list of users that needs to be specified on a file called users.csv (see csv format below). The file needs to be on the same folder as the script.
  • The script runs Exchange PowerShell commands, so make sure you’re connected to the Exchange PowerShell (online or on premises)
  • The script will export the results to a file statistics.csv
  • The script will ask you for a source or destination parameter. The statistics for the recoverable deleted items folders will only be included if you select Destination.
  • This script is provided as is and there’s no guarantee that it will work in your environment.

The Users.csv file:

5

Note: Make sure you name the column A “EmailAddress”

The Script:

Copy the text below (in red) to a notepad. Save it as .ps1 and run it from a PowerShell session connected to Exchange.

#Start Script
Param
(
[Parameter(Position=0,Mandatory = $true)][ValidateSet(‘Source’,’Destination’,’MigrationWiz’)][String]$Location,
[Parameter(Mandatory = $false)][String]$Folder = “.”,
[Parameter(Mandatory = $false)][String]$UsersCSV = “Users.csv”,
[Parameter(Mandatory = $false)][String]$OutputCSV = “Statistics.csv”
)
Get-Date
$ErrorActionPreference = “SilentlyContinue”
$SourceFile = $Folder + “\” + $UsersCSV
$OutputFile = $Folder + “\” + $OutputCSV
$mailboxes = Import-Csv -Path “$SourceFile”
$CSV = @()
Foreach ($mailbox in $mailboxes)
{
Write-Host “Working on $($mailbox.EmailAddress)”

$AllStats = Get-MailboxFolderStatistics $($mailbox.EmailAddress)| Where-Object {$_.foldertype -eq ‘root’}

$ContactStats = Get-MailboxFolderStatistics $($mailbox.EmailAddress) -FolderScope Contacts
$TotalContactsItems = ($ContactStats | select -expand ItemsInFolder |Measure-Object -Sum).Sum

$CalendarStats = Get-MailboxFolderStatistics $($mailbox.EmailAddress) -FolderScope Calendar
$TotalCalendarItems = ($CalendarStats | select -expand ItemsInFolder |Measure-Object -Sum).Sum

$RSSFeeds = Get-MailboxFolderStatistics $($mailbox.EmailAddress) -FolderScope RssSubscriptions -ErrorAction SilentlyContinue
If ($RSSFeeds)
{
$TotalRSSFeeds = ($RSSFeeds | select -expand ItemsInFolder | Measure-Object -Sum).Sum
}
Else
{
$TotalRSSFeeds = 0
}

# In the source, we don’t need to know the Size of the RecovableItems
If ($Location -eq “Destination”)
{
$recoverableItems = Get-MailboxFolderStatistics $($mailbox.EmailAddress)| where {$_.FolderType -eq ‘RecoverableItemsRoot’}
[double]$recoverableSize = $recoverableItems.FolderAndSubfolderSize.TrimEnd(” bytes)”).Split(“(“)[1].replace(‘,’,”) |%{“{0:N2}” -f ($_ /1mb)}
}

$TotalMailItems = $AllStats.ItemsInFolderAndSubfolders – $TotalContactsItems – $TotalCalendarItems – $TotalRSSFeeds

$ISSize = $AllStats.FolderAndSubfolderSize.tostring()
[double]$ISSize =$ISSize.TrimEnd(” bytes)”).Split(“(“)[1].replace(‘,’,”) |%{“{0:N2}” -f ($_ /1mb)}

$Output = New-Object PSObject
$Output | Add-Member -MemberType NoteProperty -Name “User” -Value $mailbox.EmailAddress
$Output | Add-Member -MemberType NoteProperty -Name “Location” -Value $Location
$Output | Add-Member -MemberType NoteProperty -Name “TotalItems” -Value $AllStats.ItemsInFolderAndSubfolders
$Output | Add-Member -MemberType NoteProperty -Name “MailboxSize MB” -Value $ISSize
$Output | Add-Member -MemberType NoteProperty -Name “TotalContactItems” -Value $TotalContactsItems
$Output | Add-Member -MemberType NoteProperty -Name “TotalCalendarItems” -Value $TotalCalendarItems
$Output | Add-Member -MemberType NoteProperty -Name “TotalMailItems” -Value $TotalMailItems
$Output | Add-Member -MemberType NoteProperty -Name “TotalRSSFeeds” -Value $TotalRSSFeeds
If ($Location -eq “Destination”)
{
$Output | Add-Member -MemberType NoteProperty -Name “RecovableSize in MB” -Value $recoverableSize
$Output | Add-Member -MemberType NoteProperty -Name “RecovableItems” -Value $recoverableItems.ItemsInFolderAndSubfolders
$Output | Add-Member -MemberType NoteProperty -Name “Total Mailbox Size including Recovable” -Value $($ISSize + $recoverableSize)
$Output | Add-Member -MemberType NoteProperty -Name “Total Mailbox Count including Recovable” -Value $($AllStats.ItemsInFolderAndSubfolders + $recoverableItems.ItemsInFolderAndSubfolders)
}
$CSV += $Output
Write-Host “Completed $($mailbox.EmailAddress)” -ForegroundColor Green
}

$CSV | export-CSV “$OutputCSV” -NoTypeInformation
#End Script

To run the script you need to specify the location parameter:

.\MailboxFolderStats.ps1 -Location Source

.\MailboxFolderStats.ps1 -Location Destination (includes recoverable deleted items statistics)

As always I hope the above is helpful. Thanks for reading.