Finally moving your Enterprise Public Folders to the cloud? Understand protocols and options (spoiler alert! some are no more than wishful thinking)

If your first thought when you read this blog title, was “but do people still use Public Folders?”, the answer is YES!

Not only corporations still use Public Folders, but also they want to move them to the cloud and keep using them.

For the past year or two, working for a company that owns an outstanding migration product, MigrationWiz, I’ve seen a trend of large organizations worldwide getting to their final stages of the transition to the cloud. Usually, what that means in practical terms, is that they are addressing the more complex and harder to move or coexist workloads, which also translates to Public Folder migrations, amongst other things.

For many years Microsoft made it easy for Microsoft 365 users to access Public Folders on premises, and not that long ago they enabled on premises users to access Microsoft 365 Public Folders.

What Microsoft never supported, although it was technically possible, was two source of authorities for a Public Folder infrastructure, within a Hybrid Deployment. But all this flexibility pushed Public Folder migrations to the back of the “moving to the cloud” workload queue.

So now that we have a little bit of context on Public Folder migrations, lets address the options, when you decide to finally make the move.

How much data do you have to move?

From my experience, there are fundamental differences between two main types of migrations, you might be faced with. My first exercise, when trying to help a customer or a partner, plan their Public folder migration, is to determine if it’s a small or a large migration. Let me share with you where do a draw the line between both:

• Small migrations: Migrations of up to 50GB of data, 1000 folders and 100k items
• Large migrations: Anything above that

The main differences between a large and a small migration

Depending on the volume of data, or migration project can be simplified, or very complex. Here’s some bullet points on why:

• Small migrations can easily be done with lift and shift protocols, like OA (Outlook Anywhere) or MRS (Mailbox Replication Service), however if you are looking at data transformation – this topic justifies a dedicated blog post, so won’t get into much detail on it here, but you can characterize migrating from Public Folders to Shared Mailboxes as an example of data transformation – and/or filtering or mapping the data differently from the source, you should use an EWS based migration product.
• Small migrations do not require pre migration split work, to balance data between multiple destination mailboxes. In fact the lift and shit protocols do not require that.
• Remember the wishful thinking mentioned on this blog title? Well trying to execute large migrations with lift and shift protocols, is exactly that. If you think you can migrate 2TB of Public folders with MRS, there’s no better way of saying this so I’ll just go for it: “YOU’RE WRONG”.
• In addition to the above, large migrations also require pre migration work, that will basically focus on splitting the data into smaller chunks and pointing it to multiple target mailboxes, just so we never rely in the Microsoft auto splitting process for Public Folders.

Available protocols

Now that we outlined some of the biggest differences between large and small migrations, lets talk about the available protocols to migrate:

Outlook Anywhere

Outlook Anywhere is an old legacy protocol, that amongst other things, can be used to migrate Public Folders from Exchange 2010 to Exchange Online. But what are the main problems with Outlook Anywhere?

• as stated above, the protocol is legacy and Microsoft actually stopped supporting it, in Exchange Online, back in 2017.
• OA migrations are lift and shift and not feasible for large volumes of data
• Setting up and/or troubleshooting OA can become complex really fast, in an Exchange version that is EOS since October 2020
• Previous patching of the Exchange Server can be needed, before proceeding
• Obviously it goes without saying but, OA does not support EXO (Exchange Online) as source

Mailbox Replication Service

MRS is a well established “migration-ish” protocol, that is very well know for it’s use in Hybrid migrations. For Exchange 2013+ that is the method used. Once again, where can things go wrong here?

• With MRS, just like with OA, you’ll follow a lift and shift migration and face all the challenges that come with it, for large volumes of data
• Being a lift and shift doesn’t only make it nearly impossible for some scenarios, it also makes it a non suitable option for mergers, acquisitions, hosted exchange and any scenario where you either have multi tenancy or you don’t want to target the entire Organization, in source or destination
• The permissions the MRS requires can also be a road block, for the scenarios described in the bullet point above

Exchange Web Services

With Exchange Web Services the approach to migrate the data will be completely different. EWS will target each item in the source and create it in the destination (obviously this is done in batches of items). What that gives you is a whole bunch of options, such us:

• When to migrate which data (folders, items) and to where (i.e PF to Shared mailboxes)
• Split the source data to optimize speeds
• Resume the migration from where it failed, when it does (believe me, at some point if your structure is huge, it will fail)
• Migrate Terabytes of data
• Scope source and destination permissions as needed
• Target Public Folders in Hosted Exchange, per tenant
• Migrate from Exchange Online

The bottom line

I hope that now that you read this, if you’re thinking about moving your decades old Public Folders into M365, at least you know the options you have.

There are pros to MRS (not so much to OA, to be honest), and if you have 5GB of PFs to migrate, an up to date Exchange with hybrid in place, then you can use it, but this blog post is focused in Enterprise customers, with hundreds of GBs or even TBs of data to move, and being very pragmatic, from my knowledge, that’s just not possible to do, with MRS.

Do let me know if you have any questions. Thank you for reading!

Are you considering Exchange 2019 as a “hybrid” management server in Exchange Online environments with objects synced from on premises Active Directory?

If you happen to manage an Exchange Online environment where most or all users (and other objects) are synced from your local Active Directory, you know that, for your management tasks to be executed in a supported and easy way, you need two things:

• The local Active Directory Schema extended to the latest (recommended) Exchange version
• At least one Exchange management server, to execute the management actions from

Because you need the schema extended, to match the cloud Exchange attributes, in your management server, it’s also logic that you would try and keep your management server with the latest version possible. With that said, you should plan to update your Exchange server on premises, whenever a new version is made available.

Seems simple, right? Well it was that simple, until Exchange 2019 came out and Microsoft decided to not provide Exchange Server Hybrid keys for it.

In the past, Microsoft had a specific site where you would get the Hybrid keys from. In theory, to be compliant, for any Exchange on premises version that was used for management and/or hybrid purposes only, and that did not host any mailbox, you would be able to license it for free.

But in July 2018 in the tech community article “Hybrid Configuration Wizard and licensing of your on-premises server used for hybrid” Microsoft explains how you can now use the Hybrid Wizard to license your Exchange server for free, but also states “Please note that HCW does not provide a ‘hybrid key’ for Exchange Server 2019. If you need a hybrid key, the latest version that it is available for is Exchange Server 2016.”

I know this is not new, but managing synced organizations has been and will continue to be a hot topic, for many different reasons, so I decided to blog about it, again.

Why not extend the free licensing to Exchange 2019?

It’s public that Microsoft still has a strong focus on providing Exchange 2019 as the Exchange version for organizations that do not want to move to the cloud, and this licensing decision is for sure related to that, in my opinion.

Is the Hybrid Wizard the best option to license your server?

I think that the Microsoft move from the website to the Wizard, to obtain licenses for hybrid server versions until 2016, is a clever one because it allows the licensing process to be easier to control, however, not all Exchange on premises in this environments can be truly characterized as “Hybrid”.

Many organizations either never had Exchange on premises or don’t rely on any type of interaction with their on premises Exchange, that could truly define it as a “Hybrid server”. Mail flow is fully in the cloud, all hardware and applications on premises interact directly with Exchange Online and free/busy between the cloud and on premises is not required because no objects are hosted from on premises.

So now you not only ask those type of customers to install an Exchange Server, just so they can manage their synced objects in a supported way, but you also ask them to run the Hybrid Wizard in a technically “non-hybrid” environment.

What’s my best option to keep my management server up to date?

The answer is simple: To stay fully up to date, you should update to 2019 and pay for a Standard license.

But if you don’t want to do that, at least for now managing the objects with Exchange 2016 is also a very valid option. Keep the 2016 version for as long as it’s officially supported and tackle the upgrade when you really need to have it done to stay in that supported scenario.

Exchange room booking and recurring meetings was finally simplified

If you follow the Microsoft Exchange Team blog, you probably noticed this post from around 1 month ago, “Easier Room Booking in Outlook on the Web”.

I know it’s been a month, but I haven’t blogged my 2 cents around this, so here it goes.

Why this change

This was an old ask from the Community, so well done for the Exchange Team (and in this case more specifically the Calendar Team) for making this happen.

Selecting a room

The initial focus is on user experience as it relates to room filtering. You can use filters like room location (allows multiple locations), room availability and room features (Audio, Video, etc).

Recurring meetings and room availability

This is one of the major changes implemented. Although Exchange has mechanisms to allow you to coordinate the availability of all meeting attendees, the availability of meeting rooms for the entire series was always a challenge.

The Exchange Team is addressing the above by having Exchange perform an availability query for all meeting dates, until it finds one unavailable, and let you know for how many instances the room is available.

Multiple rooms

In my opinion this is the second major change. For Geo diverse teams, with attendees in multiple office locations, you can select “browse more rooms” and add a local room for each of the attendees locations.

How does an Admin implement this

Basically by leveraging the Set-Place cmdlet (only available in Exchange Online), to define the room characteristics.

Bottom line

I really like this new feature. If I had to point out some negatives, those would be the fact that it’s not support for Exchange on premises, it was launched as an Outlook Web Access feature only (for now – it’s in the road map to make it available for Outlook) and also, in my opinion, the Exchange Team should look at allowing the Organizer to select an additional room(s) when the one selected does not cover all instances.

Finally just want to point out the -GeoCoordinates parameter in the Set-Place cmdlet. It’s really cool and allows you to enter the coordinates of the room and integrate with Bing Maps!

 

How to use MigrationWiz to migrate Public Folder calendars into mailbox calendars

It’s very common to see, in Public Folder migrations, customers that want to migrate and transform that data. But how exactly is that done?

If you’re familiar with MigrationWiz, you’ll know that to migrate data all you have to do is follow some simple steps, like configuring access to source and destination, creating the migration project and defining, within the project, what’s the source and the destination.

The steps above are as simple as they sound, however, to transform data, you’ll need to do some advanced configurations. MigrationWiz gives you flexibility that probably no other tool does, by allowing you to filter or map (I’ll elaborate in a second), which are the foundation features to transform data, but to do so properly, you need to configure your project accordingly.

So how exactly should you configure a project, to migrate a Public Folder calendar into a mailbox calendar?

I won’t give you details about the basic steps to create a project, you can look for the migration guides in the BitTitan HelpCenter, but basically you need to create a normal Public Folder project and do some changes to it.

The first and more basic change you need to do is to set mailbox as a destination.

Within the advanced options of your MigrationWiz project, go to the Destination settings and select “Migrate to Shared Mailbox”.

Now that you have your destination defined, add the Calendar Public Folder that you want to migrate, to your MigrationWiz project, and the correspondent destination mailbox address.

So now that you have your 1:1 matching done in the project, can you migrate? The answer is no, but lets see what happens if you do.

What you are seeing above is the PowerShell output that lists all folders, after the migration, for the destination mailbox. So what happened?

Basically instead of putting all data into the default calendar folder at the destination, we created 2 new folders, of type IPF.Appointment (Calendar folders), in that mailbox.

What this means for the end user is that he will see 2 new calendars, “Folder1” that will be empty since it had no calendar data at the source and “MyCalendarFolder1” that will have all data. Additionally the default Calendar folder won’t have any migrated data.

The above is rarely the intended goal, so just migrating is usually not the solution. You’ll need some additional configurations. Lets get to it.

Edit the line item you added previously and in the Support options add a Folder mapping.

The regex in this folder mapping basically moves all source data to the destination folder called “Calendar”. Since the mapping is in place and it has a defined destination, we no longer create any folders in the destination. It’s also the mapping that makes all data be copied into that destination folder.

So with the configuration above all data will be into what eventually would be the folder you want. If you adjust the filter you can put it in whatever folder you want, having in mind that if the folder doesn’t exist we will create it.

Hope that helps and happy migrations!!

 

Manage your Exchange Online from the Azure Cloud Shell

The Microsoft Exchange product group, recently announced that you can now manage your Exchange Online, via the Azure Cloud Shell.

If you’re not familiar with the Azure Cloud Shell, it’s basically a browser-based Shell experience in the cloud, fully maintained by Microsoft and that you can leverage to manage your Azure and now also Exchange Online subscriptions.

It’s a step towards running automation from anywhere, with minimal to no effort, which to me is one of the next big things coming to us as IT Consultants.

I wrote a blog article recently, on how to use a multi factor authentication account to connect to Exchange Online, and what Microsoft just did was to provide, by default, the Exchange Online Remote PowerShell module, in the Azure Shell. Smart idea and I bet an awesome quick win for them.

So is there any gotchas?

The quick answer is not any major one, but I still want to point a few things. The first one is that you need an Azure Subscription, otherwise you’ll see the below.

Although many Organizations embracing the Microsoft cloud are already using Office 365 AND Azure, some are not. Some just use Office 365 and it’s good to point out that if you want to leverage this new feature, it’s time for you to create that Azure subscription. The only cost you’ll have with using Azure Cloud Shell, is Azure Storage (also mandatory) cost, which is almost insignificant.

Another smaller thing but also worth pointing out, is MFA (Multi Factor Authentication), as Microsoft expects that you have MFA enabled across all accounts. I guess that’s directly related to the fact that this module you’re leveraging is for login with MFA enabled admin accounts.

Finally Microsoft also points out that they will claim sessions that are inactive for more than 20 minutes. Have that in mind when you build your automation, or just when you have your session open for daily work. This is an expected behavior for this type of cloud and container based Shell.

What else should you know?

I am not going to transcribe the article I pointed you to, in the top of this article, but I just want to highlight the main takeaways:

• You can access the Azure Cloud Shell in several different ways, but via the Azure portal (portal.azure.com) or directly via the Shell portal (shell.azure.com), are the two main ones.
• All Exchange Online cmdlets should be available.
• RBAC fidelity is intact
• This does not mean there’s plans to decommission the current Exchange PowerShell module (yet?) 🙂
• You’ll use the Connect-EXOPSSession cmdlet to start the session
• Microsoft provides SSO (Single Sign-On) just so you don’t have to login twice (i.e Azure Portal and Exchange Online) Yay!!!

And that’s it, enjoy!!!

 

How do you plan and execute a successful Public Folder migration?

From all the years as a consultant, and now directly in the migration business, helping partners successfully plan and execute migrations, Public Folders as always been the one of the most challenging workloads I had to deal with.

There’s always a lot of questions when you have to execute such migrations, so I decided to write a blog post about it, where I am going to try and address as much as possible.

To try and keep it as organized as possible, and because there are so many different scenarios, I will divide this post into three main sections: General migration considerations, Migrating Hybrid Public Folders and Migrating Public Folders cross organization.

We will then also discuss some more generic questions, such as why use a third party tool vs the Microsoft native tool.

General migration considerations

This blog post is focused both on Hybrid and cross organization Public Folder migrations. Some steps however are exactly the same, regardless of what the migration scenario is. Those steps are described below. After reading this section you can then focus on your specific scenario in the sections that follow.

Prepare your On Premises Environment

One of the first things you need to look at is to the On Premises Public folder structure, to check if there’s any inconsistencies or invalid folders. The best way to that is of course via scripting, and you should use this excellent script from Aaron @Microsoft, called IDFix for Public Folders. Download it, run it and fix everything that the script highlights as needing to be fixed.

You should also make sure you create a report with all mail enabled Public Folders and address, and to do so you can leverage the Get-MailPublicFolder cmdlet.

How to migrate Public Folder access permissions, as well as Send-As and Send-on-behalf rights

Public Folder permissions should be migrated by the migration tool, provided of course identities match between on premises and Exchange Online (which should of course be true for Hybrid scenarios), or between premises  in cross organization migrations.

As for the Send-As and Send-on-behalf rights, the best option is to export them from the source system and import them into the destination system, once the migration is completed. Since this is not PowerShell code I’ve focused on recently, I did a quick research online and found this article online where you can find the code to export and import those access rights.

Note: I am not the author of the code below and I am only putting it directly in my blog post just so it’s easier for you to locate it and copy it. The code was taken from the article mentioned in the line above, written by Aaron Guilmette.

Export Send-As

Get-MailPublicFolder -ResultSize Unlimited | Get-ADPermission | ? {($_.ExtendedRights -Like "Send-As") -and ($_.IsInherited -eq $False) -and -not ($_.User -like "*S-1-5-21-*")} | Select Identity,User | Export-Csv Send_As.csv -NoTypeInformation

Export Send-on-behalf

Get-MailPublicFolder | Select Alias,PrimarySmtpAddress,@{N="GrantSendOnBehalfTo";E={$_.GrantSendOnBehalfTo -join "|"}} | Export-Csv GrantSendOnBehalfTo.csv -NoTypeInformation

$File = Import-Csv .\GrantSendOnBehalfTo.csv
$Data = @()
Foreach ($line in $File)
    {
    If ($line.GrantSendOnBehalfTo)
        {
        Write-Host -ForegroundColor Green "Processing Public Folder $($line.Alias)"
        [array]$LineRecipients = $line.GrantSendOnBehalfTo.Split("|")
        Foreach ($Recipient in $LineRecipients)
            {
            Write-Host -ForegroundColor DarkGreen "     $($Recipient)"
            $GrantSendOnBehalfTo = (Get-Recipient $Recipient).PrimarySmtpAddress
            $LineData = New-Object PSCustomObject
            $LineData | Add-Member -Type NoteProperty -Name Alias -Value $line.Alias
            $LineData | Add-Member -Type NoteProperty -Name PrimarySmtpAddress -Value $line.PrimarySmtpAddress
            $LineData | Add-Member -Type NoteProperty -Name GrantSendOnBehalfTo -Value $GrantSendOnBehalfTo
            $Data += $LineData
            }
         }
    }
$Data | Export-Csv .\GrantSendOnBehalfTo-Resolved.csv -NoTypeInformation

Import Send-As

$SendAs = Import-Csv .\SendAs.csv
$i=1
foreach ($obj in $SendAs) 
    { 
    write-host "$($i)/$($SendAs.Count) adding $($obj.User) to $($obj.Identity)"
    Add-RecipientPermission -Identity $obj.Identity.Split("/")[2] -Trustee $obj.User.Split("\")[1] -AccessRights SendAs -confirm:$false; $i++
    }

Import Send-on-behalf

$GrantSendOnBehalfTo = Import-Csv .\GrantSendOnBehalfTo-Resolved.csv
$i=1
Foreach ($obj in $GrantSendOnBehalfTo)
    {
    Write-host "$($i)/$($grantsendonbehalfto.count) Granting $($obj.GrantSendOnBehalfTo) Send-On-Behalf to folder $($obj.PrimarySmtpAddress)"
    Set-MailPublicFolder -Identity $obj.PrimarySmtpAddress -GrantSendOnBehalfTo $obj.GrantSendOnBehalfTo
    $i++ 
    }

Migrating Hybrid Public Folders

This scenario, when compared to the cross organization migration, is far more complex, because besides moving the data you will also have to worry about things like mail flow, user public folder access, etc. But lets address one thing at the time.

Microsoft Official guidance to configure Hybrid Public Folders

If you’re reading this article because you’re planning to migrate your Hybrid Public folders, chances are you already read and executed the Microsoft guidance to make your on premises Public Folders available to Exchange Online users, under a Hybrid deployment. Configure legacy on-premises Public Folders for a Hybrid Deployment is the article for legacy public folders and Configure Exchange Server Public Folders for a Hybrid Deployment is the one for modern Public Folders.

Both articles are focused on the hybrid coexistence and not the migration planning of the Public Folders, but they are important to mention as they impact the migration planning, based on what type of coexistence you configured and steps you followed.

Public Folder end user access in the context of a hybrid migration

When planning a Public Folder migration, under a hybrid scenario, one of the most important things you need to consider is, end user access. With that in mind, consider the following:

• On premises users cannot access Exchange Online Public Folders
• Exchange Online users can access on premises public folders and/or Exchange Online Public folders, although you cannot configure a single user to access both, you can configure some users to have access to on premises folders and some to see them locally, in Exchange Online.

Have the two principles in mind, during your planning. The Public Folder access for Exchange Online users is complex and by itself worthy of a dedicated blog post.

The Microsoft official guidance, mentioned in the previous section, explains how you configure Exchange Online users to access on premises Public Folders.

The bottom line of this section is, make sure you move all users to Exchange Online, before you consider moving the Public Folders, and if you don’t, make sure that the users left on premises do not require any Public Folder access.

Public Folder mail flow coexistence before, during and after the migration. How do you handle mail enabled Public Folders.

Another very important component of your Public Folder migration is the mail flow coexistence, or to be more precise, the way you deal with the mail enabled Public Folders.

Mail Enabled Public Folders before the migration

When you follow the guidance provided by Microsoft, you will be asked to execute the Sync-MailPublicFolders script.

This script enables Exchange Online users to send emails to on premises mail enabled Public Folders, by creating mail objects in Exchange Online with the primary and all other SMTP addresses that those folders have on premises. This objects are not actual Exchange Online Public Folder nor they are visible in the Exchange Online Public Folder tree. They also allow those on premises Public Folders to be present in the Exchange Online GAL (Global Address List), and once a user in Exchange Online emails that folder, the email gets forwarded to Exchange On Premises.

Mail Enabled Public Folders during the migration

During the Public Folder migration, whether it’s a single or multiple pass (with pre-stage + full migration) migration strategy, you should not change the Public Folder mail flow. That means that you should not mail enable the Public Folders in Exchange Online (chose a tool that gives you that option). Actually as you will see below, there are things that you need to do in Exchange Online, before mail enabling the Public Folders.

Mail Enabled Public Folders after the migration

Once your migration (or the pre-stage) is completed, you should transition the Public Folder mail flow to Exchange Online. To do so, you should follow these steps:

1. Start the pre-stage or full migration and wait for it to be completed
2. Once the migration pass is done, go to Exchange Online and delete all mail objects created by the Sync-MailPublicFolders script (NOTE: this will temporarily break mail flow between Exchange Online users and mail enabled Public Folders, online or on premises)
3. Mail enable the Exchange Online Public Folders, either via a script or using the migration tool. Make sure you add all addresses from the on premises to the online Public Folders
4. Run a full migration pass if in step 1 the pass that you ran was a pre-stage

To elaborate a little bit more in step 2, the reason that you need to delete those objects is because you need to avoid conflicting addresses, when enabling the mail enabled Public Folders in Exchange Online, and those objects are not associated with the new EXO Public Folders.

Migrating Public Folders cross Organization

Migrating Public Folders cross organization is not as complex, and you’ll see why in the sections below. This scenario can include migrations such as:

• Exchange Online to Exchange Online
• Hosted Exchange to Exchange on premises or Exchange Online
• Exchange on premises to Exchange on premises

When to migrate users and Public Folders

Usually this Public Folder migrations cross organization come as an additional step to a migration that also includes mailboxes.

Although there’s no 100% correct answer, when it comes to the question of what should be migrated first, mailboxes or Public Folders, in this cases normally the best option is to migrate mailboxes first and Public Folders last. The main reason for that is because you should migrate the Public Folders when they’re not being used anymore, allowing you to do a clean single pass migration of all the data.

Public Folder end user access and mail flow coexistence

This is where things gets simple, for this type of scenarios. There’s no Public Folder access cross organization (unless the user is using the credentials for the 2 systems) and although technically you can configure mail flow between any two email systems, it’s not something you should consider for the majority of the cases.

Mail enabled Public Folders can and should be created at the destination during the folder hierarchy creation.

Why use a third party tool to migrate Public Folders

That’s probably the question I get the most, working for a third party migration tool company, that has an amazing Public Folder migration tool, BitTitan. And here is a list of reasons:

• Migrate large volumes of data: Migrating 2, 5 or 10GB is easy with any tool, but not all tools can deal with Terabytes of Public Folder data.
• Migrate parts of the structure or prioritize data: Either by targeting just specific parts of the Public Folder hierarchy or by using folder filtering. This is a very commonly used feature in tools like BitTitan MigrationWiz.
• Flexibility on handling mail enabled Public Folders: As explained in the Hybrid mail flow section of this posts, you might need some flexibility on how to handle mail enabled Public Folders during the migration. MigrationWiz will mail enable in the destination all Public Folders that are mail enabled at the source, but you can also suppress that option, and should in some scenarios.
• Data transformation: While planning a migration of Public Folders, some customers want to take that opportunity to also move that data into a different structure, which can be shared or resource mailboxes, office 365 groups, etc. That is something that can be successfully done with tools that are flexible enough to perform that transformation (i.e in many cases requires recipient mapping, folder mapping, folder filtering, etc), like MigrationWiz.
• Supported sources and destinations: Exchange 2007+ to Exchange 2007+, including of course Exchange Online and hosted as source and/or destination – this is the answer most customers want to hear from the support ability stand point of a third party tool, to migrate their Public Folders, and that is something they won’t get with the native tool.

The bottom line

While reading this post, before publishing it, I always get the feeling that there’s so many other things that I could mention and talk about, but I do think that it addresses the core concerns of most Public Folder migrations, and hopefully it addresses yours.

Nevertheless, if you do have any questions don’t hesitate to reach out.

 

 

 

Exchange 2013: When your Load Balancer marks your Client Access Server as offline.. and all services are up and running

Just recently, I was asked to help with an issue in an Exchange 2013 Organization. The problem was that the KEMP Load Balancer, that balances requests for multiple Exchange protocols, between several Exchange 2013 servers, was marking one of the CAS servers as being offline.

The server was not being marked as offline for all services, it was just for one, Outlook Web Access (OWA).

Before I go into server details, let me give you a visual representation of what the above means. In my case we had a KEMP Load Balancer, but this can apply to any load balancer.

What you can see above is the KEMP admin portal. On the left hand side if you go to Virtual Services and click “View/Modify Services”, you’ll see a table (that I conveniently don’t show, just so I don’t have to cover all information in it anyway). In that table you will have, per server, one or more “Real Servers”, and when that real server health cannot be verified, it will show in red.

What that means from an operational perspective, is that no request from that protocol (https/smtp/sip etc) will be sent to that server, meaning you can be in a situation of single point of failure and you won’t know it.

Now lets have a look at how those real servers health is checked.

If you click to modify the virtual service, in the “Real Servers” section you will see the check parameters. Basically what KEMP does is to get an https request to https://server/owa/healthcheck.htm

If that request fails, you should see a page not found and the server is marked as being offline. The below is the result you should expect from a healthy server.

In the URL above, you should see the server name. Below the 200 OK you should see the server FQDN.

Now that I explained, very high level, how the process works, lets drill down to the problem that I had.

Before detailing the problem that I had, I just want to point out that there are many reasons for a server to be marked as offline by a load balancer, and that you should check the obvious ones first, such as if all services are running, if Exchange is coming back as healthy when using tools such as the Get-ServerHealth or Exchange cmdlet, System Center Operations Manager (SCOM) if you have it, etc.

In my case there were no obvious reasons, plus I found it strange that only the https protocol was having issues, so this is what I did:

1 – I browsed to the https website that KEMP uses and noticed that I was getting page not found, which explains KEMP marking the server as not available

2- I then used another very handy Exchange PowerShell cmdlet, the Get-ServerComponent, to check which components were in an Active or Inactive state

See below an example of a good output for the server components, with all of the relevant ones being active.

But what I was getting was not what you see above. I was seeing the “OwaProxy” component as inactive, and that was a problem.

3- So what I did next was to look at the Exchange 2013 Health Mailboxes, searching for inconsistencies

And I found them…

When I ran a Get-Mailbox -Monitoring, to list all Health mailboxes, I found that some were corrupt, due to being associated with missing databases or missing mailbox servers.

This Exchange Organization has been downsized for the past year and during that cleanup those mailboxes were forgotten. If you have the same issue, don’t worry. Although it’s important to keep those mailboxes in a good state, you can easily recreate them, and that leads me to my next step.

4- Recreate the health mailboxes

So instead of me doing a lot of copy past into my blog post, check this Exchange 2013/2016 Monitor mailboxes article, from the product group blog, that not only has a lot of information about those mailboxes, but also has a perfect step by step at the bottom, on how exactly you recreate them.

And that’s what I did, recreate them, re-tested the URL https://server/owa/healthcheck.htm, which now worked perfectly and my load balancer was no longer marking my CAS server as unavailable in my https virtual service.

Hopefully this article can help you resolve your issue!

 

What does it really mean to be in the last year of support for Microsoft Exchange 2010?

The Microsoft Exchange product team, sent out a reminder on January 14th 2019, regarding the end of support for Exchange 2010. It will happen in 1 year! Don’t think one year is a long time, read the below to have my perspective of what this means to Microsoft but more importantly what this means to you, and start planning and executing the best roadmap for you. Check here what the Microsoft recommended roadmap is.

Let me start by outlining the simple reasons (in my perspective, of course) of why Exchange 2010 is going out of support:

• Age and Versions – Believe it or not, Exchange 2010 RTM was launched in November 2009, almost 10 years ago. To add to that, we had 3 new versions being launched since: 2013, 2016 and 2019.
• Office 365 and Hybrid compatibility – Exchange online servers get upgraded and the same compatibility standards apply. This means that it makes perfect sense that, as Exchange Online versions evolve, On Premises supported versions tend to be raised as well. Today, if you look at the Hybrid Deployment Prerequisites official information, you’ll see that Exchange 2010 based hybrid deployment is still supported, but I am sure that this won’t last long, and being in a Hybrid deployment will require at least one Exchange 2013, very soon. Logic says that once you start the On Premises upgrade path, by installing higher versions, you should be ready as well to move services to the higher versions and think about decommissioning the lower ones.

Moving on, what exactly does this announcement means, from the Microsoft perspective?

• No more support, bug or security fixes or time zone updates – Basically Microsoft here tells you that you’re on your own, should you choose to keep Exchange 2010 in your Exchange Organization.
• Move to Office 365, move to Office 365 (that’s actually not a typo or duplicate, it’s to show how determined Microsoft is to have you move to Exchange online) or upgrade your Exchange Servers On Premises to a newer version (Exchange 2013+) and decommission all Exchange 2010 servers from your Organization – It’s no secret that for Microsoft, the best option for most organizations, is to move to Exchange Online. Making a product with 10 years and 3 newer versions end of support, is not a consequence of that, but they do take the opportunity to call out those companies, that still have Exchange 2010 or older versions, and tell them to move to the cloud. I do agree that this will be the best option for most of the organizations out there, just not the best option for all, and the reasons are so vast that I could write an dedicated and extremely long blog post about it. In summary probably 95%+ of the organizations, are a perfect fit for Exchange Online.
• If you need help, use the Microsoft FastTrack (did I mentioned that they highly recommend that you go to Exchange Online? 🙂 ), or search for a partner here – This part of the article, the “What if I need help?” section, is actually interesting because they don’t even mention anything directly for if you need help to upgrade your On Premises servers. They basically say that, if you need help, the FastTrack can help you or you can hire a partner, again very Office 365 focused (and I can’t stress this enough: I agree that moving to Office 365 is the best move for most organizations). You can however use the partner search link to, if you choose to, hire one of many great partners out there that will upgrade your Exchange On Premises, if they feel that you’re not the best fit for online, or give you a migration path to Exchange Online and Office 365, better than the FastTrack can provide, if they identify that is the case, by using a third party tool.

And finally, what is probably the most important part, what exactly does this mean to you and your Exchange Organization?

• You don’t have to upgrade, or take any action for that matter, but you should – Your current Exchange 2010 servers will not stop working, and believe me I’ve recently seen Exchange 2003 or older versions still sending and receiving emails, but you should start planning now to either upgrade and/or decommission your Exchange 2010 servers. Do not go into an unsupported scenario, with a service as critical as email should be for your organization. 
• It doesn’t matter if your Exchange 2010 servers aren’t hosting mailboxes and/or processing Client Access Services  – This can affect you because you have servers with mailboxes and active CAS services, but that’s not the only scenario. If you have some old Exchange 2010 Servers, as part of an Exchange 2013 or Exchange 2016 organization, you just recently had one reason to remove them – Exchange 2019 – and now consider this as one more. Don’t keep non supported servers in your Organization. If it’s because of that printer or app using the server to relay email, or because of an old backup or email signature software that lets you stuck to 2010, upgrade that as well or get rid of it.
• It’s not only about if you’re planning to go to Office 365, it’s also if you’re already there and under a Hybrid – If you’re planning to go to Office 365, getting rid of the Exchange 2010 in the process is a good idea, but what if you’re already there and under a Hybrid deployment? If that’s your case, plan and execute. The ideal scenario is always towards having Exchange 2019 (the latest available version) Hybrid servers, and that won’t happen before you remove those 2010. If your Hybrid is a 2010 based hybrid than the urgency is even bigger: Don’t wait!

Hopefully all of the above makes sense. If not, please reach out to me. It’s time to move on from Exchange 2010. Time to upgrade or migrate, whatever suits best your needs.

Populate Exchange mailboxes using Exchange Web Services

I just came across a very nice and handy script, that allows you to quickly populate your Exchange mailbox with test data, leveraging EWS.

Being in the migration business and having to do all sorts of tests and demos all the time, this to me is a very useful script, and I am assuming if you’re reading this post, there’s a chance that you also need it.

So this PowerShell script to generate mailbox test data over a period of time, when compared to other methods that leverage SMTP, is great because you can actually populate the mailbox with “old” data.

The parameters you can specify can also be found in the page of the author of the script. You can specify the target mailbox, number of days to fill the mailbox with, messages per day and message size, EWS endpoint or to use autodiscover and some EWS API specific parameters.

Above you can see and example of the script execution.

And this is how it’s going to look, as it executes.

The script should work perfectly both in Exchange Online and On Premises, as long as it’s a version that supports EWS.

Disclaimer: Unlike many other scripts I blog about, I am not the author of this script. I’ve seen it in the Technet Gallery, used it, and though I should blog about the experience. If you have any issues and/or recommendations or evens kudos to sent, refer to the author, identified in the Technet gallery page.

 

The differences in quotas and how to handle Public Folder migrations to Exchange 2013+ On premises vs Exchange Online

When we talk about migrating Public Folders, and believe me I talk about that a lot, the usual assumption is that the migration is to Exchange Online. That is true most of the times, but not all.

BitTitan MigrationWiz is a tool that adapts to all sorts of scenarios, the rule being if you have an Exchange 2007+ in the source and destination, and that of course includes Exchange Online, that MigrationWiz will be able to migrate your Public Folders. That being said I find myself discussing scenarios such as migrating Public Folders from Exchange on premises or Exchange Online, to a destination Exchange On Premises.

For those used to migrate Public Folders into Exchange Online, you would know that one of the main things to take into account is the volume of data to be migrated and how to make that work seamlessly, since the Public Folder mailboxes in Exchange Online have quotas you can’t change, as you can see here.

One of the things that makes MigrationWiz such a good tool to migrate large volume of Public Folder data, into Exchange Online, is that we will automatically split the Public Folder data into multiple mailboxes, therefore preventing a migrate failure and delay if one of the mailboxes gets full during the migration. This is done via support and explained in all the relevant migration guides, such as this one.

So the question now is, would this process be necessary when the destination are Exchange On Premises 2013+?

Answer: No. You can’t change Public Folder mailbox quotas in Exchange Online, but you can change them in Exchange On premises, so you don’t need to automatically split the data into multiple destination mailboxes.

And should I use a single On Premises mailbox for a large volume of data?

Answer: You can, but you shouldn’t. If you have 100, 150 or 200GB of Public Folder data, then yes a single mailbox approach seems reasonable, but if you have more than that you should think about having multiple mailboxes, for reasons like backup and restore management, among others. Another reason for having multiple Public Folder mailboxes might be if you have a multi region Exchange Organization and you want to provide localized access to Public Folders.

Why do you reference the destination as being Exchange 2013+?

Answer: Because this blog post focuses specifically in modern public folders (mailbox vs database).

Now lets have a look at a Public Folder mailbox in Office 365:

As you can see above the quotas are well defined and cannot be changed.

How does that look in Exchange On premises?

Above you can see 2 Public Folder mailboxes. One comes with the quota set to unlimited, which is the default when you create a new Public Folder mailbox in Exchange 2013, and for the other one I’ve set the limits to 150GB, with the following command:

Set-Mailbox -PublicFolder <mailboxname> -ProhibitSendReceiveQuota 150GB -ProhibitSendQuota 150GB -IssueWarningQuota 150GB -UseDatabaseQuotaDefaults $false

Note: Don’t forget to set the database quota defaults to false, if you want the new quotas to apply at the mailbox level.

As you can see above there are differences between Exchange Online and On Premises, and the control you have over both. Consider them when planning your migration.