Finally moving your Enterprise Public Folders to the cloud? Understand protocols and options (spoiler alert! some are no more than wishful thinking)

If your first thought when you read this blog title, was “but do people still use Public Folders?”, the answer is YES!

Not only corporations still use Public Folders, but also they want to move them to the cloud and keep using them.

For the past year or two, working for a company that owns an outstanding migration product, MigrationWiz, I’ve seen a trend of large organizations worldwide getting to their final stages of the transition to the cloud. Usually, what that means in practical terms, is that they are addressing the more complex and harder to move or coexist workloads, which also translates to Public Folder migrations, amongst other things.

For many years Microsoft made it easy for Microsoft 365 users to access Public Folders on premises, and not that long ago they enabled on premises users to access Microsoft 365 Public Folders.

What Microsoft never supported, although it was technically possible, was two source of authorities for a Public Folder infrastructure, within a Hybrid Deployment. But all this flexibility pushed Public Folder migrations to the back of the “moving to the cloud” workload queue.

So now that we have a little bit of context on Public Folder migrations, lets address the options, when you decide to finally make the move.

How much data do you have to move?

From my experience, there are fundamental differences between two main types of migrations, you might be faced with. My first exercise, when trying to help a customer or a partner, plan their Public folder migration, is to determine if it’s a small or a large migration. Let me share with you where do a draw the line between both:

  • Small migrations: Migrations of up to 50GB of data, 1000 folders and 100k items
  • Large migrations: Anything above that

The main differences between a large and a small migration

Depending on the volume of data, or migration project can be simplified, or very complex. Here’s some bullet points on why:

  • Small migrations can easily be done with lift and shift protocols, like OA (Outlook Anywhere) or MRS (Mailbox Replication Service), however if you are looking at data transformation – this topic justifies a dedicated blog post, so won’t get into much detail on it here, but you can characterize migrating from Public Folders to Shared Mailboxes as an example of data transformation – and/or filtering or mapping the data differently from the source, you should use an EWS based migration product.
  • Small migrations do not require pre migration split work, to balance data between multiple destination mailboxes. In fact the lift and shit protocols do not require that.
  • Remember the wishful thinking mentioned on this blog title? Well trying to execute large migrations with lift and shift protocols, is exactly that. If you think you can migrate 2TB of Public folders with MRS, there’s no better way of saying this so I’ll just go for it: “YOU’RE WRONG”.
  • In addition to the above, large migrations also require pre migration work, that will basically focus on splitting the data into smaller chunks and pointing it to multiple target mailboxes, just so we never rely in the Microsoft auto splitting process for Public Folders.

Available protocols

Now that we outlined some of the biggest differences between large and small migrations, lets talk about the available protocols to migrate:

Outlook Anywhere

Outlook Anywhere is an old legacy protocol, that amongst other things, can be used to migrate Public Folders from Exchange 2010 to Exchange Online. But what are the main problems with Outlook Anywhere?

  • as stated above, the protocol is legacy and Microsoft actually stopped supporting it, in Exchange Online, back in 2017.
  • OA migrations are lift and shift and not feasible for large volumes of data
  • Setting up and/or troubleshooting OA can become complex really fast, in an Exchange version that is EOS since October 2020
  • Previous patching of the Exchange Server can be needed, before proceeding
  • Obviously it goes without saying but, OA does not support EXO (Exchange Online) as source

Mailbox Replication Service

MRS is a well established “migration-ish” protocol, that is very well know for it’s use in Hybrid migrations. For Exchange 2013+ that is the method used. Once again, where can things go wrong here?

  • With MRS, just like with OA, you’ll follow a lift and shift migration and face all the challenges that come with it, for large volumes of data
  • Being a lift and shift doesn’t only make it nearly impossible for some scenarios, it also makes it a non suitable option for mergers, acquisitions, hosted exchange and any scenario where you either have multi tenancy or you don’t want to target the entire Organization, in source or destination
  • The permissions the MRS requires can also be a road block, for the scenarios described in the bullet point above

Exchange Web Services

With Exchange Web Services the approach to migrate the data will be completely different. EWS will target each item in the source and create it in the destination (obviously this is done in batches of items). What that gives you is a whole bunch of options, such us:

  • When to migrate which data (folders, items) and to where (i.e PF to Shared mailboxes)
  • Split the source data to optimize speeds
  • Resume the migration from where it failed, when it does (believe me, at some point if your structure is huge, it will fail)
  • Migrate Terabytes of data
  • Scope source and destination permissions as needed
  • Target Public Folders in Hosted Exchange, per tenant
  • Migrate from Exchange Online

The bottom line

I hope that now that you read this, if you’re thinking about moving your decades old Public Folders into M365, at least you know the options you have.

There are pros to MRS (not so much to OA, to be honest), and if you have 5GB of PFs to migrate, an up to date Exchange with hybrid in place, then you can use it, but this blog post is focused in Enterprise customers, with hundreds of GBs or even TBs of data to move, and being very pragmatic, from my knowledge, that’s just not possible to do, with MRS.

Do let me know if you have any questions. Thank you for reading!

Populate Exchange mailboxes using Exchange Web Services

I just came across a very nice and handy script, that allows you to quickly populate your Exchange mailbox with test data, leveraging EWS.

Being in the migration business and having to do all sorts of tests and demos all the time, this to me is a very useful script, and I am assuming if you’re reading this post, there’s a chance that you also need it.

So this PowerShell script to generate mailbox test data over a period of time, when compared to other methods that leverage SMTP, is great because you can actually populate the mailbox with “old” data.

The parameters you can specify can also be found in the page of the author of the script. You can specify the target mailbox, number of days to fill the mailbox with, messages per day and message size, EWS endpoint or to use autodiscover and some EWS API specific parameters.

EWS1

Above you can see and example of the script execution.

EWS2

And this is how it’s going to look, as it executes.

The script should work perfectly both in Exchange Online and On Premises, as long as it’s a version that supports EWS.

Disclaimer: Unlike many other scripts I blog about, I am not the author of this script. I’ve seen it in the Technet Gallery, used it, and though I should blog about the experience. If you have any issues and/or recommendations or evens kudos to sent, refer to the author, identified in the Technet gallery page.

 

Understanding the admin authentication options when using the EWS API in your migration project

One of the major concerns of the consultant or the project manager, on a migration project, is how long will it take to migrate the mailboxes, and what timelines can they define as reasonable for the migration (or the migration batch) to be finished.

Having that in mind, the conversation around it needs to focus on two major factors:

  • What authentication method should we use
  • How can we minimize (or avoid) throttling

So let’s start by talking about the authentication method.

What are the available authentication methods?

Bittitan MigrationWiz, which of course leverages the Exchange Web Services (EWS) as the API to migrate to and from Exchange 2010+, allows you to use two type of admin authentication methods:

  • Delegation
  • Impersonation

Note: MigrationWiz also allows non admin authentication methods, which we will not discuss on this blog post.

What is delegation and how do I configure it?

Delegation is the authentication method in which the entire migration will be done from the calling account (admin account), that needs to have full access to all of the mailboxes that are being migrated. The requirements for delegation are:

  • admin account needs to be mailbox enabled
  • full access is required to all of the mailboxes being migrated (explicit permissions)
  • a throttling policy should be created and associated with the admin account

In the Bittitan community website you will be able to find instructions on how to configure delegation on the source and target Exchange systems. You can also find there how to create the throttling policy.

By default the MigrationWiz project will try and use delegation, so there’s no other change needed.

What is impersonation and how do I configure it?

Impersonation is the authentication method in which the migration will be done on behalf of the user account, that is being impersonated by the calling account (admin account). For this to happen the admin account needs to be granted rights to impersonate the user accounts being migrated. the requirements for impersonation are:

  • the admin account does not need a mailbox
  • impersonation rights need to be granted to the admin account, over all of the users being migrated
  • a throttling policy is not required but highly recommended

In the Bittitan community website, you will also find instructions on how to configure impersonation on the source and target Exchange systems. The instructions to create the throttling policy are the same you can find on the section above.

The default behavior from MigrationWiz is not to use impersonation, so please make sure you follow the MigrationWiz steps of the article above, to configure impersonation at the project level.

Should I use Delegation or Impersonation?

Now that I described what is Delegation and what is Impersonation, let’s discuss which one should you use for each scenario.

In my professional opinion, that I will detail below and explain why, this is what you should do:

  • If the Exchange system is Exchange Online (Office 365) use Impersonation
  • If the Exchange system is Hosted Exchange (online but not in Office 365) use Delegation
  • If the Exchange system is on premises use Delegation

So now let’s break it down per system.

Exchange Online (Office 365)

The main reason that you should use Impersonation when authenticating against Exchange Online is simple: You cannot create a throttling policy in Office 365 and impersonation is less subject to throttling than delegation. 

I don’t think I need to explain why you can’t create a throttling policy in Office 365, but why is impersonation less subject to throttling? Well the explanation is logic: The subscriptions will be charged against the throttling budget of the target mailbox and not the calling account (admin account), which in other words means that the admin account is doing the migration impersonating each target account and the throttling is being charged to the target accounts, making the limits much more flexible and the migration faster.

Another good thing about Exchange Online is that when you set the impersonation rights, you are of course setting them within the boundaries of your tenant, which is not necessarily true for hosted Exchange systems, as you’ll see below.

Hosted Exchange (not in Office 365)

Now when we look at a hosted Exchange and what admin authentication methods we can have, the main thing you need to keep in mind is that, we will have what the Hoster is willing to configure. And what might that be? Short answer will be Delegation.

Don’t get me wrong, if you manage to get a Hoster (or if you are part of a Hoster Exchange management team and you’re also driving the migration) to either create a throttling policy or configure impersonation associated with a management scope, then what I recommend is:

  1. If you can create a throttling policy then use Delegation and associate that throttling policy to the admin account
  2. If you can’t create a throttling policy but you can enable impersonation with a scope (explanation below) than enable and use Impersonation
  3. If none of the above is possible, then use Delegation

Note: when you set impersonation, if you don’t use a management scope, that will allow the admin account to impersonate any account on that hosted Exchange. That might (and should) be considered a security breach by the Hoster and therefore not possible to configure. Although possible, many Hosters might not be willing to configure impersonation with a management scope, or configure impersonation at all.

On Premises Exchange

Finally when the system is on premises Exchange, you will be able to do all necessary tasks, so why use delegation?

To be able to reach maximum speeds you will still have to create a throttling policy (if you can), so between impersonating and using delegation I do think that delegation and a throttling policy is the best way to go.

Remember when I said “Impersonation is the authentication method in which the migration will be done on behalf of the user account, that is being impersonated by the calling account (admin account).”? Well that’s not 100% true in all cases. Depending on your Exchange version, the subscription might be charged to the calling account, which in essence makes impersonation as effective as delegation in terms of throttling. See the table below:

Exchange version EWSMaxSubscriptions throttling budget accounting
Exchange Online Charged against the target mailbox.
Exchange 2013 Charged against the target mailbox.
Exchange 2010 SP3 Charged against the target mailbox.
Exchange 2010 SP2 Charged against the calling account. Starting with Exchange 2010 SP2 RU4, the budget is charged against the target mailbox.
Exchange 2010 SP1 Charged against the calling account.
Exchange 2010 Charged against the calling account.

Source: EWS throttling in Exchange

From the above table you will see that if you have an Exchange system older than Exchange 2010 SP2 RU4, impersonation and delegation will have the same impact from the throttling perspective. We know that Exchange Online does have a version newer than the one just mentioned, but that is not necessarily true for Hosted Exchange or Exchange On Premises, so have that in mind when planning the authentication methods.

So let me outline the reasons that should make you choose delegation when migrating from an on premises Exchange:

  • the speed of the migration will be highly dependent on the throttling policy, that you can and should create, as well as monitor the Exchange performance during the migration
  • Implementing delegation is easier, specially in cases where you want the admin account to just have rights over a subset of the mailboxes. It’s much easier to just give full access to some mailboxes when compared to give impersonation rights to just some users
  • the way throttling reacts when using impersonation and delegation is different. In my opinion when using impersonation you’re more likely face ErrorServerBusy errors (if you go over the limit of concurrent migrations) and that causes normally more migrations to fail. When using delegation the failures are more likely to happen on the accounts that caused the admin account to over the maximum subscriptions allowed

I know that the explanation above might be a little confusing, specially the last bullet point, but I do highly recommend that you read the EWS throttling in Exchange article, and if you have another idea on what method to use feel free to share it.

I do think that impersonation makes sense only on those scenarios where you cannot fully implement proper delegation, with throttling policies and processes defined to monitor the Exchange server resources during the migration window.

Some useful articles around this subject

The importance of EWS impersonation while using an application account: you will can read the author outline how impersonation is better from an application perspective, and delegation is more oriented towards user access. I also mentions that user access can be controlled and revoked by the end user, which can be true. But on a migration project MigrationWiz will use user access to get into the mailbox and move the data to a destination mailbox, which is exactly what delegation provides you. He also outlines how less complex and global the impersonation setting can be, which is true, but on a project when you only need to migrate a subset of the mailboxes, and where you don’t want to give global access, impersonation will be significantly more complex. In essence this article is a good read that outlines the positives of impersonation and why it’s a good option for application level access and not user level access.

Impersonation and EWS in Exchange: This is an excellent article and a must read, that details how application level impersonation works in Exchange.

So what is the bottom line? At the end of the day it’s still your decision of whether to use delegation or impersonation. I’d say that for some scenarios like Office 365, the decision is a no-brainer (impersonation of course), and in some other scenarios it will depend on what can you configure and what is the simpler and more effective configuration.

Like I stated above I am always more inclined to decide for delegation against Exchange on premises and hosted Exchange, and impersonation against Office 365.

As always I hope this article is helpful, and feel free to share your thoughts.