Let’s start from the end: We receive a call from a user who asks us to recover a deleted mail items. Before we start to investigate: How the mail was deleted or by whom,
the most import questions are: What are the available options for recovering the deleted mail items? And, is it still possible to recover the deleted mail items?
The good news is that Exchange Online enables us (or the user) to recover deleted mail items very easily.
In this article we will review the architecture of the single item recovery, the way we use for recovering mail items, and the Exchange Online “by design” restrictions and defaults, that relate to the Deleted mail items.
|Expand All Headers||Collapse All Headers|
Before we begin
The issue of “Deleted email recovery in Office 365 (Exchange Online) is a little bit tricky because of two reasons:
- Multiple Exchange services that relate to the subject of: Recover deleted mail items.
One of the challenges that I had, when I set down to write this article was: how to put the information in the “right order”?
The subject of “recovering mail items” is related to couple of Exchange services\technologies and, understanding of the relationship that exists between all of the different “parts” can be confusing.
- Misinformation and Misconception
From reading Office 365 subscribers questions about the subject of: “Exchange Online and Deleted email recovery”, in Office 365 related blogs (such as: http://community.office365.com), I have noticed that, there is a lot of “Misinformation and Misconception” related to this subject.
I know that most of us have just want to find a “quick answer” for the issue of the “recover Deleted email items” instead of “bla bla articles” but, I think that it’s important to read all the information presented in this article, so we will be able to have a firm understanding about the subject of: Exchange Online and Deleted email recovery, and to be able to provide the “right answers” for our customers.
Part 1: Exchange Online and deleted email recovery options
When using Mail services based on Exchange server, we have 2 options for: recovering mail items:
- Backup infrastructure – Backing up Exchange server Mailboxes, requires a special backup agent (software), that “knows how to access” and backup the Exchange server database and, hardware (Backup tape, storage) for the backed up data. The “Backup option”, enable the Exchange administrator to restore information from a specific point of time, based on the organization\user requirements.
- Single item recovery (Recoverable Items Folder) – Single item recovery is a built-in Exchange server feature/technology that enables us to recover deleted mail items, in an easy and efficient way, without the need for using a complicated backup infrastructure. The recovery of deleted mail items is implemented in a “one click” operation, by the user himself, or by the Exchange administrator.
Exchange Online verse Exchange On-Premises
Although Exchange Online is based on the technology of Exchange server, it’s important to understand that when we use “cloud services” such as: Exchange Online, we are subject to “Inherent limitations”, because verses Exchange On-Premises, that enable us to choose, or to set values for each of the services as we like, Exchange Online infrastructure include predefined settings and predefined limitation that apply for this services.
Exchange Online limitations
Now, this is the part of the less good news: when using Exchange Online (verses Exchange On-Premises) we have to consider the following limitations:
The subject of “Online backup” or “cloud backup” is quite new, and until now, I did not see a formal solution offered by the major backup software vendors.
In other word: when using Exchange Online, we will need to use “On-Premises backup solution” (or other Backup solutions) for backing up Mailbox’s (and mail items) stored in the Exchange Online server.
2. Single item recovery (Recoverable Items Folder)
As mentioned, Exchange includes a built-in feature described as: “Single item recovery” that enables us to recover deleted mail items. When using Exchange On-Premises, the Exchange administrator has the option to set the value of the “Time Windows” for Single item recovery. When using Exchange Online, the default value for Single item recovery is: 14 days. Let’s make it simple, when we are dealing with a scenario of: deleted mail items, we can recover the mail items only in case that the “deleted mail items age” is less than 14 days!
3. Subscription Plan
In the next section we will review Exchange services that relate to the subject of Recovering mail items. Service such as: “Search Mailbox” and “Litigation hold”, are available only for “E” (Enterprise) subscription.
4. Deleted item Policy – Predefined/Limited options
Another example could be Deleted item Policy. When using Exchange On-Premises, the Exchange administrator can set the value of this policy, based on the organization needs. When using Exchange Online, we are restricted to specific value that we will review later on.
Exchange Online and “General misconceptions”
The most popular misconceptions, related to Exchange Online and data recovery are:
1. Microsoft can recover for me mail items whenever I need!
The source for this misconception is that: when we read, or hear about the “high availability of cloud services” (such as Exchange Online) and the “insurance” that we have regarded scenarios of “disaster”, we automatically “translate” this information to the assumption that: deleted mail items will always be available for us. It’s troth that Microsoft have infrastructure for backing up all the “customer information” and these “backups” could serve for restoring data in case of “disaster” such as storage corruption, server hardware failure or even a catastrophic event of “Data center” failure but, this ability can be used only for a scenario’s of “disaster” and not for a scenario of recovering a specific deleted mail item from any point of time.
You can read additional information in the following links:
2. When using the option of “Archive”, I can always recover deleted mail items!
The source of this misconception is that: usually when using the term “Archive”, we associate this term with “backup” or “saving data”. The “Archive” option enables us to optimize the performance of outlook client, by saving “old mail items” to additional online Mailbox (the Archive). In case that a user deletes mail items from the Archive Mailbox, the mail items will be deleted like any other “standard” mail item from the user Mailbox. To make it simple: Exchange Online Archive doesn’t serve as a storage\solution for deleted mail items.
The causes for “Deleted mail item”
There are couples of “causes” that lead to the scenarios of: “deleted mail item”
1. Deletion performed by the User
This is the most common cause. A user decides to “clean the mess” in his Mailbox, delete unnecessary mail, “empty” the deleted item folder, and after a while, discover that he needs a mail item that was deleted.
2. Exchange Online Retention Policy
Exchange Online includes built-in policy (Retention policy) that serve for mail item management. The policy is implemented by using “Tags” that include instruction about “what to do” regarding mail items that reach a specific age. One of these tags, relate to the “Deleted items Folder”, and by default, each mail item that reaches the age of 30 days will be deleted.
The “other” section includes all the additional possibilities such as: mail that deleted by mistake (Human error), third party application that “hooked” to the outlook client, Anti-virus applications that Identifies specific mail items as mail items that should be deleted and so on.
Exchange Online components – Recovering Mail items
The subject of: “Recovering Mail items” is related to a number of Exchange services or components. So, let’s start with a basic description of each of the “parts” and in next sections we will review each of the parts in more details.
- Recoverable Items Folder - A special hidden folder in the user Mailbox that serves for keeping deleted mail items
- Deleted items policy -The “Deleted items policy” is “attached” to the: Recoverable Items Folder. By default, deleted mail items will be “removed” (deleted) from the Recoverable Items Folder after 14 days.
- in-place eDiscovery & hold - A very powerful Exchange Administrative tool that enables us to:
1. Search all of the Exchange Mailbox’s
2. Save and export mail items that were found by the discovery process. The “context” of the “in-place eDiscovery & hold” tool to the Recoverable Items Folder, is that in the scenario of “Hard Delete” (we will review to the concept of “Hard Delete” later in this chapter), we will need to use the option of in-place eDiscovery & hold for recovering deleted mail items.
- Retention policy - The Retention policy is an Exchange server feature, which serves for managing mail items that reach a specific age. In Exchange Online (verses Exchange on the premises), the Retention policy includes predefined setting that will automatically delete each mail item stored in the “Deleted Items Folder”, when the mail item reach the age of 30 days.
- Litigation Hold - Litigation Hold is an Exchange service that enables us to “Freeze” deleted mail items in a specific Mailbox. When using the option of Litigation Hold, mail items that were deleted, will be kept forever in a hidden folder (Purges Folder), and in case that we want to recover this mail item, we will need to use the option of “in-place eDiscovery & hold” (“standard user” doesn’t have access to this special folder).
Office 365 – Plan P verses Plan E subscription
Before we continue on with the review of additional component that relate to the subject of: recovering deleted Mail items, it’s important to mention that there is a significant difference between the Office 365 subscription plans.
The major difference is between the Plan P subscription verses Plan E subscription.
Plan P subscription (“P” stand for professional, in the current time the name was change to Small Business and Small Business Premium) was created for small business verse Plan E that stand for “Enterprise”. Plan P subscription (Small Business Premium) is cheaper the Plan E subscription but offer less option and features.
The following diagram displays the option that’s available only when an “E” (Enterprise) subscription is purchased. In other word, in case that you have purchased “Plan P” (Small business) subscription, option such as: Mailbox Search, Litigation Hold and the option of “extending” the default “Deleted items policy” is not available.
You can read more information about the features and the services that include in each of the Office 365 subscription plans by using the following links:
Part 2: Recovering deleted mail items – Architecture,
“How to” and Exchange services
Recoverable Items Folder
The “Recoverable Items Folder” is just an additional part of the user Mailbox. This folder, is hidden by design (The Recoverable Items Folder doesn’t “appear” in the standard Outlook folder view).We can think about the Recoverable Items Folder as a “failsafe mechanism”, which enables us (or the user) to recover a mail that was deleted from the Deleted Items Folder.
The main purpose of the Recoverable Items Folder is to: simplify to the task of recovering deleted mail items, instead of using complicated backup and restore procedures.
Recoverable Items Folder structure
Recoverable Items Folder includes the following 3 sub folders:
- Deletion Folder
- Purges Folder
- Versions Folder
You can read additional information about Recoverable Items Folder, by using the following link:
Recoverable Items Folder and Mailbox quota
Each time when a new Mailbox is created, Exchange allocates a dedicated storage quota for the Recoverable Items Folder. The quota that is allocated to the user Mailbox is not affected in any way, by the quota that allocated to the Recoverable Items Folder. In Exchange Online, each Mailbox has a storage limit of 50 GB and, additional 30 GB storage allocated for the Recoverable Items Folder.
To visualize the “existence” of the Recoverable Items Folder and Mailbox quota, let’s use the PowerShell command:
Deleted mail item “Life Cycle”
To understand better the purpose, and that way that Recoverable Items Folder “work”, let’s review what happened when a user delete a mail item.
Step 1 – Delete Mail item
As we all know, when a user deletes a mail item, the mail item is not “deleted” but instead, the mail item is “moved” to the “Deleted items Folder”.
Theoretically, the Deleted mail item, should stay in the Deleted items Folder ”forever” (until the user decides to empty the content of the “Deleted mail Folder”) but, in Exchange Online, there is an implementation of retention policy named: Default MRM, that include Tag with a predefined settings that relate to the “Deleted items Folder”. Each mail item that reaches the “age” of 30 days will be deleted automatically from the “Deleted items Folder”.
Step 2 – Deletions Folder and “Soft Delete”
This is the part in which we start to use the “Recoverable Items Folder”. The operation of deleting mail items from the “Deleted items Folder”, described as: “Soft Delete”. The meaning of “Soft Delete” is that: the mail item is not deleted, but instead, the mail item is moved to an additional hidden folder named: Deletions Folder (sub folder in the Recoverable Items Folder). This folder is not visible to the user and doesn’t appear in the List of the “standard” outlook mail folder. Soft delete is implemented when a user access (open) the “deleted items Folder” and delete mail items, or by choosing the Keyboard key combination: SHIFT + DELETE.
In Exchange Online environment, the “deleted mail item” will be kept in the “Deletions Folder” for 14 days.
Step 3 – Purges Folder and “Hard Delete”
As mentioned, a user has the ability to view the content of the “Deletions Folder” and, the ability to delete mail items stored in the “Deletions Folder”. When a user deletes mail items from the “Deletions Folder”, the operation described as: “Hard delete”, because after the mail item is deleted from the “Deletions Folder”, a “standard outlook user” will not be able to access/see anymore the deleted mail item. If you thought that: “this is the end my friend”, the good news is that- the mail item is still exists. When mail item is deleted from the “Deletions Folder”, the mail item is moved to an additional folder named: “Purges Folder”.
Only Exchange administrator (with the required permissions), has the ability to access the content of the “Purges Folder”, by using the “in-place eDiscovery & hold” option in the Exchange server management Web interface.
Recovering deleted mail item options
In the next section we review the “How to” part that relate to Recovering deleted mail item. The operation of “Recovering deleted mail item” could be implemented by the outlook user, and by the Exchange Online administrator. Most of the “recovery” operation could be executed by the outlook user by himself with the exception of “Hard Delete”.
When mail item is “Hard deleted”, only the Exchange Online administrator, can recover the mail item by using the Exchange in-place eDiscovery & hold. There a couple of options and “tools” that we can use for: Recovering deleted mail item. We can classify these options as:
In this section we will provide only a basic review about the concept and the use of the Retention policy. Retention policy is an Exchange service which used for implementing efficient management of mail items. The “Management of mail item” concept is implemented by: deleting “unnecessary mail items” from the user Mailbox, or move “old mail items” to the Mailbox Archive (in case that we use the option of Archive Mailbox).
The “Retention policy” is actually a collection of “Retention Tags”. Each tag includes instructions about:
Exchange Online, includes Retention policy named: Default MRM, which is applied by default to each of the Exchange Online Mailbox’s.
One of the tags that includes in the Default MRM retention policy is the: “Deleted mail items”. This Tag is attached to the Deleted item’s Folder and,
it’s configured to delete each of the mail items that is older than 30 days.
Users (and Exchange administrator) that don’t know about this default will assume that deleted items will stay in the “Deleted item Folder” forever.
Deleted items policy
Additional “element” that we should know is the “Deleted items policy”. The “Deleted items policy” is “attached” to the: Deletion Folder and the Purges Folder and its purpose is to “clean” the content of this folder (by deleting mail items stored in this folder), when the deleted item reaches a specific age.
(The default value of the Deleted items policy is: 14 days).
Deleted mail scenario
To demonstrate the “flow” of deleted mail items, let’s use the following scenario:
User deletes some mail items (the deleted mail items saved in the Deleted items Folder). When the mail items reach the age of “30 days” the mail items will be deleted (because of the use of the “deleted mail items tag). In other words, we can say that the mail items will be moved to the “Deletion Folder” (a sub folder in the Recoverable Items Folder).
The mail’s items that were deleted (moved to the “Deletion folder”) will stay in the folder for 14 days and then, will be permanently deleted, (there is no option for recovering these mail items).
Configure the Retention policy tag
The good news is that we can change the default Tag that relate to the “Deleted items Folder” very easily. We can choose one of the following options:
- “Extend” the time period of the “Deleted mail items Tag”
- Disable the “Deleted mail items Tag”
- Remove/Disable the “attachment” of the Default MRM policy from user Mailbox.
1. “Extend” the time period of the “Deleted mail items Tag”
In the Exchange management interface, on the left menu bar, choose the compliance management menu and then choose the retention Tags menu. In the screenshot, you can see a list of different retention tags. The tag that we are interested in called: Deleted item.
We can change the default value of the deleted item’s tag from 30 days to any other value that will suit our needs.
2. Disable the “Deleted mail items Tag”
We can disable the “Deleted mail items Tag” by choosing the option: Never. The meaning is that the tag will be disabled (or not active).
3. Remove/Disable the “attachment” of the Default MRM policy from user Mailbox.
An additional option is to “Unassigned” the Default MRM policy for a specific Mailbox. When choosing the Mailbox properties, in the “Mailbox settings” section we can see that the Default MRM policy is “attached” to the Exchange Online Mailbox.
We can choose the option of: “No Policy” to disable the Default MRM policy (including the retention tag for the Deleted items Folder).
You can read additional information about Retention policy by using the following link:
Default Deleted item policy
The Deleted item policy is an: Exchange server policy,
which “Attached” to the Recoverable Items Folder. The purpose of the “Deleted item policy” is to “clean” the storage of the Recoverable Items Folder,
by “removing”(deleting) the mail items after a specific amount of time. In other words, we can relate to the Recoverable Items Folder as a “Temporary storage for deleted mail items”. The default value of the deleted item policy is: 14. The meaning is that deleted mail items will be saved in the Recoverable Items Folder for 14 days.
An interesting fact, which is usually unknown for Exchange Online Administrators, is that: in case that the organization purchase “E” (Enterprise) subscription, we have the ability to change the Default value of the Deleted item policy.
As mentioned, the default value of the Default deleted item retention policy is: 14 days. We can easily “extend” this value up to 30 days. By doing so, we get a “wider window of opportunity” for recovering deleted mail items.
Theoretically, this value can be extended even more than 30 days but, to be able to use this option, you should contact Microsoft Office 365 support team, and get more information about the available options.
The “extension” of the Default deleted item retention policy to 30 days cannot be implemented by using the Exchange Online Web interface. The only option that we have is by using PowerShell.
In the following screen shot we see the default values of Exchange Online Mailbox. You can see that the value of the property: RetainDeletedItemsFor is 14 days.
To change this value to 30 days, we can use the following PowerShell command:
In the following screenshot, you can see that the value of the RetainDeletedItemsFor was updated in now the value is: 30 days.
In case that we want to test this “Hard limit”, let’s use the PowerShell with a value greater than 30 days. In the following example we try to use the PowerShell command with a value of 31 days.
The result is the following error:
The operation on mailbox John failed because it’s out of the current users write scope.
The value of the properties ‘RetainDeletedItemsFor’ exceeds the maximum allowed for user John with license ;BPOS_S_Enterprise’
For your convenience, I have “Wrapped” all of the PowerShell commands that was reviewed, In a PowerShell Script named: Retention-Policy.ps1Download Script
The option of Litigation Hold was created for a special scenario, in which a company suspects that a specific employee performs illegal action or actions that need to be investigated. In this scenario, the basic assumption is that this user will try to “cover his tracks” be deleting items that could incriminate him or reveal his actions.
The solution for this scenario described as: Litigation Hold. The option of Litigation Hold is available only for Plan E2, E3, and E4 Subscriptions.
When we use Exchange to “put a Mailbox” in a Litigation hold, a special flag will be attached to the Mailbox. The purpose of this flag is to change the default behavior of the Exchange server who relates to Deleted mail item policy.
In case that we put a Mailbox in a Litigation Hold, each time the user deletes or changes a mail item, this mail item will be kept forever in the Recoverable Items Folder. Mail item that was deleted will be kept in the Purges Folder, and mail item that were updated will be kept in the Versions Folder.
Access to deleted mail items is implemented in by using the in-place eDiscovery & hold.
Another way to look at the Litigation hold is as a “way to overcome the Deleted item Retention Policy,” and provide the option to recover each of the mail items that was sent to and from the user’s Mailbox, instead of using the “Traditional Backup” solution.
It’s important to understand that this was not the original purpose of the Litigation Hold. The “real use” of Litigation hold was designed as a solution for a limited period that should help us to keep information\data about the company employee who is “under investigation.” In case that you still want to use the option of “Litigation hold” as a “Backup Solution,” you should consider elements such as the Mailbox size. When using the option of the Litigation Hold for a long period of time, the size of the Mailbox could become “Huge” and affect the outlook user experience by slowing outlook performance, cause a synchronization problem and so on.
You can read more information about Litigation hold by using the following link:
Summary and Recap
In this article, we reviewed Exchange server (and especially Exchange Online) component and services, which relate to the subject of: Deleted mail items and the task of recovering deleted mail items.
To summarize all we have learned, let’s mention again the “flow” of deleted mail items and the charters of the different components:
When a user deletes mail items, the deleted mail item is “moved” or saved in the “Deleted items Folder”. By default, the mail item will be deleted after 30 days.
(The value of the “30 days” is determined by the Exchange Online default Retention policy – the Default MRM).
In this stage the deleted mail item is “moved” to the Recoverable Items Folder.
We can describe 3 optional scenarios:
Scenario 1: using the Outlook\OWA “Recovery Deleted Items” option to recover the deleted mail item
By using the Outlook\OWA “Recovery Deleted Items” option, the user can recover the deleted mail item. The deleted mail items are available to the user for a period of 14 days. (This stage describes as: ”Soft Delete”, because the user still have access to the deleted mail items). In case that the user decides to recover the mail item, the mail item will be “moved” back to the “Deleted items Folder”
Scenario 2: using the Outlook\OWA “Recovery Deleted Items” option to delete the mail item
In case that the user decides to delete the mail item, the mail item will be “moved” to the Purges Folder. This stage described as: “Hard Delete”, because the user doesn’t have any more the option to recover the mail item. Only Exchange administrator has access to this folder, and the mail item could be recovered by using the “in-place eDiscovery & hold” option.
Scenario 3: no user intervention
In case that the user delete mail and doesn’t try to recover the mail items, the deleted mail item stay in the “Deleted items Folder” for 30 days. After that, the deleted mail items will be moved for the “Deletion Folder”, stay for an additional 14 days and then permanently deleted.
When a user asks to recover mail items, we can say if the deleted mail items are still “recoverable” by using the “44 days formula”. (Deleted could be considered as a “recoverable” for 44 days).The number “44” is created from the: 30 days that deleted mail items will be “Kept” in the Deleted mail Folder until he will be deleted + the 14 days (The “Deleted items policy”), until the mail item will be permanently deleted (When the retention period for deleted items expires, items are permanently removed from the Exchange Online Mailbox).
Part 3 - Recovering deleted mail items | Client Side
In the following section we review the process of recovering deleted mail items that performed from the “Client Side”, by the user himself.
1. Recovering deleted mail items by using outlook client.
The task of recovering mail items, called by implemented in a very simple way, by the outlook user, without the need for “administrative involvement”. All that is required from the user is to: choose the Folder menu, and choose the “Recover deleted items” icon.
In the window that appears, we can see a list of all the deleted items (the mail items that stored in the “Deletion Folder”).
When choosing the option of “Recover selected items”, the mail item will be restored back to the “Deleted items Folder”.
2. Recovering deleted mail items by using OWA client.
The same concept could be implemented by using the OWA mail client. The only difference is that: to be able to use the “Recover deleted items” menu option, we need to first choose the “Deleted items Folder”.
3. Recovering deleted mail items by using MFCMAPI utility.
The MFCMAPI is a very powerful tool that each Exchange administrator should know. You can download the MFCMAPI tool by using the following link: http://mfcmapi.codeplex.com
By using MFCMAPI tool, we get an “Under the hood view” of the Mailbox content, and we are able to see the “real physical structure” of the Exchange Mailbox. The MFCMAPI tool can serve for many purposes, but in this article, I would like to focus in the option that we have that relate to the “Recoverable items Folder”.
How to use the MFCMAPI
To be able to display the “Recoverable Items Folder” folder, choose the menu: Tools, and check the following options:
- Use the MDB_ONLINE flag when calling OpenMsgStore
- Use the MAPI_NO_CACHE flag when calling OpenEntry
We will need to choose the required outlook mail profile of the specific Mailbox that we want to explore. In our example we will choose “John” mail profile.
Double click on the name of the Mailbox (represented by the user Email address)
To “expand” the tree view of the Exchange Mailbox folder, click on the small triangle (Position to the left of the “Root container”).
In the following screenshot we can see the “physical structure” of the Mailbox.
- “Recoverable items Folder” – this is the folder structure that cannot be seen, when we use the outlook client.
- Top of information store – this is the “standard” Mailbox folder such as: Inbox folder, calendar and so on.
Recovering deleted mail items by using MFCMAPI tool
Before we begin to describe the operation of: Recovering deleted mail items by using MFCMAPI tool, it’s important to mention that recovering mail items that were “Hard deleted” can be implemented by using one of the following options:
In case that you have purchased “Plan P Subscription” the option of in-place eDiscovery & hold is not available for you. In other word, for Plan P Subscription, to only option for recovering “Hard deleted” Mail items is using the MFCMAPI tool
In case that the user perform “Hard Delete”, we can recover the deleted mail items by exploring the content of the “Purges Folder” (that is not visible when we use outlook client).
To recover deleted mail item stored in the “Purges Folder”, double click on the “Purges Folder”. In the windows that appear, you can see the content of the “Purges Folder” and we have the ability to recover mail items by choosing options such as: Export message, Open message, Copy message etc.
We really want to know what you think about the article