What are the available options for recovering the deleted mail items? And, is it still possible to recover the deleted mail items?
The good news is that Exchange online enables us
(or the user) to recover deleted mail items very easily. In this article we will review the architecture of the single item recovery, the way we use for recovering mail items, and the Exchange online “by design” restrictions and defaults, that relate to the Deleted mail items.
- Exchange server - Deleted email recovery options
- Exchange online verse Exchange On-Premises
- Exchange online and “General misconceptions”
- The causes for “Deleted mail item”
- Exchange online components - Recovering Mail items
- Office 365 - Plan P verses Plan E subscription
Part 2 - Recovering deleted mail items – Architecture, “How to” and Exchange services
- Recoverable Items Folder
- Deleted mail item “Life Cycle”
- Recovering deleted mail item
- Retention policy
- Deleted item policy
- Litigation hold
- Summery and Recap
Part 1 - Exchange online and deleted email recovery options
Before we begin
The issue of “Deleted email recovery in Office 365 (Exchange online) is a little bit tricky because of two reasons:
1. Multiple Exchange services that relate to the subject of: Recover deleted mail items
One of the challenges that I had, when I set down to write this article was: how to put the information in the “right order”?
The subject of “recovering mail items” is related to couple of Exchange services\technologies and, understanding of the relationship that exists between all of the different “parts” can be confusing.
2. Misinformation and Misconception
From reading office 365 subscribers questions about the subject of: “Exchange online and Deleted email recovery”, in office 365 related blogs (such as: http://community.office365.com), I have noticed that, there is a lot of “Misinformation and Misconception” related to this subject.
I know that most of us have just want to find a “quick answer” for the issue of the “recover Deleted email items” instead of “bla bla articles” but, I think that it’s important to read all the information presented in this article, so we will be able to have a firm understanding about the subject of: Exchange online and Deleted email recovery, and to be able to provide the “right answers” for our customers.
Part 1: Exchange online and deleted email recovery options
Exchange server - Deleted email recovery options
When using Mail services based on Exchange server, we have 2 options for: recovering mail items:
1. Backup infrastructure
Backing up Exchange server Mailbox’s , requires a special backup agent (software), that “knows how to access” and backup the Exchange server database and, hardware (Backup tape, storage) for the backed up data. The “Backup option”, enable the Exchange administrator to restore information from a specific point of time, based on the organization\user requirements.
2. Single item recovery (Recoverable Items Folder)
Single item recovery is a built-in Exchange server feature/technology, that enables us to recover deleted mail items, in an easy and efficient way, without the need for using a complicated backup infrastructure. The recovery of deleted mail items is implemented in a “one click” operation, by the user himself, or by the Exchange administrator.
Exchange online verse Exchange On-Premises
Although Exchange online is based on the technology of Exchange server, it’s important to understand that when we use “cloud services” such as: Exchange online, we are subject to “Inherent limitations”, because verses Exchange On-Premises, that enable us to choose, or to set values for each of the services as we like, Exchange online infrastructure include predefined settings and predefined limitation that apply for this services.
Exchange online limitationsNow, this is the part of the less good news: when using Exchange online (verses Exchange On-Premises) we have to consider the following limitations:
Exchange online and “General misconceptions”The most popular misconceptions, related to Exchange online and data recovery are:
The source for this misconception is that: when we read, or hear about the “high availability of cloud services” (such as Exchange online) and the “insurance” that we have regarded scenarios of “disaster”, we automatically “translate” this information to the assumption that: deleted mail items will always be available for us. It’s troth that Microsoft have infrastructure for backing up all the “customer information” and these “backups” could serve for restoring data in case of “disaster” such as storage corruption, server hardware failure or even a catastrophic event of “Data center” failure but, this ability can be used only for a scenario’s of “disaster” and not for a scenario of recovering a specific deleted mail item from any point of time.
You can read additional information in the following links:
- Office 365 E1 and E2 plans (Exchange Online Plan 1) Mail Item Recovery Limitations & Solutions
- Office 365 Backup & Recovery
The source of this misconception is that: usually when using the term “Archive”, we associate this term with “backup” or “saving data”. The “Archive” option enables us to optimize the performance of outlook client, by saving “old mail items” to additional online Mailbox (the Archive). In case that a user deletes mail items from the Archive Mailbox, the mail items will be deleted like any other “standard” mail item from the user Mailbox. To make it simple: Exchange online Archive doesn't serve as a storage\solution for deleted mail items.
There are couples of “causes” that lead to the scenarios of: “deleted mail item”
The causes for “Deleted mail item”
Anti-virus applications that Identifies specific mail items as mail items that should be deleted and so on.
Exchange online components - Recovering Mail items
The subject of: “Recovering Mail items” is related to a number of Exchange services or components. So, let’s start with a basic description of each of the “parts” and in next sections we will review each of the parts in more details.
Recoverable Items Folder
A special hidden folder in the user Mailbox that serves for keeping deleted mail items
Deleted items policy
The “Deleted items policy” is “attached” to the: Recoverable Items Folder. By default, deleted mail items will be “removed” (deleted) from the Recoverable Items Folder after 14 days.
A very powerful Exchange Administrative tool that enables us to:
- Search all of the Exchange Mailbox's
- Save and export mail items that were found by the discovery process
The “context” of the “Multi-Mailbox Search” tool to the Recoverable Items Folder, is that in the scenario of “Hard Delete” (we will review to the concept of “Hard Delete” later in this chapter), we will need to use the option of Multi-Mailbox Search for recovering deleted mail items.
The Retention policy is an Exchange server feature, which serves for managing mail items that reach a specific age. In Exchange online (verses Exchange on the premises), the Retention policy includes predefined setting that will automatically delete each mail item stored in the “Deleted Items Folder”, when the mail item reach the age of 30 days.
Litigation Hold is an Exchange service that enables us to “Freeze” deleted mail items in a specific Mailbox. When using the option of Litigation Hold, mail items that were deleted, will be kept forever in a hidden folder (Purges Folder), and in case that we want to recover this mail item, we will need to use the option of “Multi-Mailbox Search” (“standard user” doesn’t have access to this special folder).
Office 365 - Plan P verses Plan E subscription
Before we continue on with the review of additional component that relate to the subject of: recovering deleted Mail items, it’s important to mention that there is a significant difference between the office 365 subscription plans.
The major difference is between the Plan P subscription verses Plan E subscription.
Plan P subscription (“P” stand for professional) was created for small business verse Plan E that stand for “Enterprise”.
Plan P subscription is cheaper the Plan E subscription but offer less option and features.
The following diagram displays the option that's available only when an “E” (Enterprise) subscription is purchased. In other word, in case that you have purchased “Plan P” (Small business) subscription, option such as: Mailbox Search, Litigation Hold and the option of “extending” the default “Deleted items policy” is not available.
You can read more information about the features and the services that include in each of the office subscription plans by using the following links:
- Office 365 for Enterprise Service Descriptions
- Office 365 for professionals and small businesses Service Description
Part 2: Recovering deleted mail items – Architecture, “How to” and Exchange services
Recoverable Items Folder
The “Recoverable Items Folder” is just an additional part of the user Mailbox. This folder, is hidden by design (The Recoverable Items Folder doesn’t “appear” in the standard Outlook folder view).
We can think about the Recoverable Items Folder as a “failsafe mechanism”, which enables us (or the user) to recover a mail that was deleted from the Deleted Items Folder.
The main purpose of the Recoverable Items Folder is to: simplify to the task of recovering deleted mail items, instead of using complicated backup and restore procedures.
Recoverable Items Folder structureRecoverable Items Folder includes the following 3 sub folders:
- Deletion Folder
- Purges Folder
- Versions Folder
You can read additional information about Recoverable Items Folder, by using the following link:
Recoverable Items Folder and Mailbox quota
Each time when a new Mailbox is created, Exchange allocates a dedicated storage quota for the Recoverable Items Folder. The quota that is allocated to the user Mailbox is not affected in any way, by the quota that allocated to the Recoverable Items Folder. In Exchange online, each Mailbox has a storage limit of 25 GB and, additional 3O GB storage allocated for the Recoverable Items Folder.
In the screenshot, you can that Exchange allocate a “dedicated” quota for the Recoverable Items Folder that described as: RecoverableItemsquota
Deleted mail item “Life Cycle”
To understand better the purpose, and that way that Recoverable Items Folder “work”, let’s review what happened when a user delete a mail item.
As we all know, when a user deletes a mail item, the mail item is not “deleted” but instead, the mail item is “moved” to the “Deleted items Folder”.
Theoretically, the Deleted mail item, should stay in the Deleted items Folder ”forever” (until the user decides to empty the content of the “Deleted mail Folder”) but, in Exchange online, there is an implementation of retention policy named: Default MRM, that include Tag with a predefined settings that relate to the “Deleted items Folder”. Each mail item that reaches the “age” of 30 days will be deleted automatically from the “Deleted items Folder”.
This is the part in which we start to use the “Recoverable Items Folder”. The operation of deleting mail items from the “Deleted items Folder”, described as: “Soft Delete”. The meaning of “Soft Delete” is that: the mail item is not deleted, but instead, the mail item is moved to an additional hidden folder named: Deletions Folder (sub folder in the Recoverable Items Folder). This folder is not visible to the user and doesn’t appear in the List of the “standard” outlook mail folder. Soft delete is implemented when a user access (open) the “deleted items Folder” and delete mail items, or by choosing the Keyboard key combination: SHIFT + DELETE.
In Exchange online environment, the “deleted mail item” will be kept in the “Deletions Folder” for 14 days.
If you thought that: “this is the end my friend”, the good news is that- the mail item is still exists.
When mail item is deleted from the “Deletions Folder”, the mail item is moved to an additional folder named: “Purges Folder”.
Only Exchange administrator (with the required permissions), has the ability to access the content of the “Purges Folder”, by using the “Multi-Mailbox Search” option in the Exchange server management Web interface.
Recovering deleted mail item
In the next section we review the “How to” part that relate to Recovering deleted mail item. The operation of “Recovering deleted mail item” could be implemented by the outlook user, and by the Exchange online administrator. Most of the “recovery” operation could be executed by the outlook user by himself with the exception of “Hard Delete”.
When mail item is “Hard deleted”, only the Exchange online administrator, can recover the mail item by using the Exchange Multi-Mailbox Search. There a couple of options and “tools” that we can use for: Recovering deleted mail item. We can classify these options as:
- Recovering deleted mail items – “Client Side”
- Recovering deleted mail items – “Server Side”
In the following section we review the process of recovering deleted mail items that performed from the “Client Side”, by the user himself.
The task of recovering mail items, called by implemented in a very simple way, by the outlook user, without the need for “administrative involvement”. All that is required from the user is to: choose the Folder menu, and choose the “Recover deleted items” icon.
In the window that appears, we can see a list of all the deleted items (the mail items that stored in the “Deletion Folder”).
When choosing the option of “Recover selected items”, the mail item will be restored back to the “Deleted items Folder”.
The same concept could be implemented by using the OWA mail client. The only difference is that: to be able to use the “Recover deleted items” menu option, we need to first choose the “Deleted items Folder”.
The MFCMAPI is a very powerful tool that each Exchange administrator should know. You can download the MFCMAPI tool by using the following link: http://mfcmapi.codeplex.com
By using MFCMAPI tool, we get an “Under the hood view” of the Mailbox content, and we are able to see the “real physical structure” of the Exchange Mailbox. The MFCMAPI tool can serve for many purposes, but in this article, I would like to focus in the option that we have that relate to the “Recoverable items Folder”.
How to use the MFCMAPI
To be able to display the “Recoverable Items Folder” folder, choose the menu: Tools, and check the following options:
- Use the MDB_ONLINE flag when calling OpenMsgStore
- Use the MAPI_NO_CACHE flag when calling OpenEntry
To “enter” into the Mailbox, choose the menu: Session –> Logon
We will need to choose the required outlook mail profile of the specific Mailbox that we want to explore. In our example we will choose “John” mail profile.
Double click on the name of the Mailbox (represented by the user Email address)
To “expand” the tree view of the Exchange Mailbox folder, click on the small triangle (position to the left of the “Root container”).
In the following screenshot we can see the “physical structure” of the Mailbox.
1. “Recoverable items Folder” - this is the folder structure that cannot be seen, when we use the outlook client.
2. Top of information store – this is the “standard” Mailbox folder such as: Inbox folder, calendar and so on.
Recovering deleted mail items by using MFCMAPI tool
Before we begin to describe the operation of: Recovering deleted mail items by using MFCMAPI tool, it’s important to mention that recovering mail items that were “Hard deleted” can be implemented by using one of the following options:
- Multi-Mailbox Search
- MFCMAPI tool
In case that you have purchased “Plan P Subscription” the option of Multi-Mailbox Search is not available for you. In other word, for Plan P Subscription, to only option for recovering “Hard deleted” Mail items is using the MFCMAPI tool
In case that the user perform “Hard Delete”, we can recover the deleted mail items by exploring the content of the “Purges Folder” (that is not visible when we use outlook client).
To recover deleted mail item stored in the “Purges Folder”, double click on the “Purges Folder”. In the windows that appear, you can see the content of the “Purges Folder” and we have the ability to recover mail items by choosing options such as: Export message, Open message, Copy message etc.
The Multi-Mailbox Search is a “Server Side” Exchange service, which we can use for recovering mail items that was “Hard Deleted”.
Items in the Purges subfolder in the Recoverable Items Folder are indexed and discoverable. Administrators (or discovery managers) can use the Multi-Mailbox Search service to search for purged items.
We can use the Multi-Mailbox Search for:
1. Create a Log file – that includes information about the mail items that was found
2. Save a copy of the mail items – beside of “finding” mail items, we can use an option that will create a copy of the mail items that was found by the Multi-Mailbox Search and save a copy of these mail items in a special system Mailbox named: Discovery Mailbox.
The “Multi-Mailbox Search” is a very powerful tool that can serve for many purposes. In this section, we will focus of the “relationship” that exists between the Multi-Mailbox Search, relating to the task of: “recovering deleted mail items” and, the way that we use this tool.
To demonstrate the use of the Multi-Mailbox Search, let’s use the following scenario: user named John call us and ask if we can try to find for him a mail that was deleted by mistake.
Step 1 - Access the Exchange management Web interface
To use the “Multi-Mailbox Search” option, we will need to login to the office 365 management portal by using a user account that has administrator permissions (Global Administrator).
In the “Admin Home Page” choose the option Mange (under the Exchange header).
Step 2 - Discovery Management role group
The user account which uses the option of: Multi-Mailbox Search, will have to be assigned with a special permission.
The Exchange server use a “dedicated role” describes as: Discovery Management. Only users that were “assigned” to this rule group will be able to access\use the Multi-Mailbox Search. By default this “role” is empty, meaning that in the first time (before we assign this permission), even the Global Administrator cannot use the Multi-Mailbox Search.
(The icon of the Multi-Mailbox Search is not visible if the logged in user is not added as a member to the Discovery Management role).
So, the first task is to assign the rule of Discovery Management to the required user account (in most of the scenarios only an Individual user accounts, such as the Global Administrator will have this privilege).
Choose the menu: Roles and Auditing, and under the “Role group” choose the Discovery Management role.
Add the required user account to this group role.
Step 3 - Creating a “new search”
Choose the menu: Mail control.
A “new icon” named: Discovery will “appear”
For creating a new search chose the option: New.
Step 4 - Configuring the search parameters
In the following window we need to add\choose the specific parameters for our search. We can narrow down the search results by specifying a “date range”, “Message subject” and more. (In this example, we will choose only the basic options).
1. Mailbox to search
In our scenario, we need to find mail items that were “Hard Deleted” by a user named: John.
In the “Mailbox to search” choose the required Mailbox name.
2. Search name, Type, and Storage location
In the “Search name, Type, and Storage location” section, we need choose the following options:
- “*search name” – we can type a name that we chose. This “name” will be used as the name for “Search folder” that will be created in the Discovery Mailbox.
- Copy the results to the Mailbox where the search results are stored - The default of the Search Mailbox is to create a log that includes information about the mail items that was found. Because we need to recover the deleted mail items, we need to choose the option: “Copy the search results to the destination Mailbox”. When using this option, a copy of the mail items that will be found is saved in the Discovery Mailbox. Later on, we will use this “Mail item copies” for recovering the deleted mail items.
- Search Mailbox name - The last parameter that we need to choose is the “Search Mailbox name”. In the “Select a Mailbox which to store the search result”.
By default Exchange online use a predefined Search Mailbox named:
“Discovery Search Mailbox”
Step 5 - Access the completed search task
At the end of the “search process”, in the right side of the Web console, we can see (hopefully) that the search complete successfully. (The “Time period” for the search to be completed, depend on the number of Mailbox’s that we search, the Mailbox size etc.).
By default, to be able to see the search result (the mail items), we will use the OWA mail client interface to see the “content” of the Discovery Mailbox.
Choose the option: open.
1. OWA Mail client “First Time configurations”
When using the Search Mailbox at the first time, we will need to choose the required language and the Time zone settings.
2. Viewing the content of the Discovery Mailbox
In the following screenshot, you can see that a “special folder” was created (using the name that we choose in the former step for the search).
Now, we have the option to see all of the mail items that were found. The “result” includes a copy of all the mail items that “exist” in John Mailbox.
To be able to find the required deleted mail items (mail items that were “Hard Deleted”), we will need to use some kind of “Filter” such as: filter mail by date, by sender and so on, to be able to find the specific mail items that we are looking for.
You can read more information about the Search Mailbox by using the following link:
In this section we will provide only a basic review about the concept and the use of the Retention policy. Retention policy is an Exchange service which used for implementing efficient management of mail items. The “Management of mail item” concept is implemented by: deleting “unnecessary mail items” from the user Mailbox, or move “old mail items” to the Mailbox Archive (in case that we use the option of Archive Mailbox).
The “Retention policy” is actually a collection of “Retention Tags”. Each tag includes instructions about:
- “What to do with a mail item”- the option are: delete or archive
- “When”- the “when” is implemented when the mail item reach a specific “age”
Exchange online, includes Retention policy named: Default MRM, which is applied by default to each of the Exchange online Mailbox's.
One of the tags that include in the Default MRM retention policy is the: “Deleted mail items”.
This Tag is attached to the Deleted items Folder, and it’s configured to delete each of the mail items that is older than 30 days.
Users (and Exchange administrator) that don’t know about this default will assume that deleted items will stay in the “Deleted item Folder” forever.
Deleted items policy
Additional “element” that we should know is the “Deleted items policy”.The “Deleted items policy” is “attached” to the: Deletion Folder and the Purges Folder and, it’s purpose is to “clean” the content of this folders (by deleting mail items stored in this folders), when the deleted item reaches a specific age. (The default value of the Deleted items policy is: 14 days).
Deleted mail scenario
To demonstrate the “flow” of deleted mail items, let’s use the following scenario:
User deletes some mail items (the deleted mail items saved in the Deleted items Folder). When the mail items reach the age of “30 days” the mail items will be deleted (because of the use of the “deleted mail items tag). In other word we can say that the mail items will be moved to the “Deletion Folder” (a sub folder in the Recoverable Items Folder).
The mails items that were deleted (moved to the “Deletion folder”) will stay in the folder for 14 days and then, will be permanently deleted, (there is no option for recovering this mail items).
Configure the Retention policy tag
The good news is that we can change the default Tag that relate to the “Deleted items Folder” very easily. We can choose one of the following options:
- “Extend” the time period of the “Deleted mail items Tag”
- Disable the “Deleted mail items Tag”
- Remove/Disable the “attachment” of the Default MRM policy from user Mailbox.
1. “Extend” the time period of the “Deleted mail items Tag”
In the Exchange management interface, choose the Mail control menu and then choose the retention Tags icon.
In the screenshot you can see a list of different retention tags.
The tag that we are interested in called: Deleted item.
We can change the default value of the deleted item's tag from 30 days to any other value that will suit our needs
2. Disable the “Deleted mail items Tag”
We can disable the “Deleted mail items Tag” by choosing the option: Never.
The meaning is that the tag will be disabled (or not active).
An additional option is to “Unassigned” the Default MRM policy for a specific Mailbox. When choosing the Mailbox properties, in the “Mailbox settings” section we can see that the Default MRM policy is “attached” to the Exchange online Mailbox.
We can choose the option of: “No Policy” to disable the Default MRM policy (including the retention tag for the Deleted items Folder).
You can read additional information about Retention policy by using the following link:
Default Deleted item policy
The Deleted item policy is an: Exchange server policy, which “Attached” to the Recoverable Items Folder. The purpose of the “Deleted item policy” is to “clean” the storage of the Recoverable Items Folder, by “removing”(deleting) the mail items after a specific amount of time. In other word, we can relate to the Recoverable Items Folder as a “Temporary storage for deleted mail items”.
The default value of the deleted item policy is: 14. The meaning is that deleted mail items will be saved in the Recoverable Items Folder for 14 days.
An interesting fact, which is usually unknown for Exchange online Administrators, is that: in case that the organization purchase “E” (Enterprise) subscription, we have the ability to change the Default value of the Deleted item policy.
As mentioned, the default value of the Default deleted item retention policy is: 14 days. We can easily “extend” this value up to 30 days. By doing so, we get a “wider windows of opportunity” for recovering deleted mail items.
Theoretically, this value can be extended even more than 30 days but, to be able to use this option, you should contact Microsoft Office 365 support team, and get more information about the available options.
The “extension” of the Default deleted item retention policy to 30 days cannot be implemented by using the Exchange online Web interface. The only option that we have is by using PowerShell.
In the following screenshot we see the default values of Exchange online Mailbox. You can see that the value of the property: “RetainDeletedItemsFor” is 14 days.
To change this value to 30 days, we can use the following PowerShell command:
In case that we want to test this “Hard limit”, let’s use the PowerShell with a value “larger” then 30 days. In the following example we try to use the PowerShell command with a value of 31 days.
The result is the following error:
The option of Litigation Hold was created for a special scenario, in which a company suspect that a specific employee perform Illegal action or actions that need to be investigated.
In this scenario, the basic assumption is that this user will try to “cover his tracks” be deleting items that could incriminate him or reveal his actions.
The solution for this scenario described as: Litigation Hold.
The option of Litigation Hold is available only for Plan E2, E3, E4 Subscriptions.
When we use Exchange to “put a Mailbox” in a Litigation hold, a special flag will be attached to the Mailbox. The purpose of this flag is to change the default behavior of the Exchange server that relates to Deleted mail item policy.
In case that we put a Mailbox in a Litigation Hold, each time the user delete or change a mail item, this mail item will be kept forever in the Recoverable Items Folder. Mail item that was deleted will be kept in the Purges Folder, and mail item that was updated will be kept in the Versions Folder.
Access to deleted mail items is implemented in by using the Multi-Mailbox Search.
Another way to look at the Litigation hold is as a “way to overcome the Deleted item Retention Policy”, and provide the option to recover each of the mail items that was sent to and from the users Mailbox, instead of using the “Traditional Backup” solution.
It’s important to understand that this was not the original purpose of the Litigation Hold. The “real use” of litigation hold was designed as a solution for a limited period that should help us to keep information\data about the company employee that is “under investigation”.
In case that you still want to use the option of “litigation hold” as a “Backup Solation”, you should consider elements such as the Mailbox size. When using the option of the Litigation Hold for a long period of time, the size of the Mailbox could become “Huge” and affect the outlook user experience by slowing outlook performance, cause a synchronization problem and so on.
You can read more information about Litigation hold by using the following link:
Summary and Recap
In this article we reviewed Exchange server (and especially Exchange online) component and services, which relate to the subject of: Deleted mail items and, the task of recovering deleted mail items.
To summarize all we have learned, let’s mention again the “flow” of deleted mail items and, the charters of the different components:
When user deletes mail items, the deleted mail item is “moved” or saved in the “Deleted items Folder”. By default, the mail item will be deleted after 30 days. (The value of the “30 days” is determined by the Exchange online default Retention policy the Default MRM)
In this stage the deleted mail item is “moved” to the Recoverable Items Folder.
We can describe 3 optional scenarios:
(This stage describes as: ”Soft Delete”, because the user still have access to the deleted mail items). In case that the user decides to recover the mail item, the mail item will be “moved” back to the “Deleted items Folder”
When a user asks to recover mail items, we can say if the deleted mail items are still “recoverable” by using the “44 days formula”. (Deleted could be considered as a “recoverable” for 44 days).
The number “44” is created from the: 30 days that deleted mail items will be “Kept” in the Deleted mail Folder until he will be deleted + the 14 days (The “Deleted items policy”), until the mail item will be permanently deleted (When the retention period for deleted items expires, items are permanently removed from the Exchange Online Mailbox).
For your convent, I have “Wrapped” all of the PowerShell commands that was reviewed, In a PowerShell Script named: Retention-Policy.ps1
We really like to know what is your opinion on the Article