Fire Drill definition

In the corporate world, the meaning of fire drill has been modified to suggest that any activity that is a waste of time is called a “fire drill”.

Thursday, October 31, 2013

Enterprise Vault MAPI troubleshooting

An often overlooked component in Enterprise Vault (EV) Server health is MAPI connectivity.

Main EV Components that use MAPI:
Agent Client Broker
Archive Task
Exchange Provisioning Task
Journal Task
Public Folder Task
Retrieval Task

One of the main culprits to MAPI connectivity issues is the 100 session limit in the Outlook 2007 SP3 client.  When EV was originally designed there was no MAPI session limitation in Outlook 2003, Outlook 2007 first introduced a 32 session limit, but this was hotfixed and later rolled up into SP3.  Latest EV Deployment Scanner checks for the existence of this hotfix or SP3 when you run it.

Outlook 2010 removes this limit, but it is not on the compatibility list (yet).

The first article to troubleshoot possible MAPI issues is a good start: Troubleshooting Exchange connectivity issues with Enterprise Vault (EV)

Common Event IDs indicating MAPI related issues include: 3411, 3431, 3432, 3230, 2263, 2246

Decoding the error codes:

Error Code Log Information
8004010F mapi_e_not_found
80040111 mapi_e_login_failed
8004011C mapi_e_unconfigured
8004011D mapi_e_failoneprovider
80040700 mapi_e_ambiguous_recip
80004005 mapi_e_call_failed

Warning: Incorrect use of the Windows registry editor may prevent the operating system from functioning properly. Great care should be taken when making changes to a Windows registry. Registry modifications should only be carried-out by persons experienced in the use of the registry editor application. It is recommended that a complete backup of the registry and workstation be made prior to making any registry changes.

Sorry, couldn't help myself.

Here are a few registry keys that I have been implementing in our client environments to ensure MAPI stays healthy:

1) If you are currently archiving/journaling from an Exchange 2010 environment, you should change the following registry key:

Key Location Settings

DS Server
HKEY_CURRENT_USER
\Software
  \Microsoft 
    \Exchange
      \Exchange Provider        
Value name: DS Server
Data type: REG_SZ (string)
Value data: FQDN of the CAS server

If you're using a load balancer in front of your CASarray, you do not want to use that FQDN due to MAPI timeout (F5, Barracuda) issues, this does introduce a single point of failure, but if your CAS server is down, you have bigger problems than EV archiving to fix.

Check out the More Information/Exchange 2010 Server section of the following TechNet article for Microsoft's take on the subject: http://support.microsoft.com/kb/319206 - Oh, NSPI...

2) The next three live in the Agents registry key, here's a reference of most of the keys: http://www.symantec.com/docs/HOWTO32040

32-bit: HKLM\Software\KVS\Enterprise Vault\Agents\
64-bit: HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\KVS\Enterprise Vault\Agents

Value Name: ProfileExpire
Value: Change from 3 to 1
The default for this setting is 3 . This represents the number of days before systematically created Outlook Profiles are expired.  Changing this value to 1 (day) will expire the systematically created Outlook Profiles faster, improving overall Outlook performance.

Value Name: RestartOnMAPIMutexError
Data Type: DWORD
Value: 1

Here's another article, which references RestartAllMAPITaskIntervalMins, I tested it, and it was restarting tasks at different times of the day and generating extra archiving reports as a result: http://www.symantec.com/docs/TECH68433

Value Name: RemoveEmbeddedAttachments
Data Type: DWORD
Value: 0
Note: this one isn't documented really well, I came across it from the Symantec Connect forums, and confirmed it's usefulness with Symantec Support

3) At the end of our maintenance window, I have a scheduled task that restarts the Task Controller service nightly on each EV server, this will clear/reset the MAPI resources in use or resources that could not be cleared successfully.

4) Reduce the total number of threads used by all the tasks on a given Enterprise Vault Server. Each thread of a Mailbox Archiving task can hold two MAPI sessions (one for archiving and the other for synchronization) and hence setting the task to 5 threads/connections, actually means 10 profiles/sessions in the Outlook cache.

5) The Vault Service Account (VSA) temp directory contains many ExchangePerflog*.dat files and is filling up the drive.

HKEY_LOCAL_MACHINE\SOFTWARE\KVS\Enterprise Vault\AdminService
Value Name: DelFileTypes
Data Type: String
Value: ExchangePerf*.dat;EV$*.*;*.tmp,*.dat
Reference: http://www.symantec.com/docs/TECH61099

No comments:

Post a Comment