Exchange Geek's Weblog

I'm a Geek!

Archive for the ‘MBX’ Category

How to Recover a Mailbox Server that is a part of a DAG

Posted by Milind Naphade on 13/10/2011

A day before yesterday, I was asked if an Exchange Server 2010 mailbox server which is a part of a DAG can be recovered using disaster recovery setup options.

Due to considerable changes in the ways Exchange 2010 handles the clustering components it is little tricky to recover a crashed server that was a part of some DAG in an exchange organization. In Exchange 2007 you could run setup.com /m:recovercms to recover a clustered mailbox server node. But, in exchange server 2010 you have to do it a little different way. Let us take a look at how to do this. It is highly recommended that you make a note of your database copy deployment layout before you go ahead.

For example:

Let us say you have a 2 member DAG with a database:

DAG Name: DAG01

Node1: E2K10SP101

Node2: E2K10SP102

Database: Mailbox Database

In my case the server E2K10SP102 crashed and the OS could not be recovered at all.

1. Remove Database Copies Configuration

When you add a member into DAG and deploy database copies across the members, this information is stored in active directory. Since this information is not removed automatically when either of the members a DAG crash or encounter an irrecoverable problem we need to clean this information first. During the cleanup process you essentially remove the information of the database copy that was configured to be peer of the server that is no longer available.

To perform the cleanup of the database copy run below command using EMS

Remove-MailboxDatabaseCopy –Identity “Mailbox Database\E2K10SP102

Right after you hit enter you will be prompted to confirm your action and you might also see your Management Shell screen filled up with warning messages. This is totally okay. In ideal situations, the server will try to clean up this information from another member’s local configurations as well. Since your other server is no longer available the local configuration data from other node will not be cleaned up. This will delete the database copy information from AD though. You need to repeat this for every database copy if you have more than one.

2. Remove DAG Member

Next step is to remove the crashed DAG member from the configuration.

Remove-DatabaseAvailabilityGroupServer –Identity DAG01 –MailboxServer E2K10SP102 ConfigurationOnly

ConfigurationOnly is very important here. If you miss this parameter in the command the server will try to contact the other node to remove the cluster service configuration and will fail since the other server is already gone. Keep in mind that we are removing all of this information from AD and not the other node so this parameter should not be missed.

3. Remove Cluster Service Configurations

As we all know, although DAG do not use failover clustering mechanism; it still uses MSCS failover clustering components. With said that there are some cluster service configurations in service control database and quorum needing cleanup before the crashed server could be brought back into the DAG. To clean up cluster service configurations open command prompt as a privileged user if you have UAC enabled on the server and simply type below command:

cluster.exe DAG01 /Node E2K10SP102 /Evict

Once you have completed the cleanup. Next steps are to rebuild the server, add it back to DAG and configure database copies.

4. Rebuild the Server– If you have completed fixing your hardware issues (If at all the server had any  🙂 ), it is time to install the OS similar to what patch levels  and drivers what you have on the currently working node.

5. Reset the computer account – After your server OS is built correctly and you have made sure the path level is matching to the currently working box, reset the computer account of this server in AD and join the computer with same name in the domain.

6. Install Software Prerequisites for Exchange 2010 -If you had more than one server roles installed on the server you should install all applicable prerequisites for each server role.

7. Recover the Server – Insert Exchange 2010 RTM / SP1 disk (depending upon what release of Exchange you are running) and install exchange server binaries using command line setup. Run setup.com /m:recoverserver from command prompt.

8. Add the server to DAG – After the installation completes, reboot the server once and add the server in DAG using below cmdlet

Add-DatabaseAvailabilityGroupServer –Identity DAG01 –MailboxServer E2K10SP102

This will initiate the cluster service configuration on local as well as remote computer.

9. Add Mailbox Database Copies – If you have reached this step that means you are almost done and need to execute following command to database copy configuration completed.

Add-MailboxDatabaseCopy –Identity “Mailbox Database” –MailboxServer E2K10SP102 –ActivationPreference :2

Done! You just need to monitor how the replication of this copy is going; using Get-MailboxDatabaseCopyStatus –Identity “Mailbox Database”

Hope this information helps!

Posted in Exchange 2010, MBX | 4 Comments »

What happened to System Attendant Mailbox in Exchange 2010?

Posted by Milind Naphade on 12/10/2009

A one word answer to this question is: REMOVED.

As we all know that System Attendant mailbox was used to publish the free/busy info and the link monitoring service in earlier versions of Exchange. In Exchange 2010 it the mailbox object for System Attendant has been removed. Yet, you can locate the directory object for System Attendant in Active Directory. Per a post published by Dave, the mailbox object for system attendant has been removed but can be seen in AD. This change was done due to the new high availability architecture of Exchange 2010. They observed problems in failovers while having the mailbox object for System Attendant.

How will this affect:

If you are running an application that uses System Attendant to connect to the servers/databases you will still be able to connect but not logon. That means you wont be able to open the system attendant mailbox anymore as the actual related mailbox doesn’t exist.

The way around it:

1. Use the per mailbox database System mailbox (See Additional Resources for more information about this mailbox).

2. Change your application to rely on a user created account and mailbox.

This has been described pretty well at he System Attendant mailbox has been removed from Exchange 2010

And do read the additional resource section to understand the facts about special mailboxes in Exchange.

Posted in Active Directory, Exchange 2010, MBX | Comments Off on What happened to System Attendant Mailbox in Exchange 2010?

Configuring Message Moderation for Email Delivery to DLs in Exchange 2010

Posted by Milind Naphade on 28/08/2009

We all know that we can set restrictions on distributions list so that the restrictions can stop unauthorized people from sending emails to a specific DL. For Exchange 2003 and Exchange 2007 we could only select these restrictions from a combination of “From authenticated users only”

Exchange 2010 offers an additional feature that can allow moderation of emails sent to a particular DL. This would be a useful feature to most of the managed DLs which are critical and emails sent to them should be authentic and scrutinized before they hit the DL member mailboxes. Let’s take a look at the configuration of this feature.

1. Open EMC and locate the DL to be set for moderation settings. Right click on the DL you wish to be moderated for email messages sent to it and click Properties. A dialog box pops up, then select Mail Flow Settings Tab on the properties dialog box. Select Message Moderation option. image

2. After selecting Message Moderation option from the properties dialog of DL you see another propertied dialog where you can specify the actual moderator of the messages sent to this DL. Here you can do 2 things,

First, you can assign a moderator to review all emails and then approve them to be delivered to DL members and then some exceptional senders who wont require an approval from moderator of the DL. One thing common between both these options is only a user having exchange mailbox can be added to these lists. and then, the last section is to specify the moderations notifications. If you want to let email senders know the status of their sent emails whether they were approved or declined you can configure it using “Notify all senders when their messages aren’t approved”. If you want the senders to know about the status only when the message was not approved then you can select the option to “Notify senders in your organization only when their message is not approved” and the last option can be chosen so that nobody gets an approval or denial notifications.

image

3. Once this is configured you can test it by sending an email to this DL. In my case I had the administrator as a moderator to the DL named “All Users” (The name of moderator isn’t visible on the dialog box above).

image

4. The moderator of this DL gets an email asking the approval. The moderator can preview the original message in the preview pane of OWA.

image

5. The moderator can open the original message which is sent as an attachment along with the approval request. Please see the mouse cursor in above screen shot. When the “View original message”

 imageoption is clicked the original message will open up in another window. Here in the original message window you get options to approve or reject the message delivery. You will notice that the sender of this message is not the original sender. The sender name appears “Microsoft Exchange Approval Assistant on behalf of AD RMS SVC

image

6. Now depending upon the settings you have chosen in step 2 the approval or rejection notices will be sent to original sender of the message.

image 

Here is something that you would like to know before you consider configuring message moderation for distribution lists, if you have your exchange 2010 servers installed in a mixed mode environment then message moderation may not work correctly sometimes. This may be because of the DL’s expansion server. Exchange 2010 Transport is also rewritten with many new components in it and message moderation is one of them. Exchange 2007 and 2003 transport wont understand this configuration as this is an Exchange 2010 specific feature. So, if you want to use this feature for any DLs in your organization  you must set your Exchange 2010 server as an expansion server for the DL or all of them.

.

Posted in Exchange 2010, MBX, Transport | 1 Comment »

Exchange 2010 High Availability – Best Copy Selection

Posted by Milind Naphade on 20/08/2009

Everyone who has started playing around the Exchange 2010 Beta must be knowing this concept by now. Compared to its predecessors Exchange 2007 and 2003, Exchange 2010 has been completely rewritten on this part. High availability is handled the way different than it was in any other version. Why companies spend so much of money out of their IT budgets to plan, deploy and configure high availability of services is because to make sure that their business critical applications wont go down and loss of money due to down services should not happen. Exchange being a messaging system has become one of the most critical applications for enterprises. Most of the communication happens via emails. Documents exchange, Voice mails on your phone, business communications, newsletters and much more is sent and received using messaging systems in every company. Consider a scenario where your company is a financial firm and needs continuous email communication to the customers, partners and other government authorities which should be available 24x7x365 days. In such a high demanding environment failure of even a single server for a long time may lead into loss of company revenue and company’s customer dissatisfaction. To overcome such scenarios Exchange have been providing the HA capabilities to administrator and architects. Right from Exchange 2003’s failover clustering to Exchange 2007’s CCR the Exchange HA has been evolving according to market needs and business requirements. This is one of the great features I ever liked in Exchange. Lets take a look at how the whole stuff is handled here.

Exchange 2003 and 2007 provided the HA based on Windows Clustering where the whole server as an object used to failover to another node of the cluster. However with the exchange server 2010, things have changed dramatically. Now the failover wont occur at the server level instead of that the store schema has been re-structured in such a way that the failover can occur at the database level only. This is possible with the help of Database Availability Group which is commonly known as DAG.

In short, DAG is a group of servers and databases which will provide the high availability. It still uses the Microsoft Clustering Services but do not rely on it completely. Instead of using the server level failover the clustering services are used to only group the nodes of a cluster. What will failover is the only a problematic database. Article Understanding Mailbox Database Availability provides an overview of how this whole stuff works. If you read the above linked article you will understand that a database will have multiple copies on one or more servers which will be passive copies and will not be used for production connectivity. However, these copies are continuously in-sync with production copy of them and are always updated. When a failure on  production copy of database is detected, one of the passive copies are activated and start working as a production copy for the clients. This is known as  failover.  On the other hand, if one of the passive copies are activated manually by an administrator the process is known as swithover.

As far as swithovers are concerned the administrator knows which copy to bring online. But, if the failure occurs and the administrator is not available to monitor or recover the situation then what? Don’t you worry about that. The failover is managed by the exchange store itself.

Exchange 2010’s Exchange Replication Service which monitors the databases time to time and determines their health if some sort of database failure is detected then the process of failover is started. To failover the databases in stead of the whole server a component called Active Manger is added as a part of Exchange Replication Service which replaces the cluster’s server level failover behavior. Now, when a failover occurs and you have more than one copies of a single database added to multiple servers; exchange has to decide and choose the best copy available to mount among all. This process is known as Best Copy Selection. How Best copy selection works is on a simple basis of choosing the best available copy among all database copies of a particular information store. But, hold on. This selection process includes 10 different criteria which are used to select a best of copy. Again, the one and only Active Manager manages all this stuff. Let’s see how does Active Manager decides which copy to pick up and activate.

So now as you already know that Active Manger will select the best copy and initiate the failover it looks for healthy database copy first of all, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource, and that meets all of the following sets of criteria:

  • It has a content index with a status of Healthy
  • It has a copy queue length that is < 10 log files and
  • It has a replay queue length < 50 log files

If Active Manager finds no copy meeting any of the above criteria, then it will try to locate a database copy that meets the next set of criteria:

  • It has a content index with a status of Crawling
  • It has a copy queue length that is < 10 log files and
  • It has a replay queue length < 50 log files

If Active Manager finds no copy meeting any of the above criteria, then it will try to locate a database copy that meets the next set of criteria:

  • It has a content index with a status of Healthy and
  • It has a replay queue length of < 50 log files

If Active Manager finds no copy meeting any of the above criteria, then it will try to locate a database copy that meets the next set of criteria:

  • It has a content index with a status of Crawling and
  • It has a replay queue length of < 50 log files

If Active Manager finds no copy meeting any of the above criteria, then it will try to locate a database copy that meets the next set of criteria:

  • It has a replay queue length of < 50 log files

If Active Manager finds no copy meeting any of the above criteria, then it will try to locate a database copy that meets the next set of criteria:

  • It has a content index with a status of Healthy and
  • It has a copy queue length < 10 log files

If Active Manager finds no copy meeting any of the above criteria, then it will try to locate a database copy that meets the next set of criteria:

  • It has a content index with a status of Crawling and
  • It has a copy queue length < 10 log files

If Active Manager finds no copy meeting any of the above criteria, then it will try to locate a database copy that meets the next set of criteria:

  • It has a content index with a status of Healthy.

If Active Manager finds no copy meeting any of the above criteria, then it will try to locate a database copy that meets the next set of criteria:

  • It has a content index with a status of Crawling.

Even after doing so much of matching with defined criteria if Active Manager fails finding any copy meeting above criteria then it will try to activate any database copy with a status of Healthy, DisconnectedAndHealthy, SeedingSource, or DisconnectedAndResynchronizing. At the end if Active Manager does not find any copy that meeting any of the criteria the automatic activation (failover) will not occur.

Another good question may come up asking what if Active Manger find more than one copy which matches the above criteria? Answer would be, if more than one database copy meets all of the above criteria, then the configured value for ActivationPreference is consulted, and the database with the lowest value is activated and mounted.

 

I would like to thank Scott for presenting this information in his introductory video and my friend Amit for suggesting me to write up on it. Thanks both!

Posted in Exchange 2010, MBX | 3 Comments »