Notes/Domino Best Practices: Sizing
This document contains a checklist to use when you need a new Domino Mail server, or are getting your first Domino server. While there is no one single answer to sizing, we discuss things to consider about how big your Domino server should be.
This document is one in a series of documents included in the Domino Best Practices Master Checklist (#7008523).
This is a living document and will continue to evolve over time.
|Domino Server Sizing Checklist|
1. The basics of sizing
2. Factors that can affect your sizing
3. References to learn more
It is important to understand that there are numerous topics related to sizing. For our discussion, we will assume the following definitions:
Sizing - the process of determining the required server machines for a specific application at a point in time
Performance - the process of tuning an existing environment to obtain optimal response, improved efficiency, and/or addressing specific issues.
Capacity planning - the process of monitoring usage of your system, understanding changes that happen over time, and planning how you would address the changes to your environment.
Sizing is subtly different from either performance tuning or capacity planning, but all three are interrelated. If you have a large Domino installation with many servers, your Domino architecture is also a significant factor in sizing. You must weigh all these factors to be sure you have a healthy Domino environment.
This document addresses sizing, yet presumes that you have your hands around your performance tuning, capacity planning, and architecture. You may be planning for a new implementation of Domino, planning a server consolidation (a significant change in architecture), implementing clustering to a different site for disaster recovery purposes, or just replacing old, out of date equipment with new. What you need to determine is exactly what new equipment you should buy. How many machines, with how many processors, what speed, how much memory, how much disk? And this for each machine involved, as they may be different sized machines needed, and maybe even different hardware/OS platforms for the different machines.
In one internal IBM system it was measured that average peak processor usage per day in August was 20% less than the peaks in February; the probable cause, August is the prime time for family vacations, so fewer people are working. Not only were processor peaks down, but also number of messages, concurrent users, or almost any measurement. Only by monitoring your Domino environment over a year or two will you really know your sizing needs (again, this ties in with capacity planning). The key here is to understand that if you look at your system on just one day or one week, and presume all weeks are the same, or that you have established your peak, you may miss your mark by a significant distance if you use that information to size your new servers.
You need to have a solid architecture before starting the process of sizing. While you may not know the “size” of specific boxes or absolute number of machines, you should know how many specific, distinct “boxes” you want with different functions. After sizing, you may yet grow in a horizontal mode by having multiple machines that are needed for a logical “box” in an architecture, or you could have a large machine with multiple LPARs (logical partitions), each with multiple DPARs (Domino partitions), that end up representing a “box” in an architecture diagram.
Some hardware vendors, like IBM, do have internal resources with Domino sizing skills that they can call on to help their sales teams do sizing. Again, ask your vendor.
All sizing tools and methodologies are based around some concept of the “typical user”; in the real world there is no such thing as a “typical user”. Your company will be unique. Different groups in your company will have different ‘profiles’ of how they use Domino. That sizing, whether you do it yourself or get someone to assist you, will have some significant margin of error. As a result, you should not be too aggressive about sizing unless you understand your environment and your users, and the potential for changes to a very detailed level. It is often worthwhile to perform sizing, then compare this to the resources you are using today. If the ‘total resources’ from your sizing vary greatly (high or low) from what you have today, you need to investigate and determine why.
|Sizing to address current issues|
First, if you have “response time problems”, understand if it is really caused by the limits of your current system, for instance exhausting the processor. Understand that there are other possible culprits. Could you have memory problems (paging, swapping)? Do you have I/O bottlenecks (waiting on disk I/O)? Have you completely drilled down on all aspects of your current performance issue to understand the underlying cause of the problem? Or are you just looking at a symptom that may be several steps removed from the real problem? There are many things that can influence response times, such as the network or the workstation itself, especially when the workstation tries to perform certain functions with very large mail files.
Second, understand that if you do have a system that is “constrained” by lack of resources, you most likely have significant ‘pent up demand’ from your users. They may react to this by trying to move their usage to ‘off times’ when performance is better, or they may just use ‘less mail’ than they would like. If you do not take this pent up demand into account as you size you may find that your users will take advantage of the new hardware, go back to using Domino as much as they want, at the times that they want, and your new machine that you thought had sufficient ‘growth room’ is suddenly ‘consumed’ as soon as you roll it out.
|Adding new features|
If you do not run transaction logging, but plan to turn it on with a new release/hardware, understand that it takes some resources. Understand what "roaming users" do and how you set them up can change the amount and type of required resources. Consider also full text indexing, SSL, newer versions of the Notes Client. or implementing local replicas. These are all great functions, but all come with some potential impact, maybe more, maybe less, on server resources. While this is getting into performance and capacity planning, understand that if you have ‘plans’ for these kind of things, then when sizing for new equipment, you should take these factors into consideration.
When you work on sizing your Domino servers, you must consider what the users are doing, how they do it, how often they do it. For mail, an application you get with Domino, a lot is known. For custom applications you build yourself, no one knows but you. While almost any option can effect the amount of resources needed for your server, taking a minute to think about "how much work might that cause" will give you valuable insight.
Here is a list of "big items" that are frequently encountered when sizing mail servers, and a brief explanation why and how they can have impact. But consider more than just these factors, since really almost any option you can implement will have some impact, one way or the other.
Client used: The client machine a user uses to interact with Domino changes what Domino must do. Notes is "client server" oriented, off-loading work to the client workstation; Domino Web Access (DWA) is much more "server centric" doing a majority of the work on the server, and requiring added tasks there such as dynamic translation of documents to HTML, generating URLs for documents in your inbox and so on. If you look at Notesbench.org, or some of the performance links, you will see that a specific server will handle fewer DWA users than it can handle Notes users.
How 'weighty' are the users: Clearly a casual user, with a small mail file, only a few mail items coming/going each day, will have less impact on your server than a 'power user' with huge a mail file, hundreds of incoming/outgoing items each day, and a heavy use of calendar. While there is no specific way to exactly describe light, medium, heavy users, think about that concept. All your users will not likely fit any one description; most organizations have a variety.
Concurrent active users at peak 15 minute period: Clearly sizing is an attempt to determine peak needs, and then provide sufficient hardware/architecture to handle that load. Therefore, having a good understanding of how many users need to be served at peak time (doing what, with which client, ...) is critical. Understand that this could be a specific time of day, a specific day of the week, or even have large seasonal swings depending on your business. Be sure to know the biggest peak over long periods, not just "today", and size for that big peak.
Mail file size: As mail files have grown at a tremendous rate over the last few years, their size has had impact on the servers. One of the reference articles addresses this. Understand that it is not just processor resources; many are finding that with larger mail files, and huge disk drives, they are suffering from I/O delay. Domino mail does a lot of I/O, and it is random I/O, so the number and speed of your disk drives becomes a sizing concern as well. If this is really becoming an issue, do you have plans for quotas, archiving, or other controls that will have to be accounted for in the sizing?
Clustering: This can affect your sizing efforts in several ways. First, when sizing, you have to look at "worst case"; how many users do I need to support when my failures occur at peak time. If you have a complex cluster architecture, then you may have to think a bit about what sets of users fail where, and how many machines failing are you trying to handle at once. If you do not load balance the clusters in some way, or again if you have a cluster architecture that is not a simple one to one, understand that cluster replication creates an added workload, beyond the normal workload, and you have to add that into your calculation.
Local replicas: Using local replicas can change your equation for sizing, and it can, if done right, reduce needed server resources. With a local replica the normal workload of user activities is moved to the workstation. It is replaced with the load of replication. Therefore, you need to pay attention to the replication interval you or your users specify, and if the users are actually replicating more than one database to your server. Consider that essentially all users will now be active. If their Notes client is up it will be replicating during the scheduled time frame. Think about xxx users replicating n times an hour, multiplied by however many files. If the total number of replications an hour takes your breath away, think about adjusting the interval. We would recommend 10 or 15 minutes in most cases, but the user can always force replication from their replication tab if they need to. And a warning: if your users keep a local replica going "just in case" but use the server copy of mail for normal work, then you have to account for both work loads. With local replicas you are likely to see a "consistent workload" as replication happens consistently all day, with much smaller peaks than you would see with users accessing the server copy.
Port Encryption/SSL: With so many concerns today about security you may implement these. Understand that if done by Domino you are using software to encrypt and decrypt each message that flows, using sophisticated formulas. Clearly this can have server impact. Some do use "devices" in their network to do SSL, rather than adding this workload to their Domino server.
Full Text Indexing: This is a useful feature; makes things easier for your users to find. However, it will increase the size of the mail files to contain the index. There is, of course, processor impact as well. It takes resources to build and maintain the index, and, once there, your users will "search", thus adding a new function to their work scenario, requiring more server capacity.
Roaming user: This function can have more or less impact depending on your implementation choices. Understand that it adds a replication workload (back to the local replication discussion on interval), and when a user signs on to a "new" workstation it causes the creation of new replicas as they download their information from the server.
Transaction Logging: Transaction logging is a great function, has lots of advantages and many different options, but is clearly something that will take some amount of resources. Understand that you are writing updates a second time, so server resources are used. You should plan on having a separate disk drive for this, as it is written sequentially.
Anti virus / anti spam: Often these applications are implemented on your Domino server with third party software. You should ask the vendor for advice on the impact, and it may well vary based on parameters you chose. Understand what it going to happen. Every message is going to be "evaluated, analyzed, and tested" to try and determine if it is acceptable. Clearly this takes processor time, and it is probably dependent somewhat on the amount of traffic flow in and out of your server.
Agents: Look at agents you implement (or add to the Domino system with add on products), as well as your users, if you allow them to create/run agents. Evaluate agents you write/add before putting them into production. Look at the agent manager load on your system today. Take that into account in your sizing. And if you let users write agents, understand that users do whatever users want to do, and sometimes things they really did not mean to do. Add something into your sizing to account for that, and monitor what really happens.
Outside applications: There are many Lotus partners who sell "add on" functions for Domino. These are great things, and provide great functions, but are something you must account for in your sizing. They may have minimal impact, or they may have a lot of impact. Sending and receiving faxes via Domino is probably not much impact from the occasional fax, but would be a concern if you have users who regularly "fax" to large groups. Mail interfaces to cell phones probably don't have much impact if this is just for a handful of executives; however, if you have a significant number of users, think about how many times the mail file is opened, scanned for new mail and closed (number of users, number of times a minute/hour). Understand that these functions typically run for every user enrolled, all day, every day, and they create a workload over and above normal user load. Look at each application you add to your mail server and consider how much impact it might have, and ask the vendor who supplies it for their assistance.
Growth: Here we venture a bit into the world of capacity planning, but clearly you do not want to go through a sizing process only to find that by the time you have implemented the new hardware it is already "consumed" at peak times. If you have been doing capacity planning over time, take your growth rates (processor, I/O, mail file size, number of users, and so on) into consideration, and add a year or two of anticipated growth to your sizing. Plan ahead.
Be sure to understand the limitations and constraints that your chosen operating system, hardware platform, and architecture bring into play. As Domino is still a 32 bit application, you will find at times that memory is a constraint. More/faster processors may not be the answer. It may be that you need more machines with just two processors, or more Domino partitions (DPARs) on a single machine where that is a reasonable choice (probably not for Windows). Sizing is really about more than processors; it requires a proper melding of processor, memory, I/O, architecture, and understanding the overall picture. When you are getting new hardware because you have some performance problem, be sure you really understand what the problem is, and that your new solution really solves the problem.
And lastly, when you perform sizing, take time to go back and compare the new equipment to the old. Overall, do you have more "horsepower"? Does it seem reasonable at an overall level, the new compared to the old? If you were having performance problems on your existing hardware, understand that your users would really like to do more but have been frustrated by poor performance. There is some amount of "pent up demand", and when you bring in the new equipment you will 'release' that demand and may well consume what you thought was going to be your 'headroom' for some time. Therefore, during sizing, add some resources to handle this pent up demand.
|Sizing Resources||Date Published|
|Redpaper - Domino 7 Server Consolidation: Best Practices to Get the Most Out of Your Domino Infrastructure
Since the introduction of Domino 7, IBM/Lotus has been highlighting the capabilities of Domino to now do more with less - ultimately using Domino 7 more efficiently to better leverage existing investment - and ultimately reduce the Total Cost of ...
|17 Aug 2006|
|Redpaper - Domino 7 Performance Tuning Best Practices to Get the Most Out of Your Domino Infrastructure
Performance tuning, or optimization, is the process of modifying a system's configuration in order to improve its efficiency at meeting specified functional goals. The systems in question can be a single application, one Domino server, a collection of ...
|09 Aug 2006|
|Sizing your IBM Lotus Domino mail servers
Get practical sizing information to help you plan your Lotus Domino mail environment. See which factors most affect sizing, learn how to plan for growth, and find out what information the Statrep.nsf database can provide.
|28 Jun 2006|
|Best practices for large Lotus Notes mail files
Learn how you can manage the ever-growing mail files of your Lotus Notes users, conserving your system resources while ensuring your Notes users continue to enjoy high performance and reliability.
|11 Oct 2005|
|Redbook - Lotus Domino 6 for z/OS: Performance Tuning and Capacity Planning
This IBM Redbook helps z/OS system programmers and Domino administrators to monitor and tune Domino 6 for z/OS. It identifies factors that affect performance and provides recommendations to tune the configuration and parameter settings for optimal ...
|29 Nov 2004|
|Redpaper - Domino for IBM eServer xSeries and BladeCenter Sizing and Performance Tuning
This IBM Redpaper will guide and assist you in selecting the correct server configuration for your selected Domino system, whether the system is for Web, application, or mail hosting. This Redpaper provides performance tuning techniques and best ...
|12 May 2004|