IBM Support

Open Mic Webcast: Notes/Domino Database Maintenance Best Practices - 30 September 2015 [Q&A, Presentation, Audio, Video attached]

Webcast


Abstract

Join us for a presentation on Notes/Domino Database Maintenance Best Practices. We will look at best practices for maintaining system databases vs. mail databases vs. custom databases. We will look at the Domino traditional maintenance tasks and clarify when those are needed. We will also outline some best practices and guidelines for using the 9x. Features DBMT and Compact Replication.

After a presentation, attendees will be given an opportunity to ask our panel of experts questions. Throughout the event, attendees will also be encouraged to comment or ask questions in the IBM Connections Meetings Web chat. Join us for this interactive, educational, lively session. Follow us on Twitter @IBM_ICSsupport.

Content

  • Topic: Notes/Domino Database Maintenance Best Practices
  • Date: Wednesday, September 30, 2015
  • Time: 11:00 AM EDT (15:00 UTC/GMT, UTC-4 hours) for 60 minutes
Presentation
OpenMic_DBMaintenaceBestPractices_093015.pdfOpenMic_DBMaintenaceBestPractices_093015.pdf


Audio Replay

ftp://public.dhe.ibm.com/software/lotus/info/Domino/OpenMic_Domino_DB_Maintenance_Sept2015.mp3
  • 56 minutes 57 seconds
  • Q&A begins at 49:33

Video recording

A video was made from combining the slides and demo and recorded audio from the session. You can download that video from this FTP site:
ftp://public.dhe.ibm.com/software/lotus/info/Domino/OpenMic_NDBestPractices_Sep2015.mp4


Q&A

Q1. What is DBMT? Is it an built-in feature? Where do you get it?
- DBMT stands for Database Maintenance Tool. It is a new feature to Domino 9.0 that allows you to configure the regular maintenance options that you are familiar with to run automatically on schedule to keep your databases running smoothly.
- DBMT ships with 9.0.x in your Domino Server kit. It's the dbmt process on UNIX; ndbmt.exe on Windows
- For more information, see the presentation attached above and you can also check this technote http://www-01.ibm.com/support/docview.wss?uid=swg27039379


Q2. What is the folder limit in Notes 9.0.1?
- There is not a predetermined limit but the database is limited by the number of "handles" that can be used

- There is not specific folder limit. Personally I have a few thousand folders in my mail file. However, there are performance and display issues that will be encountered when you have a large number of folders. So, in practice, a lower number of folders is recommended. There are also limits on the total number of design elements that you can have in a database (in the 10,000+ range) so you definitely want to avoid haven't that many.


Q3. What is the number of handles?
The number of handles is limited to 10,495. Notes can run out of memory handles when creating the sitemap for the list of views and folders in a database.


Q4. Per technote 1097770 , the number of handles is limited to 10,495. What does it mean?
That means that you definitely wouldn't want to have more than 10,000 folders in a database. In reality, though, I would recommend not having more than 5000. As mentioned, I have a few thousand and things like moving one folder to another can take many seconds.


Q5. What's the best way to figure out how many folders a user has?
There is no built-in tool to easily see the number of folders in a database. You can count them with the Domino Designer client, or with a short LotusScript program, though. There is also a tool called NotesPeek that will give you a count of each type of design element in a database.


Q6. How many documents can a database view can handle??
There is no limit to the number of documents that can display in a view.


Q7. IBM Support told me that notes.ini parameters have a limit of 256 characters, not 128. Is that specified anywhere in writing to confirm one way or the other?
- I am still looking but it looks like it is 256 byte limit: http://www.ibm.com/developerworks/lotus/library/ls-LotusScript_Connectors/
- As far as I remember the variable including value can be 256 characters, but the value can have 128 characters only.
- Some variables, such as JavaUserClasses. have workarounds for that limit. More information in - http://www.ibm.com/support/docview.wss?uid=swg21193964
- SPR# LMAN7V4HR9 - Notes.ini only gets 254 chars of the allowed 256 chars in custom INI variable. http://www-01.ibm.com/support/docview.wss?uid=swg1LO60217


Q8. When will the 64GByte physical file size limitation be lifted? We have apps which take 60+ Gbyte (even with all attachments stored in DAOS).
We are working on moving Views outside the .NSF in a later release of Domino


Q9. Is the DB Size hard limit of 64 GB for the NSF only or NSF+DAOS?
- It is NSF only... you can go beyond 64Gbyte if you enable DAOS.
- Just NSF... So go well beyond 64 Gig when DAOS enabled as the attachments are stored outside the .NSF


Q10. Does MailFileDisableCompactAbort work in 8.5?
Don't quote me but I believe so. The other LocalDelivery is new to 9.0.x.


Q11. If I set create_R9_Databases=1, could I just let DBMT eventually run which would do the compact -c?
YES! One of the nice features. I use it when doing ODS Upgrades all the time. with multiple threads...Helps reduce fragmentation as well at the same time...


Q12. To be clear...any reason not to just let it happen vs. forcing dBMT to run?
No reason.


Q13. How to convert ODS for daoscat.nsf after upgrading from 8.5 to 9, from ODS 51 to 52
No real need to convert it, but if you choose to do so: Quit Domino, put in the notes.ini create_R9_Databases=1, delete daoscat.nsf, and then issue Tell DAOSMgr Resync Force


Q14. Could you run DBMT with compact archive option?
- No. A separate schedule for your server based archiving needs to be maintained.
- DBMT does not support archiving. Should still use compact to schedule your archiving on the Domino server.
- You can create a program document to set a schedule for compact -a

    Tip: Run compact -a before you run DBMT if you can (or on alternate days). That's why I choose to schedule DBMT instead of at server startup.

Q15. How do you compact a custom database with scheduled agents?
The NotesDatabase class in LotusScript has Compact and CompactWithOptions methods that can be used in an agent.


Q16. Does DBMT run a different compact program than the one used by a load compact command?
DBMT uses some common code in the copy style compaction but it does have some additional smarts - like to avoid the fragmentation..DBMT compact is little more efficient than the load compact.


Q17. Can we have a GUI for DBMT some day?
Most likely not. But once you grok the parameters it's really not too bad.


Q18. Do I HAVE to use U.S. time format (am/pm), or does DBMT support the system's local?
System's local as far as I know..


Q19. What is the solution for error "unable to extend ID table"?
This technote has more information about the "Unable to extend an ID table" error -- http://www.ibm.com/support/docview.wss?uid=swg21413921


Q20. What is the recommendation for defragging Windows-based Domino servers?
This Wiki article provides some guidance about Windows fragmentation -


Q21. Is there any tech notes on defrag per system/OS?
No, steps involved would vary depending on the SAN, vendor, operating, and data.


Q22. Is there a way to tell Domino to disconnect from a database and close all file handles and not reopen them?
So you can issue a DBCache Flush on the server to release the Handles on the database; however, some tasks will not release like compact, updall, fixup until completed or if a user is actively working in the database. But there would be no way to prevent from opening while the server is running if in the data folder. No way to disconnect 100%.


Q23. Normally I would enable transaction logging on ALL databases, but some system database have in the past been suggested to be not transaction logging enabled. Does anybody have an updated list?
There has been discussion between customers, support and our development team regarding what should and should not be transactionally logged. Here is a list that is around on the internet:

    > names.nsf
    > log.nsf
    > mail.box(es)
    > admin4.nsf --Disable TXNs on admin4 if this is not the administration server

    > busytime.nsf
    > clusbusy.nsf
    > event4.nsf
    > statrep.nsf
    > statlog.nsf
    > webadmin.nsf
    > dbdirman.nsf
    > reports.nsf
    > mtstore.nsf
    > activity.nsf
    > ddm.nsf
    > 3rd party DBs - anti-virus DBs, spam DBs, etc.
    >
    > * You can do this by going to database -> properties -> advance -> disable > transactional logging.

From past developer:
    I say to always transaction log important databases. names.nsf is very important, your server doesn't do so well w/o it. mail.box should be logged as well (if you value mail), it is the first point of entry for email. admin4.nsf should be logged on your admin server. log.nsf needs special handling, if it is unlogged and over 1GB in size, the fixup that is run on it following restart will have a major impact on server startup. So you either need to rename it & let a new one be recreated or keep it small (we just run w/ txn logging enabled on it, especially w/ circular logging).
    I should mention, all concurrency improvements that are being made depend on the db being transactionally logged.
    With DAOS you will need to transactionally log your mail boxes


Q24. That is exactly the "list" that rumours around the internet. I also understand that many of the new benefits introduced by all new ODS requires translog enabled. I am still not confident with this unofficial list and the reason why the listed databases should not be translogged
On all my IBM Development domains I use Translogging on everything. In our SCN service we run Translogging on all databases.


Q25. You mention that enable transaction logging on a database helps improve performance. I don't think that has been the case in our environment so we usually disable it. One would think that logging transactions would take time so it could hurt db performance. Have any performance studies been done comparing dbs that have transaction logging enable versus disabled?

- Try to switch off your server by hard (=powerOff) and measure the time it takes to restart with Transaction Logging enabled vs. disabled. You'll see Transaction logging will greatly help to recover quickly

- Transactional logging does provide performance benefits throughout numerous customer environments. There are 3 types of logging Archival, Circular and Linear with Circular not requiring any backup product and requires 4 GB disk space to consume, archival depending on amount of transactions, number of incremental backups and size this will require a larger store and linear is the least used option. However it will allow a faster recovery should an outage occur as well as day to day operations. It does require a separate drive or dedicated space on the SAN

- Alone a 2 GB Database consistency check can take anywhere from 10 minutes + to complete which will delay startup or performance


Q26. Could IBM make a server task, that created the .ind files, instead of the powershell script?
You can dump the directories via command line:

Example of NSF dump into a text file by Size for the Domino Data Folder :

A) Pathed out to the Data folder on my test Server --> cd\lotus\domino\data

B) Issued the following command --> dir *.nsf /B /OS > C:\Lotus\Domino\Data\Datadump.txt <-- Dumps it in bare format and by size smallest to largest
Or Issued the following command --> dir *.nsf /B /O-S > C:\Lotus\Domino\Data\Datadump.txt <-- Dumps it in bare format and by size largest to smallest

C) If I want to dump out the contents of the Mail Directory I would path out to the Mail Folder --> Cd\Lotus\Domino\Data\Mail

D) Issue the following command --> dir *.nsf /B /OS >c:\Lotus\Domino\Data\MailDump.txt <-- Dumps it in bare format and by size smallest to largest
Or Issued the following command --> dir *.nsf /B /O-S > C:\Lotus\Domino\Data\MailDump.txt <-- Dumps it in bare format and by size largest to smallest


Resources

Transaction logging - http://www.ibm.com/support/docview.wss?uid=swg27009309

Technote 1663952 - UPDALL appears to hang when processing a database - http://www.ibm.com/support/docview.wss?uid=swg21663952

SPR GFUR9N6LD4 http://www.lotus.com/ldd/fixlist.nsf/SPRNum/E834F75711C0250785257DCC0074AC1F?OpenDocument&ca=drs-fo

SPR JPAI9G7NP2 http://www.lotus.com/ldd/fixlist.nsf/Public/F9C5303D5AFC0B1B85257CFC006E05AE?OpenDocument

SPR JPAI9VYQ8T http://www.lotus.com/ldd/fixlist.nsf/8ed1b46cfdba8957852570c90054623b/ebce9e7615d5f02f85257e5b00005e91?OpenDocument

SPR JPAI9WMMG5 http://www.lotus.com/ldd/fixlist.nsf/8ed1b46cfdba8957852570c90054623b/f8a722002b31d30685257e65006e23de?OpenDocument

Create_R9_Databases=1 - http://www.lotus.com/ldd/dominowiki.nsf/dx/CREATE_R9_DATABASES


About Open Mic Webcasts
IBM Collaboration Solutions (Lotus) Support wants to help you take full advantage of your products. Join us for our Open Mic Webcast series as technical experts share their knowledge and answer your questions. These webcasts are designed to address specific topics and provide an in-depth and focused technical exchange in a convenient online webcast format. For more information about our Open Mic webcasts, visit the IBM Collaboration Solutions Support Open Mics page.

Original publication date

2015/8/18

Document information

More support for: IBM Domino

Software version: 9.0

Operating system(s): Linux, Windows

Reference #: 7046303

Modified date: 30 September 2015