Server unresponsive when user opens or deletes very large email.

Technote (troubleshooting)


Problem


Server unresponsive when user opens or deletes very large email.

Symptom

When customer (who's mail file is DAOS enabled) opens or deletes a very large email (>100mb), the server is unresponsive for hours.
During this time we see long held locks on the mail file.

Locking thread's can be seen to have the following call stack:

############################################################
### thread 13/57: [ nSERVER: 0b84: 0bd0]
### FP=0x097fa54c, PC=0x7c82847c, SP=0x097fa4e4
### stkbase=0x09800000, total stksize=262144, used stksize=23324
############################################################
...
...
@[ 5] 0x60b1e5a5 nnotes.DAOSReadObject+885 (1213e4a7,1093a38,240e000,f000)
@[ 6] 0x609a5a5d nnotes.NSFDAOSReadObject+61 (97fb1dc,97fae57,240e000,f000)
@[ 7] 0x6005503a nnotes.ReadObject+5642 (97fb1dc,18442e,240e000,f000)
@[ 8] 0x600b34ae nnotes.DbReadObject@20+30 (97fb1dc,18442e,240e000,f000,0)
@[ 9] 0x600b23c0 nnotes.iNSFDbReadObject@20+224 (97fb1dc,18442e,240e000,f000,0)
@[10] 0x600e4565 nnotes.NSFDbReadObject@20+165 (611,18442e,240e000,f000,0)
@[11] 0x60a8b36b nnotes.NSFObjectStreamReadExt@20+123 (29,1094684,2000,97fb280,0)
@[12] 0x60a8ba5d nnotes.NSFObjectStreamRead@12+29 (29,1094684,2000)
@[13] 0x6034a8b3 nnotes.GetMimeEntityDataFromItem+771 (2e,0,240e000,a000)
@[14] 0x6034b581 nnotes.MIMEGetEntityDataExt@32+289 (2e,107ac98,0,240e000,0,97fb310,f10f10,ffffffff)
@[15] 0x6034b606 nnotes.MIMEGetEntityData@28+38 (2e,107ac98,0,240e000,0,97fb338,f10f10)
@[16] 0x6034b6f6 nnotes.MIMEGetEntityDataStream@16+230 (2e,107ac98,0,97fb730)
@[17] 0x607ffb5d nnotes.MIMEGetDecodedEntityDataStream@16+93 (2e,107ac98,0,97fb730)
@[18] 0x6034c112 nnotes.GetMimeText+82 (2e,97fb8e8,138,1)
@[19] 0x6034c43b nnotes.MIMEGetText@28+139 (2e,60eefc5c,107ac98,1,0,97fb8b8,f10f10)
@[20] 0x607f8e35 nnotes.ConvertItemToAbstract+229 (2e,60eefc5c,4,97fec94)
@[21] 0x6094c7a3 nnotes.NoteUpdateImpl@56+20531 (97ff800,2e,8008045,0,0,97ff038,f10f10,ffffffff,0,0,0,0,0,0)
@[22] 0x60956832 nnotes.iNoteUpdate2@24+546 (97ff800,25,12002d5,0,0,97ff3e4)
@[23] 0x6096333e nnotes.DispatchNoteUpdate@36+1534 (97ff800,1,8008045,0,0,97ff644,f10f10,ffffffff,0)
@[24] 0x60963ec6 nnotes.NSFNoteUpdateExtended3@28+1366 (0,8008045,0,0,0,97ff86c,f10f10)
@[25] 0x609642ae nnotes.NSFNoteUpdateExtendedFlags2@16+94 (611,8008045,0,0)
@[26] 0x601396c4 nnotes.NSFNoteUpdateExtended@8+20 (25,8008045)
@[27] 0x60b212d2 nnotes.iNSFDbDeleteNotesExtended@16+2050 (0,2fdb,8008045,0)
@[28] 0x6089be94 nnotes.NSFDbDeleteNotesExtended@16+164 (0,2fdb,0,0)
@[29] 0x1005b89d nserverl.ServerDbStampNotes@8+669 (7b100001,c0340001)
@[30] 0x10021c63 nserverl.DbServer@8+2515 (a47c0033,7b100001)
@[31] 0x10037277 nserverl.WorkThreadTask@8+1655 (5b4a63c,0)
@[32] 0x100018ce nserverl.Scheduler@4+750 (0)
@[33] 0x6015577f nnotes.ThreadWrapper@4+175 (0)
[34] 0x77e6482f kernel32.GetModuleHandleA+223 (0,0,0,0)


Cause

This is documented in SPR VDES8G4GUG

Environment

all

Diagnosing the problem

Although DAOS seems to be a general characteristic of this problem (i.e. as seen in the call stack), Domino Development indicated that the issue is related more to MIME than DAOS. Since soft-delete is enabled, deleting a single document will have absolutely no effect on DAOS, as (from the DAOS standpoint) the document and attachment reference still exist, and no DAOS changes are involved.

This behaviour was deemed to be working as designed.

The note contained in the email that was the source of the lock (i.e. the large one being opened or deleted) is a very large Mime note. This means that when the user tries to open or delete it while accessing the mail file directly on the server, the note has to be converted. This operation during internal tests took hours to complete which resulted in Locks on the server console. This is expected behaviour.

Resolving the problem

Work-a-round:
(1) As a prevention measure however, users should be deleting very large
mails like this one on their local replica to avoid contention on the
server side.

(2) if you know the specific emails causing this issue - address these emails by either blocking them or if you have control over them, significantly reduce their size.

Rate this page:

(0 users)Average rating

Add comments

Document information


More support for:

IBM Domino

Software version:

8.5.3

Operating system(s):

Linux, Windows

Reference #:

1653079

Modified date:

2013-10-15

Translate my page

Machine Translation

Content navigation