Domino View Accessibility on Large Databases under Heavy Load

Technote (troubleshooting)


Problem

Large, shared databases with heavily accessed views while under note update activity load can suffer slowdowns.

Symptom

The contention for exclusive update access can seriously impact throughput, with many users experiencing hourglass icons while the view is updated. If nsds are taken (even via the inexpensive -stacks option, several threads will be observed to have contexts like these:
0x76e0165a ntdll.ZwDelayExecution+10 (35b9ef18,44c,b935ced0,1)
0x7FEFD511203 KERNELBASE.SleepEx+179 (bb8ef618,0,0,0)
0x00456302 nnotes.OSDelayThread+50 (1,18b5009,4138d00,47d046)
0x0222212a nnotes.NIFUpdateCollectionNext+1994 (19100010,7FE28600013,103,0)
0x10074db3 nserverl.ServerOpenCollection+2919 (64a8000a,28600013,35b9fb60,35b9fb60)
0x10043a96 nserverl.DbServer+2442 (d01c0193,64a8000a,35b9fd0c,50)
0x10064628 nserverl.WorkThreadTask+2324 (c6520038,0,78c,abc0df4)
0x10001b21 nserverl.Scheduler+969 (0,0,0,0)
0x00456586 nnotes.ThreadWrapper+330 (0,0,0,0)
0x7694652d kernel32.BaseThreadInitThunk+13 (0,0,0,0)
0x76ddc521 ntdll.RtlUserThreadStart+33 (0,0,0,0)


0x76e0165a ntdll.ZwDelayExecution+10 (12c828,717,778bced0,2)
0x7FEFD511203 KERNELBASE.SleepEx+179 (7c3fe0a8,0,0,0)
0x00f56302 nnotes.OSDelayThread+50 (33e1d14,18b5009,302a56d0,787d5500)
0x02d2212a nnotes.NIFUpdateCollectionNext+1994 (14,18b5009,d008,0)
0x02d21165 nnotes.NIFUpdateCollection+497 (0,18b5009,56AEBD0D030500,70F00C885257338)
0x02c600c9 nnotes.NIFGetCollectionUpdated+845 (32,2b0d009,745,736)
0x02c66b76 nnotes.NIFOpenCollectionExtended2+5682 (7759,12d3b0,12e020,0)
0x004c1020 nlsxbe.ANView::ANVOpenView+716 (0,0,0,1)
0x004d07a8 nlsxbe.ANView::ANVFindDocByKey+236 (0,0,12d6f0,f56a2e)
0x004d04dc nlsxbe.ANView::ANVFindDocByKey+268 (c436,1050,c436,0)
0x004c7104 nlsxbe.ANView::ANDispatchMethod+1112 (0,0,0,f7003e)
0x0044cfd9 nlsxbe.ANCLASSCONTROL+497 (400,30cd011,0,815e978)


0x77ef0a3a ntdll.ZwWaitForSingleObject+10 (24f,24e952c,196009,1009)
0x77d704ff kernel32.WaitForSingleObjectEx+223 (14f0,0,0,207)
0x00e2e75c nnotes.WaitOnNativeSemaphoreCounted+100 (22dd270,25e80001,8d92bb8,e0a9a6)
0x00e2f8fe nnotes.OSLockReadFRWSem+918 (8e86ec0,f1009,0,2b680001)
0x02b995d8 nnotes.LockCollectionRead+1844 (8ea0bea,536F72000F1009,5A000085257602,52576C900638E3B)
0x02b75e65 nnotes.NIFGetCollectionUpdated+825 (32,149809,231,362)
0x02b7cba6 nnotes.NIFOpenCollectionExtended2+7134 (546459,e2ceb4,b,e2ceb4)
0x1388cf08 nlsxbe.ANView::ANVOpenView+700 (13a79658,2658000c,13a79658,d)
0x1388d36e nlsxbe.ANView::ANGetPropBoolean+198 (10bf3e00,10c23c40,0,1380acb5)
0x139017c3 nlsxbe.ANotes::ANJPropGetBoolean+67 (78631780,10bf3e00,787,0)


0x77ef0a3a ntdll.ZwWaitForSingleObject+10 (2bc,24e952c,0,1009)
0x77d704ff kernel32.WaitForSingleObjectEx+223 (1554,0,0,1b8)
0x00e2e75c nnotes.WaitOnNativeSemaphoreCounted+100 (4ecf41d0,2702d58,2c9,1)
0x00e2e266 nnotes.OSLockWriteFRWSem+302 (8e86ec0,e2cc6c,0,8e86ea0)
0x02b99d86 nnotes.LockCollection+462 (0,f1009,63b6fc38,4ecf41d0)
0x02b999f0 nnotes.LockCollectionWrite+316 (8e86ea0,f1009,2bd,2bc)
0x02c27226 nnotes.NIFUpdateCollectionNext+226 (14,f1009,12808,0)
0x02c26a71 nnotes.NIFUpdateCollection+497 (8ea0bea,536F72000F1009,5A000085257602,52576C900638E3B)
0x02b75e3c nnotes.NIFGetCollectionUpdated+784 (32,112809,144,362)
0x02b7cba6 nnotes.NIFOpenCollectionExtended2+7134 (546459,e2ceb4,3e,e2ceb4)
0x1388cf08 nlsxbe.ANView::ANVOpenView+700 (13a79658,d158000c,13a79658,d)
0x1388d36e nlsxbe.ANView::ANGetPropBoolean+198 (1092f200,16aa5d60,0,1380acb5)
0x139017c3 nlsxbe.ANotes::ANJPropGetBoolean+67 (7888e370,1092f200,787,0)


with only one or a handful of threads in this state (with ProcessNote and that which it calls):

0x76e0137a ntdll.ZwReadFile+10 (1,1bf165e,241de148,44ca49)
0x7FEFD511A7A KERNELBASE.ReadFile+122 (78c,667c,241de180,6406ceb8)
0x76941559 kernel32.ReadFile+89 (25ef6254,2000,c,5e9b2b60)
0x004726e7 nnotes.OSFDFileRead+47 (241de148,2000,1e5485e8,241de148)
0x01bf22ee nnotes.FileReadOSFD+510 (2000,21e09d7c,2000,241de364)
0x01bf1cc7 nnotes.NSFFileSeekReadC+371 (2,329e009,25ef6218,1e69c59)
0x01e483c2 nnotes.DBCONTReadPage+1258 (16600036C81,e0000d6,3b967a0,e0000d6)
0x01e6a071 nnotes.UBMSetupPage+2405 (6406d900,e0000d6,c018403f,e0000d6)
0x01e6a718 nnotes.UBMReplaceBuffer+1404 (16600036C81,8598,0,0)
0x004226cc nnotes.UBMAutoIngestGroupPage+1684 (6406da30,6406da80,1e9bb0e8,16600036C81)
0x01e67814 nnotes.UBMClock+64 (3b967a0,7eb0018,0,0)
0x01e67439 nnotes.UBMPinExtended+6145 (a,5e9b2b60,bcddb5a8,0)
0x01e65bd5 nnotes.UBMPin+25 (241de148,a,5e9b2b60,ba1f7608)
0x01999bac nnotes.BUFPin+1072 (6406dfe8,5010329E009,329e009,ba1f71c0)
0x01f7ebce nnotes.iNSFSlotPin+614 (329e009,329e009,329e009,e168)
0x01bc81f1 nnotes.NSFSlotPinCtx+845 (0,329e009,35f1cc9,e168)
0x01ee7d93 nnotes.NSFSlotChangePinCtx+347 (360000000B2,329e009,551962,e168)
0x01ebe564 nnotes.BTFindKeyNode+532 (329e009,228a3e3,0,be309)
0x0198c5eb nnotes.BTKeySearchExtCtx+611 (600000000,549b411,b935d310,100457B01)
0x0227f250 nnotes.FindEntry+664 (0,549b409,549b411,1e9bdee)
0x0228391b nnotes.LocateCollatedPosition+6163 (3,18b5009,1,198409)
0x022ac575 nnotes.FindNote+2233 (0,18b5009,551962,0)
0x022ba6aa nnotes.ProcessNote+3766 (241de148,18b5009,241de14c,199a292)
0x02225c54 nnotes.UpdateCollection+11988 (0,18b5009,4138d00,47d046)
0x022229f5 nnotes.NIFUpdateCollectionNext+4245 (30380010,7FEEF300011,103,0)
0x10074db3 nserverl.ServerOpenCollection+2919 (e528001a,ef300011,6406fb60,6406fb60)
0x10043a96 nserverl.DbServer+2442 (cf1401a6,e528001a,6406fd0c,50)
0x10064628 nserverl.WorkThreadTask+2324 (c65200af,0,78c,abc0df4)
0x10001b21 nserverl.Scheduler+969 (0,0,0,0)
0x00456586 nnotes.ThreadWrapper+330 (0,0,0,0)
0x7694652d kernel32.BaseThreadInitThunk+13 (0,0,0,0)
0x76ddc521 ntdll.RtlUserThreadStart+33 (0,0,0,0)

The problem may clear up when update load diminishes or persist as load remains the same or increases.

Cause

All threads using Domino views must first open those views prior to searching or reading the entries therein. By default, these open operations (executing the Domino C API NIFOpenCollection call) refresh the view to include any database changes that occurred since the last access.

When document updating is heavy, all threads performing this refresh enter into a tournament to update the view, causing only one thread to actually perform the update while the others wait.

Using the C API, the OPEN_NOUPDATE flag can be utilized to open the view without refreshing, but in the Lotusscript and Java APIs, even with autoupdate turned OFF, the refresh behavior will occur.


Environment

All

Diagnosing the problem

If performance against large, heavily-updated Domino databases degrades, take an nsd and compare stacks with those in this technote.

Resolving the problem

If you are using Lotusscript or Java with the backend classes, consider persisting open (Notes)View objects so that refresh doesn't occur, calling the refresh() method when absolutely necessary. Also, turn off autoupdate, as it makes the update frequency even higher.

During view design, consider using this refresh option:




perhaps in conjunction with updall program documents that update the view more frequently than once per hour.

At this writing, Domino development is currently devising a genuine fix to the issue. Inquiries about the work and the problem at large can be directed through SPR
JCUS8MXLA2.



Rate this page:

(0 users)Average rating

Document information


More support for:

IBM Domino
Database

Software version:

6.0, 6.5, 7.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 8.0, 8.0.1, 8.0.2, 8.5, 8.5.1, 8.5.2, 8.5.3

Operating system(s):

AIX, AIX 64bit, IBM i, Linux, Linux iSeries, Linux zSeries, Windows, Windows 64bit, i5/OS, z/OS

Software edition:

All Editions

Reference #:

1624464

Modified date:

2013-02-08

Translate my page

Machine Translation

Content navigation