IC68037: AUTOCONF MAY CAUSE SOLARIS TO CRASH DURING UNLOAD OF DEVICE DRIVER WHEN THE DEVICE DRIVER TRACE IS ON

APAR status

• Closed as program error.

Error description

• Using Tivoli Storage Manager device driver on Solaris
  (TIVsmSdev).
  Autoconf has already been run and configured devices. On the
  next execution of autoconf,
  it will try and unload the device driver. If the device driver
  trace is on, Solaris will crash.
  
  Customer/L2 Diagnostics:
  
  Solaris crash analysis.
  
  -1m16.54s| adsm scsi_probe: Target 6742336 Lun 0, HP Ultrium
  4-SCSI
  -1m16.54s| mt2 at fp3: name w500104f000aeaeba,0, bus address
  65e240
  -1m16.54s| mt2 is
  /pci@0,0/pci8086,3604@1/pci10df,fc22@0,1/fp@0,0/mt@w500104f000ae
  aeba,0
  -1m16.54s| mt22 at fp14: name w500104f000aeaeea,0, bus address
  66e140
  -1m16.54s| mt22 is
  /pci@0,0/pci8086,3605@2/pci8086,3500@0/pci8086,3514@1/pci10df,fc
  22@0/fp@
  0,0/mt@w500104f000aeaeea,0
  -1m16.54s| adsm scsi_probe: Target 6676992 Lun 0, HP Ultrium
  4-SCSI
  -1m16.53s| adsm scsi_probe: Target 6742272 Lun 0, HP Ultrium
  4-SCSI
  -1m16.53s| mt3 at fp3: name w500104f000aeaebd,0, bus address
  65e200
  -1m16.53s| mt3 is
  /pci@0,0/pci8086,3604@1/pci10df,fc22@0,1/fp@0,0/mt@w500104f000ae
  aebd,0
  -1m16.53s| mt23 at fp14: name w500104f000aeaeed,0, bus address
  66e100
  -1m16.53s| mt23 is
  /pci@0,0/pci8086,3605@2/pci8086,3500@0/pci8086,3514@1/pci10df,fc
  22@0/fp@
  0,0/mt@w500104f000aeaeed,0
  -1m13.48s| pseudo-device: lb16383
  -1m13.48s| lb16383 is /pseudo/lb@16383
  -1m9.77s| pseudo-device: op16383
  -1m9.77s| op16383 is /pseudo/op@16383
  0s|
  | panic[cpu5]/thread=fffffe84c7a59e80:
  0s| mutex_tryenter: bad mutex, lp=ffffffffc072d008
  owner=fffffe84c7a59e80 thread=fffffe84c7a59e80
  
  SolarisCAT(vmcore.0/10X)> panic
  panic on CPU 5
  panic string: mutex_tryenter: bad mutex, lp=ffffffffc072d008
  owner=fffffe84c7a59e80 thread=fffffe84c7a59e80
  ==== panic user (LWP_SYS) thread: 0xfffffe84c7a59e80 PID: 4391
  on CPU: 5
  affinity CPU: 5 ====
  cmd: /usr/sbin/rem_drv mt
  t_procp: 0xfffffe84c6b14368
  p_as: 0xfffffe84c6e27b68 size: 2842624 RSS: 1212416
  hat: 0xfffffe84c6e26c90
  cpuset: 5
  zone: global
  t_stk: 0xfffffe80007f1f10 sp: 0xfffffe80007f1c80 t_stkbase:
  0xfffffe80007ed000
  t_pri: 19(TS) t_tid: 1 pctcpu: 0.034482
  t_lwp: 0xfffffe84c75b7cd0 lwp_regs: 0xfffffe80007f1f10
  mstate: LMS_SYSTEM ms_prev: LMS_SYSTEM
  ms_state_start: 0.000946550 seconds earlier
  ms_start: 0.009185552 seconds earlier
  psrset: 0 last CPU: 5
  idle: 0 ticks (0 seconds)
  start: Thu Apr 8 10:58:49 2010
  age: 0 seconds (0 seconds)
  syscall: #152 modctl(, 0x0) (sysent: genunix:modctl+0x0)
  tstate: TS_ONPROC - thread is being run on a processor
  tflg: T_PANIC - thread initiated a system panic
  T_DFLTSTK - stack is default size
  tpflg: TP_TWAIT - wait to be freed by lwp_wait
  TP_MSACCT - collect micro-state accounting information
  tsched: TS_LOAD - thread is in memory
  TS_DONT_SWAP - thread/LWP should not be swapped
  pflag: SMSACCT - process is keeping micro-state accounting
  SMSFORK - child inherits micro-state accounting
  
  pc: unix:vpanic_common+0x165: addq $0xf0,%rsp
  
  unix:vpanic_common+0x165()
  unix:0xfffffffffb856242()
  unix:mutex_panic+0x6f()
  unix:mutex_vector_tryenter+0x2f()
  mt:ps_lock_ndelay+0x9()
  mt:dd_start_trace+0x171()
  mt:_fini+0xc8()
  genunix:moduninstall+0x120()
  genunix:modunrload+0x98()
  genunix:modunload+0x10()
  genunix:modctl_modunload+0x48()
  genunix:modctl+0xc8()
  unix:_syscall_invoke+0x30()
  -- switch to user thread's user stack --
  
  TSM Versions Affected:
  TSM Servers V6.2 on Solaris
  
  Initial Impact: High
  
  Additional Keywords:
   zz62 dev driver device TIVsmSdev panic abend crash sun unload
  autoconf trace ddtrace
  

Local fix

• Disable device driver tracing;
  
  (ie on Unix)
  cd /opt/tivoli/tsm/devices/bin
  
  ddtrace end tapedd
  ddtrace end librarydd
  ddtrace end opticaldd
  
  Then rerun autoconf.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All Tivoli Storage Manager device driver     *
  *                 users.                                       *
  ****************************************************************
  * PROBLEM DESCRIPTION: See error description.                  *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available. This      *
  *                 problem is currently projected to be fixed   *
  *                 in level 6.2.1. Note that this is subject    *
  *                 to change at the discretion of IBM.          *
  ****************************************************************
  *
  

Problem conclusion

• This problem was fixed in the Tivoli Storage Manager device
  driver.
  
  Affected platforms: Sun Solaris
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R62S PSY

     UP