DCE Trace Facilities and DFS dfstrace

Technote (troubleshooting)


DCE Trace Facilities and DFS dfstrace

Resolving the problem

DCE offers a number of tracing features, as described on this page. (If you've been referred here by a support rep who wants you to enable "serviceability debug tracing", you may want to jump ahead to the section on serviceability examples.) The tracing features are extensive but extremely quirky, as you can see by reading this page...


DCE Messaging
dced and secd Log Files
DCE Auditing
DCE Serviceability Tracing
Kerberos and syslog
OS Features
Solaris truss
AIX trcrpt
Solaris snoop
AIX tcpdump and iptrace
Debuggers and showProcInfo
DFS dfstrace tracing

DCE Messaging

"Messaging" refers to informative output that is classified into five categories: fatal, error, warning, notice, and verbose notice. The messaging facility is described in the OSF Application Development Guide -- Core Components, chapter 3. As used by the DCE services itself, it is capable of logging messages to files that are stored by default in /opt/dcelocal/var/svc, and whose disposition is most easily controlled via the /opt/dcelocal/var/svc/routing file.

By default, only fatal, error and warning messages are actually kept; messages of type notice and verbose notice are discarded. The default messages are stored in files named fatal.log, error.log, and warning.log in the /opt/dcelocal/var/svc directory. You can cause DCE to retain notice and verbose notice messages too, by editing the /opt/dcelocal/var/svc/routing and adding lines like these:


    (Changing the routing file does not affect running daemons, so you must stop and restart DCE in order for this change to take effect.) You could choose to send the messages to any arbitrary files, and/or to processes' standard output and/or standard error. The /opt/dcelocal/var/svc/routing file has a lengthy description of how to do this, as does man svcroute.

    One use of NOTICE class messages is to track CDS skulk failures, which look like this:

    1996-12-09-08:10:05.928-05:00I----- cdsd(14712) NOTICE cds server
    event_lib.c 201 0x20023720 A skulk failed for directory /.../ps4.
    dce.company.com: status = 282108908.

    Security-server propagation failures are NOTICE_VERBOSE messages, like this:

    1996-12-09-08:20:38.070-05:00I----- secd NOTICE_VERBOSE sec rs_prop
    rs_prop_tasks.c 1885 0x2004bfa0 Propagation failed : Connection request

    You must monitor disk-space usage in /opt/dcelocal/var/svc; DCE will not automatically clean up old log files or protect you from filling up a disk partition with log messages. The "FILE.N.M" syntax (see below) can help control the length of your logfiles. Make sure you have a strategy for monitoring them, especially if you enable notice or verbose notice messages.

    There are a couple of other ways to control the messaging facility, besides /opt/dcelocal/var/svc/routing:

    • The dcecp log modify command; for example:
      log modify /.:/subsys/dce/sec/machine_rgy -change {NOTICE_VERBOSE:FILE:/tmp/verbose.secd}

      Here (and in many of the examples that follow) you would substitute one of your machine names in place of "machine". This will tell a running security server to start sending verbose notice messages to the specified file. (In contrast, changes made to /opt/dcelocal/var/svc/routing affect all processes, and they take effect only when a process starts.) You can specify a string binding instead of a CDS name if desired. The dcecp log show command will display the current message dispositions.

      By the way, in many of the examples on this page we direct trace output to files in /tmp. This is a bad idea if you think your machine might reboot, because usually /tmp is cleaned up when the machine restarts and this will result in removal of your trace output file. To prevent this, you may want to create a directory like /usr/trace to store trace output. No matter where you choose to store it, you should also make certain that you have adequate free disk space; some of the output files can be big.
    • Most DCE daemons can take a command line argument like
      -w NOTICE:FILE:/tmp/notices to control message routing.

    You can use dcecp log modify to turn messaging off without stopping a process:

    log modify /.:/subsys/dce/sec/machine_rgy -change {NOTICE_VERBOSE:DISCARD:}

    dced and secd Log Files

    The DCE endpoint mapper process, dced, keeps a log file in /opt/dcelocal/var/dced/dced.log. This logfile is created anew each time dced is started; old messages from the previous instance of dced are not saved.

    Similarly, on security server machines, secd keeps a log file in either /opt/dcelocal/var/security/secd.log or in /opt/dcelocal/var/security/adm/secd/secd.log (depending on the platform). This logfile is also created anew whenever secd is started, and old messages are not retained.

    DCE Auditing

    DCE has an Audit feature that's used by security and DTS by default; it's described in more detail in the OSF documentation, and at this link.
    As used by security and DTS, audit trail files are kept in directories /opt/dcelocal/var/security and /opt/dcelocal/var/adm/time respectively. The audit trail files are in binary format; you can translate to English via the dcecp audtrail show command:

    audtrail show /opt/dcelocal/var/security/sec_audit_trail -to /tmp/secd.at

    You'll have to run man sec_audit_events to interpret the event numbers that you'll find in the resulting /tmp/secd.at file. As with messaging, you need to monitor the disk space used by audit trail files; it's even more important with audit files since they tend to get big quickly. There are environment variables that you can use to either disable the auditing feature entirely or to limit the length of the audit trail files; see our page on DCE environment variables for details. (Look for the variables whose names start with DCEAUDIT.)

    DCE Serviceability Tracing

    In addition to the messaging facility and the audit facility, DCE also has a general-purpose facility to control "serviceability debug" messages. It looks like the messaging facility in many ways. Don't ask me why three similar-but-not-quite-identical facilities exist! Anyway, serviceability is generally used to get fairly low-level debug-type messages; these are typically useful only to support personnel and others who have access to DCE source code. You'll probably only use this feature as directed by your vendor's DCE support staff.

    A serviceability routing format looks like this:


    Here the component and subcomponent names are compiled into the program being traced; level runs from 1 (few messages) to 9 (many messages), DESTINATION is typically FILE or STDOUT (or DISCARD), and the additional-attribute is most commonly a file name. DESTINATION and additional-attribute work exactly as in the messaging facility.

    You can enable serviceability tracing in several ways:

    • Via dcecp log modify (as with messaging):
      log mod /.:/subsys/dce/sec/machine_rgy -change {rpc:general.1:FILE:/tmp/secd.log}

      This would turn on level-1 tracing for component RPC, subcomponent general, and send the output to the specified file.

    • Via the /opt/dcelocal/var/svc/routing file (also same as with messaging). For example, you could enable CDS tracing by adding a line like this:

      You would have to stop and restart CDS processes in order for the above change to the routing file to have an effect.

    • By setting an environment variable before starting the process to be traced. To control component xyz, you must set environment variable $SVC_XYZ_DBG. (Nothing like the messaging facility!) For example:

      setenv SVC_RPC_DBG 'rpc:general.1:FILE:/tmp/secd.log'
      unsetenv SVC_RPC_DBG
      This is equivalent to the dcecp log modify tracing above, except that using the environment variable requires you to stop and restart the process to be traced. (Warning: some DCE start scripts, like for example the scripts that start AIX DCE 2.2 and AIX and Solaris DCE 3.1, seem to sanitize their environment before starting the DCE daemons. So, beware that the startup scripts may defeat the variables -- you may need to start daemons manually, or use other techniques to set tracing, on these platforms.)

    • cdsd, cdsadv, and gdad can all take command-line arguments of the form:
      -d 'rpc:general.1:FILE:/tmp/rpc.log'; no other DCE daemon accepts such an argument, however. (This is similar to the -w flag that all DCE daemons can use to control messaging.)

    Common destinations for the trace output are as follows:
    • FILE:filename when sending trace output to a named file use , as in the example above.
    • FILE.N.M:filename allows you to control the size of the trace output. N files consisting of M lines each will be kept, using the specified filename as a base. For example, FILE.10.100:/tmp/foo would result in 10 files named /tmp/foo.1, /tmp/foo.2, and so on. Each file would contain 100 lines of output. After the 10th file had been created, tracing would begin to overwrite the data in the first file. In this way, you'd always have the last 900 to 1000 lines of trace output available.
    • STDOUT and STDERR have the obvious meanings; you must specify a null destination-attribute, as in 'rpc:general.1:STDOUT:'. (Setting $SVC_RPC_DBG to this value would cause level-1 RPC general tracing to go to standard output.)
    • DISCARD can be used to disable tracing, as described in the next bullet.

    You can turn tracing off in a couple of ways:

    • Via dcecp log modify, with a destination of DISCARD, like:
      dcecp -c 'log modify /.:/subsys/dce/sec/machine_rgy -change {rpc:*.9:DISCARD:}'

      This turns off all RPC tracing in the named security server; it will work even if the only tracing you enabled was, say, rpc:general.1.
    • If you used the routing file to start tracing, then you would have to stop the processes being traced, remove the line from the routing file, then restart the processes.
    • If you enabled tracing via a DCE_XYZ_DBG environment variable, unset that variable. You'll also have to either stop and restart or log modify any processes that are running with the variable set, of course.

    Some sample uses of serviceability tracing:

    • rpc:general.1 tracing: this appears in DCE 2.0 and later for Solaris, and in DCE 3.1 and later for AIX (but not in AIX DCE 2.2).
      This tracing produces a line for each incoming and outgoing RPC. It's not free -- with UDP the tracing amounts to about 400 bytes of information per RPC and will cut your RPC performance roughly in half (RPCs will take twice as long); with TCP, the trace data is about 1200 bytes per RPC and RPCs will take approximately 4 times as long as without tracing. Still, the information you get can be indispensable in some cases. (By the way, the tracing should not cause your processes to grow in size -- the byte-size estimates here are estimates of the amount of trace data emitted.)

      To monitor a security server's RPCs, you might do this:

      dcecp -c 'log modify /.:/subsys/dce/sec/machine_rgy -change {rpc:general.1:FILE:/tmp/secd.log}'

      To see what RPCs occur during a DCE login, you could do this:

      setenv SVC_RPC_DBG 'rpc:general.1:FILE:/tmp/dcelogin.dbg'
      dce_login cell_admin
      exec kdestroy # Terminate new DCE shell and logout of DCE
      unsetenv SVC_RPC_DBG

      To monitor RPCs handled by a CDS server, you could do this:

      cdscp disable server # Stop running CDS server
      cdsd -d 'rpc:general.1:FILE:/tmp/cdsd.log'

      You can get even more trace output by turning the number "1" up; setting it to "5", for example. Also you could ask for "auth" subcomponent tracing in addition to "general" tracing (rpc:general.1,auth.1 will get level-1 tracing for both subcomponents); or you could go whole hog and get rpc:*.9 tracing, which is the maximum available (level 9 tracing on all subcomponents).
    • To watch the "secval" part of dced (the module that maintains the hosts/machine/self identity), enable dhd:secval.1 tracing. You can do this via dcecp:
      dcecp -c 'log modify /.:/hosts/machine/self -change {dhd:secval.1:FILE:/tmp/secval.log}'

      Output will not be particularly large. You may want to do dcecp -c secval deactivate then dcecp -c secval activate after enabling tracing, to stop and restart the secval service in order to get tracing from a known point.
    • The design of the CDS client software makes tracing CDS client activity a bit tricky. In general, CDS requests go to the local cdsadv process (known as "the advertiser"), which forwards them to your personal cdsclerk process (your "clerk"). The advertiser will create you a clerk if one is not currently assigned to you. Finally your cdsclerk performs the CDS work you requested.
      To trace your CDS accesses, you really need to trace your clerk. However, none of the tracing commands can be directed at a clerk. They can be directed to the advertiser though, so the usual procedure is to kill your existing clerk (if any), then make a trace change request to the advertiser. Your next CDS access will trigger creation of a new clerk process, and that new clerk will inherit the advertiser's tracing attributes.

      A good general trace setting for CDS client activity is cds:*.6, although you may need full tracing (cds:*.9) for some types of investigations. Here's how to enable cds:*.9 tracing:
      • Become UNIX root.
      • Turn on tracing in the advertiser:
        dcecp -c 'log modify /.:/hosts/machine/cds-clerk -change {cds:*.9:FILE.10.1000:/tmp/cdscl}'

        This will direct the trace output to a set of 10 files with basenames of /tmp/cdscl, and each of the files will hold 1000 lines of tracing.
      • Use ps to identify any running CDS clerk processes, and kill them. Any CDS accesses from this point on will require creation of a new clerk process, and that clerk will inherit the new tracing directive from the advertiser.
      To disable the tracing, do this to tell the advertiser to stop tracing:

      dcecp -c 'log modify /.:/hosts/machine/cds-clerk -change {cds:*.9:DISCARD:}'

      ...then kill any running clerks.
    • To trace a CDS server, cds:server.2 is often enough. (Sometimes you'll want cds:*.6, but not often; only very rarely would you need a setting higher than 6.) Do this:
      dcecp -c 'log modify /.:/hosts/machine/cds-server -change {cds:server.2:FILE:/tmp/cdsd.log}'
    • For gdad, try:
      dcecp -c 'log modify /.:/hosts/machine/cds-gda -change {cds:gda.5:FILE:/tmp/gdad.log}'
    • To trace DTS, start with this:
      dcecp -c 'log modify /.:/hosts/machine/dts-entity -change {dts:*.1:FILE:/tmp/dts.log}'
      Depending on what you're doing, you may need to use dts:*.9 instead of *.1. Also, turning in NOTICE messages during a synchronization can be helpful, like this:
      dcecp -c 'log modify /.:/hosts/machine/dts-entity -change {NOTICE:FILE:/tmp/notice.dts}'


    The DEBUG_SEC environment variable, if set to a number from 1 to 9, will trace a process' binding to security servers. Setting it to "1" is usually enough. You can watch the security binding code obtain bindings from CDS, fail over to the /opt/dcelocal/etc/security/pe_site file if CDS isn't up (or if BIND_PE_SITE is set), and generally track its progress in finding a security server. This variable can be helpful if you notice that dce_login is slow, for example: it can help determine if the problem is with CDS (i.e. trouble finding the list of security servers) or with security (trouble talking to some security server(s) after obtaining the list from CDS).

    To test drive it, do this:

    setenv DEBUG_SEC 1
    dce_login cell_admin
    exec kdestroy # Terminate new DCE shell and logout of DCE
    unsetenv DEBUG_SEC

    Kerberos and syslog

    Kerberos uses the UNIX syslog feature to record LOG_INFO messages from facility local6 regarding some ticket-granting events. To enable it, do the following:
    • Add a line like this to the /etc/syslog.conf config file:
      local6.info /var/adm/info6
      (You must use one or more TABs, not spaces, as whitespace.)
    • touch /var/adm/info6 to create the log file
    • kill -HUP `cat /etc/syslog.pid` to tell syslog to re-read the config file

    Remove the local6.info line from the config file and repeat the kill -HUP to make the logging stop.

    OS Features

    Your operating system probably has tracing facilities that can be useful in DCE troubleshooting. For example:

    Solaris truss
    The Solaris truss command allows you to see all system calls made during the execution of a particular process. The command if very easy to use: just type "truss command", where "command" is some UNIX command that you want to trace. For example, try "truss ls".

    There are several useful command-line options to truss (use man truss to get the complete story):
    • To follow child processes, use the -f flag. This is useful if you're tracing a process that daemonizes itself, for example.
    • To see thread ID numbers, use the -l flag. This is almost always the right thing to do when tracing DCE multithreaded processes, since it allows you to see which thread is making which system call.
    • To restrict tracing to only certain system calls, use the -t option followed by a list of system calls to trace. For example, -topen (to trace just opencalls) or -tread,write (to trace read and write calls).
    • To omit tracing of signals, use -s\!all. (When tracing DCE processes, the signal tracing is often just distracting noise.)
    • To send the trace output to a file, use -o outfile-filename.
    • To attach truss to a running process, use -p PID.

    • Suppose you want to watch all open calls made by dce_login and by the shell that it spawns:
      truss -f -l -topen -s\!all -o /tmp/dce_login.trace dce_login cell_admin
    • To watch all write calls made by process ID 7097:
      truss -f -l -twrite -s\!all -p 7097
      You must be either the owner of PID 7097 or be root to do this. The output will go to your screen since -o was not specified. You can stop the tracing via control-C in the window where you're running truss; doing so will not affect the process being traced. (That is, the control-C will not kill process 7097.)

    AIX trcrpt
    AIX has a system-call tracing facility, but it's more difficult to use since it can't be directed at a single process -- turning on the tracing will cause the system to trace all calls made by all processes on the machine, and you can't tell which process made which call from the trace output. The relevant commands are trace, trcstop, and trcrpt; see the AIX documentation for usage information.

    Solaris snoop
    snoop is a "network sniffer" program that can be used to analyze network packets. It takes advantage of the fact that all network packets on a subnet are visible to all machines on that subnet. Normally, a machine's network card is programmed to pay attention only to those packets directed to that machine and to ignore all others. But it's possible to put the card in "promiscuous mode", in which case it will examine all packets that it can, even those that are not addressed to it. This allows any machine on a particular subnet to display any network packets that go to or from any other machine on the subnet. The Solaris snoop command is used to do this. Some examples:
    • To trace all packets going between machines box1 and box2, use "snoop box1 box2". You must be root on a machine that's on the same subnet as either box1 or box2 to do this. (So you could do it from box1 or box2, but you don't have to be on either of those machines.)
    • Suppose box3 is running some DCE server process that's using UDP port 47028 and TCP port 41554. Use this command (as root, on any machine that's on the same subnet as box3) to watch all packets involving either of those two ports:
      snoop -t a 'box3 ( ( udp port 47028 ) or ( tcp port 41554 ) )'
    • You can use the -o option to direct snoop's output to a file, and use the -i option to display the contents of such a file. (The file will be in a binary format so you can't just cat it.)

    AIX tcpdump and iptrace
    The AIX tcpdump command is similar to Solaris' snoop. Alternatively, the combination of iptrace and ipreport can be used on AIX. See the AIX documentation for details; in particular, try "man tcpdump" and "man iptrace".

    To trace all packets going between machines box1 and box2 using tcpdump, use "tcpdump -x host box1 and box2".

    (This same tcpdump program is usually available on Solaris too, if you'd prefer to use it instead of snoop.)

    iptrace can only be used to trace packets going to or from the local machine. It writes binary output to a file that must be interpreted via ipreport. Note that iptrace daemonizes itself, and you must kill it when you want the tracing to stop. To trace all traffic between the local machine and box3, use "iptrace -a -s box3 -b /tmp/trace". Locate the iptrace process via ps and kill it to stop the tracing, then use "ipreport /tmp/trace" to see the results. Alternatively, you could also use the startsrc and stopsrc commands to stop and start ipreport.

    IBM's DCE support team has a tool called SMT that can extract DCE RPC information from iptrace output.

    Debuggers and showProcInfo
    The ultimate tracing tool is a debugger like dbx. Even without source code, dbx can tell you a lot. Everyone knows that it's useful for analyzing core dumps, but it can also help troubleshoot processes that are apparently hung. The idea is to get a stack trace of all threads in the process, wait a minute or so, then do it again. Maybe even wait another minute and get a third set of stack traces. (DCE Support's showProcInfo script can be a big help when you want stack traces of all threads in a process.) Anyway, running diff or diff3 on the two or three sets of stack traces can help determine which threads are making progress and which aren't, and it can indicate troubles with threads that are stuck waiting for some lock, for example.

    For whatever it's worth, a thread that's making an outbound RPC (that is, one that's acting as a client) will have a stack trace that looks like this:

    [.] ...
    [5] rpc__cn_call_transceive(...) or rpc__dg_call_transceive(...)
    [6] greet_fn(...)
    [7] main(...)

    Here we see that main called greet_fn. A call to rpc__cn_call_transceive would indicate that greet_fn is a client RPC stub function, and that it's in the process of making an RPC via a connection-oriented protocol (e.g. TCP/IP). A call to rpc__dg_call_transceive would indicate that greet_fn is making the RPC via a datagram protocol (UDP/IP). Threads that are sitting in rpc__cn_call_transceive or rpc__dg_call_transceive are threads that are in the process of making RPCs. (Or perhaps waiting for results from an RPC that they just made.)
    A server's call-executor threads start in cthread_call_executor; any thread that includes cthread_call_executor is a server-side thread that's dedicated to handling incoming RPCs. Here's a server thread that's accepting an RPC to greet_fn:

    [.] ...
    [4] greet_fn(...)
    [5] op0_ssr(...)
    [6] rpc__dg_execute_call(...) or rpc__cn_call_executor(...)
    [7] cthread_call_executor(...)
    [8] pthread_start(...)

    As with client-side threads, rpc__cn_call_executor would indicate that the incoming RPC is via TCP, and rpc__dg_execute_call would indicate UDP.
    It's worth noting that the Solaris pstack command can be used as a cheap substitute for showProcInfo, if you can't run showProcInfo because for example dbx or perl are unavailable.

    DFS dfstrace

    What is dfstrace used for?
    dfstrace is used by system administrators to diagnose problems within the DFS kernel or within server processes.

    Basic Guidelines for Running dfstrace

      1. Become local root on the machine

      2. Set environment variables and ensure that required packages/filesets are installed on the machine to be traced, as follows:

        For DFS 1.1/2.0:
        setenv NLSPATH /opt/dce{1.1|2.0}/nls/msg/en_US.ASCII
        setenv LANG C

        For DFS 3.1:
        Ensure IDFSenUSm is installed
        setenv NLSPATH /opt/dcelocal/nls/msg/%L/%N
        setenv LANG C

        Ensure dfs.msg.en_US.client.rte is installed
        Ensure dfs.msg.en_US.server.rte is installed if the machine is a DFS server
        setenv NLSPATH /usr/lib/nls/msg/%L/%N
        setenv LANG en_US
        setenv LC__FASTMSG false (that's a double underscore "__")

        Note: When tracing specific DFS processes (ex: icl trace), if the above environment variables are not already set as shown, then you will need to stop the DFS processes using the command stop.dfs, set the environment variables in your shell, and then restart the DFS processes using the command start.dfs.

      3. Determine which event sets are currently activated and which sets need to be activated.

      An event set is a module designed to track specific events within the DFS kernel or within one or more server processes.

      "dfstrace lsset" indicates which event sets are currently activated.

      # dfstrace lsset
      Available sets:
      cm: active (Cache Manager package - DFS kernel event set)
      krpc: active
      fx: active (File exporter package - DFS kernel event set)
      fshost: active (File exporter host package - DFS kernel event set)
      xops: active (vnode-to-fileset synchronization package - DFS kernel event set)
      episode/anode: dormant (LFS anode package - DFS kernel event set)
      episode/logbuf: dormant (LFS buffer/logging package - DFS kernel event set)
      episode/allocation: dormant
      episode/vnopsVerbose: dormant
      episode/vnopsBasic: active
      zlc: active
      vnl: dormant
      tkc: active (Token cache package - DFS kernel event set)
      tkm/conflictQ: dormant
      tkm/grants: dormant
      tkm: active (Token manager package - DFS kernel event set)
      tkm/recycle: dormant
      tkm/revokes: dormant
      tpq: dormant

      Notes: The active state for the event set indicates that tracing is enabled. The dormant state for the event set indicates that tracing is disabled. Herein, the event set releases its claim to space occupied by the logs to which it sends data.

      "dfstrace setset" is used to change the state of a set. If any of the event sets that you want to trace has logging disabled (state "dormant"), you will have to change the state to "active" in order to enable tracing. Tracing is immediately enabled for that event set once the event set's state is set to "active".

      # dfstrace setset -set cm krpc -active
      (sets active the cm and krpc event sets)

      # dfstrace setset -set fx xops fshost krpc -active
      (sets active the fx, xops, fshost, and krpc event sets)

      If you are running dfstrace at the request of DFS Support, you will likely be given a list of event sets to activate. Use the dfstrace setset command to activate all event sets required by Support to troubleshoot your particular problem.

      4. Use "dfstrace clear" to clear server process or kernel trace logs (i.e. If you want to clean up and start fresh). THIS IS NOT ADVISED IF YOU ARE TRYING TO CAPTURE HISTORY OF AN ISSUE. However, if you are recreating a problem which you can reliably reproduce, you will probably want to clear out the trace buffer logs before attempting to reproduce the problem. To do this, simply issue:

      # dfstrace clear

      5. Issue the DFS or OS command that you want to trace, or initiate the activity that you want to trace. You typically want to trace the activity or the command to completion.

      6. Immediately after the command returns or the activity completes, dump the server process or kernel trace logs.

      Use "dfstrace dump" to dump the logs; to dump the trace logs to a file, issue:

      # dfstrace dump -file filename

      7. To turn off tracing, issue the dfstrace setset command again, specifying the same event sets that you set as active previously, but now issue the command with the "-inactive" switch.
      # dfstrace setset -set cm krpc -inactive
      # dfstrace setset -set fx xops fshost krpc -inactive

    The above represents the basics of running dfstrace. If you are running dfstrace at the request of DFS Support, you may be given additional/different instructions to follow when collecting dfstrace data.

    Cross Reference information
    Segment Product Component Platform Version Edition
    Storage Management DFS AIX, Solaris 3.1

    Document information

    More support for:

    Distributed Computing Environment

    Software version:

    3.1, 3.2

    Operating system(s):

    AIX, Solaris

    Reference #:


    Modified date:


    Translate my page

    Content navigation