Extending sensor discovery scope with Simplified Model

You can easily extend the discovery scope of a sensor by using new functions of the Simplified Model. The following procedure shows an example configuration, on which you can base customization of your own sensors.

1. Create custom server template by using Custom Servers pane in Discovery Management Console.
   Custom Servers pane contains templates of sensors that can be run by generic server sensor or as custom application server sensor.
   Open the details for MongoDB server. It is enabled. It stores objects of the AppServer type. In the Identifying Criteria section, you must specify a file name. When generic server sensor finds a process that contains this file name, it runs the custom sensor. In this case, the file name is mongod.exe.

   You can now run a discovery but only RuntimeProcess objects are discovered.

2. Configure extension for template sensor.
   1. Open the TADDM home directory and go to the dist/etc/templates/commands directory. Create a command file with the same name as the template defined in Custom Servers, in this case, MondoDB.
      The content of this file is executed when the custom sensor is run.
      The MongoDB file contains the following line:

      SCRIPT[com.ibm.cdb.core.jython253_2.5.3]:etc/templates/commands/extension-scripts/mongodb.py

      It means that the mongodb.py script is executed when the sensor is run.
   2. Configure the mongodb.py script.
      The following sections provide brief description of the most important elements of this script.

      The default beginning of the sensor

      (os_handle, result, appserver, seed, log, env) = sensorhelper.init(targets)

      This section specifies parameters that are passed by platform that is called targets:

      • os_handle - an operating system object.
      • result - the result, where the objects are attached.
      • appserver - the app server object that represents runtime processes that match the template's condition.
      • seed - the seed object that triggers the sensor.
      • log - logger.
      • env - environment settings.

      Creating the main object - defining class and hierarchy attributes

      mdbserver = ModelFactory.newInstance(Class.forName('com.collation.platform.model.topology.simple.SSoftwareServer'))
      mdbserver.setHierarchyDomain('app.db.mongodb')
      mdbserver.setHierarchyType('MongoDBServer')

      In this section, a new type of class is used, SSoftwareServer. There are also new attributes, which position a CI that is represented by this object on the maps of domains. These attributes provide necessary details for the sensor.

      • HierarchyDomain - specifies levels of domains. In this case, in the app.db.mongodb value, app is an application server, db is a database server, and mongodb is a specific database.
      • HierarchyType - specifies the type of the object. In this case, mongodb that is used in the HierarchyDomain attribute represents a database server, so the HierarchyType attribute is set to MongoDBServer.

      OpenId - a new generic naming rule attribute

      print 'Primary IP : ' + seed.getPrimaryIpAddress().getStringNotation()
      print 'Port       : ' + str(seed.getPort())
      id = OpenId().addId('IP' , seed.getPrimaryIpAddress().getStringNotation()).addId('port' , str(seed.getPort()))
      ID = OpenId.generateDisplayName(id)
      mdbserver.setInstanceID(ID)
      mdbserver.setOpenId(id)
      mdbserver.setHostComputerSystem(os_handle.getComputerSystem())

      You must name the object that you created. To do so, you must define a naming rule and specify attributes that provide unique values. For servers, naming rule is usually based on two attributes. They are the primary service access point that is created from the server's primary IP address, and a port, on which this service listens on.

      The new OpenId attribute is a wrapper for a string attribute that gathers all values that are used for GUID calculation. This new attribute arranges the values in the same order every time. The addId method adds objects and set them as the value of the OpenId attribute.

      There are also other, predefined attributes set for this sensor, for example instance ID, or host computer system to attach the server to a computer system that is passed by a platform as an operation system object attribute.

      Setting the result object

      print 'Setting mongodb server'
      result.setServer(mdbserver)

      Set the result object with the MongoDB server. If during the discovery the sensor fails, at least this information is stored.

      Extract databases function

      def extractDatabases(mgbserver, dbListLines):
        databases = list([])
        dbDir = None
        currentDatabase = None
        XD = None
        XA = None
      
        for dbLine in dbListLines:
          print 'Processing line : ' + dbLine
          if "DATABASE" in dbLine :
            print 'Found a database ' + dbLine
            #Create physical files
            print 'Attach as files'
            dbLineTokens = dbLine.split(" ")
            dbName = dbLineTokens[2].strip()
            dbDir = dbLineTokens[4].strip()
            print "Database name " + dbName
            print "Database directory " + dbDir
            
            #Create deployable components
            print "Create deployable component for database"
            database = ModelFactory.newInstance(Class.forName('com.collation.platform.model.topology.simple.SDeployableComponent'))
            database.setHierarchyDomain('app.db.mongodb')
            database.setHierarchyType('Database')
            database.setOpenId(OpenId().addId("databaseName", dbName))
            database.setName(dbName)
            database.setSoftwareServer(mgbserver)
            databases.append(database)
            currentDatabase = database
            XA = ExtendedAttributesData()
            XA.attachTo(currentDatabase)
            XD = ExtendedInstanceData()
            currentDatabase.setXD(currentXD)
         
          elif dbDir is not None and dbDir in dbLine and "Metadata for" in dbLine:
            print "Found file " + dbLine + " to read for XD"
            databaseConfFileLineTokens = dbLine.split(" ")
            fileType = databaseConfFileLineTokens[3]
            fileName = databaseConfFileLineTokens[5]
            print "Instance type " + fileType
            print "Files content to read " + fileName
      
            if not fileName.endswith("bson"):
              cnfFileContent = readFileContent(fileName) 
              print "Read content : " + cnfFileContent
              XD.addInstance(fileType, ExtendedInstanceData.INSTANCE_FORMAT.json, 1, cnfFileContent)
              print "Added instance"
      
              if "index" in cnfFileContent :
                XA.addAttribute("hasDatabaseIndexDef", "true")
                xml = XA.toXML()
                print "Add XA marker for database " + dbName + " with XML " + xml
      
        print "Attach files and deployable components"
        mgbserver.setDeployedComponents(tuple(databases))
        XA = ExtendedAttributesData()
        XA.addAttribute("Markers", DatabasesCnt", str(len(databases)))
        XA.attachTo(mgbserver)
        print "Add XA marker for server with xml " + xml 

      This function uses two containers for data. One contains extended attributes and the other extended instances.

      In the #Create deployable components section, sensor creates object for discovered database by using the SDeployableComponent class type. The HierarchyDomain is set to app.db.mongodb because it is the same domain, but the HierarchyType attribute is set to Database. The naming rule is based on the database name, so the openId attribute is set to databaseName. A predefined attribute name is set (database.setName(dbName)), and the component is attached (databases.append(database)) to a server with software server attribute (database.setSoftwareServer(mgbserver)).

      What is most important in this section, are two data containers. These containers are created for the newly created database. The XA attribute contains ExtendedAttributesData object for extended attributes and the XD attribute contains ExtendedInstanceData object for large chunks of data.

      ExtendedAttributesData

      By using the XA = ExtendedAttributesData() attribute, the sensor creates an ExtendedAttributesData object and adds it to a CI. By using XA.addAttribute method, it adds the attributes to this object, which are name and value. For example, in the extended attribute for the server object, sensor creates ExtendedAttributesData object, and adds the attributes, which are DatabaseCnt and category Markers. To convert the attribute into layout that can be stored, the sensor calls the toXML method.

      ExtendedInstanceData

      By using the XD = ExtendedInstanceData() attribute, the sensor creates an ExtendedInstanceData object. In this sensor, the XD attribute is used to store the contents of files that represent the inner structure of the database. The content of the database is read by the readFileContent function that is defined in the following way:

      def readFileContent(fileName):
        print "Open file to read : " + fileName
        f = open(fileName, 'r')
        content = ""
        for line in f:
          content = content + line + "\n"
        return content

      The following method is used to store the data:

      XD.addInstance(fileType, ExtendedInstanceData.INSTANCE_FORMAT.json, 1, cnfFileContent)

      This particular addInstance method contains the following parameter:

      • fileType - specifies the instance type.
      • ExtendedInstanceData.INSTANCE_FORMAT.json - specifies the instance format.
      • 1 - means true, and specifies that the content is visible on the details pane.
      • cnfFileContent - specifies the instance content.