Customizing the Python data collector for on-premises applications

By modifying files in the data collector package, you can set the environment variables to customize the monitoring for your Python application.

Two files are provided to customize data collector settings, global.environment and config.properties. After you change the settings in these files, restart the Python application for the change to take effect.

By modifying the global.environment file, you can customize the connection between the data collector and the Cloud APM server. If you want to use another Cloud APM server instead of the default one, or the key file or its password is changed, modify the Cloud APM server to reconnect the data collector to the Cloud APM server.

By modifying the config.properties file, you can customize data collector behaviors according to your needs, such as enabling or disabling method trace.

The global.environment configuration file

Table 1 shows the environment variables that you can set in the global.environment configuration file and the correlated descriptions. You can find the global.environment file in the etc folder where your Python data collector is installed, for example, /root/.pyenv/versions/3.5.2/lib/python3.5/site-packages/ibm_python_dc/etc directory.

Table 1. Supported environment variables in the global.environment file
Variable name Importance Value Description
APM_BM_GATEWAY_URL Optional
  • https:
    //<server ip or hostname>:
    443
  • http:
    //<server ip or hostname>:80
 The target on-premises server gateway URL.
APM_KEYFILE_PSWD Optional Password of the key file The key file password that is paired with the key file.
Note: Set this variable only when you configured the Gateway to use HTTPS.
APM_KEYFILE_URL Optional http://<hosted http server>:<port>/keyfile.p12 The URL to download the key file.
Note: Set this variable only when you configured the Gateway to use HTTPS.

The config.properties file

Table 2 shows the environment variables you can set in the config.properties configuration files and the correlated description. You can find the config.properties file in the installation directory of your Python data collector, for example, /root/.pyenv/versions/3.5.2/lib/python3.5/site-packages/ibm_python_dc directory.

Table 2. Supported environment variables in the config.properties file
Variable name Importance Value Description
KPG_ENABLE_DEEPDIVE Optional
  • False
  • True
  • False: The default value. If you set this variable to False, diagnostics data will not be collected.
  • True: If you set this variable to True, diagnostics data will be collected.

The default level is True.

If you do not set this variable, diagnostics data will not be collected.
KPG_DD_CONFIG_FILE Optional File name of the diagnostics monitoring configuration file. File name of the diagnostics monitoring configuration file. The default file name is kpg_dd_config.xml.
Note: After you customize the settings in this file, you must put it in the application root directory.

If you do not set this variable, the default configuration file kpg_dd_config.xml in the data collector package will be used.
KPG_DD_APP_PATH Optional Path to the Python application. The path to the Python application or the module for which the data collector collects diagnostics data. Separate the paths of different Python applications and modules that you want to monitor with a semi-colon ;.

If you do not set this variable, the data collector will collect diagnostics data for requests and the modules that your application use. Data of requests in Python lib will not be collected.
KPG_DD_SECURITY_FILTER Optional
  • True
  • False
  • True: The default value. If you set this variable to True, values (such as the passwords) will be masked in SQL statements and parameters will not be displayed in group widget Request Context.
  • False: If you set this variable to False, values in SQL statements will not be masked and parameters will be displayed in group widget Request Context.

If you do not set this variable, values (such as the passwords) will be masked in SQL statements and parameters will not be displayed in group widget Request Context.
KPG_GC_STATS Optional True All statistical functions of python garbage collection are enabled. When you set this value to True, it equals running the following command: 

gc.set_debug(gc.DEBUG_STATS |
gc.DEBUG_COLLECTABLE |
gc.DEBUG_UNCOLLECTABLE | 
gc.DEBUG_INSTANCES | 
gc.DEBUG_OBJECTS )

To disable KPG_GC_STATS, delete this environment variable. Do not set it to False.

The default value is True.

Note: Never set KPG_GC_STATS=True in your formal product environment. It is only for the debug mode. And make sure enough memory is assigned to the application.
KPG_LOG_LEVEL Optional
  • DEBUG
  • ERROR
  • INFO
  • DEBUG: Only useful debug information will be printed in the log, for example, collected data, data that is sent to server, and server response.
  • ERROR: Only information about exceptions and very unexpected situations will be printed in the log.
  • INFO: The summary information about the data collector for the user to know what it is doing will be printed in the log.

The default value is ERROR.
KPG_LOG_TOCONSOLE Optional
  • Y
  • True
  • Any other value that is not False
 The log will be printed to console and you can see the log by running the command cf logs <appname>.

The default value is True.
KPG_SAVE_ALL Optional True All unreferenced objects will be saved into gc.garbage, and you need to clear gc.garbage every minute (the data collector does this for you). When the value is set to True, it equals running the following command: 
gc.set_debug(gc.SAVE_ALL)
To disable KPG_SAVE_ALL, delete this environment variable. Do not set it to False.

The default value is True.

Note:
Never set KPG_SAVE_ALL=True in your formal product environment. It is only for the Debug mode. And make sure enough memory is assigned to the application.
APM_GW_PROXY_CONNECTION Optional http://<server ip or hostname>:port The HTTP or HTTPS proxy that the Python data collector uses to send monitoring data.