Configuring dynamic cache (DynaCache) to use the WebSphere eXtreme Scale dynamic cache provider

The dynamic cache engine is the default cache provider for the dynamic cache APIs and framework. However, dynamic cache allows WebSphere eXtreme Scale to act as its core caching engine. You can configure dynamic cache to use WebSphere eXtreme Scale as your cache provider instead of the default dynamic cache engine. Configuring dynamic cache to use WebSphere eXtreme Scale lets you leverage transactional support, improved scalability, high availability, and other WebSphere eXtreme Scale features without changing your existing dynamic cache caching code.

Before you begin

  • Read the Introduction: Dynamic cache topic for an overview of WebSphere eXtreme Scale functionality.
  • Determine whether using WebSphere eXtreme Scale is beneficial to the applications running in your application servers.

    The WebSphere eXtreme Scale features significantly increase the distributed capabilities of the dynamic cache function beyond what is offered by the default dynamic cache engine and data replication service. With WebSphere eXtreme Scale, you can create caches that are truly distributed between multiple servers, rather than just replicate and synchronize caches between the servers. WebSphere eXtreme Scale caches are transactional and highly available, ensuring that each server sees the same dynamic cache content. WebSphere eXtreme Scale also offers a higher quality of service for cache replication than what is provided through the data replication service (DRS).

    However, these advantages do not mean that the eXtreme Scale dynamic cache provider is the correct choice for every application. Use the decision trees and feature comparison matrix provided in the topic about configuring the dynamic cache instance in the WebSphere eXtreme Scale Version documentation for a overview of this cache provider.

If you decide that is beneficial for your applications, use the information about the dynamic cache provider in the WebSphere eXtreme Scale documentation to determine the appropriate WebSphere eXtreme Scale topology for your caching deployment.

If you use WebSphere eXtreme Scale, instead of the default dynamic cache engine, dynamic cache has the following limitations:
  • No disk cache support. The following custom properties do not work:
    • com.ibm.ws.cache.CacheConfig.enableDiskOffload
    • com.ibm.ws.cache.CacheConfig.diskOffloadLocation
    • com.ibm.ws.cache.CacheConfig.flushToDiskOnStop
    • com.ibm.ws.cache.CacheConfig.htodCleanupFrequency
    • com.ibm.ws.cache.CacheConfig.htodDelayOffload
    • com.ibm.ws.cache.CacheConfig.htodDelayOffloadEntriesLimit
    • com.ibm.ws.cache.CacheConfig.htodDelayOffloadDepIdBuckets
    • com.ibm.ws.cache.CacheConfig.htodDelayOffloadTemplateBuckets
    • com.ibm.ws.cache.CacheConfig.diskCachePerformanceLevel
    • com.ibm.ws.cache.CacheConfig.diskCacheEvictionPolicy
    • com.ibm.ws.cache.CacheConfig.diskCacheHighThreshold
    • com.ibm.ws.cache.CacheConfig.diskCacheLowThreshold
    • com.ibm.ws.cache.CacheConfig.diskCacheSize
    • com.ibm.ws.cache.CacheConfig.diskCacheSizeInGB
    • com.ibm.ws.cache.CacheConfig.diskCacheEntrySizeInMB
    • com.ibm.ws.cache.CacheConfig.explicitBufferLimitOnStop
    • com.ibm.ws.cache.CacheConfig.lruToDiskTriggerTime
    • com.ibm.ws.cache.CacheConfig.lruToDiskTriggerPercent
    Avoid trouble: The disk offload function should be disabled prior to configuring extremeScale. To disable the disk offload function:
    1. In the administrative console, click Servers > Server Types > WebSphere application servers > server_name > Container Services > Dynamic cache service.
    2. Locate the Enable disk offload property and unselect this property if it is selected.
    If you forget to disable disk offload prior to configuring extremeScale as the cache provider, use the following wsadmin commands to disable the disk offload function:
    1. Issue the following command to find the config object for the server.
      wsadmin>$AdminConfig list DynamicCache
      (cells/optimusprimeCell01/nodes/optimusprimeNode01/servers/server1|serve
      r.xml#dynamic_cache_name)
      
    2. Issue the following AdminConfig modify command to set diskoffload to false:
      wsadmin>$AdminConfig modify
      (cells/optimusprimeCell01/nodes/optimusprimeNode01/servers/server1|serve
      r.xml#dynamic_cache_name) {{enableDiskOffload false}} 
    3. Issue the following command to save this configuration change:
      wsadmin>$AdminConfig save
  • No DRS replication (push or push-pull support). The following custom properties will not work:
    • com.ibm.ws.cache.CacheConfig.enableReplicationAcks
    • com.ibm.ws.cache.CacheConfig.enableCacheReplication
    • com.ibm.ws.cache.CacheConfig.replicationDomain
    • com.ibm.ws.cache.CacheConfig.cacheEntryWindow
    • com.ibm.ws.cache.CacheConfig.cachePercentageWindow
    • com.ibm.ws.cache.CacheConfig.cacheInvalidateEntryWindow
    • com.ibm.ws.cache.CacheConfig.cacheInvalidatePercentWindow
    • com.ibm.ws.cache.CacheConfig.filterTimeOutInvalidation
    • com.ibm.ws.cache.CacheConfig.filterLRUInvalidation
  • The alias API feature is not supported for Object cache.
  • Event listener is supported. When firing any event, WebSphere® eXtreme Scale always sets the sourceOfInvalidation to REMOTE.
  • Disable dependency ID, com.ibm.ws.cache.CacheConfig.disableDependencyId, and templates, com.ibm.ws.cache.CacheConfig.disableTemplatesSupport, are not supported.
  • No PMI support.
  • The following CacheStatistic counters are supported:
    • CacheHits
    • CacheLruRemoves
    • CacheMisses
    • CacheRemoves
    • ExplictInvalidationsFromMemory
    • MemoryCacheEntries
    • TimeoutInvalidationsFromMemory
  • NioMap - skipMemoryAndWriteToDisk will not work because the disk cache is not supported. In addition, the DistributedNioMapObject.release() is not called to release byteBuffer to the NIO buffer management.

Complete the following actions to enable the WebSphere eXtreme Scale dynamic cache provider.

Procedure

  1. Install the WebSphere eXtreme Scale client, or the WebSphere eXtreme Scale client and server packages in your application server for the remote server and the other topologies respectively.
    [z/OS]Avoid trouble: Only the WebSphere eXtreme Scale client installation, and the remote topology are supported.
  2. Designate the WebSphere eXtreme Scale dynamic cache provider as your cache provider.

    Each cache instance can be individually configured to use WebSphere eXtreme Scale. The dynamic cache engine is the default cache provider for a cache instance. Cache instances configured with WebSphere eXtreme Scale can coexist with cache instances configured with DRS.

    Complete one of the following actions to designate the WebSphere eXtreme Scale dynamic cache provider as your cache provider:

    1. Use the administrative console to designate the WebSphere eXtreme Scale dynamic cache provider as your cache provider.
      1. In the administrative console, click Servers > Server Types > WebSphere application servers > server_name.
      2. Under Container Services, click Dynamic cache service server_name, and then, in the Cache provider field, select WebSphere eXtreme Scale dynamic cache provider .
      3. Click OK.
    2. Add the cacheProviderName property to the cacheinstances.properties file that is bundled in an enterprise application, and set the property to com.ibm.ws.objectgrid.dynacache.CacheProviderImpl.

      For example, for cache.instance.26, you would add the following line to the cacheinstances.properties file:

      cache.instance.26.cacheProviderName = com.ibm.ws.objectgrid.dynacache.CacheProviderImpl
    3. Add the following Factory APIs to the code for an enterprise application:
      Properties p = new Properties();
                      ==>p.put(CacheConfig.CACHE_PROVIDER_NAME, CacheConfig.CACHE_PROVIDER_OBJECT_GRID);
                      DistributedMap map1 = DistributedMapFactory.getMap("myMap", p);
                      DistributedMap map2 = DistributedObjectCacheFactory.getMap("myMap2", p);
  3. Configure the replication setting for the cache instance.

    With the WebSphere eXtreme Scale dynamic cache provider you can have local cache instances, and replicated cache instances.

    If you are only going to use local cache instances, go to the last step, and save your configuration changes. In a local cache the WebSphere eXtreme Scale container is co-located with the JVM; that is, the WebSphere eXtreme Scale container and WebSphere Application Server share the same JVM heap.

    If you are going to use replicated caches, complete one of the following actions, depending on how you have created the cache instance:

    1. On the Java virtual machine > Custom properties page in the administrative console, click New again.
      Enter com.ibm.ws.cache.CacheConfig.enableCacheReplication in the Name field, and true in the Value field, and then click OK.
    2. Add the enableCacheReplication property to the cacheinstances.properties file that is bundled in an enterprise application, and set the property to true.
      For example, for cache.instance.26, you would add the following line to the cacheinstances.properties file:
      cache.instance.26.enableCacheReplication = true
    3. Add the following Factory APIs to the code for an enterprise application:
      Properties p = new Properties();
                      p.put(CacheConfig.CACHE_PROVIDER_NAME, CacheConfig.CACHE_PROVIDER_OBJECT_GRID);
                      ==>p.put(CacheConfig.ENABLE_CACHE_REPLICATION, "true");
                      DistributedMap map1 = DistributedMapFactory.getMap("myMap", p);
                      DistributedMap map2 = DistributedObjectCacheFactory.getMap("myMap2", p);
  4. Configure the WebSphere eXtreme Scale replication topology.

    The only required configuration parameter for the WebSphere eXtreme Scale dynamic cache provider is the cache topology parameter.

    1. On the Java virtual machine > Custom properties page in the administrative console, click New again.
      Enter com.ibm.websphere.xs.dynacache.topology in the Name field, and one of the following values in the Value field:
      • embedded
      • embedded_partitioned
      • remote
      Avoid trouble: If you specify embedded_partitioned, you must also add the com.ibm.websphere.xs.dynacache.num_initial_containers custom property to your JVM settings, and set this property to an integer that is equal to or slightly less than the total number of server instances that are accessing this distributed cache instance.
    2. Add the com.ibm.websphere.xs.dynacache.topology property to the cacheinstances.properties file that is bundled in an enterprise application, and set the property to embedded, embedded_partitioned, or remote.
      For example, for cache.instance.26, you would add the following line to the cacheinstances.properties file:
      cache.instance.26.com.ibm.websphere.xs.dynacache.topology = embedded
      Avoid trouble: If you specify embedded_partitioned, you must also add the com.ibm.websphere.xs.dynacache.num_initial_containers property to the cacheinstances.propertiesfile, and set this property to an integer that is equal to or slightly less than the total number of server instances that are accessing this distributed cache instance.

      For example, if a dynamic cache service is shared between grid members, then the variable should be set to the number of grid members.

    3. Add the following Factory APIs to the code for an enterprise application:
      Properties p = new Properties();
                      p.put(CacheConfig.CACHE_PROVIDER_NAME, CacheConfig.CACHE_PROVIDER_OBJECT_GRID);
                      p.put(CacheConfig.ENABLE_CACHE_REPLICATION, "true");
                      ==>p.put("com.ibm.websphere.xs.dynacache.topology", "embedded");
                      ==>p.put("com.ibm.websphere.xs.dynacache.num_initial_containers", "3");
                      DistributedMap map1 = DistributedMapFactory.getMap("myMap", p);
                      DistributedMap map2 = DistributedObjectCacheFactory.getMap("myMap2", p);

    See the Configuring the catalog service in WebSphere Application Server topic in the WebSphere eXtreme Scale documentation for a description of how to start the catalog service process in a WebSphere Application Server environment.

  5. Configure the eXtreme Scale catalog service grid.

    When you run a catalog service grid, you must set the catalog.services.cluster custom property for the catalog service endpoints.

    See the topic Starting the catalog service process in a WebSphere Application Server environment in the WebSphere eXtreme Scale Version 7 documentation for a description of how to start the catalog service process in a WebSphere Application Server environment.

    1. In the administrative console, click System administration > Cell > Custom properties > New.
    2. Enter catalog.services.cluster in the Name field, and the appropriate server_name:host_name:client_port:peer_port:listener_port value in the Value field.
      • server_name is the fully qualified name of the WebSphere process, such as the cell name, node name, or server name, of the server that hosts the catalog service. Example: cellA\node1\nodeagent
      • host_name is the name of the hosting server.
      • client_port is the port that is used for peer catalog grid communication.
      • peer_port is the port that is used for peer catalog grid communication.
      • listener_port is the listener port. This port must match the BOOTSTRAP_ADDRESS value that is defined in the server configuration.
      Following is an example of a valid value:
      cellA\node1\nodeagent:host.local.domain:6600:6601:2809,cellA\node2\
      nodeagent:host.foreign.domain:6600:6601:2809
    3. Click OK.
  6. Click Save to save all of your configuration changes.
  7. Restart all the application servers that you configured to use WebSphere eXtreme Scale.
  8. Configure custom key objects.

    When you are using custom objects as keys, the objects must implement the Serializable or Externalizable interface. If you are using custom objects with the embedded or embedded partitioned topologies, you must place the objects on the shared library path, in the same way that you do if you are using the default dynamic cache provider. For more information, see the topic Using the DistributedMap and DistributedObjectCache interfaces for the dynamic cache.

    If you are using the remote topology, you must place the custom key objects on the CLASSPATH for the stand-alone WebSphere eXtreme Scale containers. See the steps to start a container process in the WebSphere eXtreme Scale Version 7.0 Administration Guide for more information. This publication is available on the WebSphere eXtreme Scale library page

  9. Configure eXtreme Scale container servers.

    The cached data is stored in WebSphere eXtreme Scale containers. These containers can run inside or outside of a WebSphere Application Server process. The eXtreme Scale provider automatically creates containers inside a WebSphere Application Server process when you are using embedded or embedded partitioned topologies for a cache instance. No further configuration is needed for these topologies.

    When you use the remote topology, you must start up stand-alone eXtreme Scale containers before you start the WebSphere Application Server instances that access the cache instance. The WebSphere eXtreme Scale Version 7.0 Administration Guide documents the steps that you need to complete to start stand-alone containers.

    Avoid trouble: Make sure that all the containers for a specific dynamic cache point to the same catalog service endpoints.

    The dynacache-remoteobjectgrid. xml and dynacache-remote-definition.xml configuration files for the stand-alone eXtreme Scale Dynamic Cache provider containers are located in the install_root/customLibraries/ ObjectGrid/dynacache/etc directory if WebSphere eXtreme Scale is installed on top of the WebSphere Application Server, or in the install_root/ObjectGrid/dynacache/etc directory if you are using a stand-alone version of WebSphere eXtreme Scale. Make copies of these files to edit and use when launching stand-alone containers for the eXtreme Scale dynamic cache provider. The value specified for the numIntitialContainers parameter in the dynacache-remote-deployment.xml file should be based on the number of container processes being run.

    The following example illustrates a UNIX-based command line entry that launches a stand-alone container for the WebSphere eXtreme Scale dynamic cache provider:
    startOgServer.sh container1 -objectGridFile ../dynacache/etc/dynacache-remoteobjectgrid.
    xml -deploymentPolicyFile ../dynacache/etc/dynacache-remotedeployment.
    xml -catalogServiceEndpoints MyServer1.company.com:2809
    Avoid trouble: The set of container processes needs to have enough free memory to service all the dynamic cache instances configured to use the remote topology. Any WebSphere Application Server process that shares the same or equivalent values for catalog.services.cluster must use the same set of stand-alone containers. The number of containers and number of machines they reside on needs to be sized appropriately. See the topic Capacity planning and high availability in the WebSphere eXtreme Scale Version 7.0 Product Overview for additional details. This publication is available on the WebSphere eXtreme Scale library page.
  10. Verify that the WebSphere eXtreme Scale dynamic cache provider is correctly configured.
    If the WebSphere eXtreme Scale dynamic cache provider is correctly configured, the system log contains a number of messages similar to the following messages:
    DYNA1001I: WebSphere Dynamic cache instance named "{0}" initialized successfully using cache provider "{1}".
    DYNA1071I: The cache provider \"{0}\" is being used.
    If the configuration and initialization of the cache instance with WebSphere eXtreme Scale fails the dynamic cache runtime reverts to the default dynamic cache cache provider, and you should see messages similar to the following message in the system log.
    DYNA1066E: Unable to initialize the cache provider \"{0}\". 
    The Dynamic cache will be used to create the cache instance \"{1}\" 
    instead of the configured cache provider.