Video: Configuring an auto-scalable cluster for Liberty elasticity

The following transcript is for the Configuring an auto-scalable cluster for Liberty elasticity video, which demonstrates how to configure a collective to support Liberty server elasticity. This transcript is the video storyboard. Audio describes narration and captions. Onscreen Action describes the content shown in the video.

Video Configuring an auto-scalable cluster for Liberty elasticity

Table 1. Title page. Show title and then animation that illustrates autonomic provisioning and then starting and stopping of servers based on resource use.
Scene Audio Onscreen Action
1 This video shows how to configure an auto-scalable cluster for Liberty elasticity. Show title Configuring an auto-scalable cluster for Liberty elasticity.
2 With Liberty elasticity, you have autonomic starting and stopping of cluster members based on scaling policies and resource use, just like with JVM elasticity.

But you also have autonomic provisioning of new cluster members to other hosts.

Even if a host doesn't have Liberty installed, Liberty elasticity can autonomically install on that host what's called a stack of Liberty software--the Liberty runtime, an IBM JRE, and a Liberty server with your applications--and then autonomically start and stop those installed Liberty servers and their applications. This gives you greater capacity to handle spikes in resource demand.

What you see here is a scaling controller provisioning a stack group of Liberty software to hosts registered with the controller when resource use goes up, and then stopping or starting the provisioned cluster member servers and their applications in response to resource demands.

You determine what Liberty software needs to be in a stack group based on what Liberty software is available on a registered host.
Show heading Liberty elasticity auto scaling.

Show information Autonomic starting and stopping of cluster members based on scaling policies and resource use (JVM elasticity). Show animation of a scaling controller starting and stopping Liberty server cluster members on a registered host computer. When an arrow representing resource use points upward, show scaling member servers on the registered host going from stopped to running. Then, when the arrow representing resource use points downward, show scaling member servers on the registered host going from running to stopped.

Show information + Autonomic provisioning of new cluster members to other hosts. Show animation of a scaling controller provisioning new scaling members to two registered hosts and then starting and stopping those member servers based on the position of an arrow representing resource use.

Show information What's in a stack group?. Show animation of a stack of Liberty software moving to two registered hosts. The registered host without an IBM JRE that supports Liberty servers in JAVA_HOME path receives:
  • Liberty runtime (wlp)
  • IBM JRE (wlp.jre)
  • Liberty server package (wlp.usr)
The registered host with an IBM JRE that supports Liberty servers in JAVA_HOME path receives:
  • Liberty runtime (wlp)
  • Liberty server package (wlp.usr)
Table 2. Demo configuring a scaling controller. Show configuring a scaling controller.
Scene Audio Onscreen Action
3 I begin by creating a server. At the wlp/bin directory of my Liberty installation, I run the server create command and name the server myController. Show heading 1 Configure a scaling controller.

Show a command window open at wlp/bin, and run a command that creates a server:

server create myController
Then show the message Server myController created.
4 To make this server a collective controller, I run the collective create command. The command, by default, writes output to a console screen. You can specify the --createConfigFile option to write the output to a file instead. In the command window open at wlp/bin, run a command that makes the server a collective controller:
collective create myController --hostName=controller_host_name --keystorePassword=myController
5 I copy the output and paste it into the server.xml file of myController. Show the generated controller configuration. Then, show marking and copying the configuration in the command window. And finally, show opening an editor on the wlp/usr/servers/myController/server.xml file and pasting the configuration into the file before the </server> tag.
6 I specify adminUser and adminPassword for the controller administrative user name and password.

I add host="*" to the httpEndpoint element. This is optional.

To make myController a scaling controller, I add the scalingController-1.0 feature to a featureManager element.

I also add the adminCenter-1.0 feature so I can view my collective in Admin Center.

And finally, add a scaling policy. I begin with a scaling policy that is disabled and has a minimum and maximum of one server.
In the editor, show the following changes to the controller server.xml file:
  1. Add an administrative user name and password.
    <quickStartSecurity userName="adminUser" userPassword="adminPassword"/>

    Show information Add an administrative user name and password.

  2. Add host="*" to the httpEndpoint element.
  3. Add features to make myController a scaling controller and to enable administration of myController in Admin Center to a featureManager element.
    <feature>scalingController-1.0</feature>
    <feature>adminCenter-1.0</feature>

    Show information The scalingController-1.0 feature is required for a scaling controller and then The adminCenter-1.0 feature is optional for a scaling controller but needed for this example to use Admin Center.

  4. Add a scaling policy that is disabled and has a minimum and maximum of one server.
    <scalingDefinitions>
       <defaultScalingPolicy enabled="false" min="1" max="1"/>
    </scalingDefinitions>

    Show information Add a scaling policy. Start with the scaling policy disabled.

  5. Save the file.
7 To start myController, run the server start command at a wlp/bin command line. At an SSH service command line, enter commands that move to the wlp/bin directory and then start the controller.
cd /cygdrive/c/wlp/bin

./server start myController

Show information On Windows computers, using an SSH service to start servers enables a controller to connect to member servers with SSH.

The command runs and messages in the command window show myController started.
Table 3. Demo going to the Servers page of the Admin Center Explore tool. Show opening a browser on Admin Center, logging in, and displaying the Servers page of the Explore tool.
Scene Audio Onscreen Action
8 To view myController in Admin Center, open a browser on the https://controller_host_name:controller_port_number/adminCenter/ URL.

If your browser prompts you to confirm that the connection is trusted, specify an exception or otherwise enable the connection to Admin Center.

Go to the Explore tool Servers page.
Show heading Optional: View servers in Admin Center.

In a web browser:

  1. Open the browser on the https://controller_host_name:9443/adminCenter/ URL.
  2. Select to trust the connection.
  3. On the Admin Center login page, enter adminUser for the user name and adminPassword for the password. In the editor, highlight the user name and password in the scaling controller server.xml file at the same time to show needed login values.
  4. On the Toolbox page, select Explore icon.
  5. In the Explore tool, select the Servers panel. The Explore tool shows one running server, myController.
Table 4. Demo configuring a scaling member. Show creating a server, joining it as a member server to the controller, and then configuring it to be a scaling member.
Scene Audio Onscreen Action
9 I now make a scaling member. At a wlp/bin command prompt, I run the server create command and name the server myMember. Show heading 2 Configure a scaling member.

Show command window open at wlp/bin, and run:

server create myMember
Then show the message Server myMember created.
10 To join this server to the collective, I run the collective join command. You can get the --host, --port, --user and --password values from the controller server.xml file. For --keystorePassword, set a value to use for the member keystore password. The --hostName parameter is optional. In command window open at wlp/bin, run:
collective join myMember --host=controller_host_name --hostName=host_name --port=9443 --user=adminUser --password=adminPassword --keystorePassword=myMember
After a message asking whether to accept the certificate chain displays, enter y.
11 I now copy the output and paste it into the myMember server.xml. You can specify the --createConfigFile option to write the output to a file instead. Show marking and copying the generated configuration in the command window. Then, show opening an editor on the wlp/usr/servers/myMember/server.xml file and pasting the configuration into the file before the </server> tag.
12 In the member server.xml file, I set the RPC user and password, which are the user name and password that I enter to log on to this Windows computer. The user must have administrative privileges on the computer.

In this example, the controller and member are on the same host, so I need to change the port numbers. If your member server is on a different host from the controller, ensure the ports are unique on that host.

Add the scalingMember-1.0 feature to make the member server a scaling member.

Add a hostSingleton element and set its port to a number that is unique on the host. The host singleton port is used by all Liberty servers on the same host to elect a leader. The job of the leader is to report aggregated scaling metrics for all servers on the host. If the leader stops running, another leader is elected.

In the editor, show the following changes to the member server.xml file:
  1. Set the RPC user and password, which is the user name and password that you enter to log on to the computer. The user must have administrative privileges on the computer.
    <hostAuthInfo rpcUser="admin_user_id"
                  rpcUserPassword="admin_user_password" />
  2. Change the port numbers from 9080 to 9081 and 9443 to 9444 so that the member ports are different from the controller ports and from other ports used on the computer.
    <httpEndpoint id="defaultHttpEndpoint"
                  httpPort="9081"
                  httpsPort="9444" />
  3. Make myMember a scaling member by adding scalingMember-1.0 to a featureManager element.
    <feature>scalingMember-1.0</feature>

    Show information The scalingMember-1.0 feature is required for a scaling member. You can add both clusterMember-1.0 and scalingMember-1.0 or add only scalingMember-1.0. You don't need clusterMember-1.0 because the scalingMember-1.0 feature enables the clusterMember-1.0 feature.

  4. Add a hostSingleton element and set its port to 5164 or any other port that is unique on the computer.
    <hostSingleton name="ScalingMemberSingletonService" port="5164" />

    Show information A scaling member server.xml file must define a hostSingleton element with a port. All scaling members on the same host must use the same port. You can specify any port number, but the port number must be unique on the host computer. This example uses port number 5164.

  5. Save the file.
On the Admin Center Explore tool Servers page, myMember is shown as stopped.
13 Start myMember. At an SSH service command line on the wlp/bin directory, start the scaling member.
./server start myMember
The command runs and the messages showing myMember started display in the command window. On the Admin Center Explore tool Servers page, myMember is shown as started.
Table 5. Demo verifying that JVM elasticity auto scaling is working. Show enabling the scaling policy, with the auto scaling icon displaying on the Servers page of the Admin Center Explore tool, and then disabling the scaling policy.
Scene Audio Onscreen Action
14 I want to test the auto scaling. In the controller server.xml file, I enable the scaling policy and save the file. Show heading Optional: Test the scaling configuration.

In the editor open on the controller server.xml file, enable auto scaling. Change:

<defaultScalingPolicy enabled="false" min="1" max="1"/>
to:
<defaultScalingPolicy enabled="true" min="1" max="1"/>

On the Admin Center Explore tool Servers page, the auto scaling icon Auto scaling icon displays on the myMember card, indicating that the scaling policy is enabled.

Show information When scaling policies are enabled, scaling members have an auto scaling icon in Admin Center.
15 At this time, the configuration has JVM elasticity. The scaling controller can start and stop scaling members based on scaling policies and resource use. To verify that JVM elasticity auto scaling is working, change the min value to 0 in the editor:
<defaultScalingPolicy enabled="true" min="0" max="1"/>

Show information Setting the minimum to 0 (zero) stops the scaling member.

On the Admin Center Explore tool Servers page, myMember status changes from started to stopped, indicating that JVM elasticity auto scaling is working.
16   In the editor open on the controller server.xml file, change the min value back to 1:
<defaultScalingPolicy enabled="true" min="1" max="1"/>

Show information Setting the minimum to 1 (one) restarts the scaling member.

On the Admin Center Explore tool Servers page, myMember status changes from stopped to started, indicating that JVM elasticity auto scaling is working.
17 Because I haven't yet set up the configuration for Liberty elasticity, I disable the scaling policy. In the editor open on the controller server.xml file, disable auto scaling.
<defaultScalingPolicy enabled="false" min="1" max="1"/>

Show information Testing confirms that auto scaling is working. For this example, temporarily disable scaling policies until the stack group and its installables and packages are defined..

On the Admin Center Explore tool Servers page, the auto scaling icon Auto scaling icon is no longer shown on the myMember card, indicating that scaling policies are disabled.
Table 6. Demo adding a stack group cluster name to the member server.xml file. Show assigning the member to myStackGroup.cluster1. By default, cluster members are in defaultCluster.
Scene Audio Onscreen Action
18 In the member server.xml file, I add a statement that puts the member in a cluster named myStackGroup.cluster1. I am going to make the myStackGroup directory and put installables and packages for Liberty elasticity auto scaling into it. You can use the default stack group name, defaultStackGroup. cluster1 is the name of the server package that I'm going to make.

If your collective has other dynamic cluster members, add the same statement to the other member server.xml files so they belong to the same cluster.

Show heading 3 Add the scaling member to a cluster named stack_group_name.provisionable_cluster_name.

In the editor open on the member server.xml file, add a statement that puts the member in a cluster named myStackGroup.cluster1:

<clusterMember name="myStackGroup.cluster1"/>

Show information myStackGroup is the name of the stack group directory that will hold installables and packages for Liberty elasticity auto scaling and cluster1 is the name of the server package that the scaling controller will provision to registered hosts.

On the Admin Center Explore tool Servers page, the name of the cluster for myMember changes from defaultCluster to myStackGroup.cluster1.
Table 7. Demo registering a target host. Show running the registerHost command to register a target host with the collective controller.
Scene Audio Onscreen Action
19 I want Liberty elasticity auto scaling to install cluster1 on a different host.

I run the collective registerHost command and specify a Linux® machine for the target host.

You can run additional registerHost commands and register more than one target host.
Show heading 4 Register a host, to which the scaling controller will provision servers based on scaling policies.

Show a command window open at wlp/bin, and then run a command to register the target host:

collective registerHost target_host_name --host=controller_host_name --port=9443 --user=adminUser --password=adminPassword --rpcUser=target_host_OS_user --rpcUserPassword=target_host_OS_user_password
After a message asking whether you want to accept the certificate chain displays, enter y. Then show a message confirming the host registered successfully.
20   In Admin Center, select Dashboard icon, the Dashboard icon, to return to the Explore tool Dashboard, select the Hosts panel, show that the registered host has been added, and then return to the Servers page.
21 If the registered host has no Liberty software, which is the case here, then the stack group needs to contain a Liberty runtime, IBM JRE, and a Liberty server package with applications. Show the file system of the registered host.

Show information For this example, the registered host does not have Liberty or an IBM JRE that supports Liberty installed. Liberty elasticity auto scaling will need to provision to the registered host:

  • Liberty runtime installable (wlp)
  • JRE installable (wlp.jre)
  • Server package (wlp.usr)
Table 8. Demo creating the stackGroups/myStackGroup directories. Show creating the wlp/usr/shared/stackGroups/myStackGroup directory with its installables and packages subdirectories.
Scene Audio Onscreen Action
22 I'm now going to create the myStackGroup directory. This directory name is the same as what's in the cluster name.

I copy the defaultStackGroup directory, and rename it.

I'll be putting Liberty software that I want the scaling controller to install on the registered host in the installables and packages subdirectories.
Show heading Optional: Create your own stack group subdirectories if you don't want to use the default stack group.

Show a file system browser and move to the wlp/usr/shared/stackGroups directory. Copy the defaultStackGroup directory, paste it into the stackGroups directory, and rename the pasted directory myStackGroup. Then show the installables and packages subdirectories.

Table 9. Demo creating the Liberty runtime installable and adding it to the stackGroups/myStackGroup/installables directory. Show running a package command to create wlp.855.zip and then adding the file to the wlp/usr/shared/stackGroups/myStackGroup/installables directory.
Scene Audio Onscreen Action
23   Show heading 5 Create a Liberty runtime installable named the wlp.name.zip naming convention.

Show a file system browser and move to the wlp_installable directory. Show that the directory is empty.

24 To create a Liberty runtime archive, I run the server package command with the --include=wlp option. This creates an archive that contains the wlp directory, but does not contain the usr directory. The archive name, wlp.855.zip, follows the naming convention. Show a command window open at wlp/bin, and then run a package command to create a Liberty runtime installable named wlp.855.zip to go in the wlp_installable empty directory:
server package --include=wlp --archive=c:\wlp_installable\wlp.855.zip
The command runs and the messages show c:\wlp_installable\wlp.855.zip packaging completed.
25 Copy the archive to the stack group installables directory. In the file system browser, copy wlp.855.zip and paste it into the wlp/usr/shared/stackGroups/myStackGroup/installables directory.
Table 10. Demo creating the JRE installable and adding it to the stackGroups/myStackGroup/installables directory. Show compressing the contents of the java directory of a Liberty JRE installation to create jre.17.zip and then adding the file to the wlp/usr/shared/stackGroups/myStackGroup/installables directory.
Scene Audio Onscreen Action
26 To create an IBM JRE that supports Liberty servers, zip up the contents of the java directory of the JRE used for Liberty.

Move the archive to the stack group installables directory.

Show heading 6 Create a JRE installable named with the jre.name.zip naming convention if the registered host doesn't have or use an IBM JRE that supports Liberty servers.

Show a file system browser and move to the java directory of a Liberty JRE installation. Show compressing the contents of the java directory to create jre.17.zip. Then copy the file and paste it into the wlp/usr/shared/stackGroups/myStackGroup/installables directory.

Show information: The JRE installable must be appropriate for the operating system of the registered hosts

Table 11. Demo creating the cluster1 server package and adding to the myStackGroup/packages directory. Show running the create command to create a server named cluster1, pasting an application into the dropins directory of cluster1, running a package command to create cluster1.zip, and then adding the file to the wlp/usr/shared/stackGroups/myStackGroup/packages directory.
Scene Audio Onscreen Action
27 To create the cluster1 server package, create a server named cluster1. Show heading 7 Create a server package, which the scaling controller will provision to the registered host based on scaling policies. This example packages a server named cluster1 that has an application in its dropins directory..

Show a command window open at wlp/bin, and run:

server create cluster1
Then show the message Server cluster1 created.
28 Add one or more applications to it. In a file system browser, paste an application into wlp/usr/servers/cluster1/dropins to add an application to the server.
29 And run the server package cluster1 command with the --include=usr option. This creates a server package that contains the usr directory, but does not contain all the Liberty runtime files. In the command window at wlp/bin, run:
server package cluster1 --include=usr
Then show the message confirming package creation and its location.

Show information: For Windows registered hosts, create a server.env file that sets JAVA_HOME to the JRE location on the registered hosts. Place the server.env in the same directory as the server.xml in the server package ZIP. Example server.env file contents: JAVA_HOME=C:\wlp.jre\jre.17.zip\jre

30 Copy the zip file to the stack group packages directory. In the file system browser, copy wlp/usr/servers/cluster1/cluster1.zip and paste it into the wlp/usr/shared/stackGroups/myStackGroup/packages directory.
Table 12. Demo setting a user name and password for the stack manager in the scaling controller server.xml file. Show adding a stackManager element with the administrative user name and password to the scaling controller server.xml file.
Scene Audio Onscreen Action
31 In the controller server.xml file, add a statement that sets the user name and password for the stack manager. Show heading 8 Add a stackManager element to the scaling controller configuration.

In the editor open on the scaling controller server.xml file, add a statement that sets the administrative user name and password for the stack manager:

<stackManager controllerUser="adminUser" controllerUserPassword="adminPassword" />
Then, save the file.
Table 13. Demo changing the scaling policy to force the scaling controller to provision a new server on the registered host. Show changing the scaling policy in the scaling controller server.xml file to enable auto scaling and require a minimum of two servers. Then show a server added to the Admin Center Explore tool Servers page.
Scene Audio Onscreen Action
32 I'm now ready to test the Liberty elasticity auto scaling. I can force the scaling controller to provision the Liberty software to the registered host by increasing the min and max values and enabling the scaling policy.

After I save the changes, I wait for Admin Center to display the provisioned member server.

As you can see, the scaling controller installed a cluster1 member on the registered host then started the member and its application.
Show heading 9 Set scaling policy min and max values that force the scaling controller to provision a new server, and enable scaling policies.

In the editor open on the scaling controller server.xml file, increase the min and max values from 1 to 2 and enable auto scaling by changing enabled from false to true:

<scalingDefinitions>
   <defaultScalingPolicy enabled="true" min="2" max="2"/>
</scalingDefinitions>

Then, save the changes.

Show information Provisioning installables and a server package to a registered host can take several minutes.

The Admin Center Explore tool Servers page shows a cluster1 server provisioned on the registered host. Its status changes from stopped to started.
33 Liberty elasticity auto scaling is working so I set the min and max values to what I want for the auto scaling. The provisioned member will start and stop based on scaling policies and resource use. In the editor open on the scaling controller server.xml file, decrease the min value from 2 to 1:
<scalingDefinitions>
   <defaultScalingPolicy enabled="true" min="1" max="2"/>
</scalingDefinitions>

Then, save the changes.

Show information This example sets min=1 and max=2 for the minimum and maximum number of member servers available for auto scaling. You can set a higher maximum, such as max=4, to enable additional cluster1 member servers to be provisioned on the registered host. The scaling controller can start provisioned member servers when resource use increases and stop the member servers when resource use decreases..

The Admin Center Explore tool Servers page shows the provisioned cluster1 server status change from started to stopped, or from stopped to started, indicating that Liberty elasticity auto scaling is working.
Table 14. Show new directories and files on the registered host. Show that the registered host now has the wlp, wlp.jre, and wlp.usr directories for the Liberty runtime and JRE installables and cluster1 server package on the registered host.
Scene Audio Onscreen Action
34   Show heading Registered host before auto scaling with a screen capture of a file system browser on the registered host, which has no wlp directories. Show heading Registered host after auto scaling with a screen capture of the file system browser on the registered host with the wlp, wlp.jre, and wlp.usr directories.
35 The registered host now has the wlp, wlp.jre, and wlp.usr directories for the Liberty runtime and JRE installables and cluster1 server package. Show heading Registered host after auto scaling. Show a screen capture of the file system browser on the registered host with the wlp, wlp.jre, and wlp.usr directories for the Liberty runtime and JRE installables and cluster1 server package highlighted. Show information Liberty runtime installable, IBM JRE installable, and cluster1 server package.
Table 15. Conclusion. Show where to find more information about auto scaling with Liberty elasticity.
Scene Audio Onscreen Action
36 For more information about auto scaling in Liberty, see the Liberty documentation. Show the following text:

For more information about Liberty auto scaling, see:

WebSphere Application Server Liberty documentation on IBM Knowledge Center
http://www.ibm.com/support/knowledgecenter/
Update: Since this video was made, IBM Documentation has replaced IBM Knowledge Center.

For more information, see Configuring provisionable clusters for Liberty elasticity.