Pacbase road map: Part 3 – Repository functions

By Nicolas Dangeville
Chief Architect, Rational Team Concert for System z
Rational

Repository functions are key factors in the IBM® VisualAge® Pacbase environment to providing scalability and productivity for large organizations. Nicolas Dangeville, Chief Architect of Rational Team Concert for System z®, discusses how the Pacbase repository functions are exposed and enhanced in the IBM Rational® Software Delivery Platform (SDP).

This article, the third in a series, describes the new features included in the updated IBM® Rational® Software Delivery Platform (SDP). These new features provide enterprise-wide, multi-criteria search and impact analysis functions across the Pacbase artifacts. The article also includes details about the level of extensibility that the solution offers.

The functions of the Pacbase repository

The Pacbase repository is one of the key capabilities of the Pacbase development tool. The Pacbase repository enables classification, storage, and management of the Pacbase artifacts. It also provides powerful search and end-to-end impact analysis across these artifacts. The combination of the Pacbase programming model and the repository functions provides a great level of control and scalability for the IT system.
The Pacbase repository provides two complementary set of services: 1) storage and version management, and 2) impact analysis and search.

Storage and version management

The Pacbase repository is a central database that you can use to store, manage and assign a version to Pacbase artifacts (such as data definitions, program model instances and the user specific code, and macros). The repository also provides classification mechanisms through the definition of hierarchical libraries that store artifacts based on their level of reusability in the whole organization.

Impact analysis and search

The Pacbase repository understands the artifacts that it stores, and therefore is able to perform multi-criteria search and impact analysis operations. Search can combine both technical and semantic criteria. An example of impact analysis is searching for all programs that reference a certain data element. You can therefore evaluate the impact of a change over all the artifacts in the system: an example of multi-criteria search is finding all the model instances that reference the concept of “Customer” or any of its synonyms. This search answers artifacts that might not be directly related at the model level but contribute to a common concept.

The nature of artifacts and their management will significantly evolve in the new offering composed of Rational Developer for System z (RDz) and Rational Team Concert for System z (RTCz). In the following sections, the new structure is described, including how configuration and search capabilities are provided and expanded.

Managing the artifacts in the new solution

Nature of artifacts in the new solution

A major difference between Pacbase as it is today and the new solution is in the way model instances and user-specific code are physically stored. In Pacbase, they both are entries in the repository, and don’t have any physical representation. In the new solution, model instances are serialized in XML and stored in files, which are physically located in Eclipse projects. User-specific code is now inserted directly in the COBOL generated code, with a lot of assistance and tools, as explained in Pacbase futures: Part 2. Therefore, user-specific code is stored with the generated code in files.

Given that both model and code are saved in files, the configuration management functions must now be performed against source files (whether they contain XML serialized model instances or source code). Providing configuration management functions (such as fine grained or coarse grained versioning, branching, compare and merge) is precisely the purpose of Source and Configuration Management (SCM); the SCM functions of RTCz are well-suited to perform those functions.

RTCz enables collaboration and common processes for the team. RTCz will provide the SCM in a multi-stream environment that exceeds the versioning capabilities of Pacbase (sessions). In particular, the concepts of working sessions and historical sessions map well to the concepts of Streams and Snapshots in the SCM of RTCz.

Classification

Classification in Pacbase is achieved by structuring the Pacbase repository in a tree of libraries, each library being able to store Pacbase elements. Classification is helpful because it gives a different visibility to a Pacbase element, and therefore defines its domain of applicability and its level of reuse. Because of the parent/child relationship of libraries, it is possible to dispatch or override characteristics of a model instance at two or more levels of libraries in the hierarchy.

In the new solution, location information of the model elements will be expressed by two complementary mechanisms:

• The logical concept of namespace, which is defined through a package. The package is a well-known concept in many programming languages, such as Java: a model instance is created within a package. The uniqueness of a model instance is defined by the combination of its package, its name and its type.
• A physical placeholder for the serialized model files, which is an Eclipse project that contains folders, nested to represent the package. Serialized model instances belong to the folder that matches the name of the package of the model instance.

Figure 1 shows a sample project, viewed from the resource explorer that displays the physical files and folders in the workspace. You can see that a project DemoLoc.Pro contains nested folders that contain files. These files are serialized model instances.

Figure 1 - projects viewed in the resource explorer 
Figure 2 shows the same project viewed by the design explorer. The design explorer displays the logical contents of the project, with model instances located in packages.

Figure 2 - projects viewed through the design explorer 
As in languages such as Java, a path (here called designPath) indicates the search path to be used to locate and resolve references from one model instance to the other. The design path indicates which projects can be explored from a given project, and in which order.

Redefinition of a model instance becomes possible by defining the same model instance (same package, name and type) in two different projects. The designPath provides the appropriate resolution, based on the location of the caller.

The classification mechanism in the new solution relies on the clear definition of namespaces for model instances and the physical storage of these serialized instances within projects and folders. The resulting configuration is therefore a combination of folders and files that SCMs can easily manage.

A guiding principle in this new solution is the consistency with concepts used in mainstream programming languages. This principle facilitates the adoption of this solution for existing Pacbase users and new developers, and therefore broadens the community.

Search and Impact analysis

You have seen so far that configuration management for Pacbase artifacts will be achieved by reusing SCM functions, as developers do today with any programming language. These functions however do not cover the search and impact capabilities across a complete system.

A dedicated component, on top of the RTCz SCM, provides search and impact analysis among artifacts in the SCM. This capability will be available for Pacbase artifacts and for non-Pacbase artifacts, providing increased integration.

This new component of RTCz works by collecting metadata about the sources in the RTCz SCM. This metadata is then stored in a dedicated database. The database provides a query language, called SPARQL, which is a W3C specification. Search and impact analysis requests are translated into SPARQL queries that answer references to the source files in the context of a RTCz Stream.

Metadata is collected by scanning the files. The metadata is tightly linked with the artifacts and the structure of the RTCz SCM. Therefore, a set of metadata is associated with each stream; metadata can be versioned along with SCM snapshots.

The following example shows how a user will work with these repository functions. The example shows where maintenance should be performed. It starts with a developer who receives a defect about a program.

Use the repository functions to find the program
The developer has to perform a fix that will be part of a maintenance release of the application. He therefore knows that the fix must be done in the maintenance stream. His workspace is currently empty and he wants to find which projects he needs to load to gain access to the model and specific code of the program.

Assume, for the purpose of the example, that the naming convention of the company gives him the first three letters of the program. Assume that the program to be fixed involves security.

The developer performs a repository search against the maintenance stream. The criteria used for the query will therefore involve artifacts of type "Program", starting with a given prefix and using the keyword “security” or any of its synonyms.

The query will return a number of results. Each result provides the name and label of the program, and the project where it is located. The developer can now identify the program to work on.

The developer views the appropriate program and selects the project that contains it. He triggers an action to load it in his workspace. This action will make the model instances and sources available in the developer’s workspace.

Perform maintenance on the program
The required program fix involves a change in the model representation of the program and some user-specific code change.
When the fix is complete, the developer creates a change set and attaches it to the defect in RTCz. He delivers his change set to the stream.

The RTCz server will now finds that some indexes are out of synch and reparses the changed files; these changed files include the fix the developer has made. The metadata for the model instance of the program is updated.

Perform impact analysis
Now the developer notices that a Data Element display representation must be updated. Care must be taken when changing a Data Element because it might be used by many programs directly or indirectly. When considering the change, the developer performs an impact analysis query.

Two different options are available for impact analysis:

1. The first option follows the relationships between model instances. This option is the classic impact analysis. Such query result is based on structural dependencies and is made possible because the repository collects dependency information for each model instance.
2. A second narrower option indicates which generated programs use a given data element, or any other model instance. This query can detect, for example, which programs are out of synch and must be generated. This type of query result is based on generation links. It is made possible because the repository collects the model instances actually visited by a given generator for a given program.

Through these examples, you have seen a few functions that the metadata collection and queries capabilities in RTCz will provide:

• Multi-criteria search
  The metadata being collected is defined by the scanner being used to parse the sources. This metadata can contain attributes valid for every model element (name, label) or attributes valid for a certain type of model element (type of a data element or length of a numeric data element).
  Any combination of properties that has been captured in the metadata can be queried. Regular expressions and wildcards can be used in the query.
• Thesaurus
  A thesaurus can be centrally defined in the repository. It provides a definition and synonyms for a given keyword. When a keyword is used as a criterion in a query, its synonyms are automatically be used.
• Impact analysis
  By using impact analysis, you can browse the dependency links that have been found during parsing and identify all the artifacts that contain a reference to a given artifact. Impact analysis can be scoped to find the direct and the indirect uses to follow dependencies defined in the object or to narrow the analysis to the generation links, and therefore detect generated programs that are out of synch.

Extensibility

The mechanism to perform a search on impact analysis queries relies on three complementary pieces:

• Metadata associated with an artifact managed by the RTCz SCM
• A parser that can obtain the metadata from the artifact
• A query language and predefined queries

Each of these pieces is customizable and extensible, providing a significant amount of flexibility:

Update interactively the metadata
The metadata captured in the repository can be visualized and edited by a user. Therefore, if a captured dependency is not relevant, or if it has not been captured correctly, the developer can fix it. Updating the metadata is not limited to dependencies. For example, users can add attributes for tagging files with specific characteristics that can be used afterwards in queries.

Define the scanner to be used
Metadata collection is performed by an extensible architecture; one or several parsers can be involved. By default, the RTCz server is configured with a scanner that captures the dependencies between model instances, and some key characteristics of the instances that are helpful in standard queries (such as a data element type and data length).

For each artifact, stored metadata includes common attritutes like name, label, and keywords. Parsers can provide more information.

A user-defined scanner can replace the standard scanner or be added to it.

Define your own queries
The solution comes with a set of predefined queries that are valuable in the context of Pacbase instances. Such queries are about searches that combine criteria or impact analysis. You can create new queries based on any attribute that has been captured in the metadata. The tool provides support in creating new queries or permits to write new ones queries manually by using the SPARQL query language. The queries can be run interactively from the developer’s workstation. You can also script requests that can be run as a batch program by using the query API of the server.

Future directions

The technology used for the repository functions in RTCz can enable more advanced features in future releases of the solution. Here are some directions that are being considered because of the value that they bring to the user.

End to end impact analysis
RTCz and the new solution for Pacbase share a common technology to manage and store dependency data. They also share a common representation of such data. Therefore, it becomes conceivable to create queries, such as impact analysis queries, that start from a Pacbase model element, reach the model element of a program, follow a traceability link to its generated code, and reach callers of the program written in a native language (for example, COBOL). Impact analysis queries can cross the boundary of models and source.

Dependency set and sparse loading queries
For a given program, the knowledge of the dependencies can be used to build a minimum list of all the artifacts that are required to represent the program. This list can include all the model instances that are referenced directly and indirectly from the program. This list can be used to perform a minimal load of the artifacts from the SCM to the developer’s workstation, which optimizes the amount of files to retrieve.

Summary

This article examined the functions of the Pacbase repository, where there are a combination of SCM-related functions, and search and impact analysis functions. The article included a discussion of the new nature of the artifacts in the new solution, how SCM and classification can be achieved by the RTCz SCM, and how metadata collection on top of the RTCz SCM can enable flexible queries for searches and impact analysis.

Note:
The information regarding IBM’s product plans provided herein is intended to outline general product direction and should not be relied on in making a purchasing decision. This information is for informational purposes only and may not be incorporated into any contract. The information is not a commitment, promise, or legal obligation to deliver any material, code or functionality. The development, release, and timing of any features or functionality described for our products remains at our sole discretion.