IBM InfoSphere Streams Version 4.1.0

Operator DSPFilter

SPL standard and specialized toolkits > com.ibm.streams.timeseries 3.1.0 > com.ibm.streams.timeseries.analysis > DSPFilter

Primitive operator image not displayed. Problem loading file: ../../image/tk$com.ibm.streams.timeseries/op$com.ibm.streams.timeseries.analysis$DSPFilter.svg

DEPRECATED: The com.ibm.streams.timeseries.analysis.DSPFilter operator is deprecated and is replaced by the com.ibm.streams.timeseries.analysis.DSPFilter2 operator. The deprecated operator might be removed in a future release.

The Digital Signal Processing (DSP) filter operator performs a digital filtering operation on an input time series. Digital filtering is a pervasive techniques use to enhance or reduce certain characteristics of the time series. It can be used for countless data processing, for example, data smoothing, trending, enhancements, amplification, and frequency resolution.

Digital filtering is a linear process that performs a weighted sum of current and past inputs and past past outputs in order to produce the current output. The set of coefficients applied to input values are referred to as a(k) and the set of coefficients applied to past output values are referred to as b(k). k is the time-lag, with k=0 meaning the current value, k=1 meaning the past value and so on. The choice of a(k) and b(k) determines the type of filtering to be performed. Typical type of filters include the high-pass filter which removes slow-varying components, the low-pass filter which removes high-varying components, and the band pass filter which removes both high-varying and low-varying components.

The DSPFilter operator is a univariate operator that supports expanding time series. Each component in the vector time series is processed independently of other components and all components are processed in parallel. The filter parameters can automatically be estimated for low-pass and high-pass filters, given the user-provided cut-off frequency. The algorithm used for filter estimation is the Butterworth filter design of order 2. The type of filter is specified by filterType parameter. For non-supported filter type, external software can be used to estimate the filter coefficients. The estiimated coefficients are then passed to the operator by using the xcoef parameter for the a(k) values and the ycoef parameter for the b(k) values. The estimated coefficients are passed as list of parameter values in {lag, value} pairs. These parameter values are used by the DSPFilter operator to filter the input time series.

Behavior in a consistent region

  • The operator is not supported in a consistent region. A warning occurs when you compile your streams processing application.
  • The operator cannot be the start of a consistent region. An error occurs when you compile your streams processing application.

Exceptions

The DSPFilter operator throws an exception and terminates in the following cases:

  • The input time series dimension grows over time.
  • The input time series dimension exceeds the maxDimension parameter value.

Run time signal control

The control port is an optional input port where you can send control signals to control the behavior of an operator. You can change the behavior of the operator at run time without having to recompile your application. The control port supports the following control signals:
  • Load: Initialize the model with the provided coefficients. If you provide incorrect coefficients, the operator logs a warning message in the log file and continues to predict values by using the older coefficients.
  • Monitor: Write the coefficients to the optional monitor output port for later use or for further analysis.
Examples
These examples demonstrate the use of the DSPFilter operator.

Summary

Ports
This operator has 2 input ports and 2 output ports.
Windowing
This operator does not accept any windowing configurations.
Parameters
This operator supports 12 parameters.

Required: inputTimeSeries

Optional: xcoef, ycoef, coefParameterFile, filterType, samplingRate, cutOffFrequency, filteredOnly, maxDimension, partitionBy, controlSignal, inputCoefficient

Metrics
This operator does not report any metrics.

Properties

Implementation
C++
Threading
Always - Operator always provides a single threaded execution context.

Input Ports

Ports (0)

Consumes timeseries data for filtering against the DSPFilter. The inputTimeSeries parameter specifies the name of the attribute on this port that contains the time series data. The accepted data types are float64 and list<float64>.

Properties

Ports (1)

Accepts control signals to control the behaviour of the operator. The controlSignal parameter specifies the name of the attribute on this port that contains the control signal type. The inputCoefficient parameter specifies the attribute name on this port that contains the new input coefficents.

Properties

Output Ports

Assignments
This operator allows any SPL expression of the correct type to be assigned to output attributes.
Output Functions
DSPFilterFunc
<any T> T AsIs(T v)

The default function for output attributes. By default, this function assigns the output attribute to the value of the input attribute with the same name.

<any T> T filteredTimeSeries()

This function returns the filtered time series data. The return type is the same as the type of the time series input.

Getcoeff
map<rstring,map<uint32,float64> > coefficients()

This function returns the current coefficients for the DSPFilter.

<any T> T AsIs(T v)

The default function for output attributes. By default, this function assigns the output attribute to the value of the input attribute with the same name.

Ports (0)

Submits a tuple containing the result of the filter. This port will submit a tuple each time the input time series is filtered. Custom output functions are used to specify the submitted data. The output tuple attributes whose assignments are not specified are assigned from input attributes.

Properties

Ports (1)

Submits a tuple containing the coefficients used by the filter. This port will submit a tuple each time a Monitor signal is consumed on the input control port. The coefficients() output function is used to assign the value of the coefficients to an attribute. The expected type of the attribute is map<rstring,map<uint32,float64> >.

Properties

Parameters

This operator supports 12 parameters.
xcoef

This parameter of type map<uint32, float64> specifies the a(k) coefficients in the form of a series of {lag:value}, separated by a comma and enclosed in curly braces.Non-specified lags are assumed to be zero. For example: xcoef:{1u:1.0,3u:0.6} specifies a(0)=0.0, a(1)=1.0, a(2)=0, a(3)=0.6.

Properties

ycoef

This parameter of type map<uint32, float64> specifies the b(k) coefficients in the form of a series of {lag:value}, separated by commas and enclosed in curly braces. The value at lag 0 should always be 1.0 (b(0)=1.0).Non-specified lags are assumed to be zero. For example: ycoef:{0u:1.0,2u:0.6} specifies b(0)=1.0, b(1)=0.0, b(2)=0.6.

Properties

coefParameterFile

This parameter of type rstring specifies the file that contains the parameters for the filters. The path is relative to the data folder or is an absolute path. The file is an ASCII file that contains the keywords (xcoef and ycoef) followed by a column. If xcoef and ycoef are not provided as parameter values, they must be present in the coefParameterFile or an exception is thrown.

The following example shows the format of the file contents:

xcoef: {1:2.0,4:0.6}
ycoef: {0:2.0,4:0.6}
Properties

inputTimeSeries

This mandatory parameter is an attribute expression, which specifies the name of the attribute that contains the time series data in the input tuple. The supported data types are float64 and list<float64>.

Properties

filterType

This optional parameter of type custom literal specifies the list of filters that you can apply on the input time series. When this parameter is specified, the values for xcoef and ycoef parameters are ignored. The DSPFilterFinite operator estimates the coefficients based on the type of filter that you choose and the input time series data. The supported values for the filterType parameter are HighPass and LowPass.

Properties

samplingRate

This is an optional parameter of type float64. This parameter is required if you specify the filterType parameter. The DSPFilterFinite operator uses the sampling rate to estimate the coefficients for applying the filtering operation. The sampling rate is specified in Hertz (Hz).

Properties

cutOffFrequency

This is an optional parameter of type float64. This parameter is required if you specify the filterType parameter. The cutoffFrequency parameter specifies the cutoff frequency that you can use when applying the filtering operation. For example, if the filterType parameter is set to LowPass and the cutOffFrequency parameter is set to 5.0 Hz, the DSPFilter operator filters all the signal components greater than 5.0 Hz.

Properties

filteredOnly

This is a parameter of type boolean. When the parameter value is true, the output contains only the time series values for which the coefficient was provided and filtering was performed. The input time series index for which the coefficient values were not provided is not returned. The default is parameter value is false, which returns all the time series dimension data points.

Properties

maxDimension

This is an optional parameter of type uint32, which triggers the time series expansion mode. The parameter specifies the maximum number of time series that can be included in the process in real time. When the parameter is not specified, the number of time series (as estimated from the first tuple) is assumed to be constant.

Properties

partitionBy

Specifies the name of the attribute that contains the key values that are associated with the time series values in the input tuple.

Properties

controlSignal

This optional parameter is an attribute expression that specifies the name of the attribute in the control port, which holds the control signal. The supported control signals are: TSSignal.Monitor, TSSignal.Load.

Properties

inputCoefficient

This optional parameter is an attribute expression that specifies the name of the attribute in the control port, which ingests the coefficients that are used for loading the model. If this parameter is not specified, by default, the inputCoefficient attribute is used. If the default attribute or the inputCoefficient parameter is not provided, the operator throws an exception. If the attribute or the parameter value does not contain valid coefficients, the load operation fails and the operator logs a warning message for each failed operation. The operator continues to predict values by using the older coefficients. The supported type is map<rstring,map<uint32,float64>>.

Properties

Code Templates

DSPFilter
stream<${schema}> ${outputStream} = DSPFilter(${inputStream}) 
{
	param
		inputTimeSeries:		${timeSeriesExpression};
		xcoef: ${xcoefSpecification};
		ycoef: ${ycoefSpecification};
	output
		${outputStream}: ${outputExpression};
}

Libraries

No description for library.
Command
Library Name: tsatapi
Library Path: ../../../impl/lib
Include Path: ../../../impl/include/