Plugins
TsAggregate
@Plugin(type = SparkCompute.PLUGIN_TYPE)
@Name("TsAggregate")
@Description("A time series aggregation stage that aggregates a sparse time series leveraging a user-defined "
+ "tumbling window. Suppose a certain point in time refers to 09:00 am and a window of 10 minutes is "
+ "defined, then all points in time falling into the window [09:00, 09:10] are collected and their "
+ "associated values aggregated.")
public class TsAggregate extends TimeCompute {
...
}
Parameters
Time Field | The name of the field in the input schema that contains the time value. |
Value Field | The name of the field in the input schema that contains the value. |
Group Field | The name of the field in the input schema that specifies data groups. |
Aggregation Method | The name of the aggregation method. Supported values are 'avg', 'count', 'mean' and 'sum'. Default is 'avg'. |
Window Duration | The time window used to aggregate intermediate values. Default is '10 minutes'. |
TsACF
@Plugin(type = SparkCompute.PLUGIN_TYPE)
@Name("TsACF")
@Description("A time series computation stage that determines the ACF (Auto Correlation Function) "
+ "of a time series. This stage does not transform the input dataset, but computes and persists "
+ "its ACF for ease of use in subsequent stages.")
public class TsACF extends TimeCompute {
...
}
Parameters
Dataset Name | The unique name of the ACF dataset. |
Time Field | The name of the field in the input schema that contains the time value. |
Value Field | The name of the field in the input schema that contains the value. |
Algorithm Configuration | |
Maximum Lag | The maximum lag value. Use this parameter if the ACF is based on a range of lags. Default is 1. |
Discrete Lags | The comma-separated sequence of lag value. Use this parameter if the ACF should be based on discrete values. This sequence is empty by default. |
Correlation Threshold | The threshold used to determine the lag value with the highest correlation score. Default is 0.95. |
TsInterpolate
@Plugin(type = SparkCompute.PLUGIN_TYPE)
@Name("TsInterpolate")
@Description("A time series interpolation stage that adds missing values. This stage "
+ "interpolates missing values from the last non-null value before and the first "
+ "on-null value after the respective null value.")
public class TsInterpolate extends TimeCompute {
...
}
Parameters
Time Field | The name of the field in the input schema that contains the time value. |
Value Field | The name of the field in the input schema that contains the value. |
Group Field | The name of the field in the input schema that specifies data groups. |
TsResample
@Plugin(type = SparkCompute.PLUGIN_TYPE)
@Name("TsResample")
@Description("A time series resampling stage that turns a sparse time series into "
+ "an equidistant time grid. This stage supports data values that belong to "
+ "different categories or groups, say sensor devices. Then resampling is performed "
+ "on a per group basis. Note, resampling may lead to missing values for intermediate "
+ "points in time.")
public class TsResample extends TimeCompute {
...
}
Parameters
Time Field | The name of the field in the input schema that contains the time value. |
Value Field | The name of the field in the input schema that contains the value. |
Group Field | The name of the field in the input schema that specifies data groups. |
Time Interval | Distance between subsequent points of the time series after resampling. The value specifies a certain time interval in seconds. Default is 30. |