Data Processing Language (DPL)
Teragrep’s Data Processing Language (DPL) is selected by entering %dpl
on the paragraph’s code editor.
By default, DPL uses US time format (MM/DD/YYYY:HH:mm:SS) for compatibility reasons.
Data Search
With DPL you can make data searches by using different kind of criteria. The simplest dpl command contains only the dataset’s name.
%dpl
index="f17"
To narrow down the results, you can add date range by using either earliest=
, latest=
or both.
%dpl
index="f17" earliest="01/08/2020:00:00:00" latest="01/08/2020:04:00:00"
You can also add keywords to narrow down the results. The following example uses keyword 997913837433
which will fetch all records that contain it.
%dpl
index="f17" 997913837433
You can use multiple criteria at the same time. The following example searches all records that:
-
are from the dataset
f17
, specified withindex=
-
with a specific stream and host, specified with
sourcetype=
andhost=
-
within a predefined date range, between
01/08/2020:02:00:00
and01/08/2020:02:00:00
, specified withearliest=
andlatest=
-
match to keywords
200000762939453
and997913837433
%dpl
index="f17" sourcetype=log:f17:0 AND host=sc-99-99-14-162 earliest="01/08/2020:00:00:00" latest="01/08/2020:04:00:00" 200000762939453 997913837433
To see more available DPL commands, read transforms, aggregations and evaluations.
Pivot Tables
You can create pivot tables with dataset’s name, chart
and an aggregation. chart
and the aggregation needs to be separated from the dataset’s name with |
. The following example will use count()
aggregation.
%dpl
index=f17 | chart count(_raw)
As in picture, the example will create a single column table. You can aggregate more than one column, just use comma between aggregations.
%dpl
index=f17 | chart count(_raw), earliest(_time)
Use by
to group your aggregated values.
%dpl
index=f17 | chart count(_raw), earliest(_time) by host
You can use data search criteria with aggregations. The following example aggregates all data:
-
from the dataset
f17
with specific streamlog:f17:0
, on date range12/31/1970:10:15:30
-12/31/2022:10:15:30
-
which creates a pivot table from aggregated column
_raw
-
and is grouped by
host
.
%dpl
index=f17 sourcetype=log:f17:0 earliest="12/31/1970:10:15:30" latest="12/31/2022:10:15:30" | chart count(_raw) by host
See the comprehensive list of DPL aggregations here.
Data output
DPL produces the data into a Spark Query named after the paragraph’s unique identifier. (You can see each paragraph’s identifier from the Paragraph Settings menu.) This output can then be processed with other interpreters that belong to the same interpreter group.