Data Processing Language (DPL)

Teragrep’s Data Processing Language (DPL) is selected by entering %dpl on the paragraph’s code editor.

By default, DPL uses US time format (MM/DD/YYYY:HH:mm:SS) for compatibility reasons.

Archive Access

DPL automatically connects to the Teragrep Archive without any extra configuration.

Data Search

With DPL you can make data searches by using different kind of criteria. The simplest dpl command contains only the dataset’s name.

%dpl
index="f17"

To narrow down the results, you can add date range by using either earliest=, latest= or both.

%dpl
index="f17" earliest="01/08/2020:00:00:00" latest="01/08/2020:04:00:00"

You can also add keywords to narrow down the results. The following example uses keyword 997913837433 which will fetch all records that contain it.

%dpl
index="f17" 997913837433

You can use multiple criteria at the same time. The following example searches all records that:

  • are from the dataset f17, specified with index=

  • with a specific stream and host, specified with sourcetype= and host=

  • within a predefined date range, between 01/08/2020:02:00:00 and 01/08/2020:02:00:00, specified with earliest= and latest=

  • match to keywords 200000762939453 and 997913837433

%dpl
index="f17" sourcetype=log:f17:0 AND host=sc-99-99-14-162 earliest="01/08/2020:00:00:00" latest="01/08/2020:04:00:00" 200000762939453 997913837433

Data search example

To see more available DPL commands, read transforms, aggregations and evaluations.

Pivot Tables

You can create pivot tables with dataset’s name, chart and an aggregation. chart and the aggregation needs to be separated from the dataset’s name with |. The following example will use count() aggregation.

%dpl
index=f17 | chart count(_raw)

DPL pivot table

As in picture, the example will create a single column table. You can aggregate more than one column, just use comma between aggregations.

%dpl
index=f17 | chart count(_raw), earliest(_time)

DPL pivot table

Use by to group your aggregated values.

%dpl
index=f17 | chart count(_raw), earliest(_time) by host

DPL pivot table

You can use data search criteria with aggregations. The following example aggregates all data:

  • from the dataset f17 with specific stream log:f17:0, on date range 12/31/1970:10:15:30 - 12/31/2022:10:15:30

  • which creates a pivot table from aggregated column _raw

  • and is grouped by host.

%dpl
index=f17 sourcetype=log:f17:0 earliest="12/31/1970:10:15:30" latest="12/31/2022:10:15:30" | chart count(_raw) by host

DPL pivot table

See the comprehensive list of DPL aggregations here.

Data output

DPL produces the data into a Spark Query named after the paragraph’s unique identifier. (You can see each paragraph’s identifier from the Paragraph Settings menu.) This output can then be processed with other interpreters that belong to the same interpreter group.