Skip to content

klogs

The klogs plugin can be used together with the kobsio/klogs output plugin for Fluent Bit. You can then use the specified Query Syntax to get the logs from ClickHouse.

Logs

Configuration

The klogs plugin can be used within the hub or cluster. To use the klogs plugin the following configuration is needed:

Field Type Description Required
name string The name of the klogs plugin instance. Yes
type klogs The type for the klogs plugin. Yes
options.address string Address of the ClickHouse instance. Yes
options.database string The name of the database. Yes
options.username string Username to access a ClickHouse instance. No
options.password string Password to access a ClickHouse instance. No
options.dialTimeout string ClickHouse dial timeout. The default value is 10s. No
options.connMaxLifetime string ClickHouse maximum connection lifetime. The default value is 1h. No
options.maxIdleConns number ClickHouse maximum number of idle connections. The default value is 5. No
options.maxOpenConns number ClickHouse maximum number of open connections. The default value is 10. No
options.settings map Additional settings which should be applyed to the ClickHouse connection. No
options.materializedColumns []string A list of materialized columns. See kobsio/klogs for more information. No
plugins:
  - name: klogs
    type: klogs
    options:
      address:
      database:
      username:
      password:
      dialTimeout:
      connMaxLifetime:
      maxIdleConns:
      maxOpenConns:
      settings:
        # e.g.
        # receive_timeout: 600
      materializedColumns:

Insight Options

Note

The klogs plugin can not be used within the insights section of an application.

Variable Options

Note

The klogs plugin can not be used to get a list of variable values.

Panel Options

The following options can be used for a panel with the klogs plugin:

Field Type Description Required
type string Set the type which should be used to visualize your logs. This can be logs or aggregation. Yes
showChart boolean If this is true the chart with the distribution of the log lines in the selected time range will be shown. No
queries []Query A list of queries, which can be selected by the user. This is only required for type logs. Yes
aggregation Aggregation Options for the aggregation. This is only required for type aggregation. Yes

Query

Field Type Description Required
name string A name for the klogs query, which is displayed in the select box. Yes
query string The query which should be run against ClickHouse. See Query Syntax for more information on the syntax. Yes
fields []string A list of fields to display in the results table. If this field is omitted, the whole document is displayed in the results table. No
order string Order for the returned logs. Must be ascending or descending. The default value for this field is descending. No
orderBy string The name of the field, by which the results should be orderd. The default value for this field is timestamp. No

Aggregation

Field Type Description Required
query string The query, which should be used for the aggregation. Yes
chart string The visualization type for the aggregation. This can be pie, bar, line or area. Yes
sliceBy string Field which should be used for slicing in a pie chart. No
sizeByOperation string Operation to size the slices. This can be count, min, max, sum or avg. No
sizeByField string When the sizeByOperation is min, max, sum or avg, this must be the name of a field for the sizing of the slices. No
horizontalAxisOperation string The operation for the chart. This must be time or top. No
horizontalAxisField string The name of the field for the horizontal axis. No
horizontalAxisOrder string The order of the top values. Must be ascending or descending. No
horizontalAxisLimit number The maximum number of top values, which should be shown. No
verticalAxisOperation string The operation for the vertical axis. This can be count, min, max, sum or avg. No
verticalAxisField string When the verticalAxisOperation is min, max, sum or avg, this must be the name of a field for the vertical axis. No
breakDownByFields []string A list of field names, which should be used to break down the data. No
breakDownByFilters []string A list of filters, which should be used to break down the data. No

Usage

Query Syntax

Operators

kobs supports multiple operators which can be used in a query to retrieve logs from ClickHouse:

Operator Description Example
( and ) Multiple terms or clauses can be grouped together with parentheses, to form sub-queries. cluster='kobs-demo' _and_ (namespace='bookinfo' _or_ namespace='istio-system')
_not_ Exclude the term from the query. cluster='kobs-demo' _and_ _not_ namespace='bookinfo'
_and_ Both terms must be included in the results. namespace='bookinfo' _and_ app='bookinfo'
_or_ The result can contain one of the given terms. namespace='bookinfo' _or_ namespace='istio-system'
_exists_ The field can not be null container_name='istio-proxy' _and_ _exists_ content_request_id
= The field must have this value. namespace='bookinfo'
!= The field should not have this value. namespace!='bookinfo'
> The value of the field must be greater than the specified value. content_response_code>499
>= The value of the field must be greater than or equal to the specified value. content_response_code>=500
< The value of the field must be lower than the specified value. content_response_code<500
<= The value of the field must be lower than or equal to the specified value. content_response_code<=499
=~ The value of the field is compared using ILIKE. content_upstream_cluster=~'inbound%'
!~ The value of the field is compared using NOT ILIKE. content_upstream_cluster!~'inbound%'
~ The value of the field must match the regular expression. The syntax of the re2 regular expressions can be found here. content_upstream_cluster~'inbound.*'

Default Fields

In the following you can find a list of fields which are available for each log line. Consider to filter you logs by these fields, to keep your queries fast:

  • timestamp: The timestamp for when the log line was written.
  • cluster: The name of the cluster as it is set by Fluent Bit.
  • namespace: The namespace of the Pod.
  • app: The value of the app or k8s-app label of the Pod.
  • pod_name: The name of the Pod.
  • container_name: The name of the container from the Pod.
  • host: The name of the host where the Pod is running on.
  • log: The complete log line as it was written by the container.

Examples

Logs
---
apiVersion: kobs.io/v1
kind: Application
metadata:
  name: default
  namespace: default
spec:
  description: The default application is an application to test all available kobs plugins.
  dashboards:
    - title: Logs
      inline:
        rows:
          - autoHeight: true
            panels:
              - title: Istio Logs
                plugin:
                  name: klogs
                  type: klogs
                  options:
                    type: logs
                    showChart: true
                    queries:
                      - name: Istio Logs
                        query: >-
                          namespace='kobs' _and_ app='hub' _and_
                          container_name='istio-proxy' _and_
                          content_upstream_cluster~'inbound.*'
                        fields:
                          - pod_name
                          - content_authority
                          - content_route_name
                          - content_protocol
                          - content_method
                          - content_path
                          - content_response_code
                          - content_upstream_service_time
                          - content_bytes_received
                          - content_bytes_sent
                h: 6
                w: 12
                x: 0
                'y': 0

Logs

Aggregations
---
apiVersion: kobs.io/v1
kind: Application
metadata:
  name: default
  namespace: default
spec:
  description: The default application is an application to test all available kobs plugins.
  dashboards:
    - title: Logs
      inline:
        rows:
          - panels:
              - title: Number of Logs per App
                plugin:
                  name: klogs
                  type: klogs
                  options:
                    type: aggregation
                    aggregation:
                      query: cluster='dev-de1'
                      chart: bar
                      horizontalAxisOperation: top
                      horizontalAxisField: app
                      horizontalAxisOrder: descending
                      horizontalAxisLimit: 10
                      verticalAxisOperation: count
                h: 7
                w: 6
                x: 0
                'y': 0
              - title: Log Levels for MyApplication
                plugin:
                  name: klogs
                  type: klogs
                  options:
                    type: aggregation
                    aggregation:
                      query: >-
                        cluster='dev-de1' _and_ app='myapplication' _and_
                        container_name='myapplication'
                      chart: pie
                      sliceBy: content_level
                      sizeByOperation: count
                h: 7
                w: 6
                x: 6
                'y': 0
              - title: Request Duration for MyApplication by Response Code
                plugin:
                  name: klogs
                  type: klogs
                  options:
                    type: aggregation
                    aggregation:
                      query: >-
                        cluster='dev-de1' _and_ app='myapplication' _and_
                        container_name='istio-proxy' _and_ content_response_code>0
                      chart: line
                      horizontalAxisOperation: time
                      verticalAxisOperation: avg
                      verticalAxisField: content_duration
                      breakDownByFields:
                        - content_response_code
                h: 7
                w: 12
                x: 0
                'y': 7

Aggregations