Elasticsearch
The Elasticsearch plugin can be used to retrieve logs from a configured Elasticsearch instance. You can specify a query, which is used to retireve all documents from Elasticsearch. You can also select a list of fields, to display the results in a table. From this table you can also select a single document to view the complete json representation of this document.
Configuration
To use the Elasticsearch plugin the following configuration is needed in the satellites configuration file:
Field | Type | Description | Required |
---|---|---|---|
name | string | The name of the Elasticsearch plugin instance. | Yes |
type | elasticsearch |
The type for the Elasticsearch plugin. | Yes |
options.address | string | Address of the Elasticsearch instance. | Yes |
options.username | string | Username to access an Elasticsearch instance via basic authentication. | No |
options.password | string | Password to access an Elasticsearch instance via basic authentication. | No |
options.token | string | Token to access an Elasticsearch instance via token based authentication. | No |
plugins:
- name: elasticsearch
type: elasticsearch
options:
address:
username:
password:
token:
Insight Options
Note
The Elasticsearch plugin can not be used within the insights section of an application.
Variable Options
Note
The Elasticsearch plugin can not be used to get a list of variable values.
Panel Options
The following options can be used for a panel with the Elasticsearch plugin:
Field | Type | Description | Required |
---|---|---|---|
showChart | boolean | If this is true the chart with the distribution of the Documents over the selected time range will be shown |
No |
queries | []Query | A list of Elasticsearch queries, which can be selected by the user. | Yes |
Query
Field | Type | Description | Required |
---|---|---|---|
name | string | A name for the Elasticsearch query, which is displayed in the select box. | Yes |
query | string | The Elasticsearch query. We are using the Query String Syntax for Elasticsearch. | Yes |
fields | []string | A list of fields to display in the results table. If this field is omitted, the whole document is displayed in the results table. | No |
Notification Options
Note
The Elasticsearch plugin can not be used to get a list of notifications.
Usage
Query String Syntax
The Elasticsearch plugins uses the query string syntax for filtering log lines.
Field Names
You can specify fields to search in the query syntax:
- where the
status
field containsactive
status:active
- where the
title
field containsquick
orbrown
title:(quick OR brown)
- where the
author
field contains the exact phrase"john smith"
author:"John Smith"
- where the
first name
field containsAlice
(note how we need to escape the space with a backslash)
first\ name:Alice
- where any of the fields
book.title
,book.content
orbook.date
containsquick
orbrown
(note how we need to escape the*
with a backslash):
book.\*:(quick OR brown)
- where the field
title
has any non-null value:
_exists_:title
Wildcards
Wildcard searches can be run on individual terms, using ?
to replace a single character, and *
to replace zero or more characters:
qu?ck bro*
Regular Expressions
Regular expression patterns can be embedded in the query string by wrapping them in forward-slashes ("/"
):
name:/joh?n(ath[oa]n)/
Fuzziness
We can search for terms that are similar to, but not exactly like our search terms, using the "fuzzy" operator:
quikc~ brwn~ foks~
Ranges
Ranges can be specified for date, numeric or string fields. Inclusive ranges are specified with square brackets [min TO max]
and exclusive ranges with curly brackets {min TO max}
.
- All days in 2012:
date:[2012-01-01 TO 2012-12-31]
- Numbers 1..5
count:[1 TO 5]
- Tags between
alpha
andomega
, excludingalpha
andomega
:
tag:{alpha TO omega}
- Numbers from 10 upwards
count:[10 TO *]
- Dates before 2012
date:{* TO 2012-01-01}
Boolean operators
By default, all terms are optional, as long as one term matches. A search for foo bar baz
will find any document that contains one or more of foo
or bar
or baz
. We have already discussed the default_operator
above which allows you to force all terms to be required, but there are also boolean operators which can be used in the query string itself
to provide more control.
The preferred operators are +
(this term must be present) and -
(this term must not be present). All other terms are optional. For example, this query:
quick brown +fox -news
Grouping
Multiple terms or clauses can be grouped together with parentheses, to form sub-queries:
(quick OR brown) AND fox
Example Dashboard
---
apiVersion: kobs.io/v1
kind: Dashboard
spec:
placeholders:
- name: namespace
description: The workload namespace
- name: app
description: The workloads app label
rows:
- size: -1
panels:
- title: Istio Logs
colSpan: 12
plugin:
name: elasticsearch
type: elasticsearch
options:
showChart: true
queries:
- name: Istio Logs
query: "kubernetes.namespace: {% .namespace %} AND kubernetes.labels.app: {% .app %} AND kubernetes.container.name: istio-proxy AND _exists_: content.method"
fields:
- "kubernetes.pod.name"
- "content.authority"
- "content.route_name"
- "content.protocol"
- "content.method"
- "content.path"
- "content.response_code"
- "content.upstream_service_time"
- "content.bytes_received"
- "content.bytes_sent"