#Elasticsearch

#Introduction

Elasticsearch version is automatically retrieved from root/ping endpoint. Based on this version Jaeger uses compatible index mappings and Elasticsearch REST API. The version can be explicitly provided via version: config property.

Elasticsearch does not require initialization other than installing and running Elasticsearch. Once it is running, pass the correct configuration values to Jaeger.

#Configuration

A sample configuration for Jaeger with Elasticsearch backend is available in the Jaeger repository: config-elasticsearch.yaml. In the future the configuration documentation will be auto-generated from the schema. Meanwhile, please refer to config.go as the authoritative source.

#Shards and Replicas

Shards and replicas are some configuration values to take special attention to, because this is decided upon index creation. This article goes into more information about choosing how many shards should be chosen for optimization.

#Query Time Range

Two options control the time window that the query service scans when reading spans:

For example, with time-based indices retained for 7 days and traces no longer than one hour:

max_span_age: 168h
max_trace_duration: 1h

#Index Management Strategies

Jaeger supports three index management strategies, selected with the rotation configuration under indices.<type> (for each of spans, services, dependencies, and sampling). At most one strategy can be set per index type; when no rotation is configured, time-based indices are used.

The relevant configuration fields under indices.<type>.rotation are:

#Index Rollover

Elasticsearch rollover is an index management strategy that optimizes use of resources allocated to indices. For example, indices that do not contain any data still allocate shards, and conversely, a single index might contain significantly more data than the others. Rollover feature can be enabled by configuring an alias-based rotation strategy (rotation.auto_rollover, recommended, or rotation.manual_rollover) under indices.<type> for each index type.

Rollover lets you configure when to roll over to a new index based on one or more of the following criteria:

• max_age - the maximum age of the index. It uses time units: d, h, m.
• max_docs - the maximum documents in the index.
• max_primary_shard_size - the maximum size of a single primary shard. It uses byte size units tb, gb, mb.

To learn more about rollover index management in Jaeger refer to this article.

#Initialize

The jaeger-es-rollover image address is published by the Alauda Build of Jaeger v2 cluster plugin in the jaeger-cluster-plugin-manifest ConfigMap, so that it always matches the deployed Jaeger version:

export JAEGER_ES_ROLLOVER_IMAGE=$(kubectl -ncpaas-system get configmap jaeger-cluster-plugin-manifest -o jsonpath='{.data.jaeger-es-rollover-image}')

The following command prepares Elasticsearch for rollover deployment:

nerdctl run -it --rm --net=host \
  ${JAEGER_ES_ROLLOVER_IMAGE} \
  init http://localhost:9200

If you need to initialize archive storage, add -e ARCHIVE=true.

The initializer performs the following steps for each index type (spans, services, dependencies):

1. Creates index templates that define field mappings, shard/replica settings, and index patterns (e.g., jaeger-span-*). All future rollover indices inherit their schema from these templates.
2. Creates the first rollover index (e.g., jaeger-span-000001). Subsequent rollovers increment this number.
3. Creates read and write aliases (e.g., jaeger-span-read and jaeger-span-write) pointing to the initial index. Jaeger queries via the read alias and writes via the write alias.

After the initialization, Jaeger can be deployed with the manual_rollover rotation strategy (indices.<type>.rotation.manual_rollover: {}), whose default alias names match the aliases created by the initialization. With this strategy, index rollover and retention must be driven by external jaeger-es-rollover rollover and lookback cron jobs, so use it only when ILM is not available (for example, when the Jaeger Elasticsearch account lacks ILM privileges). Otherwise, prefer the ILM-based auto_rollover strategy described below.

#ILM support

Elasticsearch ILM automatically manages indices according to performance, resiliency, and retention requirements.

ILM support is an alternative to the manual rollover + lookback + index-cleaner workflow. When ILM is enabled, Elasticsearch manages rollover and retention automatically according to the configured policy.

For example:

• Rollover to a new index by size (bytes or number of documents) or age, archiving previous indices
• Delete stale indices to enforce data retention standards

To enable ILM support:

1. Create an ILM policy in Elasticsearch named jaeger-ilm-policy.

   For example, the following policy will rollover the "active" index when it is older than 1m and delete indices that are older than 2m.

   curl -X PUT \
     http://localhost:9200/_ilm/policy/jaeger-ilm-policy \
     -H 'Content-Type: application/json; charset=utf-8' \
     --data-binary @- << EOF
   {
     "policy": {
       "phases": {
         "hot": {
           "min_age": "0ms",
           "actions": {
             "rollover": {
               "max_age": "1m"
             },
             "set_priority": {
               "priority": 100
             }
           }
         },
         "delete": {
           "min_age": "2m",
           "actions": {
             "delete": {}
           }
         }
       }
     }
   }
   EOF

2. Run the Elasticsearch initializer with ES_USE_ILM=true:

   nerdctl run -it --rm --net=host \
     -e ES_USE_ILM=true \
     ${JAEGER_ES_ROLLOVER_IMAGE} \
     init http://localhost:9200

   If you need to initialize archive storage, add -e ARCHIVE=true.

   WARNING

   While initializing with ILM support, make sure that an ILM policy named jaeger-ilm-policy is created in Elasticsearch beforehand (see the previous step), otherwise the following error message will be shown:

   "ILM policy jaeger-ilm-policy doesn't exist in Elasticsearch. Please create it and rerun init"

   The initializer performs the same steps as described in Initialize (creates index templates, seed indices, and aliases), with the following ILM-specific additions:

   • Validates that the ILM policy (jaeger-ilm-policy) exists in Elasticsearch.
   • Embeds index.lifecycle.name and index.lifecycle.rollover_alias in the index templates, so Elasticsearch automatically applies the ILM policy to every new rollover index.
   • Sets is_write_index: true on the write aliases, which is required for Elasticsearch to perform ILM-triggered rollovers.

After the initialization, deploy Jaeger with the auto_rollover rotation strategy on each index type. Rollover and retention are then managed automatically by the ILM policy, and no external cron jobs (rollover, lookback, or index-cleaner) are required:

indices:
  spans:
    rotation:
      auto_rollover: {}
  services:
    rotation:
      auto_rollover: {}
  dependencies:
    rotation:
      auto_rollover: {}
  sampling:
    rotation:
      auto_rollover: {}

When create_mappings: false is set (index templates are created by the es-rollover init job), leave rotation.auto_rollover.policy_name empty; the ILM policy reference is already embedded in the index templates by the initialization.