Using OpenTelemetry
This guide explains how your Quarkus application can utilize OpenTelemetry (OTel) to provide distributed tracing for interactive web applications.
|
准备
要完成本指南,您需要:
-
Roughly 15 minutes
-
An IDE
-
JDK 11+ installed with
JAVA_HOME
configured appropriately -
Apache Maven 3.9.6
-
Docker and Docker Compose or Podman, and Docker Compose
-
Optionally the Quarkus CLI if you want to use it
-
Optionally Mandrel or GraalVM installed and configured appropriately if you want to build a native executable (or Docker if you use a native container build)
完整源码
We recommend that you follow the instructions in the next sections and create the application step by step. However, you can skip right to the completed example.
Clone the Git repository: git clone https://github.com/quarkusio/quarkus-quickstarts.git
, or download
an archive.
The solution is located in the opentelemetry-quickstart
directory.
Creating the Maven project
First, we need a new project. Create a new project with the following command:
For Windows users:
-
If using cmd, (don’t use backward slash
\
and put everything on the same line) -
If using Powershell, wrap
-D
parameters in double quotes e.g."-DprojectArtifactId=opentelemetry-quickstart"
This command generates the Maven project and imports the
quarkus-opentelemetry
extension, which includes the default OpenTelemetry
support, and a gRPC span exporter for
OTLP.
If you already have your Quarkus project configured, you can add the
quarkus-opentelemetry
extension to your project by running the following
command in your project base directory:
quarkus extension add opentelemetry
./mvnw quarkus:add-extension -Dextensions='opentelemetry'
./gradlew addExtension --extensions='opentelemetry'
This will add the following to your build file:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-opentelemetry</artifactId>
</dependency>
implementation("io.quarkus:quarkus-opentelemetry")
Examine the Jakarta REST resource
Create a src/main/java/org/acme/opentelemetry/TracedResource.java
file
with the following content:
package org.acme.opentelemetry;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
import org.jboss.logging.Logger;
@Path("/hello")
public class TracedResource {
private static final Logger LOG = Logger.getLogger(TracedResource.class);
@GET
@Produces(MediaType.TEXT_PLAIN)
public String hello() {
LOG.info("hello");
return "hello";
}
}
Notice that there is no tracing specific code included in the application. By default, requests sent to this endpoint will be traced without any required code changes.
Create the configuration
There are no mandatory configurations for the extension to work.
If you need to change any of the default property values, here is an example
on how to configure the default OTLP gRPC Exporter within the application,
using the src/main/resources/application.properties
file:
quarkus.application.name=myservice (1)
quarkus.otel.exporter.otlp.traces.endpoint=http://localhost:4317 (2)
quarkus.otel.exporter.otlp.traces.headers=authorization=Bearer my_secret (3)
quarkus.log.console.format=%d{HH:mm:ss} %-5p traceId=%X{traceId}, parentId=%X{parentId}, spanId=%X{spanId}, sampled=%X{sampled} [%c{2.}] (%t) %s%e%n (4)
# Alternative to the console log
quarkus.http.access-log.pattern="...traceId=%{X,traceId} spanId=%{X,spanId}" (5)
1 | All spans created from the application will include an OpenTelemetry
Resource indicating the span was created by the myservice
application. If not set, it will default to the artifact id. |
2 | gRPC endpoint to send spans. If not set, it will default to
http://localhost:4317 . |
3 | Optional gRPC headers commonly used for authentication |
4 | Add tracing information into log messages. |
5 | You can also only put the trace info into the access log. In this case you must omit the info in the console log format. |
All configurations have been updated from The legacy configurations are now deprecated but will still work during a transition period. |
Disable all or parts of the OpenTelemetry extension
Once you add the dependency, the extension will be enabled by default but there are a few ways to disable the OpenTelemetry extension globally or partially.
Property name |
Default value |
Description |
|
true |
If false, disable the OpenTelemetry usage at build time. |
|
false |
Comes from the OpenTelemetry autoconfiguration. If true, will disable the OpenTelemetry SDK usage at runtime. |
|
true |
If false, disable the OpenTelemetry tracing usage at build time. |
|
true |
If false will disable the default OTLP exporter at build time. |
If you need to enable or disable the exporter at runtime, you can use the Sampler because it has the ability to filter out all the spans if needed.
Run the application
The first step is to configure and start the OpenTelemetry Collector to receive, process and export telemetry data to Jaeger that will display the captured traces.
Jaeger-all-in-one includes the Jaeger agent, an OTel collector, and the query service/UI. You do not need to install a separated collector. You can directly send the trace data to Jaeger (after enabling OTLP receivers there, see e.g. this blog entry for details). |
Start the OpenTelemetry Collector and Jaeger system via the following
docker-compose.yml
file that you can launch via docker-compose up -d
:
version: "2"
services:
# Jaeger
jaeger-all-in-one:
image: jaegertracing/all-in-one:latest
ports:
- "16686:16686" # Jaeger UI
- "14268:14268" # Receive legacy OpenTracing traces, optional
- "4317:4317" # OTLP gRPC receiver
- "4318:4318" # OTLP HTTP receiver, not yet used by Quarkus, optional
- "14250:14250" # Receive from external otel-collector, optional
environment:
- COLLECTOR_OTLP_ENABLED=true
You should remove the optional ports you don’t need them.
Now we are ready to run our application. If using application.properties
to configure the tracer:
quarkus dev
./mvnw quarkus:dev
./gradlew --console=plain quarkusDev
or if configuring the OTLP gRPC endpoint via JVM arguments:
quarkus dev -Djvm.args="-Dquarkus.otel.exporter.otlp.traces.endpoint=http://localhost:4317"
./mvnw quarkus:dev -Djvm.args="-Dquarkus.otel.exporter.otlp.traces.endpoint=http://localhost:4317"
./gradlew --console=plain quarkusDev -Djvm.args="-Dquarkus.otel.exporter.otlp.traces.endpoint=http://localhost:4317"
With the OpenTelemetry Collector, the Jaeger system and the application running, you can make a request to the provided endpoint:
$ curl http://localhost:8080/hello
hello
When the first request has been submitted, you will be able to see the tracing information in the logs:
10:49:02 INFO traceId=, parentId=, spanId=, sampled= [io.quarkus] (main) Installed features: [cdi, opentelemetry, rest-client, resteasy, smallrye-context-propagation, vertx]
10:49:03 INFO traceId=17ceb8429b9f25b0b879fa1503259456, parentId=3125c8bee75b7ad6, spanId=58ce77c86dd23457, sampled=true [or.ac.op.TracedResource] (executor-thread-1) hello
10:49:03 INFO traceId=ad23acd6d9a4ed3d1de07866a52fa2df, parentId=, spanId=df13f5b45cf4d1e2, sampled=true [or.ac.op.TracedResource] (executor-thread-0) hello
Then visit the Jaeger UI to see the tracing information.
Hit CTRL+C
or type q
to stop the application.
JDBC
The JDBC instrumentation will add a span for each JDBC queries done by your application, to enable it, add the following dependency to your build file:
<dependency>
<groupId>io.opentelemetry.instrumentation</groupId>
<artifactId>opentelemetry-jdbc</artifactId>
</dependency>
implementation("io.opentelemetry.instrumentation:opentelemetry-jdbc")
As it uses a dedicated JDBC datasource wrapper, you must enable telemetry for your datasource:
# enable tracing
quarkus.datasource.jdbc.telemetry=true
# configure datasource
quarkus.datasource.db-kind=postgresql
quarkus.datasource.jdbc.url=jdbc:postgresql://localhost:5432/mydatabase
Additional configuration
Some use cases will require custom configuration of OpenTelemetry. These sections will outline what is necessary to properly configure it.
ID Generator
The OpenTelemetry extension will use by default a random ID Generator when creating the trace and span identifier.
Some vendor-specific protocols need a custom ID Generator, you can override
the default one by creating a producer. The OpenTelemetry extension will
detect the IdGenerator
CDI bean and will use it when configuring the
tracer producer.
@Singleton
public class CustomConfiguration {
/** Creates a custom IdGenerator for OpenTelemetry */
@Produces
@Singleton
public IdGenerator idGenerator() {
return AwsXrayIdGenerator.getInstance();
}
}
Propagators
OpenTelemetry propagates cross-cutting concerns through
propagators
that will share an underlying Context
for storing state and accessing data
across the lifespan of a distributed transaction.
By default, the OpenTelemetry extension enables the
W3C Trace Context and the
W3C Baggage propagators, you can however
choose any of the supported OpenTelemetry propagators by setting the
propagators
config that is described in the
OpenTelemetry Configuration Reference.
Additional Propagators
-
The
b3
,b3multi
,jaeger
andottrace
propagators will need the trace-propagators extension to be added as a dependency to your project.
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-extension-trace-propagators</artifactId>
</dependency>
implementation("io.opentelemetry:opentelemetry-extension-trace-propagators")
-
The
xray
propagator will need the aws extension to be added as a dependency to your project.
<dependency>
<groupId>io.opentelemetry.contrib</groupId>
<artifactId>opentelemetry-aws-xray-propagator</artifactId>
</dependency>
implementation("io.opentelemetry.contrib:opentelemetry-aws-xray-propagator")
Customise Propagator
To customise the propagation header you can implement the
TextMapPropagatorCustomizer
interface. This can be used, as an example, to
restrict propagation of OpenTelemetry trace headers and prevent potentially
sensitive data to be sent to third party systems.
* /**
* Meant to be implemented by a CDI bean that provides arbitrary customization
for the TextMapPropagator
* that are to be registered with OpenTelemetry
*/
public interface TextMapPropagatorCustomizer {
TextMapPropagator customize(Context context);
interface Context {
TextMapPropagator propagator();
ConfigProperties otelConfigProperties();
}
}
Resource
A resource is a representation of the entity that is producing telemetry, it adds attributes to the exported trace to characterize who is producing the trace.
You can add attributes by setting the resource-attributes
tracer config
that is described in the OpenTelemetry
Configuration Reference. Since this property can be overridden at runtime,
the OpenTelemetry extension will pick up its value following the order of
precedence that is described in the
Quarkus Configuration
Reference.
If by any means you need to use a custom resource or one that is provided by
one of the
OpenTelemetry
SDK Extensions you can create multiple resource producers. The
OpenTelemetry extension will detect the Resource
CDI beans and will merge
them when configuring the tracer producer.
@ApplicationScoped
public class CustomConfiguration {
@Produces
@ApplicationScoped
public Resource osResource() {
return OsResource.get();
}
@Produces
@ApplicationScoped
public Resource ecsResource() {
return EcsResource.get();
}
}
Sampler
A sampler decides whether a trace should be discarded or forwarded, effectively managing noise and reducing overhead by limiting the number of collected traces sent to the collector.
Quarkus comes equipped with a built-in sampler, and you also have the option to create your custom sampler.
To use the built-in sampler, you can configure it by setting the desired sampler parameters as detailed in the OpenTelemetry Configuration Reference. As an example, you can configure the sampler to retain 50% of the traces:
# build time property only:
quarkus.otel.traces.sampler=traceidratio
# Runtime property:
quarkus.otel.traces.sampler.arg=0.5
An interesting use case for the sampler is to activate and deactivate tracing export at runtime, acording to this example:
|
Quarkus 3.0 introduced breaking changes on the configuration. Sampler related property names and values change to comply with the latest
Java OpenTelemetry SDK. During a transition period it will be possible to
set the new configuration values in the old property because we are mapping
If the sampler is parent based, there is no need to set, the now dropped
property, The values you need to set on
|
If you need to use a custom sampler there are now 2 different ways:
Sampler CDI Producer
You can create a sampler CDI producer. The Quarkus OpenTelemetry extension
will detect the Sampler
CDI bean and will use it when configuring the
Tracer.
@Singleton
public class CustomConfiguration {
/** Creates a custom sampler for OpenTelemetry */
@Produces
@Singleton
public Sampler sampler() {
return JaegerRemoteSampler.builder()
.setServiceName("my-service")
.build();
}
}
OTel Sampler SPI
This will use the SPI hooks available with the OTel Autoconfiguration. You can create a simple Sampler class:
public class CustomSPISampler implements Sampler {
@Override
public SamplingResult shouldSample(Context context,
String s,
String s1,
SpanKind spanKind,
Attributes attributes,
List<LinkData> list) {
// Do some sampling here
return Sampler.alwaysOn().shouldSample(context, s, s1, spanKind, attributes, list);
}
@Override
public String getDescription() {
return "custom-spi-sampler-description";
}
}
Then a Sampler Provider:
public class CustomSPISamplerProvider implements ConfigurableSamplerProvider {
@Override
public Sampler createSampler(ConfigProperties configProperties) {
return new CustomSPISampler();
}
@Override
public String getName() {
return "custom-spi-sampler";
}
}
Write the SPI loader text file at resources/META-INF/services
with name
io.opentelemetry.sdk.autoconfigure.spi.traces.ConfigurableSamplerProvider
containing the full qualified name of the CustomSPISamplerProvider
class.
Then activate on the configuration:
quarkus.otel.traces.sampler=custom-spi-sampler
As you can see, CDI is much simpler to work with.
Additional instrumentation
Some Quarkus extensions will require additional code to ensure traces are propagated to subsequent execution. These sections will outline what is necessary to propagate traces across process boundaries.
The instrumentation documented in this section has been tested with Quarkus and works in both standard and native mode.
CDI
Annotating a method in any CDI aware bean with the
io.opentelemetry.instrumentation.annotations.WithSpan
annotation will
create a new Span and establish any required relationships with the current
Trace context.
Annotating a method in any CDI aware bean with the
io.opentelemetry.instrumentation.annotations.AddingSpanAttributes
will not
create a new span but will add annotated method parameters to attributes in
the current span.
If a method is annotated by mistake with @AddingSpanAttributes
and
@WithSpan
annotations, the @WithSpan
annotation will take precedence.
Method parameters can be annotated with the
io.opentelemetry.instrumentation.annotations.SpanAttribute
annotation to
indicate which method parameters should be part of the span. The parameter
name can be customized as well.
Example:
@ApplicationScoped
class SpanBean {
@WithSpan
void span() {
}
@WithSpan("name")
void spanName() {
}
@WithSpan(kind = SERVER)
void spanKind() {
}
@WithSpan
void spanArgs(@SpanAttribute(value = "arg") String arg) {
}
@AddingSpanAttributes
void addArgumentToExistingSpan(@SpanAttribute(value = "arg") String arg) {
}
}
Available OpenTelemetry CDI injections
As per MicroProfile Telemetry Tracing specification, Quarkus supports the CDI injections of the following classes:
-
io.opentelemetry.api.OpenTelemetry
-
io.opentelemetry.api.trace.Tracer
-
io.opentelemetry.api.trace.Span
-
io.opentelemetry.api.baggage.Baggage
You can inject these classes in any CDI enabled bean. For instance, the
Tracer
is particularly useful to start custom spans:
@Inject
Tracer tracer;
...
public void tracedWork() {
Span span = tracer.spanBuilder("My custom span")
.setAttribute("attr", "attr.value")
.setParent(Context.current().with(Span.current()))
.setSpanKind(SpanKind.INTERNAL)
.startSpan();
// traced work
span.end();
}
SmallRye Reactive Messaging - Kafka
When using the SmallRye Reactive Messaging extension for Kafka, we are able to propagate the span into the Kafka Record with:
TracingMetadata tm = TracingMetadata.withPrevious(Context.current());
Message out = Message.of(...).withMetadata(tm);
The above creates a TracingMetadata
object we can add to the Message
being produced, which retrieves the OpenTelemetry Context
to extract the
current span for propagation.
Exporters
Default
The Quarkus OpenTelemetry extension uses its own exporter built on top of Vert.x for optimal performance and maintainability.
The exporter is automatically wired with CDI, that’s why the
quarkus.otel.traces.exporter
property defaults to cdi
.
The quarkus.otel.exporter.otlp.traces.protocol
default to grpc
and
http/protobuf
can also be used.
On Quarkiverse
Additional exporters will be available in the Quarkiverse quarkus-opentelemetry-exporter project.
Quarkus core extensions instrumented with OpenTelemetry tracing
-
quarkus-smallrye-reactive-messaging
-
AMQP 1.0
-
RabbitMQ
-
Kafka
-
Pulsar
-
-
quarkus-vertx
(http requests)
OpenTelemetry Configuration Reference
Quarkus supports the OpenTelemetry Autoconfiguration for Traces. The
configurations match what you can see at
OpenTelemetry
SDK Autoconfigure adding the usual quarkus.*
prefix.
Quarkus OpenTelemetry configuration properties now have the quarkus.otel.*
prefix.
The legacy properties with prefix quarkus.opentelemetry.*
are currently being mapped to the new ones as a default, during a transition period. See Default column in the details below.
Configuration property fixed at build time - All other configuration properties are overridable at runtime
Type |
Default |
|
---|---|---|
List of exporters supported by Quarkus. List of exporters to be used for tracing, separated by commas. Has one of the values on Default on Quarkus is Environment variable: Show more |
list of string |
|
The sampler to use for tracing. Has one of the values on Fallbacks to the legacy property Environment variable: Show more |
string |
|
Enable the The Environment variable: Show more |
boolean |
|
No Metrics exporter for now Environment variable: Show more |
list of string |
|
No Log exporter for now. Environment variable: Show more |
list of string |
|
The propagators to be used. Use a comma-separated list for multiple propagators. Has values from Default is Environment variable: Show more |
list of string |
|
If true, disable the OpenTelemetry SDK. Runtime configuration. Defaults to Environment variable: Show more |
boolean |
|
Suppress non-application uris from trace collection. This will suppress tracing of Providing a custom This is a Quarkus specific property. Suppressing non-application uris is enabled by default. Fallbacks to the legacy property Environment variable: Show more |
boolean |
|
Include static resources from trace collection. This is a Quarkus specific property. Include static resources is disabled by default. Providing a custom Fallbacks to the legacy property Environment variable: Show more |
boolean |
|
Sampler argument. Depends on the When setting the stock sampler to Defaults to Environment variable: Show more |
string |
|
The maximum length of attribute values. Applies to spans and logs. By default, there is no limit. Environment variable: Show more |
string |
|
The maximum number of attributes. Applies to spans, span events, span links, and logs. Default is Environment variable: Show more |
int |
|
The maximum length of span attribute values. Takes precedence over By default, there is no limit. Environment variable: Show more |
int |
|
The maximum number of attributes per span. Takes precedence over Default is Environment variable: Show more |
int |
|
The maximum number of events per span. Default is Environment variable: Show more |
int |
|
The maximum number of links per span. Default is Environment variable: Show more |
int |
|
The interval, in milliseconds, between two consecutive exports. Default is Environment variable: Show more |
|
|
The maximum queue size. Default is Environment variable: Show more |
int |
|
The maximum batch size. Default is Environment variable: Show more |
int |
|
The maximum allowed time, in milliseconds, to export data. Default is Environment variable: Show more |
|
|
Specify resource attributes in the following format: Environment variable: Show more |
list of string |
|
Specify logical service name. Takes precedence over service.name defined with otel.resource.attributes and from quarkus.application.name. Defaults to Environment variable: Show more |
string |
|
Specify resource attribute keys that are filtered. Environment variable: Show more |
list of string |
|
The maximum amount of time Quarkus will wait for the OpenTelemetry SDK to flush unsent spans and shutdown. Environment variable: Show more |
|
|
Sets the OTLP endpoint to connect to. If unset, defaults to Environment variable: Show more |
string |
|
OTLP Exporter specific. Will override Fallbacks to the legacy property Environment variable: Show more |
string |
|
Key-value pairs to be used as headers associated with gRPC requests. The format is similar to the Environment variable: Show more |
list of string |
|
Sets the method used to compress payloads. If unset, compression is disabled. Currently supported compression methods include Environment variable: Show more |
|
|
Sets the maximum time to wait for the collector to process an exported batch of spans. If unset, defaults to `OtlpExporterRuntimeConfig#DEFAULT_TIMEOUT_SECS`s. Environment variable: Show more |
|
|
OTLP defines the encoding of telemetry data and the protocol used to exchange data between the client and the server. Depending on the exporter, the available protocols will be different. Currently, only Environment variable: Show more |
string |
|
Comma-separated list of the path to the key files (Pem format). Environment variable: Show more |
list of string |
|
Comma-separated list of the path to the certificate files (Pem format). Environment variable: Show more |
list of string |
|
Comma-separated list of the trust certificate files (Pem format). Environment variable: Show more |
list of string |
About the Duration format
To write duration values, use the standard You can also use a simplified format, starting with a number:
In other cases, the simplified format is translated to the
|