add OTEL trace for clusterpedia-apiserver #604
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hi @KubeKyrie, DetailsInstructions for interacting with me using comments are available here. |
|
Here is the test results
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "2"
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"clusterpedia-apiserver"},"name":"clusterpedia-apiserver","namespace":"clusterpedia-system"},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"clusterpedia-apiserver"}},"template":{"metadata":{"labels":{"app":"clusterpedia-apiserver"}},"spec":{"containers":[{"command":["/usr/local/bin/apiserver","--secure-port=443","--storage-config=/etc/clusterpedia/storage/internalstorage-config.yaml","--tracing-config-file=/etc/clusterpedia/trace/tracing-config.yaml","-v=3"],"env":[{"name":"DB_PASSWORD","valueFrom":{"secretKeyRef":{"key":"password","name":"internalstorage-password"}}}],"image":"ghcr.io/kubekyrie/clusterpedia/apiserver-amd64:latest","name":"apiserver","volumeMounts":[{"mountPath":"/etc/clusterpedia/storage","name":"internalstorage-config","readOnly":true},{"mountPath":"/etc/clusterpedia/trace","name":"tracing-config","readOnly":true}]}],"serviceAccountName":"clusterpedia-apiserver","volumes":[{"configMap":{"name":"clusterpedia-internalstorage"},"name":"internalstorage-config"},{"configMap":{"name":"clusterpedia-tracing-config"},"name":"tracing-config"}]}}}}
creationTimestamp: "2023-11-30T10:09:18Z"
generation: 2
labels:
app: clusterpedia-apiserver
name: clusterpedia-apiserver
namespace: clusterpedia-system
resourceVersion: "5463952"
uid: 5d30e0d4-0fd8-4398-b7ad-075855d4772c
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: clusterpedia-apiserver
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: clusterpedia-apiserver
spec:
containers:
- command:
- /usr/local/bin/apiserver
- --secure-port=443
- --storage-config=/etc/clusterpedia/storage/internalstorage-config.yaml
- --tracing-config-file=/etc/clusterpedia/trace/tracing-config.yaml
- -v=3
env:
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
key: password
name: internalstorage-password
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: insight-agent-opentelemetry-collector.insight-system.svc.cluster.local:4317
- name: OTEL_SERVICE_NAME
value: clusterpedia-apiserver
- name: OTEL_K8S_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: OTEL_RESOURCE_ATTRIBUTES_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: OTEL_RESOURCE_ATTRIBUTES_POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: OTEL_RESOURCE_ATTRIBUTES
value: k8s.namespace.name=$(OTEL_K8S_NAMESPACE),k8s.node.name=$(OTEL_RESOURCE_ATTRIBUTES_NODE_NAME),k8s.pod.name=$(OTEL_RESOURCE_ATTRIBUTES_POD_NAME)
image: ghcr.io/kubekyrie/clusterpedia/apiserver-amd64:latest
imagePullPolicy: Always
name: apiserver
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/clusterpedia/storage
name: internalstorage-config
readOnly: true
- mountPath: /etc/clusterpedia/trace
name: tracing-config
readOnly: true
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: clusterpedia-apiserver
serviceAccountName: clusterpedia-apiserver
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
name: clusterpedia-internalstorage
name: internalstorage-config
- configMap:
defaultMode: 420
name: clusterpedia-tracing-config
name: tracing-config
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2023-11-30T10:09:18Z"
lastUpdateTime: "2023-11-30T15:02:19Z"
message: ReplicaSet "clusterpedia-apiserver-67d445b478" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2023-12-01T08:44:43Z"
lastUpdateTime: "2023-12-01T08:44:43Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 2
readyReplicas: 1
replicas: 1
updatedReplicas: 1 |
KubeKyrie
force-pushed
the
add-otel-trace
branch
from
348e9a3
to
a849b95
Compare
|
/auto-cc |
KubeKyrie
force-pushed
the
add-otel-trace
branch
2 times, most recently
from
36b79f1
to
7088ab1
Compare
ok, It has been implemented with reference to the implementation of kube-apiserver otel tracing. |
|
KubeKyrie
force-pushed
the
add-otel-trace
branch
from
7088ab1
to
babb8e2
Compare
|
The OTEL env such as OTEL_K8S_NAMESPACE is not necessary, but it's recommended to set for detailed trace requirement |
Iceber
approved these changes
Dec 5, 2023
There was a problem hiding this comment.
LGTM, Thanks for adding this feature!
/kustomize also needs to be updated
KubeKyrie
force-pushed
the
add-otel-trace
branch
2 times, most recently
from
e85a8c4
to
51888c8
Compare
Signed-off-by: KubeKyrie <shaolong.qin@daocloud.io>
KubeKyrie
force-pushed
the
add-otel-trace
branch
from
51888c8
to
2e13a70
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind feature
What this PR does / why we need it:
new support opentelemetry trace for clusterpedia-apiserver，and enhance observability capabilities
Which issue(s) this PR fixes:
Fixes #None
Special notes for your reviewer:
The dafault FeatureGate APIServerTracing is enabled, we also need provide the apiserver with a tracing configuration file with --tracing-config-file=<path-to-config>.
This is an example config that records spans for 1 in 10000 requests, and uses the default OpenTelemetry endpoint:
Detailed info can be referenced https://kubernetes.io/docs/concepts/cluster-administration/system-traces/
Does this PR introduce a user-facing change?: