Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/develop/go/workers/serverless-workers/aws-lambda.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: aws-lambda
title: Serverless Workers on AWS Lambda - Go SDK
sidebar_label: Serverless Workers on AWS Lambda
sidebar_label: AWS Lambda
description: Write a Temporal Worker that runs on AWS Lambda using the Go SDK lambdaworker package.
slug: /develop/go/workers/serverless-workers/aws-lambda
toc_max_heading_level: 4
Expand Down Expand Up @@ -122,7 +122,7 @@ It controls how much time before the Lambda deadline the Worker begins its grace
The default is `WorkerStopTimeout` + 2 seconds.

If your Worker handles long-running Activities, increase `WorkerStopTimeout`, `ShutdownDeadlineBuffer`, and the Lambda invocation deadline (`--timeout`) together.
For guidance on how these values relate, see [Tuning for long-running Activities](/serverless-workers#tuning-for-long-running-activities).
For guidance on how these values relate, see [Tuning for long-running Activities](/serverless-workers/aws-lambda#tuning-for-long-running-activities).

## Add observability with OpenTelemetry {/* #add-observability */}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: aws-lambda
title: Serverless Workers on AWS Lambda - Python SDK
sidebar_label: Serverless Workers on AWS Lambda
sidebar_label: AWS Lambda
description: Write a Temporal Worker that runs on AWS Lambda using the Python SDK lambda_worker package.
slug: /develop/python/workers/serverless-workers/aws-lambda
toc_max_heading_level: 4
Expand Down Expand Up @@ -127,7 +127,7 @@ It controls how much time before the Lambda deadline the Worker begins its grace
The default is `graceful_shutdown_timeout` + 2 seconds.

If your Worker handles long-running Activities, increase `graceful_shutdown_timeout`, `shutdown_deadline_buffer`, and the Lambda invocation deadline (`--timeout`) together.
For guidance on how these values relate, see [Tuning for long-running Activities](/serverless-workers#tuning-for-long-running-activities).
For guidance on how these values relate, see [Tuning for long-running Activities](/serverless-workers/aws-lambda#tuning-for-long-running-activities).

## Add observability with OpenTelemetry {/* #add-observability */}

Expand Down
162 changes: 162 additions & 0 deletions docs/develop/python/workers/serverless-workers/cloud-run.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
---
id: cloud-run
title: Serverless Workers on GCP Cloud Run - Python SDK
sidebar_label: GCP Cloud Run
description: Add OpenTelemetry to a Temporal Worker running on a GCP Cloud Run worker pool using the Python SDK.
slug: /develop/python/workers/serverless-workers/cloud-run
toc_max_heading_level: 4
keywords:
- serverless
- cloud run
- gcp
- google cloud
- opentelemetry
- python sdk
- worker
- serverless worker
tags:
- Workers
- Python SDK
- Serverless
- GCP Cloud Run
---

import { ReleaseNoteHeader } from '@site/src/components';

<ReleaseNoteHeader type="prerelease">
To request access during Pre-release, create a [support ticket](/cloud/support#support-ticket) or contact your account team.
APIs are experimental and may be subject to backwards-incompatible changes.
[Sign up for updates](https://temporal.io/pages/serverless-workers-updates) to be notified when Serverless Workers reach Public Preview.
</ReleaseNoteHeader>

On a [GCP Cloud Run worker pool](https://cloud.google.com/run/docs/worker-pools), you run a standard long-lived Temporal Worker.
Register Workflows and Activities the same way you would with any other Worker, and Temporal Cloud scales the pool up and down as work arrives and drains.
Worker Versioning is required for Serverless Workers; set a [versioning behavior](/worker-versioning#versioning-behaviors) on each Workflow.

The GCP-specific integration is the OpenTelemetry plugin, which exports traces and Temporal Core metrics to a collector running alongside your Worker.

For the end-to-end deployment guide covering the Worker Pool, IAM, and compute configuration, see [Deploy a Serverless Worker on GCP Cloud Run](/production-deployment/worker-deployments/serverless-workers/cloud-run).

## Add observability with OpenTelemetry {/* #add-observability */}

The `temporalio.contrib.gcp.OpenTelemetryPlugin` configures observability with defaults suited to a Cloud Run worker pool.
By default, the plugin configures:

- OTLP gRPC export to `localhost:4317`, the collector sidecar endpoint.
- `service.name` from the Cloud Run-provided `CLOUD_RUN_WORKER_POOL` environment variable.
- A replay-safe OpenTelemetry tracer provider.
- Temporal Core metrics with a 60-second export interval.

The underlying metrics and traces are the same ones the Python SDK emits in any environment.
For general observability concepts and the full list of available metrics, see [Observability - Python SDK](/develop/python/platform/observability) and the [SDK metrics reference](/references/sdk-metrics).

Pass the plugin to `Client.connect`. Opt into `add_temporal_spans=True` to trace named operations such as `RunWorkflow:GreetingWorkflow`:

<!--SNIPSTART python-cloud-run-otel-worker-->
[gcp_open_telemetry/worker.py](https://github.com/temporalio/samples-python/blob/main/gcp_open_telemetry/worker.py)
```py
# Endpoint, service name, Core metrics, and tracer provider all use the GCP
# plugin defaults. The opt-in adds named Temporal operation spans.
plugin = OpenTelemetryPlugin(add_temporal_spans=True)
client = await Client.connect(
settings.address,
namespace=settings.namespace,
api_key=settings.api_key,
tls=True,
plugins=[plugin],
)
worker = Worker(
client,
task_queue=settings.task_queue,
workflows=[GreetingWorkflow],
activities=[compose_greeting],
graceful_shutdown_timeout=WORKER_GRACEFUL_SHUTDOWN_TIMEOUT,
)
```
<!--SNIPEND-->

Named Temporal operation spans are opt-in. The endpoint, service name, tracer provider, and Core metrics use the plugin defaults, but you must set `add_temporal_spans=True` to emit spans for Workflow and Activity operations.

### Run the collector as a sidecar {/* #collector-sidecar */}

The plugin exports to a collector you run as a second container in the Worker Pool.
Use the [Google-Built OpenTelemetry Collector](https://cloud.google.com/stackdriver/docs/instrumentation/google-built-otel), which detects the Cloud Run resource, authenticates through the Worker Pool's service account, exports metrics to Google Managed Service for Prometheus, and exports traces through the Google Cloud Telemetry API.

Provide the following collector configuration. It receives OTLP on `localhost:4317`, detects the GCP resource, and routes metrics and traces to their respective pipelines:

<!--SNIPSTART python-cloud-run-otel-collector-config-->
[gcp_open_telemetry/collector-config.yaml](https://github.com/temporalio/samples-python/blob/main/gcp_open_telemetry/collector-config.yaml)
```yaml
receivers:
otlp:
protocols:
grpc:
endpoint: localhost:4317

processors:
batch/traces:
send_batch_max_size: 200
send_batch_size: 200
timeout: 5s
memory_limiter:
check_interval: 1s
limit_percentage: 65
spike_limit_percentage: 20
resource_detection:
detectors: [gcp]
timeout: 10s
transform/collision:
metric_statements:
- context: datapoint
statements:
- set(attributes["exported_location"], attributes["location"])
- delete_key(attributes, "location")
- set(attributes["exported_cluster"], attributes["cluster"])
- delete_key(attributes, "cluster")
- set(attributes["exported_namespace"], attributes["namespace"])
- delete_key(attributes, "namespace")
- set(attributes["exported_job"], attributes["job"])
- delete_key(attributes, "job")
- set(attributes["exported_instance"], attributes["instance"])
- delete_key(attributes, "instance")
- set(attributes["exported_project_id"], attributes["project_id"])
- delete_key(attributes, "project_id")
transform/set_project_id:
error_mode: ignore
trace_statements:
- set(resource.attributes["gcp.project_id"], resource.attributes["gcp.project.id"]) where resource.attributes["gcp.project.id"] != nil
- set(resource.attributes["gcp.project_id"], resource.attributes["cloud.account.id"]) where resource.attributes["gcp.project_id"] == nil and resource.attributes["cloud.account.id"] != nil

exporters:
googlemanagedprometheus:
otlp_grpc:
endpoint: telemetry.googleapis.com:443
compression: none
balancer_name: pick_first
auth:
authenticator: googleclientauth

extensions:
googleclientauth:
health_check:
endpoint: 0.0.0.0:13133

service:
extensions: [googleclientauth, health_check]
pipelines:
metrics:
receivers: [otlp]
processors: [memory_limiter, resource_detection, transform/collision]
exporters: [googlemanagedprometheus]
traces:
receivers: [otlp]
processors:
[memory_limiter, resource_detection, transform/set_project_id, batch/traces]
exporters: [otlp_grpc]
```
<!--SNIPEND-->

The metrics pipeline forwards each SDK export directly, without a batch processor, so a runtime shutdown-time export cannot collide with a periodic export on the same Managed Prometheus time series.
Traces use a dedicated five-second batch processor.

To verify telemetry after deploying, confirm that Trace Explorer contains `RunWorkflow:GreetingWorkflow` with `service.name` equal to the Worker Pool name, and that Metrics Explorer contains `prometheus.googleapis.com/temporal_workflow_completed_total/counter`.
1 change: 1 addition & 0 deletions docs/develop/python/workers/serverless-workers/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,4 @@ For the end-to-end deployment guide, see [Deploy a Serverless Worker](/productio
## Supported providers

- [**AWS Lambda**](/develop/python/workers/serverless-workers/aws-lambda) - Use the `lambda_worker` contrib package to run a Worker as a Lambda function. Covers setup, configuration, Lambda-tuned defaults, and observability.
- [**GCP Cloud Run**](/develop/python/workers/serverless-workers/cloud-run) - Run a standard Worker on a Cloud Run worker pool, and wire up OpenTelemetry with the `temporalio.contrib.gcp` plugin.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
id: aws-lambda
title: Serverless Workers on AWS Lambda - TypeScript SDK
sidebar_label: Serverless Workers on AWS Lambda
sidebar_label: AWS Lambda
description: Write a Temporal Worker that runs on AWS Lambda using the TypeScript SDK @temporalio/lambda-worker package.
slug: /develop/typescript/workers/serverless-workers/aws-lambda
toc_max_heading_level: 4
Expand Down Expand Up @@ -121,7 +121,7 @@ It controls how much time before the Lambda deadline the Worker begins its grace
The default is `shutdownGraceTime` (5s) + 2s.

If your Worker handles long-running Activities, increase `shutdownGraceTime`, `shutdownDeadlineBufferMs`, and the Lambda invocation deadline (`--timeout`) together.
For guidance on how these values relate, see [Tuning for long-running Activities](/serverless-workers#tuning-for-long-running-activities).
For guidance on how these values relate, see [Tuning for long-running Activities](/serverless-workers/aws-lambda#tuning-for-long-running-activities).

## Add observability with OpenTelemetry {/* #add-observability */}

Expand Down
Loading
Loading