Observability Cloud Setup

hango always sends traces, metrics and logs to the local Grafana LGTM container. It can also send them to Grafana Cloud and send traces to Langfuse Cloud. Each cloud is switched on by one variable in .env. When that variable is unset, the collector sends nothing to that cloud.

Destination	Signals	Switched on by	Free plan
Local lgtm (`localhost:3000`)	traces, metrics, logs	always on	not applicable
Grafana Cloud	traces, metrics, logs	`GRAFANA_CLOUD_INSTANCE_ID`	Free: 10k metric series, 50 GB logs, 50 GB traces per month, 14-day retention
Langfuse Cloud	traces	`LANGFUSE_PUBLIC_KEY`	Hobby: 50k units per month, 30-day retention

Prompt, completion and tool content is always deleted before data reaches lgtm or Grafana Cloud. Langfuse receives it only on spans that carry hango.tenant.trace_llm_content=true (a boolean); a span without that attribute is stripped (phase 3).

Before you start

cp .env.example .env

Fill in values by uncommenting lines in .env. Leave a line commented out rather than setting it to an empty value. The collector treats an empty value as set, so it skips its built-in default and can refuse to start with at least one endpoint must be specified.

Langfuse Cloud

Sign up at cloud.langfuse.com (EU region) or us.cloud.langfuse.com (US region). Data stays in the region you pick.
Create an organization, then a project, for example hango-dev.
Open the project’s Settings, then API Keys, and create a key pair. Copy the public key (pk-lf-...) and the secret key (sk-lf-...). The secret is shown once.

In .env, set:

LANGFUSE_BASE_URL=https://cloud.langfuse.com    # or https://us.cloud.langfuse.com
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...

Apply it. A plain restart does not reread .env, so recreate the collector:
```
docker compose up -d --force-recreate otelcol
```
Send a request with curl -i localhost:8080/readyz, then open Tracing in the Langfuse project. The GET /readyz trace appears within a minute.

Grafana Cloud

Sign up at grafana.com and create a stack. The stack name becomes <name>.grafana.net, and the region you pick is where data is stored.
In the Grafana Cloud portal, open your stack and find the OpenTelemetry tile. Choose Configure.
That page shows the OTLP endpoint (for example https://otlp-gateway-prod-us-east-0.grafana.net/otlp) and the instance ID (a number). Generate an API token there. It creates an access policy token with write access to metrics, logs and traces. Copy it, because it is shown once.

In .env, set:

GRAFANA_CLOUD_OTLP_ENDPOINT=https://otlp-gateway-prod-<region>.grafana.net/otlp
GRAFANA_CLOUD_INSTANCE_ID=123456
GRAFANA_CLOUD_API_TOKEN=glc_...

Recreate the collector:

docker compose up -d --force-recreate otelcol

Send a request, copy its X-Trace-Id, and open https://<name>.grafana.net/explore. Use the stack’s traces data source (named like grafanacloud-<name>-traces) and paste the trace id. Use the logs data source with {service_name="hango"} and the metrics data source with go_memory_used_bytes. Metrics arrive every 10 seconds.

Check the collector

docker compose logs otelcol | grep -E "Everything is ready|Exporting failed"

Log	Meaning	Fix
`Everything is ready` and no errors	Working, or a cloud is switched off	Check that the switch variable is uncommented in `.env`
`Exporting failed` from `otlp_http/langfuse` with `HTTP Status Code 401`	Langfuse rejected the keys	Recheck both keys and that `LANGFUSE_BASE_URL` matches the region of the project
`Exporting failed` from `otlp_http/grafanacloud` with `HTTP Status Code 401`	Grafana Cloud rejected the instance ID or token	Regenerate the token from the OpenTelemetry tile and make sure the ID belongs to the same stack
`at least one endpoint must be specified`	An endpoint variable is set but empty	Comment the line out or give it a value

How the switch works

The collector config cannot turn a pipeline on and off, so each cloud has a filter processor at the front of its pipelines. For Grafana Cloud the condition is "${env:GRAFANA_CLOUD_INSTANCE_ID:-}" == "". When the variable is unset, the condition reads "" == "" and is true for every item, so everything on that pipeline is dropped before the exporter. When it is set, the condition is false and data flows. Only the ID or public key is used in the condition, so a token never appears in an expression the collector might log. See deploy/otelcol/config.yaml.

Diagrams Docs site