Google Cloud Platform (GCP)

Send metrics and/or logs for Google Cloud Platform (GCP) services to Cardinal.

Overall Steps:

(Logs) Set up a Google Cloud Pub/Sub topic and subscription.
(Logs) Create a Google Cloud Log Router sink to route logs to the Pub/Sub topic.
(Logs / Metrics) Set up an OpenTelemetry Collector to pull logs from the Pub/Sub topic and send them to Cardinal.

Set up Google Cloud Logs

The Google Cloud Log Routing (opens in a new tab) document describes these steps in more detail.

(If you are only interested in sending metrics, skip to the next section.)

Create a Google Cloud Pub/Sub topic

Go to your Topic List (opens in a new tab) in the Google Cloud Console and click the Create Topic button.
Give it a meaningful name, such as cardinal-logs, and click Create.

Create a Google Cloud Pub/Sub subscription

In most cases a default subscription is automatically created when you create the topic. If this did not happen, you will need to manually create it.

Go to your Subscription List (opens in a new tab) in the Google Cloud Console and click the Create Subscription button.
Give it a meaningful name, such as cardinal-logs-sub.
Set the Topic to the topic you created in the previous step.
Set the Delivery type to Pull.
The remaining options can be left as default.

Route logs to the Pub/Sub topic

Go to your Log Router (opens in a new tab) in the Google Cloud Console.
Click the Create Sink button.
Give it a meaningful name, such as cardinal-logs-sink.
Click Next.
Set the Sink Service to Cloud Pub/Sub Topic.
Set the Sink Destination to the topic you created in the previous step.
Click Next.
In the Choose logs to include in sink step, you can either select specific logs or leave it as default to include all logs. For more information on this, refer to the Logging Query Language (opens in a new tab) documentation.
Click the Create Sink button to create the sink.

Using a new OpenTelemetry Collector

These steps will guide you through setting up a minimal OpenTelemetry Collector on a Google Compute Engine (GCE) VM instance to collect metrics and logs for various Google Cloud Platform (GCP) services.

(If you already have an existing OpenTelemetry Collector, skip to the next section.)

Build and publish a Collector Docker image

Create the following Collector configuration file in a new directory:

touch config.yaml

receivers:
  googlecloudpubsub: # For logs
    project: "${GCP_PROJECT_ID}"
    subscription: "${GCP_SUBSCRIPTION_NAME}"
    encoding: cloud_logging
  googlecloudmonitoring: # For metrics
    collection_interval: 1m
    project_id: "${GCP_PROJECT_ID}"
    metrics_list:
      # This is an example of how to collect all Google Cloud Run metrics.
      # You can find the full list of metrics in the Google Cloud Platform 
      # documentation – https://cloud.google.com/monitoring/api/metrics_gcp
      - metric_descriptor_filter: 'metric.type = starts_with("run.googleapis.com")'
processors:
  resource/env:
    attributes:
      - key: deployment.environment.name
        value: "${DEPLOYMENT_ENVIRONMENT}"
        action: upsert
      - key: service.name
        action: insert
        from_attribute: "gcp.service_name"
      - key: service.name
        action: insert
        from_attribute: "service_name"
exporters:
  otlphttp/cardinal:
    endpoint: "${CARDINALHQ_OTLP_ENDPOINT}"
    headers:
      x-cardinalhq-api-key: "${CARDINALHQ_API_KEY}"
service:
  pipelines:
    logs: # For logs
      receivers: [googlecloudpubsub]
      processors: [resource/env]
      exporters: [otlphttp/cardinal]
    metrics: # For metrics
      receivers: [googlecloudmonitoring]
      processors: [resource/env]
      exporters: [otlphttp/cardinal]

The environment variables will be set at runtime in a later step.

(Refer to the googlecloudpubsub (opens in a new tab) and googlecloudmonitoring (opens in a new tab) receiver READMEs for more configuration options.)

Create a Dockerfile in the same directory:

touch Dockerfile

FROM otel/opentelemetry-collector-contrib:latest

COPY config.yaml /etc/otelcol-contrib/config.yaml

Build and publish the Docker image to Google Artifact Registry:

(Replace LOCATION, PROJECT_ID, and REPOSITORY with your own values.)

export DOCKER_DEFAULT_PLATFORM=linux/amd64
 
docker build -t ${LOCATION}-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/otel-collector-gcp:latest .
docker push ${LOCATION}-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/otel-collector-gcp:latest

Deploy the Collector to a GCE VM instance

In the Google Cloud Console, navigate to the VM Instances page (opens in a new tab).
Click the Create Instance button.
On the Machine configuration step, give the instance a name, and select the e2-medium machine type.
On the OS and storage step, click the Deploy Container button.

Set the Container image to the image you built in the previous step, and add the following environment variables to the instance:

GCP_PROJECT_ID: The ID of the GCP project to collect metrics from (e.g. sandbox-124512)
GCP_SUBSCRIPTION_NAME: The name of the Google Cloud Pub/Sub subscription to collect logs from (e.g. projects/your-project-id/subscriptions/your-subscription-name)
DEPLOYMENT_ENVIRONMENT: A name for the deployment environment (e.g. dev or prod)
CARDINALHQ_OTLP_ENDPOINT: https://otelhttp.intake.us-east-2.aws.cardinalhq.io
CARDINALHQ_API_KEY: Your Cardinal API key

Container configuration

On the Security step, under the Identity and API access > Access scopes section, select Set access for each API and set the following:

Stackdriver Monitoring API: Full (for metrics)
Cloud Pub/Sub: Read Only (for logs)

Security configuration

Click the Create button to deploy the instance.
Once the instance is up and running, SSH into it, and run the following command to verify that the Collector is running and collecting metrics and logs by tailing its logs:

docker ps # Get the Container ID of the Collector
docker logs -f <CONTAINER_ID> # View the logs of the Collector

Using an existing OpenTelemetry Collector

Update your existing OpenTelemetry Collector configuration to include the yaml provided above.
Set the environment variables as shown above.
Restart the Collector to apply the changes.
Verify that the Collector is running and collecting metrics and logs by tailing its logs.

Cardinal Collector GitHub