Set up New Relic using the Kubernetes operator

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. You can use it to provision all kinds of infrastructure and services, including New Relic entities.

In this guide you'll learn how to set up New Relic for the first time with the open source New Relic Kubernetes operator. More specifically, you'll provision an alert policy with NRQL conditions in your New Relic account using Kubernetes.

Before you begin

This walkthrough assumes you’ve already deployed a Kubernetes cluster. You could even create a local cluster on your machine with kind. To use this guide, you should have some basic knowledge of both New Relic and Kubernetes. To complete the full exercise, you’ll need to:

Deploy a New Relic agent if you haven't done so yet. Install New Relic for your application.
Install kubectl and point it at the correct cluster; this determines the cluster where you’ll install the New Relic operator.
Install kustomize.

Step 1 of 3

Installing the operator on your Kubernetes cluster

First, install cert-manager, which automatically provisions and manages TLS certificates in Kubernetes.

bash

$kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.15.0/cert-manager.yaml

Next, install the Kubernetes operator.

bash

$kustomize build https://github.com/newrelic/newrelic-kubernetes-operator/config/default | kubectl apply -f -

To confirm the installation was successful, run a few kubectl commands to check the status of the Kubernetes operator. Ensure the Kubernetes operator's namespace, newrelic-kubernetes-operator-system, has been applied:

bash

$kubectl get namespaces

The output should be similar to the following, which includes the Kubernetes operator's namespace, newrelic-kubernetes-operator-system:

bash

NAME                                     STATUS   AGE
cert-manager                             Active   4m35s
default                                  Active   20m
kube-node-lease                          Active   20m
kube-public                              Active   20m
kube-system                              Active   20m
newrelic-kubernetes-operator-system      Active   3m48s

Now, make sure the Kubernetes operator's controller manager is running:

Note: Don't forget to include the --namespace (shorthand -n) option when running kubectl get pods to ensure you're inspecting resources within the correct namespace.

bash

$kubectl get pods --namespace newrelic-kubernetes-operator-system

You should see output similar to the following:

bash

NAME                                                              READY   STATUS    RESTARTS   AGE
newrelic-kubernetes-operator-controller-manager-7b9c64f58crwg9j   2/2     Running   0          157m

If your output is similar to the example shown, you’re ready for the next step. If you don’t see a pod named newrelic-kubernetes-operator-controller-manager-<hash>, double check your Kubernetes configuration to ensure you’re within the correct context and pointing to the correct cluster.

Step 2 of 3

Creating your first alert policy

To kick things off, start small. First, create an alert policy with the minimum required configuration, then add a NRQL alert condition to the policy, which will add the condition to the policy in New Relic. A minimal alert policy configuration is represented in the code below. For the sake of this walkthrough, name this file new_relic_alert_policy.yaml.

Note: For help locating your user key, check out New Relic's user key documentation.

apiVersion: nr.k8s.newrelic.com/v1
kind: AlertsPolicy
metadata:
  name: my-policy
spec:
  account_id: <your New Relic account ID>
  api_key: <your New Relic user key>
  name: 'Alert Policy Created With k8s' # Feel free to rename
  region: 'us'

Now run the kubectl apply command to create your alert policy.

bash

$kubectl apply -f ./new_relic_alert_policy.yaml

You'll see output that reads similar to the following:

bash

alertspolicy.nr.k8s.newrelic.com/my-policy created

Confirm that your alert policy was created by viewing your policies at alerts.newrelic.com/accounts/{your account ID}/policies. You can search for your new policy by its name. In this case, search for "Alert Policy Created With k8s."

You should see your new alert policy. Next it’s time to add a NRQL alert condition to the policy using the same configuration file.

Step 3 of 3

Add NRQL alert conditions to your alert policy

In the previous section you created an alert policy; now, you’ll add some alert conditions to the policy so you can trigger alerts when certain metrics are out of line. In your new_relic_alert_policy.yaml file, add a NRQL alert condition to the policy that will alert you when an application's average overall response time is above five seconds for a three minute period.

Note: To receive notifications when an alert is triggered, add notification channels to your alert policy, with this code.

# The policy from the previous steps
apiVersion: nr.k8s.newrelic.com/v1
kind: AlertsPolicy
metadata:
  name: my-policy
spec:
  account_id: <your New Relic account ID>
  api_key: <your New Relic user key>
  name: 'Alert Policy Created With k8s' # Feel free to rename
  region: 'us'

  # Add a NRQL alert condition to the policy
  conditions:
    - spec:
        type: 'NRQL'
        name: 'NRQL Alert Condition Created With k8s'
        nrql:
          query: "SELECT average(duration) FROM Transaction WHERE appName = 'YOUR APP NAME'"
          evaluationOffset: 3
        enabled: true
        terms:
          - threshold: '5'
            threshold_occurrences: 'ALL'
            threshold_duration: 180
            priority: 'CRITICAL'
            operator: 'ABOVE'
        violationTimeLimit: 'ONE_HOUR'
        valueFunction: 'SINGLE_VALUE'

With the alert condition added to the configuration, you can apply the update, which will create a NRQL alert condition and add it to your policy.

bash

$kubectl apply -f ./new_relic_alert_policy.yaml

To confirm that the NRQL alert condition was created successfully, refresh your alert policy. If you see a new alert condition added to the alert policy, it was a success.

To finish things off, you'll create and add an alert channel to your alert policy. For example, maybe you want to send an email out to your team when your alert condition is triggered.

Try it out now

We have a Kubernetes test cluster ready for you in 2 minutes. By following this on-line tutorial, you will learn how to:

Deploy the New Relic agent in a Kubernetes environment
Use the New Relic Kubernetes operator

Some tips to use the on-line tutorial window:

Accept the cookies, so you can see the menu bar.
Click anywhere in the tutorial window to start. It will take about 2 minutes for your environment to be ready.
Press CTRL-l or type clear to clear the terminal window
Click on the finish flag icon in the bottom menu to hide or show the instructions

Good luck!

Important

Some browsers automatically disable the use of iframes. If the module isn't loading please check your browser settings.

What’s next?

Nice work — now you can manage your New Relic alert policies and NRQL alert conditions with code that integrates seamlessly within your Kubernetes workflow. This provides the ability to configure and manage your alerts with a domain-specific pattern, providing consistency and maintainability. You also gain the benefits of code reviews for any potential changes moving forward. As you and your team move forward, you might need to adjust some of the configuration values to better fit your needs. The New Relic Kubernetes Operator is just one of several tools in the New Relic Developer Toolkit aimed at facilitating observability as code.