Notifications and Events Guide

Spinnaker supports several types of notifications.

Out of the box, Spinnaker allows you to configure the following types of notifications:

This is discussed in the Configuring Notifications section below.

Additionally, Spinnaker allows you to set webhooks for git triggers. See the Setting up Git Triggers in Spinnaker section.

You can also set Spinnaker to stream all its events to a downstream listener. See the Add a Webhook to Spinnaker section.

Additionally, Spinnaker is capable of handling cron-based triggers and detect changes in Jenkins builds and Docker images. This functionality will be documented at a later time.

See also <code>hal config notifications</code> .

Configuring notifications

Notification configurations are in echo.yml and settings-local.js. For changes to echo.yml, create echo-local.yml and put your changes in there. You can customize settings-local.js directly.

Where to put echo-local.yml and settings.js

If you use Halyard to configure Spinnaker, put echo-local.yml ~/.hal/{deployment}/profiles/.

For settings-local.js, follow the Custom Profile for Deck reference and place it in the following location ~/.hal/{deployment}/profiles/settings-local.js.

If you don’t use Halyard, put echo-local.yml in the same place as the current echo.yml, in /opt/spinnaker/config, and put settings-local.js in /opt/deck/html/.

Spinnaker baseURL

You need to set the spinnaker.baseUrl configuration value which is used by spinnaker templates. This should point back to the url for your spinnaker’s UI ( deck ) instance. This url is used in notifications to link back to your spinnaker instance.

Email

Email in spinnaker is provided by Spring Boot Mail starter .

The settings for some popular email providers are listed below:

Email ProviderSMTP usernameSMTP passwordSMTP server addressSMTP port (TLS)SMTP port (SSL)SMTP TLS/SSL required
Gsuite/GmailYour email addressYour email passwordsmtp.gmail.com587465yes
YahooYour email addressYour email passwordsmtp.mail.yahoo.com587465yes
Hotmail/liveYour email addressYour email passwordsmtp.live.com587-yes
OutlookYour email addressYour email passwordsmtp-mail.outlook.com58725yes

The following is an example of using hotmail to send notifications.

echo.yml:

  1. mail:
  2.   enabled: true
  3.   from: <myemail>@hotmail.com
  4. spring:
  5.   mail:
  6.     host: smtp.live.com
  7.     username: <myemail>@hotmail.com
  8.     password: hunter2
  9.     port: 587
  10.     properties:
  11.       mail:
  12.         smtp:
  13.           auth: true
  14.           starttls:
  15.             enable: true
  16.         transport:
  17.           protocol: smtp
  18. #    debug: true <- this is useful if you are mucking around with smtp properties

in settings-local.js (deck)

  1. window.spinnakerSettings = window.spinnakerSettings || {};
  2. window.spinnakerSettings.notifications = window.spinnakerSettings.notifications || {};
  3. window.spinnakerSettings.notifications.email = window.spinnakerSettings.notifications.email || {};
  4. window.spinnakerSettings.notifications.email.enabled = true;

Microsoft Teams

To enable Microsoft Teams support, add the following statement to the echo-local.yml file:

  1. microsoftteams:
  2.   enabled: true

Spinnaker supports sending notifications to Microsoft Teams using Incoming Webhooks.

To create a custom incoming webhook in Teams, follow the instructions provided here: Add an incoming webhook to a Teams channel

When configuring a Microsoft Teams notification, enter the full incoming webhook URL in the Teams Webhook URL text box. In the Edit Notification window, specify Microsoft Teams in the Notify via field and provide your Teams webhook URL. Then, select what events trigger notifications.

Slack

For slack, you need to create a custom bot user , then get the access token associated with that new bot user. Then…

  1. hal config notification slack enable
  2. echo $TOKEN_FROM_SLACK | hal config notification slack edit --bot-name $SPINNAKER_BOT --token

Note: your users will need to invite the slack bot to private rooms that want to be notified.

Twilio

For Twilio, you need to add your account credentials . Then…

  1. hal config notification twilio enable
  2. echo $TWILIO_AUTH_TOKEN | hal config notification twilio edit --account $TWILIO_ACCOUNT_SID --from $TWILIO_PHONE_NUMBER --token

Using notifications

Once notifications have been configured, you can use them to send changes in pipelines and in the manual judgment stage in Spinnaker.

To set up an application-wide notification, go to Application -> Config -> Notifications,

Click on ‘Add Notification’

Notifications and Events Guide - 图2

Enter your notification details.

You can also set Notifications at the Pipeline level ( under configuration ) and at the stage level ( by clicking on the [ ] Send Notifications for this stage checkbox.

Customizing notifications

Spinnaker will send a standard message based on the event that triggered the notification, indicating the stage (if applicable), the pipeline, the application, and the status of the event. It will also include a link to the pipeline.

For stage-level notifications, you can override the message by editing the JSON of the stage, adding a customBody field (for email) or a customMessage field (for Slack). If you are customizing an email notification, you can use Markdown or HTML to customize the format of the email. If you are using Slack, you can use a more limited range of formatting .

For email messages, you can customize the subject by adding a customSubject field.

All standard SpEL expressions will be evaluated prior to sending the notifications, and can be used in the custom notification fields.

There are two special variables available when adding a custom message or body: executionId (the ID of the execution) and link (a fully-formed URL for the pipeline or stage that triggered the notification). To use either of these fields, just wrap them in two curly brackets, e.g. {% raw %}{{link}}{% endraw %}.

An example of a custom email notification: {% raw %}

  1. {
  2.   "customSubject": "Beginning deployment to production (started by: ${trigger.user})",
  3.   "customBody": "*Pipeline parameters:* ${parameters.toString()}\n\n [View the stage]({{link}}) here.",
  4.   "notifications": [
  5.     {
  6.       "address": "spinnakerteam@spinnaker.io",
  7.       "level": "stage",
  8.       "type": "email",
  9.       "when": [
  10.         "stage.starting"
  11.       ],
  12.     }
  13.   ],
  14.   //...
  15. }

{% endraw %}

A custom Slack message: {% raw %}

  1.   {
  2.     "customMessage": "${trigger.user} started a <{{link}}|deploy to production>",
  3.     "notifications": [
  4.       {
  5.         "address": "spinnaker-prod-deploys",
  6.         "level": "stage",
  7.         "type": "slack",
  8.         "when": [
  9.           "stage.starting"
  10.         ],
  11.       }
  12.     ],
  13.     //...
  14.   }

{% endraw %}

Note: the custom fields will apply to all notifications within a stage. If you have different notifications for different events (e.g. a Slack message when the stage starts and when it completes), the custom fields will be applied to both messages.

Setting up Git Triggers in Spinnaker

Instead of listening for events from Github like in the Docker and Jenkins case, direct triggering from git requires a webhook endpoint to be set up in the corresponding VCS.

Github

Using Github as a Trigger

Go to your project’s Webhooks page, for example https://github.com/spinnaker/igor/settings/hooks

Click “Add webhook”

enter http://[spinnakerapi]/webhooks/git/github in the Payload URL, where spinnakerapi is the location of your gate installation.

select “Just the push event”

make it active

Click “Add Webhook” button

Notifications and Events Guide - 图3

Stash

Go to settings / hooks for your repository

Add a Post Receive webhook

Notifications and Events Guide - 图4

Add the following URL: http://[gate url]/webhooks/git/stash, where gate url is the url to your spinnaker api.

You are now ready to receive events from this repository.

BitBucket Cloud

Go to settings / webhooks for your repository

Click ‘Add webhook’

Add a ‘Repository Push’ or ‘Pull Request Merged’ webhook

Notifications and Events Guide - 图5

Add the following URL: http://[gate url]/webhooks/git/bitbucket, where gate url is the url to your spinnaker api.

You are now ready to receive events from this repository.

Add Git Trigger in Pipeline:

Go to Pipeline > Configuration

Select Trigger Type to be Git

Set Repository Type to be github, stash, or bitbucket

Enter your Repository organization / user

Enter your Repository name

Save your pipeline

Notes

You can access the hash of the build via expression ${trigger.hash}

Add a listening webhook to spinnaker

The echo-rest module in spinnaker allows you to set downstream listeners keeping track of Spinnaker events.

Whenever Spinnaker receives an event from Orca, igor, or echo, it will forward these events to the webhooks registered.

A useful pattern is to have an intermediate service that will filter and convert events you care about into the format you expect.

Configuration in echo.yml

  1. rest:
  2.   enabled: true
  3.   endpoints:
  4.     -
  5.       wrap: false
  6.       url: http://listener.com
  7.     -
  8.       wrap: false
  9.       url: http://listener2.com

This will post every event to listener and listener2

Wrapping

If your endpoint expects events of the form

  1.     { "eventName" : "abc", "payload" : "[event json]"}

you can wrap the event via this configuration:

  1.   -
  2.       wrap: true
  3.       flatten: true
  4.       url: http://listener3
  5.       eventName: spinnaker_events
  6.       fieldName: payload

The flatten setting will simply make the json in content and details ( see below ) into a json String.

Event types

Here is an example event:

  1. {
  2.     "details": {
  3.       "source": "orca",
  4.       "type": "orca:task:complete",
  5.       "created": "1422487582294",
  6.       "organization": null,
  7.       "project": null,
  8.       "application": "asgard",
  9.       "_content_id": null
  10.     },
  11.     "content": {
  12.       "standalone": true,
  13.       "context": {
  14.         "asgName": "asgard-staging-v048",
  15.         "credentials": "test",
  16.         "deploy.account.name": "test",
  17.         "deploy.server.groups": {},
  18.         "kato.last.task.id": {
  19.           "id": "19351"
  20.         },
  21.         "kato.task.id": {
  22.           "id": "19351"
  23.         },
  24.         "kato.tasks": [
  25.           {
  26.             "history": [
  27.             ],
  28.             "id": "19351",
  29.             "resultObjects": []
  30.           }
  31.         ],
  32.         "notification.type": "enableasg",
  33.         "regions": ["us-west-1"],
  34.         "targetop.asg.enableAsg.name": "asgard-staging-v048",
  35.         "targetop.asg.enableAsg.regions": ["us-west-1"],
  36.         "user": "clin@netflix.com",
  37.         "zones": ["us-west-1a", "us-west-1c"]
  38.       },
  39.       "execution": ...
  40.       "executionId": "62ca5574-0629-419a-b9ac-fb873aa165b2",
  41.       "taskName": "f92239a7-b57a-408d-9d72-3a77484e050b.enableAsg.monitorAsg.9568e7e5-3c37-4699-9e93-f62118adc7c6"
  42.     }
  43.   }

Events have details, which will always be the same.

details.type

The type of the event will outline where the event is coming from:

  • orca:[task type]:[status] - where task type is either ‘pipeline’, ‘stage’ or ‘task’ and status is ‘starting’, ‘complete’, ‘failed’
  • build - from igor Jenkins events
  • docker - from igor Docker events
  • git - from git web triggers

details.application

The application involved in the execution, this will only exist for orca: type events.

content.execution

For orca type events, this is the current execution that has the entire pipeline execution.

Last modified September 3, 2021: Fix e-mail table (#140) (4b672f1)