Techniques: One-to-many pipeline triggers

My DevOps & QA teams use ChatOps and Azure Pipelines extensively. In doing so, they frequently encountered a bottleneck when triggering a handful of QA pipelines simultaneously during releases. Because the pipelines are independent, this required making repeated calls to the chatbot to trigger these suites. It would go something like this…

>> chatbot run the smoke tests
>> chatbot run the integration tests
>> chatbot run the API test suite
>> chatbot run the performance test suite
(etc)

The solution:

Since the pipelines are part of the same logical grouping, we created a new “upstream” pipeline to automatically trigger the “downstream” QA pipelines. The upstream pipeline was empty - it simply served as a trigger for the others downstream. To finish it off, we gave the upstream pipeline a sensible name to reflect its function and voila: the the “release signoff scripts” pipeline was born!

Now, we simply need to type:

>> chatbot run the release signoff scripts
(this triggers all the pipelines mentioned earlier)

Here’s are the steps:

1. Create an empty upstream pipeline. Note: the repo you choose is irrelevant, so pick something lightweight to minimize Agent utilization.

2. For each downstream pipeline, click Edit, then update the pipeline Triggers menu by adding a “Build Completion” trigger that references the name of the upstream pipeline. This is what kicks off the pipeline whenever the upstream pipelines runs.

3. Test it out!

Sidenote: in order to run N-pipelines simultaneously, you’ll need to provision an equal number of parallel jobs, as well as Test Agents in the Agent Pool(s) used by your downstream pipelines.

Takeaway

Pipeline chaining is nothing new, but rather than focus on 1:1 links, we’ve made great use of 1:many chains, especially when incorporated with ChatOps workflows. I hope this sparks some ideas for how you could leverage this on your team.