Posting ADF Processing Statues in different ways

What is a ADF and Why Do You Need to Monitor?

Azure Data Factory is Azure's cloud ETL service for scale-out serverless data integration and data transformation. It offers a code-free UI for intuitive authoring and single-pane-of-glass monitoring and management. The Data Factory service provides reliable and complete view of your storage, processing, and data movement services. It helps you quickly assess end-to-end data pipeline health, pinpoint issues, and take corrective action if needed. You can also visually track data lineage and the relationships between your data across any of your sources, and see a full historical accounting of job execution, system health, and dependencies from a single monitoring dashboard.

This blog describes various options of how to create alerts and get notified on failures:

Three different Ways to receive Alert Notification from the Azure Data Factory Pipeline on Failures:

1. Sending alert notifications via Teams Channel

Create a new Microsoft Teams monitoring channel.
Create a webhook connector, the connector will generate a URL. This URL is the URL used inside ADF to send the alert to Microsoft Teams.
To enable notifications, click on Channel notifications --> All Activities.
Create a Alert Notification Pipeline and use the URL and body message as script to use for the error messages which is passed as Params.
This Alert message also can be customized with actionable buttons.
Other ways to send Alerts to Teams are using either Azure Functions or Logic Apps to parse the message and send it to the Incoming Webhook.

2. Sending alert notifications via Slack Channel

Create a new Microsoft Slack monitoring channel.
Create a webhook connector, the connector will generate a URL. This URL is the URL used inside ADF to send the alert to Slack Channel.
To enable notifications, click on Channel notifications --> All Activities.
Create a Alert Notification Pipeline and use the URL and body message as script to use for the error messages which is passed as Params.

3. Sending alert notifications via Email

In the Logic app open the designer and look for When a HTTP request is received, use POST method and add the schema of the body in JSON
Add a new step to Logic App and send an email using Send email (V2)
Configure the email related settings like TO/FROM/Subject/Pipeline properties and error message.
Create a Send Email Notification Pipeline as Web Activity. In the settings, URL, configure the Logic app url.

There are certainly other ways of monitoring the processes of the ADF pipelines in Azure Insights; logs etc, which can be leveraged by the team.