6+ years of ADF data flows/Snowflake. if you just worked on ADF pipelines, this is not for you.
An Azure Data Factory (ADF) Integration Developer is responsible for building, maintaining, and optimizing data pipelines and integration solutions using Microsoft Azure Data Factory. This role is key in data engineering and ETL (Extract, Transform, Load) processes in a cloud-based environment.
An Azure Data Factory (ADF) Integration Developer is responsible for building, maintaining, and optimizing data pipelines and integration solutions using Microsoft Azure Data Factory. This role is key in data engineering and ETL (Extract, Transform, Load) processes in a cloud-based environment.
✅ Key Responsibilities:
-
Design and Develop Data Pipelines
- Build end-to-end data workflows using ADF (v2) for batch and real-time data processing.
- Create and manage Linked Services, Datasets, Pipelines, Activities, and Triggers.
- Build end-to-end data workflows using ADF (v2) for batch and real-time data processing.
-
Data Integration
- Integrate data from multiple sources like SQL Server, Azure Blob Storage, Azure Synapse, REST APIs, SAP, Salesforce, etc.
- Handle structured and unstructured data using Data Flows, Mapping Data Flows, or custom transformations.
- Integrate data from multiple sources like SQL Server, Azure Blob Storage, Azure Synapse, REST APIs, SAP, Salesforce, etc.
-
ETL/ELT Development
- Implement data transformations using Azure Data Flows, Stored Procedures, or Azure Functions.
- Optimize performance and cost using best practices.
- Implement data transformations using Azure Data Flows, Stored Procedures, or Azure Functions.
-
Monitoring and Troubleshooting
- Set up pipeline monitoring, alerts, and logs using ADF Monitoring and Azure Monitor.
- Debug failures and optimize pipeline performance.
- Set up pipeline monitoring, alerts, and logs using ADF Monitoring and Azure Monitor.
-
CI/CD and DevOps
- Use Azure DevOps or Git integration for version control, build, and release pipelines.
- Implement ARM templates for infrastructure as code (IaC).
- Use Azure DevOps or Git integration for version control, build, and release pipelines.
-
Security and Governance
- Manage access through Azure RBAC, Key Vault, and Managed Identity.
- Ensure compliance and data security standards.
- Manage access through Azure RBAC, Key Vault, and Managed Identity.
-
Collaboration
- Work closely with Data Architects, Business Analysts, and BI developers.
- Translate business requirements into technical solutions.
- Work closely with Data Architects, Business Analysts, and BI developers.
🛠️ Required Skills and Technologies:
-
Azure Data Factory
-
Azure Synapse Analytics, Azure SQL
This is a remote position.
Compensation: $50.00 per hour
We are an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability status, protected veteran status, or any other characteristic protected by law.
(if you already have a resume on Indeed)