Primary Skills: Databricks, Azure Data Factory, Azure Function App, PySpark, Python, SQL, GitHub Job Summary We are seeking an experienced Data Engineer Lead with strong expertise in Azure cloud data engineering technologies, scalable ETL pipeline development, and modern big data processing frameworks. The ideal candidate will be responsible for designing, developing, optimizing, and leading enterprise-grade data engineering solutions using Databricks, Azure Data Factory, Azure Function Apps, PySpark, Python, and SQL. The candidate should possess hands-on experience building scalable batch and real-time data pipelines, implementing cloud-native architectures, managing large datasets, and collaborating with cross-functional teams to deliver reliable and high-performance data solutions. Key Responsibilities Data Pipeline Development Design, develop, and maintain scalable ETL/ELT pipelines using: Azure Databricks Azure Data Factory (ADF) PySpark Python Build efficient data ingestion frameworks for structured and unstructured data sources. Azure Cloud Data Engineering Develop cloud-native data solutions on Microsoft Azure. Work extensively with: Azure Databricks Azure Function Apps Azure Data Lake Azure Blob Storage Azure SQL Azure Synapse Analytics Implement secure, scalable, and high-performance data architectures. Big Data Processing Develop and optimize distributed data processing workflows using PySpark. Handle large-scale datasets with focus on: Performance optimization Cost efficiency Scalability Reliability Data Transformation & Integration Develop complex SQL transformations and data models. Integrate data from multiple enterprise systems, APIs, and third-party sources. Ensure data quality, consistency, and governance standards. Automation & CI/CD Manage code repositories and deployment workflows using GitHub. Implement CI/CD best practices for data engineering deployments. Automate operational workflows and monitoring processes. Leadership & Collaboration Lead and mentor junior data engineers. Collaborate with: Architects Product teams Business analysts DevOps teams Stakeholders Participate in design discussions and technical decision-making. Monitoring & Optimization Monitor pipeline performance and troubleshoot failures. Optimize Spark jobs, SQL queries, and data workflows. Implement logging, alerting, and operational monitoring mechanisms. Strong Hands-on Experience With Required Technical Skills Must-Have Skills Azure Data Engineering Azure Databricks Azure Data Factory (ADF) Azure Function Apps Azure Data Lake Azure Blob Storage Programming & Processing Strong Expertise In PySpark Python SQL Data Engineering Experience With ETL/ELT pipelines Batch processing Data transformation Data integration Data warehousing Version Control & CI/CD Hands-on Experience With GitHub Git workflows CI/CD pipelines Cloud & Big Data Experience Working With Distributed data processing Large-scale datasets Cloud-native architectures Skills: azure,pyspark,python,sql