Data Engineer - DaAI

About the Role

As a Data Engineer - DaAI at Infosys Limited, you will be responsible for designing, developing, and maintaining large-scale data systems and pipelines. Your primary focus will be on data ingestion, processing, and storage, ensuring seamless data flow across various platforms. You will work closely with cross-functional teams to identify data requirements and implement efficient solutions.

Key Responsibilities

• Design and develop data pipelines using Apache Spark, Airflow, and AWS Glue to ensure efficient data processing and storage.
• Develop and maintain ETL/ELT processes using Python, SQL, and dbt to extract, transform, and load data into various data warehouses and lakes.
• Collaborate with data scientists and analysts to understand data requirements and implement data solutions using PySpark and Dagster.
• Develop and maintain data quality checks and data governance policies to ensure data accuracy and compliance.
• Work with cloud-based data platforms to design and implement scalable data architectures.
• Monitor and optimize data pipeline performance to ensure high-quality data delivery.
• Stay up-to-date with emerging data technologies and trends to recommend innovative solutions to the team.

Skills & Qualifications

• Strong foundation in Python programming language and experience with data engineering frameworks like Apache Spark, Airflow, and AWS Glue.
• Proficiency in SQL and experience with data modeling, data warehousing, and data governance.
• Experience with ETL/ELT tools like dbt and ability to design and develop efficient data pipelines.
• Knowledge of cloud-based data platforms like AWS and Azure, with experience in designing and implementing scalable data architectures.
• Bachelor's degree in Computer Science, Information Technology, or related fields.
• Strong problem-solving skills, attention to detail, and ability to work in a fast-paced environment.
• Good communication and collaboration skills, with experience working in cross-functional teams.

What You'll Learn

As a Data Engineer - DaAI at Infosys Limited, you will have the opportunity to learn and work with cutting-edge data technologies and tools. You will gain hands-on experience in designing and developing large-scale data systems and pipelines, and will work closely with cross-functional teams to identify data requirements and implement efficient solutions. This role will help you develop strong problem-solving skills, attention to detail, and ability to work in a fast-paced environment.

Resume Tip

When applying for this role, make sure to highlight your experience with data engineering frameworks like Apache Spark, Airflow, and AWS Glue. Also, include any relevant projects or experiences where you have designed and developed data pipelines, and highlight your proficiency in SQL and data modeling.