AWS Data Engineer

This role is for one of the Weekday's clients

Min Experience: 8+ years

Location: Gurgaon

JobType: full-time

We are looking for an experienced and highly skilled AWS Data Engineer – Lead to join our growing data engineering team. The ideal candidate will have deep expertise in designing, building, and optimizing scalable cloud-based data platforms on AWS. This role requires strong hands-on experience with AWS data services, PySpark, and SQL, along with the ability to lead technical initiatives, mentor team members, and collaborate with cross-functional stakeholders to deliver high-quality data solutions.

As an AWS Data Engineer Lead, you will play a critical role in architecting modern data pipelines, enabling efficient data processing, and supporting analytics and business intelligence initiatives across the organization. You will work closely with data architects, analysts, data scientists, and application teams to create reliable and scalable data ecosystems.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using AWS cloud technologies.
• Build and optimize large-scale data processing frameworks using PySpark and distributed computing methodologies.
• Develop robust data ingestion and transformation workflows using AWS Glue, Lambda, and other AWS-native services.
• Manage and optimize data storage and warehousing solutions using Amazon Redshift and Amazon S3.
• Create efficient SQL queries, stored procedures, and performance tuning strategies for large datasets.
• Lead end-to-end implementation of cloud-based data engineering solutions and ensure adherence to best practices.
• Collaborate with business stakeholders and technical teams to understand data requirements and translate them into scalable solutions.
• Ensure data quality, integrity, governance, and security across the data platform.
• Monitor, troubleshoot, and enhance existing data pipelines for improved reliability and performance.
• Participate in architecture discussions and contribute to cloud modernization initiatives.
• Mentor junior and mid-level engineers by providing technical guidance, code reviews, and best practices.
• Work in Agile development environments and contribute to sprint planning, estimations, and delivery timelines.

Required Skills

Must-Have Skills

• Strong experience as an AWS Data Engineer with leadership responsibilities.
• Expertise in AWS Cloud services including:
• AWS Glue
• Amazon Redshift
• Amazon S3
• AWS Lambda
• Strong hands-on experience in PySpark for large-scale data processing.
• Advanced SQL skills with experience in query optimization and performance tuning.
• Experience in building scalable ETL pipelines and data integration solutions.
• Strong understanding of data warehousing concepts and distributed data processing.
• Experience working with structured and unstructured datasets.
• Good understanding of cloud-native architectures and best practices.

Preferred Qualifications

• Experience with workflow orchestration tools such as Apache Airflow.
• Familiarity with CI/CD pipelines and DevOps practices in cloud environments.
• Knowledge of data governance, security, and compliance standards.
• Exposure to real-time or streaming data processing frameworks is an added advantage.
• AWS certifications such as AWS Certified Data Analytics or AWS Certified Solutions Architect are preferred.

Experience & Education

• Experience: 8 – 12 years in Data Engineering with significant exposure to AWS cloud technologies.
• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field.

Desired Candidate Profile

The ideal candidate should possess strong analytical and problem-solving abilities, excellent communication skills, and the capability to lead technical discussions and delivery initiatives. A proactive mindset, ownership attitude, and passion for building scalable cloud-based data solutions are essential for success in this role.