Data Engineer

This role is for one of the Weekday's clients

Salary range: Rs 500000 - Rs 1000000 (ie INR 5-10 LPA)

Experience: 3+ yrs

Location: Hyderabad

Job Type: full-time

We are looking for a highly motivated and experienced Data Engineer with strong expertise in Snowflake, Data Warehousing, Python, and ETL/ELT development. The ideal candidate will be responsible for designing, developing, and maintaining scalable data platforms that enable efficient data processing, reporting, and analytics across the organization. This role requires a deep understanding of modern cloud-based data architectures, data integration frameworks, and performance optimization techniques to support business intelligence and data-driven decision-making.

As a Data Engineer, you will work closely with business stakeholders, analysts, data architects, and development teams to build reliable and high-performing data solutions. You will play a key role in developing end-to-end data pipelines, ensuring data quality, implementing robust data models, and optimizing Snowflake environments for performance and cost efficiency. The position offers an opportunity to work with large-scale datasets, modern cloud technologies, and enterprise-grade analytics platforms in a fast-paced and collaborative environment.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines to ingest, transform, and load structured, semi-structured, and unstructured data into Snowflake.
• Build and manage data ingestion workflows using Azure Data Factory (ADF), Python, SQL, and other integration technologies.
• Develop robust data transformation processes that ensure data consistency, accuracy, and reliability across multiple systems.
• Write, maintain, and optimize complex SQL queries, stored procedures, views, tasks, streams, and user-defined functions (UDFs) within Snowflake.
• Monitor and enhance query performance through advanced optimization techniques such as clustering keys, partition pruning, materialized views, caching mechanisms, and warehouse tuning.
• Manage Snowflake warehouse resources effectively to balance system performance, scalability, and operational costs.
• Design and implement logical and physical data models that support reporting, business intelligence, self-service analytics, and advanced analytical workloads.
• Develop dimensional models including star schemas, snowflake schemas, normalized, and denormalized structures based on business requirements.
• Collaborate with cross-functional teams to understand data requirements and translate them into scalable technical solutions.
• Implement ETL best practices, including incremental loading, change data capture (CDC), auditing, reconciliation, error handling, and performance monitoring.
• Establish data quality controls and validation processes to ensure the integrity and accuracy of enterprise data assets.
• Troubleshoot and resolve data pipeline failures, performance bottlenecks, and production issues in a timely manner.
• Create and maintain technical documentation for data flows, architecture designs, data models, and operational procedures.
• Support continuous improvement initiatives by identifying opportunities for automation, optimization, and process enhancement.
• Ensure adherence to data governance, security, compliance, and best practices across all data engineering activities.

What Makes You a Great Fit

• Strong hands-on experience with Snowflake and cloud-based data warehouse technologies.
• Proven expertise in designing, developing, and supporting enterprise-scale ETL/ELT solutions.
• Advanced SQL skills with experience in query optimization, performance tuning, and database development.
• Proficiency in Python for data processing, automation, scripting, and pipeline development.
• Experience working with Azure Data Factory (ADF) or similar cloud-based data integration platforms.
• Solid understanding of data warehousing concepts, dimensional modeling, and database design principles.
• Knowledge of incremental loading strategies, ETL auditing, data validation, and data quality management techniques.
• Experience optimizing Snowflake environments for scalability, performance, and cost efficiency.
• Strong analytical and problem-solving abilities with attention to detail and a focus on delivering high-quality solutions.
• Ability to work effectively in cross-functional teams and communicate complex technical concepts to both technical and non-technical stakeholders.
• Familiarity with Agile development methodologies and modern software engineering best practices.
• Self-motivated mindset with the ability to manage multiple priorities in a dynamic and fast-paced environment.