Principal Consultant-Senior Data Engineer

Job Description

With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world&rsquos biggest brands&mdashand we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We&rsquore harnessing the power of technology and humanity to create meaningful transformation that moves us forward in our pursuit of a world that works better for people. Now, we&rsquore calling upon the thinkers and doers, those with a natural curiosity and a hunger to keep learning, keep growing., People who thrive on fearlessly experimenting, seizing opportunities, and pushing boundaries to turn our vision into reality. And as you help us create a better world, we will help you build your own intellectual firepower.

Welcome to the relentless pursuit of better.

Inviting applications for the role of Principal Consultant, Senior DataEngineer!

In this role, the Senior Data Engineer is responsible for providing technical direction and lead a group of one or more developer to address a goal.

Responsibilities:

.Design, develop, and automate ETL processes using DBT and AWS

.Build robust data pipelines to move data from various sources to data warehouses or data lakes.

.Collaborate with cross-functional teams to ensure data accuracy, completeness, and consistency.

.Perform data cleansing, validation, and transformation to ensure data quality and integrity.

.Optimize database and query performance to ensure efficient data processing.

.Work with data analysts and data scientists to provide clean, reliable data for analysis and modelling.

.Writing SQL queries against Snowflake, developing scripts to do Extract, Load, and Transform data.

.Hands-on experience with Snowflake utilities such as SnowSQL, SnowPipe, Tasks, Streams, Time travel, Cloning, Optimizer, Metadata Manager, data sharing, stored procedures and UDFs.

.Have experience with Snowflake cloud data warehouse and AWS S3 bucket or Azure blob storage container for integrating data from multiple source system

.Should have good experience in Python/Pyspark.integration with Snowflake and cloud(AWS/Azure)

.Sound understanding of ETL tools / Data integration techniques.

.Work with business stakeholders to understand data requirements and develop ETL solutions

.Strong programming skills in languages such as Python, Java, and/or Scala.

.Experience with big data technologies such as Kafka.

.Experience with cloud computing platforms such as AWS.

.Familiarity with database technologies such as SQL, NoSQL, and/or Graph databases.

.Experience in requirement gathering, analysis, designing, development, and deployment.

.Should Have experience building data ingestion pipeline

.Good to have experience in deployment using CI/CD tools: Azure boards, Github and write automated test cases wherever applicable.

.Have experience in client facing projects.

.Have experience on Snowflake Best Practices

Qualifications we seek in you !

Minimum qualifications

.B.E./ Masters in Computer Science, Information technology, or Computer engineering or any equivalent degree with IT experience and relevant working experience as Senior Snowflake Data Engineer.

Preferred Qualifications/Skills

.Experience in IT industry

.Working experience with building productionized data ingestion and processing data pipelines in Snowflake

.Fully well-versed with data warehousing concepts.