AddressJOB DESCRIPTION
Job Summary:
We are looking for a skilled Data Engineer to join our team. The ideal candidate will have a strong background in Python, SQL, and PySpark and Databricks with experience in developing and managing data pipelines and databases.
Experience Required- Minimum 4 years (Must), Please do not apply if you do not have relevant experience.
Responsibilities:
- Design, build, and maintain efficient and reliable data pipelines and databases.
- Develop and implement ETL processes to integrate data from various sources into a unified view.
- Work closely with data scientists and analysts to ensure the quality and accuracy of data.
- Troubleshoot data-related issues and provide solutions in a timely manner.
- Manage data security, privacy, and compliance requirements.
- Monitor performance and capacity of data pipelines and databases.
- Continuously improve Data Engineering processes and practices.
- Experience in Azure Log analytics, Data Shares, Data factory and Azure synapse is must.
Requirements:
- Bachelor’s or Master’s degree in Computer Science or a related field.
- 4+ years of experience in Data Engineering.
- Strong programming skills in Python and SQL.
- Experience with Spark and Databricks and related tools such as PySpark or Spark.
- Experience in Azure Log analytics, Data Shares, Data factory and Azure synapse is must.
- Familiarity with data modeling, database design, and data warehousing concepts.
- Proficiency in Linux and command-line tools.
- Excellent problem-solving skills and attention to detail.
