Address We are seeking an experienced Azure Data Engineer with specialized skills in Azure Data Explorer (ADX) to join our team at InnoWave. The ideal candidate will have a strong background in Data Engineering and expertise in designing, implementing, and managing data solutions on the Azure platform, particularly utilizing ADX. As an Azure Data Engineer, you will play a key role in developing and maintaining data pipelines, optimizing data storage and retrieval, and driving insights and analytics for our clients.
Responsibilities:
Responsibilities:
Data Engineering:
- Design, develop, and implement scalable and efficient data pipelines on the Azure platform, utilizing Azure Data Explorer (ADX) and other Azure services.
- Build and maintain data ingestion, transformation, and processing workflows to ensure reliable and timely data delivery.
- Optimize data storage, retrieval, and query performance for large-scale data sets using ADX query optimization techniques.
- Collaborate with cross-functional teams to integrate data solutions with existing systems and applications.
Data Analysis and Insights:
- Utilize ADX and other analytical tools to drive insights and analytics from structured and unstructured data sources.
- Develop and maintain dashboards, reports, and visualizations to communicate key metrics and findings to stakeholders.
Monitoring and Optimization:
- Monitor data pipelines and systems for performance, reliability, and security, and implement proactive measures to ensure high availability and data integrity.
- Continuously optimize data processing and storage solutions to improve efficiency and reduce costs.
Documentation and Collaboration:
- Document technical specifications, design decisions, and codebase for reference and knowledge sharing.
- Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver data solutions that meet business needs.
Qualifications:
- Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience).
- Minimum of 3-5 years of experience as a Data Engineer, with specific expertise in Azure Data Explorer (ADX).
- Proficiency in Azure services, including Azure Data Factory, Azure Databricks, Azure Synapse Analytics, etc.
- Strong understanding of data modeling, ETL processes, and data warehousing concepts.
- Experience with data visualization tools such as Power BI, Tableau, etc.
- Excellent problem-solving skills and attention to detail.
- Effective communication skills and ability to work collaboratively in a team environment.
