Address
Form of workRole- Solution Engineer - Data EngineeringExperience- 8 to 12 yearsOverview
Primary focus would be to do development work within the platform on Azure Data Lake environment and other related ETL technologies, satisfying project requirements, while adhering to enterprise architecture standards.Responsibilities:Assess and recommend architecture frameworks, design and implement high-performance solutions to support data and analytical products.
Act as a subject matter expert across different digital data projects.
Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.
Build and own the automation and monitoring frameworks that captures metrics and operational KPIs for data pipeline quality and performance.
Collaborate with internal clients (data science and product teams) to drive solutioning and POC discussions.Strong technical background in data science, business intelligence or Data Engineering a...
Primary focus would be to do development work within the platform on Azure Data Lake environment and other related ETL technologies, satisfying project requirements, while adhering to enterprise architecture standards.Responsibilities:Assess and recommend architecture frameworks, design and implement high-performance solutions to support data and analytical products.
Act as a subject matter expert across different digital data projects.
Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.
Build and own the automation and monitoring frameworks that captures metrics and operational KPIs for data pipeline quality and performance.
Collaborate with internal clients (data science and product teams) to drive solutioning and POC discussions.Strong technical background in data science, business intelligence or Data Engineering a...
Role- Solution Engineer - Data EngineeringExperience- 8 to 12 yearsOverview
Primary focus would be to do development work within the platform on Azure Data Lake environment and other related ETL technologies, satisfying project requirements, while adhering to enterprise architecture standards.Responsibilities:Assess and recommend architecture frameworks, design and implement high-performance solutions to support data and analytical products.
Act as a subject matter expert across different digital data projects.
Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.
Build and own the automation and monitoring frameworks that captures metrics and operational KPIs for data pipeline quality and performance.
Collaborate with internal clients (data science and product teams) to drive solutioning and POC discussions.Strong technical background in data science, business intelligence or Data Engineering and ETL best practices.Great Knowledge of Databricks Lakehouse and Azure DataLake conceptKnowledge of Data Bricks delta concept– Delta live tables (DLT)
Strong hands-on experience in ELT– pipeline development using Azure Data factory and Databricks Autoloader.
Strong knowledge of metadata-driven data pipeline, metadata management, dynamic logic
In-depth knowledge of data storage solutions, including Azure Data Lake Storage (ADLS), and Azure Serverless SQL Pool.
Experience with data transformation using Spark, and SQL technologies.
Solid understanding of design patterns, and best practices of the cloud stack.
Experience with code management and version control using Git or similar tools.
Strong problem-solving and debugging skills in ETL workflows and data pipelines.
Strong understanding of Azure Data bricks– features and capabilities.
Knowledge of Azure DevOps and continuous integration and deployment (CI/CD) process.
Knowledge of data quality and data profiling techniques, with experience in data validation and data cleansing.
Hands-on Duties:Conducting technical sessions, design reviews, code reviews, and demos of pipelines and their functionality
Developing technical specification for Data pipelines and workflow and getting sign-off from Architect.
Developing, deploying, and maintaining workflows and data pipelines using Azure Data bricks.
Collaborating with data architects, data analysts, and other stakeholders to design and implement ETL solutions that meet business requirements.
Writing efficient and high-performing ETL code using PySpark, and SQL technologies.
Building and testing data pipelines using Azure Data bricks.
Ensuring the accuracy, completeness, and timeliness of data being processed and integrated.
Troubleshooting and resolving issues related to data pipelines and notebooks.
Performance benchmarking of data ingestion and Data flow pipeline/notebook and ensuring consistency
Primary focus would be to do development work within the platform on Azure Data Lake environment and other related ETL technologies, satisfying project requirements, while adhering to enterprise architecture standards.Responsibilities:Assess and recommend architecture frameworks, design and implement high-performance solutions to support data and analytical products.
Act as a subject matter expert across different digital data projects.
Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.
Build and own the automation and monitoring frameworks that captures metrics and operational KPIs for data pipeline quality and performance.
Collaborate with internal clients (data science and product teams) to drive solutioning and POC discussions.Strong technical background in data science, business intelligence or Data Engineering and ETL best practices.Great Knowledge of Databricks Lakehouse and Azure DataLake conceptKnowledge of Data Bricks delta concept– Delta live tables (DLT)
Strong hands-on experience in ELT– pipeline development using Azure Data factory and Databricks Autoloader.
Strong knowledge of metadata-driven data pipeline, metadata management, dynamic logic
In-depth knowledge of data storage solutions, including Azure Data Lake Storage (ADLS), and Azure Serverless SQL Pool.
Experience with data transformation using Spark, and SQL technologies.
Solid understanding of design patterns, and best practices of the cloud stack.
Experience with code management and version control using Git or similar tools.
Strong problem-solving and debugging skills in ETL workflows and data pipelines.
Strong understanding of Azure Data bricks– features and capabilities.
Knowledge of Azure DevOps and continuous integration and deployment (CI/CD) process.
Knowledge of data quality and data profiling techniques, with experience in data validation and data cleansing.
Hands-on Duties:Conducting technical sessions, design reviews, code reviews, and demos of pipelines and their functionality
Developing technical specification for Data pipelines and workflow and getting sign-off from Architect.
Developing, deploying, and maintaining workflows and data pipelines using Azure Data bricks.
Collaborating with data architects, data analysts, and other stakeholders to design and implement ETL solutions that meet business requirements.
Writing efficient and high-performing ETL code using PySpark, and SQL technologies.
Building and testing data pipelines using Azure Data bricks.
Ensuring the accuracy, completeness, and timeliness of data being processed and integrated.
Troubleshooting and resolving issues related to data pipelines and notebooks.
Performance benchmarking of data ingestion and Data flow pipeline/notebook and ensuring consistency
