Pyspark Developer - Big Data Tools

About the role :You will work on teams building a variety of Big Data analytics solutions including Big Data lakes. More specifically, you will work on:Scalable data ingestion pipelines to handle real time streams, CDC events, and batch data High-performance data processing for structured and unstructured data, and data harmonizationScheduling, orchestrating, and validating pipelinesException handling and log monitoring for debuggingCollaborate with business consultants, data scientists, engineers, and application developers to develop analytics solutionsJob Requirement :Hadoop ecosystem - HDFS, Hive, Sqoop, Kafka, ELK Stack etcSpark, Scala, Python and core/advance JavaNOSQL databases e.g. Hbase, Cassandra, MongoDBRelevant AWS or Azure components required to build Big Data solutionsGood to know: Databricks, SnowflakeAbility to develop and manage scalable Hadoop cluster environmentsGood understanding of data warehousing concepts, distributed systems, data pipelines, ETL3+ years of professional experience with at least 2 years in Big Data engineering (ref:hirist.com)