Data Engineer

Data Engineer
Role and Resposibilities :-
Develop programs in Scala, Java and Python as part of data cleaning and processing.
Conduct complex data analysis and report on results
Responsible to design and develop distributed, high-volume, high-velocity multi-threaded event processing systems using Core Java technology stack.
Prepare data for prescriptive and predictive modeling
Develop efficient software code for multiple use cases leveraging Core Java and Big Data technologies for various use cases built on the platform.
Provide high operational excellence guaranteeing high availability and platform stabilityImplement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Hadoop, Cloud computing etc.
Assist in designing, enhancing, and constructing new and existing features using performant, scalable, secure, documented, and maintainable code
Quickly produce well-organized, optimized, well-tested, and documented code
Meet deadlines and satisfy requirements.
Participate in agile ceremonies such as iteration planning, retrospective, and daily stand-ups
Their role typically falls into one of three categories: generalist, pipeline-centric, and database-centric.
Managing, optimizing, overseeing, and monitoring data retrieval, storage, and distribution.
Relevant experience with an emphasis on data integration in structured and unstructured formats such as flat files, JSON, and XML
Strong knowledge of OOP and reusable coding practices
Maintaining a data warehouse and analytics environment, and writing scripts for data integration and analysis
Process the data with Hive, Hadoop, and Spark.
Working knowledge of Python, serverless environments, microservices, and RESTful/SOAP APIs is a plus
Experienced using practices such as unit testing, integration testing, proper code documentation, and appropriate logging to develop highly maintainable and reliable code
Collaborate with other Engineers and Operations to design and implement features
Know how to build and maintain database systems, be fluent in programming languages such as SQL, Python, and R, be adept at finding warehousing solutions, and using ETL (Extract, Transfer, Load) tools, and understand basic machine learning and algorithms
Communicate effectively and efficiently with teammates and other departments
Develop reusable integrations, both inbound and outbound, between Cami.AI systems and other customer platforms.
Responsible to Ingest data from files, streams and databases. (Ref:www.freshersworld.com,freshersworld)