AddressJob Purpose: Design, Develop, and maintenance of ETL processes to move and transform data from source systems to target systems.
Key Roles and Responsibilities:
1. Design, develop, and implement ETL processes to extract, transform, and load data from source systems to target databases, data warehouses, or data lakes.
2. Develop, maintain and automate data integration workflows to ensure that data is accurately and efficiently transferred between systems.
3. Utilize ETL tools and technologies (e.g., Informatica, Talend, Apache Nifi, Apache Spark) to implement ETL processes.
4. Utilize scripting languages (Python, Bash Javascript, etc) and code editing environement (Jupyter Notebook, Viso etc.) for data extraction from Web API.
5. Build a data warehouse for the organization's internal departments utilizing various data warehousing techniques.
6. Implement data quality checks and validation processes to identify and rectify data anomalies, errors, and inconsistencies.
7. Monitor ETL jobs and workflows for errors, performance issues, and data processing bottlenecks.
8. Document ETL processes, data flow diagrams, and transformation logic for future reference and auditing.
9. Collaborate with data analysts, data scientists, database administrators, and other stakeholders to understand data requirements and ensure data integration meets SALIC business needs.
10. Ensure data security and compliance with data protection regulations during the ETL process.
11. Perform any other duties required from the Line Manager.
Qualifications :
- Bachelor's Degree in Computer Science, Information Technology, or similar fields.
- Extensive knowledge of SQL and other languages, including Python, JavaScript etc.
- Proficiency in warehousing architecture techniques, including MOLAP, ROLAP, ODS, DM, and EDW.
- Knowledge of at least one ETL tool (Informatica, SSIS, Talend, etc.)
- Knowledge of different SQL/NoSQL data storage techniques and Big Data technologies.
- Experience with data warehouse, ETL and ELT.
- Passionate about sophisticated data structures and problem solutions.
- Ability to analyse a company's data needs.
- Quickly learn new data tools and ideas.
- Clear communication skills.
- Database and ETL Tools certifications desirable but not mandatory.
