Job Opportunities in Egypt


October 17, 2024

TechiTopia

القاهرة

FULL TIME


Jd lead data engineer

Job Summary:
As the Lead Data Engineer, you will lead the data engineering team in building, testing, and optimizing sustainable data pipelines, architectures, and data lake platforms. You will collaborate closely with data architects and other stakeholders to ensure the seamless integration and transformation of data from various sources. Your role will be critical in setting the strategic direction for data engineering operations, ensuring data reliability, quality, and availability for business operations and advanced analytics.

Duties and Responsibilities

-Lead the design, development, and optimization of data engineering pipelines, integrating and streaming data from diverse sources using ETL/ELT processes.

  • Oversee the implementation of methods to improve data reliability and quality, ensuring that raw data is transformed into consistent, machine-readable formats.
  • Architect and develop data systems that enable efficient data extraction and transformation for predictive and prescriptive modelling.
  • Define and implement development, testing, release, update, and support processes for data engineering operations, including troubleshooting and bug fixing.
  • Develop and validate big data models and use cases, ensuring they are ready for data operations and accessible to end users.
  • Lead efforts to create and execute queries on structured and unstructured data sources to identify process issues or perform mass updates.
  • Ensure that batch production scheduling and report distribution processes are accurate, timely, and efficient.
  • Build and maintain processes that support data transformation, data structures, metadata management, and workload management.
  • Manage and prioritize ad hoc requests from users, including data research, file manipulation/transfer, and process issue resolution.


Job Requirements


Technical Skills
. Extensive hands-on experience in selecting and deploying appropriate CI/CD tools.
. Strong expertise in building continuous integration, continuous development, and deployment pipelines (CI/CD Pipeline).
. Advanced problem-solving, troubleshooting, and process-oriented skills.
. Proficiency in object-oriented/object function scripting languages such as Python, Java, C++, Scala, etc.
. Extensive experience with ETL/ELT technologies, including both streaming and batch processing.
. Proven experience in building and optimizing big data pipelines, architectures, and datasets.
. Hands-on experience with big data tools such as Hadoop, Hive, Impala, Spark, Kafka, etc.
. Expertise in stream-processing systems like Storm, Spark-Streaming, etc.
. Knowledge of the Cloudera Ecosystem is an advantage.
. Understanding of data models, data mining, and segmentation techniques is beneficial.

Qualifications and Experiences

Bachelor’s degree in Computer Science, Computer Engineering, or a related field.
Fluent in both written and spoken Arabic and English.
Preferred experience in leading or participating in large-scale Big Data projects.
7+ years of experience in data warehousing (DWH), ETL, and data engineering, with demonstrated leadership experience.


Skills

lead data engineer
Job poster
A1Softech company

Latest Job Opportunities


October 20, 2024

Impactyn
Senior Software Quality Control Tester

القاهرة

FULL TIME

View Details

October 20, 2024

Impactyn
Senior Backend Software Engineer (.net)

القاهرة

FULL TIME

View Details

October 20, 2024

Impactyn
IOS Developer

القاهرة

FULL TIME

View Details

October 20, 2024

Impactyn
Sales Executive

القاهرة

FULL TIME

View Details

October 20, 2024

Impactyn
Senior Front End Developer

القاهرة

FULL TIME

View Details

October 20, 2024

Impactyn
Android Developer

القاهرة

FULL TIME

View Details

October 20, 2024

EY
External Audit Manager - Cairo

القاهرة

View Details

October 20, 2024

EY
Assistant Manager - Forensic and Integrity Services - Egypt 1

القاهرة

View Details

Similar Jobs


September 10, 2024

PepsiCo
Production Shift Leader

الجيزة

View Details

September 11, 2024

PepsiCo
Sales District Leader Designate - Faisal

الجيزة

View Details

September 9, 2024

Vodafone
Business Development Sr. Lead

الجيزة

View Details

September 10, 2024

REI Call Center
Financial Lead

الإسكندرية

FULL TIME

View Details

August 25, 2024

Coca‑Cola HBC Austria GmbH
Health & Safety Lead

الإسكندرية

View Details

August 22, 2024

Coca‑Cola HBC Austria GmbH
Electrical Lead--Alex

الإسكندرية

View Details

New Jobs from This Company


October 17, 2024

TechiTopia
Jd lead data engineer

القاهرة

FULL TIME

View Details

October 17, 2024

TechiTopia
Senior Devops Engineer

القاهرة

FULL TIME

View Details

October 17, 2024

TechiTopia
Jd data architect

القاهرة

FULL TIME

View Details

October 17, 2024

TechiTopia
Principal laserfiche developer

القاهرة

FULL TIME

View Details

October 16, 2024

TechiTopia
Senior product owner

القاهرة

FULL TIME

View Details