Data-erp
Data Engineer
Intermediate
Remote
CBZ Technologies
80 000 $ - 110 000 $ (Permanent)
65 $ - 90 $ (Freelance)
Build data pipelines feeding analyses and AI products. Be the architect of data heritage.
Responsibilities
- Design and build robust and scalable data pipelines (ETL/ELT).
- Manage data warehouses and data lakes.
- Ensure data quality, reliability, and security.
- Collaborate with data scientists and analysts to prepare data.
Requirements
- Minimum THREE (3) years of experience as data engineer.
- Operated data lakes and DBs based on S3 or SQL (PostgreSQL, MySQL, Oracle).
- Developed data flows (ETL/ELT, Airflow, Streamsets, NiFi).
- Developed using common languages (Python, Java, Scala).
- Processed big data and distribution with tools like Spark, Presto, Dask, Parquet, ORC.
- Used tools like Linux, Containers, OpenShift, Kubernetes in DevOps mode.
- Developed stream processing with Spark, Kafka, etc.