Position: Adobe CDP Data Engineer
Location : Toronto (4 days a week at work)
Adobe CDP(Customer Data Platform)
About the Role:
We are looking for a Senior Data Engineer with deep experience in Google Cloud Platform (GCP) and its native data engineering toolset, including Airflow (Cloud Composer), dbt, and BigQuery. In this role, you will design, build, and optimize scalable data pipelines and data models to support advanced analytics, business intelligence, and data science initiatives.
As a senior member of the team, you will influence architecture decisions, mentor junior engineers, and collaborate with cross-functional teams to deliver high-quality, reliable data solutions that empower data-driven decision-making across the organization.
Key Responsibilities:
- Data Pipeline Development:
- Design, develop, and maintain scalable ETL/ELT workflows using Apache Airflow (Cloud Composer) and dbt.
- Build batch and streaming pipelines for ingesting, processing, and transforming large-scale datasets.
- Data Warehousing & Modeling:
- Architect and optimize data warehouses and marts in BigQuery using best practices for performance, cost-efficiency, and security.
- Develop and maintain data models (dimensional, star/snowflake schemas) to support analytics and reporting.
- Cloud-Native Architecture:
- Leverage GCP services (Pub/Sub, Dataflow, Dataproc, Cloud Storage, Bigtable, etc.) for data processing and integration.
- Ensure solutions align with cloud-native architecture principles and best practices.
- Performance & Quality:
- Implement data validation, testing, and monitoring to ensure accuracy and reliability of pipelines.
- Optimize query performance and manage costs through partitioning, clustering, and materialized views.
- Collaboration & Leadership:
- Partner with analysts, data scientists, and application teams to understand data needs and deliver fit-for-purpose solutions.
Mentor junior engineers and contribute to establishing coding standards, best practices, and governance policies.
-
Qualifications & Experience:
Required:
- 5+ years of experience as a Data Engineer, with at least 3 years in GCP environments.
- Expert-level proficiency in BigQuery, Apache Airflow (Cloud Composer), and dbt.
- Strong skills in SQL and one or more programming languages (Python preferred).
- Proven experience building scalable, production-grade ETL/ELT pipelines in cloud environments.
- Knowledge of cloud storage formats and optimization (Parquet, Avro, ORC, etc.).
Understanding of data governance, security, and compliance requirements.
-
Preferred:
- Experience with streaming data pipelines using Pub/Sub and Dataflow.
- Exposure to machine learning workflows and integration with Vertex AI.
- Familiarity with CI/CD pipelines for data engineering (GitLab, Cloud Build, etc.).
Experience in performance tuning and cost optimization for BigQuery and GCP data services.
-
Key Skills & Competencies
- Strong problem-solving and analytical skills.
- Ability to work independently and lead initiatives with minimal supervision.
- Excellent communication skills for both technical and business audiences.
- Passion for building reliable, scalable, and maintainable data systems.
