Our client is seeking a Data Engineer to join the Business Intelligence Big Data Team. In this newly created role, you will build high-performance data pipelines on the Google Cloud Platform (GCP), collaborating with Data Scientists and stakeholders to enhance analytic capabilities.
Core Responsibilities
- Design and develop high-performance data processing and ingestion frameworks.
- Partner with Business Analysts to expand data coverage and improve analytic depth.
- Deliver clean, defect-free code, participate in peer reviews, and provide post-deployment support.
- Research and recommend emerging technologies to optimize the Big Data environment.
- Independently resolve complex development issues within a large-scale IT landscape.
Technical Requirements
- GCP Expertise: Hands-on experience with BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Composer.
- Proficiency in Apache Spark (using Scala or PySpark).
- 3+ years of experience with mainstream languages such as Python, Java, or Scala.
- Demonstrated experience developing scalable Big Data ingestion frameworks.
- Experience in writing ETLs for data ingestions from multiple enterprise data sources / databases.
Key Competencies
- Strong analytical skills specifically tailored to "Big Data" environments.
- Ability to take initiative and learn new technical standards and processes quickly.
- Motivation to work independently on multifaceted, complex projects.
Pay: $45.00-$55.00 per hour
Work Location: Hybrid remote in Mississauga, ON
