Notre culture vous élève - il n'y a pas d'ego sur le chemin. Notre objectif commun? Nous voulons tous gagner pour nos clients. Nous visons à être toujours évolutifs, dynamiques et ambitieux. Nous croyons au pouvoir des connexions authentiques. Chaque employé fait partie de ce qui nous rend uniques sur le marché: agiles et dévoués.
Régulier
POSITION SUMMARY
We aren't looking for an all-knowing AI guru to define our entire organizational strategy from scratch. We are looking for a sharp, curious Specialist, QA Test Automation who can bridge the gap between strategic vision and day-to-day technical execution. Working closely with the Chapter Lead, QA — who provides the overarching AI QA strategy—you will be embedded within Cogeco’s AI squads to bring that strategy to life.
In this role, you will move past standard "pass/fail" scripts to tackle the complex world of non-deterministic outputs. You will focus heavily on probabilistic testing, curating golden datasets, and leveraging cutting-edge LLM evaluation tools like Promptfoo. If you have a solid foundation in QA or software engineering, a healthy obsession with automation, and a restless drive to learn, you’ll fit right in.
-
Embrace the Unpredictable: Implement and refine testing workflows tailored for AI powered solutions where outputs are probabilistic rather than deterministic.
-
Regression & Drift Detection: Build guardrails to identify model drift, prompt regression, hallucination risks, and safety violations before they hit production.
-
Golden Datasets: Own the creation, maintenance, and scaling of high-quality golden datasets used as the source of truth for benchmarking model changes.
-
AI Eval Tooling: Champion programmatic evaluation tools such as Promptfoo, DeepEval, or Ragas to automate the assessment of prompt variants and model performance.
-
Chapter Alignment: Partner closely with the Chapter Lead, QA to translate high-level AI QA frameworks into actionable, daily tasks for technical teams.
-
Feedback Loop: Provide real-time feedback to the Chapter Lead on how the strategy is performing in the trenches, recommending practical pivots based on real-world results.
-
Smart Pipelines: Collaborate with DevOps specialists to inject AI evaluation metrics directly into modern CI/CD pipelines, ensuring automated gates handle fluid AI responses.
-
Data Validation: Validate data pipelines feeding our models, ensuring that data integrity is maintained from ingestion to inference.
-
College diploma or Degree in Computer Science, Engineering, or a related discipline (or equivalent practical experience).
-
The Right Mindset: A strong foundation in core QA principles combined with a passion for AI/ML. You don't need a PhD; you just need to be eager to learn how LLMs behave.
-
3+ years of experience in Quality Assurance, Software Engineering, or Data Engineering.
-
Hands-on Scripting: Experience in Python or JavaScript/TypeScript (essential for configuring evaluation frameworks like Promptfoo and manipulating test data).
-
Modern QA Ecosystem: Experience with test automation frameworks, API testing, and code repositories (Git).
-
Exposure to Golden Datasets: creating, maintenance, and/or scaling of high-quality golden datasets used as the source of truth for benchmarking model changes.
-
Expsosure to AI Eval Tooling: Champion programmatic evaluation tools such as Promptfoo, DeepEval, or Ragas to automate the assessment of prompt variants and model performance.
-
CI/CD Familiarity: Experience working with pipeline tools (e.g., Bitbucket pipelines, Google Cloud run, GitHub Actions, GitLab CI, Jenkins).
-
Analytical Deep-Dives: Comfortable analyzing data inputs/outputs and debugging prompts to identify the root causes of unexpected AI behavior.
-
Tool Adaptability: Ready to quickly learn, test, and implement new open-source AI evaluation and observability tooling as the ecosystem evolves.
-
Comfortable with Ambiguity: Thrives in a high-uncertainty environment where standard QA blueprints don’t always apply.
-
Execution Focus: Excel at taking an established roadmap or framework and running with it, troubleshooting technical blockers independently.
-
Squad Partner: Able to communicate technical testing gaps clearly to both AI developers and product owners.
-
Open to constructive criticism: Open to constructive feedback, willing to experiment, and values an environment where mistakes are treated as learning milestones.
You’ll benefit from:
-
Flexibility: Yes, we think that what you do matters. At work and at home.
-
Fun: We laugh a lot, it makes every day brighter.
-
Discounted services: We provide amazing services to our clients, and you’ll get them at home, because you deserve them.
-
Rewarding Pay: Let's be honest, everybody likes to make a good salary. We offer attractive compensation packages, and it comes with a great culture.
-
Benefits: We’ve got you covered.
-
Career Evolution: Join us and we will give you the tools to achieve your career goals!
-
Technology: You have a passion for technology? Excellent, we do too. Here, you will manage, influence, play, create, fix, and shape the industry.
#LI-KN1
#LI-Hybrid
Burlington, ON
Cogeco Connexion Inc
Chez Cogeco, nous savons que des origines, des points de vue et des croyances différentes peuvent apporter une valeur inestimable à notre entreprise. Cette diversité renforce notre capacité à imaginer, à innover et à nous développer en tant qu'entreprise. Nous nous engageons donc à faire tout ce qui est en notre pouvoir pour créer un milieu d'appartenance plus diversifié et inclusif.
En créant une culture où tous nos collègues peuvent donner le meilleur d'eux-mêmes au travail, nous faisons notre part pour construire un lieu de travail et un monde plus équitables. Du développement professionnel à la sécurité personnelle, Cogeco s'efforce constamment de créer un environnement qui accueille et enrichit tout le monde. Nous faisons de la santé et du bien-être de nos collègues l'une de nos plus grandes priorités, car nous savons que des employés engagés et appréciés sont synonymes d'une meilleure expérience globale pour nos clients.
Si vous avez besoin de mesures d'adaptation, quelles qu’elles soient, en ce qui concerne notre processus de recrutement, veuillez nous contacter en toute confidentialité à [email protected]
