What is the opportunity?
AI Systems and Models Testing
Design and execute comprehensive test strategies for AI systems and models, including prompt engineering, output evaluation, and bias/safety testing. Develop deep understanding of LLM behavior—tokenization, embeddings, attention mechanisms, and inference—to anticipate failure modes. Construct effective prompts, recognize hallucinations and off-target outputs, and assess quality across accuracy, tone, coherence, and bias dimensions. Apply evaluation metrics specific to generative AI and establish appropriate thresholds.
What will you do?
Test AI systems integrated with RAG pipelines and knowledge bases, validating data quality and retrieval accuracy as they impact model outputs. Understand vector database mechanics, similarity search thresholds, embedding drift, and test edge cases including near-duplicate documents, sparse vs. dense embeddings, and performance under scale. Leverage LangChain and LangGraph frameworks to read code, understand chain and graph construction, identify failure points, and write test harnesses. Validate integration points using MCPs, testing tool availability and error handling.
Test Strategy and Planning
Define and execute comprehensive test strategies for different platforms, ensuring coverage across functional, regression, integration, and performance testing. Establish testing standards and best practices that span both traditional QA and AI-specific validation.
Test Automation and Framework Development
Design, build, and maintain automated test suites to accelerate release cycles and improve coverage. Leverage AI and ML tools to enhance test coverage, improve efficiency, and reduce regression cycles.
QA Lifecycle Validate end-to-end deal workflows including setup, structuring, processing, and distributions. Ensure data integrity across upstream and downstream systems through reconciliation testing and reporting.
Release and Regression Testing
Coordinate regression testing for platform releases, patches, and infrastructure changes. Ensure stability and backward compatibility, particularly during critical processing windows.
Cross-Functional Collaboration
Partner with developers, business analysts, and product owners to clarify requirements, identify edge cases, and ensure testability of new features. Lead defect triage sessions, prioritize issues based on business impact, and maintain clear documentation through to closure.
Quality Leadership and Reporting
Define and monitor key quality indicators including defect density, test coverage, and automation rates. Present findings to leadership and recommend improvements. Mentor junior QA team members and foster a culture of quality across the team.
Production Support Provide production support during critical processing windows, investigate incidents, and coordinate root cause analysis and remediation efforts.
What You Need to Succeed
Must-Have:
7+ years of quality assurance or quality engineering experience, with at least 3 years in a lead or senior capacity. Strong domain knowledge in Capital Markets or similar asset classes.
Technical Skills
Hands-on experience with test automation tools (Selenium, Robot Framework, Playwright, or similar). Proficiency in programming languages including Java and Python, with demonstrated framework implementation expertise. Hands-on API automation and backend system validation experience. Proficiency in database query development, data validation, and reconciliation testing. Experience with CI/CD pipelines and DevOps practices (Jenkins, GitHub, or similar).
AI and ML Competencies
Core understanding of LLM architecture and behavior. Hands-on experience with LangChain and/or LangGraph frameworks. Knowledge of RAG pipelines, vector databases, and agentic solutions. Familiarity with Model Context Protocols (MCPs) and integration testing. Understanding of bias, safety, and red-team testing methodologies for AI systems.
Soft Skills and Mindset
Demonstrated passion for automation, continuous learning, and solving open-ended problems. Excellent analytical, problem-solving, and communication skills. Ability to manage multiple priorities in a fast-paced, deadline-driven environment. Ability to mentor and lead teams while maintaining focus on quality excellence.
What’s in it for you?
Leaders who support your development through coaching and managing opportunities
Work in a dynamic, collaborative, progressive, and high-performing team
Opportunities to do challenging work
Opportunities to take on progressively greater accountabilities
Access to a variety of job opportunities across business and geographies
Education
Bachelor’s degree in computer engineering, Computer Science, or equivalent professional experience
Industry certifications (ISTQB, CSTE) a plus
#LI-ASPost
#TECHPJ
Job Skills
Application Testing, Decision Making, Detail-Oriented, Group Problem Solving, IT Quality Assurance, Long Term Planning, Predictive Analytics, Programming Languages, Software Product Testing, Test Automation
Additional Job Details
RBC CENTRE, 155 WELLINGTON ST W:TORONTO
Toronto
Canada
37.5
Full time
CAPITAL MARKETS
Regular
Salaried
2026-07-17
2026-08-14
Note: Applications will be accepted until 11:59 PM on the day prior to the application deadline date above
Our Employment Opportunities
At RBC, we are guided by living shared values of Client First, Integrity, Collaboration, Respect and Excellence and winning together as One RBC. We believe an inclusive workplace that has diverse perspectives is core to our continued growth as one of the largest and most successful banks in the world. Maintaining a workplace where our employees feel supported to perform at their best, effectively collaborate, drive innovation, and grow professionally helps to bring our Purpose to life and create value for our clients and communities. RBC strives to deliver this through policies and programs intended to foster a workplace based on respect, belonging and opportunity for all.
Join our Talent Community
Stay in-the-know about great career opportunities at RBC. Sign up and get customized info on our latest jobs, career tips and Recruitment events that matter to you.
Expand your limits and create a new future together at RBC. Find out how we use our passion and drive to enhance the well-being of our clients and communities at jobs.rbc.com.
RBC is presently inviting candidates to apply for this existing vacancy. Applying to this posting allows you to express your interest in this current career opportunity at RBC. Qualified applicants may be contacted to review their resume in more detail.
