Leverage Metrics and Benchmarks to Evaluate LLMs
Wednesday, October 23 at 4:00 PM ET | 1:00 PM PT
Webinar Summary
When it comes to generative AI technology, OpenAI is no longer the only game in town. The AI vendor marketplace is an evolving and complex landscape where the best and most cost-effective solutions are not always obvious.
Finding the right LLMs for your customers requires an evaluation of whether to buy, customize, or build models that address specific gaps and deliver specific value, all while balancing and optimizing tangible costs and measurable efficiencies against specific performance requirements.
Join this insightful session as we explore a framework for selecting the right LLM, including how to:
- Use a benchmark framework to assess LLMs along various performance axes to align the LLM with specific solution requirements.
- Leverage cost and performance metrics to ensure you have a cost-effective solution that drives organizational value.
- Employ benchmarks to assess subjective performance characteristics of models and ensure you align LLM selection with specific capability requirements.
Webinar Speaker
Justin St-Maurice
Principal Research Director
-
Dr. Justin St-Maurice is a Principal Research Director for Info-Tech Research Group. His background is rooted by formal education in systems engineering and informatics. Justin has had a career implementing technology, leading innovation, and supporting digital transformations, mostly in postsecondary and primary healthcare. He ran his own VAR business, consulting practice, and training center; navigated the height of the pandemic as an executive at a polytechnic college; and worked as a senior technical product manager at one of Canada's fastest-growing unicorns.
-
Justin has run successful research programs by studying complex socio-technical systems. He has numerous publications in systems engineering and informatics. His research leveraged cognitive work analysis to abstract critical processes in high-risk decision making during human-computer interactions. Justin has been the recipient of multiple industry awards, an Industrial Partnership Award from the Natural Sciences and Engineering Research Council of Canada, and a prestigious teaching award for his time in the classroom.
-
Justin has an Engineering Systems degree from the University of Guelph, a master’s degree in Informatics from the University of Victoria, and a doctorate in Systems Design Engineering from the University of Waterloo. He holds a variety of certifications from Microsoft and VMware and is a Certified Solution Architect with AWS. He is also a certified ICF coach and a two-time Ironman finisher.
Register for Our Webinar
About Us
Info-Tech Research Group is the world’s fastest-growing information technology research and advisory company, proudly serving over 30,000 IT professionals.
We produce unbiased and highly relevant research to help CIOs and IT leaders make strategic, timely, and well-informed decisions. We partner closely with IT teams to provide everything they need, from actionable tools to analyst guidance, ensuring they deliver measurable results for their organizations.
© Copyright Info-Tech Research Group,
