Evaluating LLM-based Solutions: Overcoming Challenges and Developing Effective Metrics
The introduction of LLMs has made solving complex tasks more widely available for companies and significantly expedited the development process. Ideas that previously would have taken months to implement can now be developed much faster. However, as tasks evolve in nature and pace, new challenges emerge. Specifically, the more diverse and human-like output a model can produce, the more difficult it is to assess its quality. In this talk, we will discuss the changing nature of tasks, the challenges this shift poses, and how to build an effective evaluation framework for your LLM-based project.
Olena is a linguist with experience in both academia and industry. She began her journey in 2008 as an ESL teacher and held the role for eleven years. The next phase of Olena’s professional journey involved delving into research: she completed her PhD in Philology at Kyiv National Linguistic University, where she also taught English and lexicology for several years. In 2019, Olena joined Grammarly as an analytical linguist, and ever since, she have worked on various projects aimed at solving problems ranging from grammatical error correction and ethical use of AI to those requiring the application of LLMs.
