CASE STUDY

Enhancing AI Performance: Optimational's Linguistic Data Annotation Solutions for Bitext

About

In the dynamic realm of artificial intelligence (AI), high-quality data annotation is crucial for training effective models. Bitext, a leader in developing enterprise Q&A systems, sought a reliable partner to provide precise linguistic data annotation to enhance their AI models. Their collaboration with Optimational significantly boosted their AI performance. Dive into this case study to discover the impact of this partnership.

The Challenge: Complex Data Annotation Requirements

Bitext faced unique challenges in their data annotation and labelling (DAL) projects:

Non-Standard Annotations: The projects required specialised annotations beyond typical categories like Named Entity Recognition (NER) and sentiment analysis.
Evolving Specifications: Annotation guidelines frequently changed based on interim results, necessitating flexibility and adaptability in the annotation process.

In search of a partner who could handle these complexities and deliver high-quality annotations, Bitext turned to Optimational.

The Solution

Optimational provided tailored linguistic data annotation and labelling solutions, addressing the specific needs of Bitext’s projects.

Customised Annotation Services: Optimational’s team delivered precise annotations, ranging from paraphrasing user prompts (following very specific and complex linguistic guidelines) to labelling pairs of queries and passages on how well the passages answered the queries and how relevant they were.
Flexible and Adaptive Processes: The team demonstrated flexibility in accommodating evolving annotation specifications, reapplying changes to already labelled data when necessary.
Proactive Issue Identification: Optimational’s team was great in pointing out potential patterns of issues, which allowed Bitext to work with the end client to improve the annotation specifications to obtain better performance in the final AI models.
Agile Methodology: An agile mindset with an iterative and incremental approach was implemented. This allowed for small daily deliveries, enabling rapid feedback and continuous improvement. Consequently, the quality of deliveries was enhanced, and errors and delays were minimised.
Scalable Team Expansion: The team was expanded continuously throughout the project to meet the desired volumes and maintain an efficient workflow. This adaptation to the changing needs of the project ensured that deadlines were met.
Flexible Timezones: Linguistic teams were on standby during weekends, and the team adapted to respond to project managers in different time zones (Asia, USA, etc.).

The Step-by-Step Process

The collaboration between Optimational and Bitext followed a structured process to ensure
high-quality and efficient data annotation:

.1

Project Briefing

Bitext provided detailed annotation guidelines and objectives for each project.

.2

Annotation Execution

Optimational’s team conducted the annotations, following the specified guidelines meticulously.

.3

Interim Reviews

Regular reviews and feedback sessions allowed for adjustments and improvements in the annotation process.

.4

Final Delivery

Annotated data was delivered promptly, meeting the accuracy thresholds and quality standards set by Bitext.

The Results

Significant Improvements in AI Model Performance

Enhanced Accuracy: The annotations met a high accuracy threshold of 90% or higher, as evaluated against gold sets defined with the end client.
Performance Boost: AI models trained with the annotated data saw performance improvements of up to 15%.
Scalability and Efficiency: Optimational’s ability to handle large projects with tight deadlines enabled Bitext to scale their human-in-the-loop (HITL) processes effectively.
Humanising AI Models: The team at Optimational played a key role in humanising the AI models, ensuring that the annotated data reflected a nuanced human understanding and contextual relevance, leading to more natural and effective AI interactions.

Positive Impact on Operations

“Working with Optimational has improved our ability to handle larger projects with tighter deadlines, while maintaining the same level of quality. Their feedback has also helped us improve our pre-annotation processes, which has allowed us to scale our HITL processes even more.”

Bitext

Bitext highlighted several benefits of working with Optimational:

High-Quality Service: Consistent delivery of accurate and relevant annotations.
Responsive Communication: Timely updates and responsiveness to issues.
Operational Flexibility: Adaptability in integrating with evolving workflows and processes.

Ongoing Collaboration and Future Prospects

Bitext plans to continue partnering with Optimational on future AI training projects, leveraging their linguistic expertise and subject-matter experts for complex annotations. The positive outcomes of this collaboration underscore Optimational’s role as a valuable partner in advancing AI technologies.

“Just keep up the good work!”