Featuring

Paul Cirstean, Head of Innovation at Yonder

This session of AI for Smart People features Paul Cirstean, Head of Innovation at Yonder. Focusing on the cost, risks, and benefits of AI integration, this session provides practical tips for choosing AI-enhanced legal technologies.

Exploring Artificial Intelligence and Machine Learning

Paul explains the origins and types of artificial intelligence (AI). Narrow AI systems are skilled at specific tasks like face detection. Current AI struggles with complex tasks like planning. Machine learning uses data to improve over time, with supervised learning using labeled data as examples.

Deep Learning, Generative AI, and Transparency

Paul and Cheryl discussed the advancements and challenges of deep learning and generative AI. Paul explained the process of training systems on vast amounts of labeled data, which allows the computer to identify and understand complex features.

Supervised Learning and System Learning

Cheryl and Paul discussed the three methods of system learning, starting with supervised learning. He highlighted the need for large amounts of labeled data, the effort invested in labeling, and the potential for introducing biases. Cheryl used the example of a contract review system, ‘Kira,’ to illustrate the concept of supervised learning, where a system needs to be trained with data before it can be used. They then planned to discuss unsupervised learning next.

Unsupervised and Reinforcement Learning Discussion

Paul and Cheryl discussed the applications and benefits of unsupervised learning and reinforcement learning systems. Paul explained how these systems can handle vast amounts of data, identifying patterns and anomalies, and are particularly useful in fields such as fraud detection and e-discovery. Cheryl highlighted the early use of these technologies in the legal tech space. Paul also introduced reinforcement learning, explaining its role in powering self-driving cars and advanced chess-playing systems, emphasizing that the system’s decision-making is based on defined rules and rewards, not independent thought.

AI Model Types and Availability of Data

Cheryl and Paul discussed the differences between base models and pre-trained or fine-tuned models in artificial intelligence. Paul explained that base models have been trained on vast amounts of public data, while fine-tuned models are further specialized using proprietary or domain-specific data. He highlighted that fine-tuning requires more investment and effort but often results in better performance at specialized tasks. Cheryl noted the limited availability of widely available data in the legal sector, leading to the predominant use of base models. Paul introduced the concept of synthetic data generated by AI to address this limitation. They also discussed the example of Harvey AI, which reportedly achieved significant success in the legal sector by fine-tuning a base model.

AI Integration Strategies and Opportunities

Cheryl and Paul discussed the integration of AI into existing products, focusing on the benefits and limitations of add-on and vertical integrations. They noted that add-on integrations are a practical solution for initial experiments and can be easily adapted, but may be isolated from the core product, limiting its learning and improvement potential. In contrast, vertical integrations offer full integration, enabling continuous learning and deeper understanding of the data and processes, but are more complex and costly. They advised that companies should consider their needs and risks before choosing between the two options.

AI Model Integration and Evaluation Considerations

Cheryl and Paul discussed the complexities and considerations of integrating AI models into their systems. Paul emphasized that companies should be wary of paying a premium for AI integration unless the company can prove they’re doing something more valuable than wrapping a base model. He also highlighted the security concerns of sending data to a cloud not owned by the company. Cheryl stressed the need not to get too committed to base models due to potential changes or breakage. The discussion also touched on the potential utility of benchmarks in assessing AI models.

Evaluating Generative AI Systems Performance

Paul and Cheryl discussed the challenges and recommendations for evaluating the performance of generative AI systems. Paul advised having a test set of one’s own to assess the accuracy and suitability of different AI solutions, referring to his own experience with buying a generative AI solution. Cheryl mentioned the importance of considering how a particular model will work for one’s use case and highlighted the usefulness of benchmarks and tools like Valves.ai’s legal bench. They both emphasized that the big companies, due to their resources and economies of scale, can offer AI systems at a lower cost and that these systems could be applied to solve vertical problems.