Trial, Proof of Concept, or Pilot? A Guide to Designing Effective Evaluation Methods for your Legal Tech

Pilots, trials, proof of concept is not just about giving your product to some users and hoping they like it.

This is the most fragile phase for any legal tech vendor - it doesn't matter what product you have. At this stage both you and your client are in negative equity. You have invested time and resources getting the attention of this prospective customer. You may have already used a significant portion of your  CAC (customer acquisition cost) getting to this point.

Equally you’re not yet at the point of delivering value to your client - you are consuming their time and resources.

So it’s important that you think about this carefully. There is a lot that goes into designing an effective evaluation method but it’s time well spent. A well structured engagement can not only impact your ability to convert all that effort you put in but you can also minimise the time (resources) you and prospects have to invest in the evaluation.

Working with one legal tech company, we reduced the average pilot period down from 3 months to 2 weeks. Happy CEO and happy clients.

Trials, Proofs of Concept (POCs), and Pilots are the most common methods for assessing the value of a product, but these terms are often used interchangeably.

For simplicity, we’ll use the term Pilot throughout this article. 

Before we get into the best methods for piloting your software we should talk a little bit about seen, proven and realised value.

The Three Types of Value: Seen, Proven, and Realised

Seen value gets you in the door. Proven value is where the decision is made. Realised value secures your long-term relationship.

Seen value is when a prospective buyer can see the potential value of your technology. This normally takes place during a demo. You might hear things like “I can see how this will save us a huge amount of time” when value is seen.

Proven value is when you prove to your prospective buyers that they will receive the value you have shown them in the demo through some method of evaluation (the crux of this article). You might hear things like “We saw an average saving of 5 hours per matter during the pilot” when value is proven.

Realised value is when a prospective client has obtained that value, typically occurring at some point after deployment of a technology. You might hear things like “we saved 1000 hours this year using XYZ technology” when value is realised.

Value Type	What It Is	When It Happens	What It Sounds Like
Seen Value	When a prospective buyer can see the potential value of your technology.	Demo	“I can see how this will save us a huge amount of time.”
Proven Value	When you prove that your technology delivers the value promised in the demo through an evaluation process.	Pilot	“We saw an average saving of 5 hours per matter during the pilot.”
Realised Value	When the client achieves measurable results post-deployment.	Deployment	“We saved 1,000 hours this year using XYZ technology.”

This article focuses on Proven Value and the evaluation methods.

Key Considerations for Selecting the Evaluation Method

1. The Client

The clients needs are the most important which is why it’s number 1 in this list.

The client has the experience of their firm / their team. They will have insights into people of influence, what has worked for them in the past and any particular nuances of the firm that you might not be aware of. Understand what is going to work for them and why. 

Through the same vein - you will know more about this specific part of the legal work than anyone else as you will have had the exposure to numerous clients and what is best practice. Share this experience and why you have designed a pilot in a certain way and what the benefits (for them) are and you’ll give yourself the best chance of influencing the process. 

Remember running a pilot is a team effort and most firms understand this - each party will need to complete numerous tasks throughout..

However, this is a fact lost on some vendors. I once met an Innovation Manager at a top 10 UK firm who managed 200 existing applications and was piloting 2 new solutions every month whilst also horizon scanning. She was a team of 2, for a 5000+ person firm.

Not every firm is structured in the same way but most innovation teams are inundated with requests so help them in every way you can and this will go a long way.

2. Scope of the Problem

Typically you will already have an outline of this in an overarching problem statement about your company but you will need to deeply understand the problem and the existing workflows of the individual users for this specific client.

You’ll also need an understanding of:

Breadth - How many people experience this problem? Ensure you have a meaningful cohort to validate your solution.

Depth - How significant is the problem for each user? Does it consume a substantial amount of time?

Frequency - How frequently does the problem occur? Is it daily, monthly, annually? Is there  seasonality trends to consider?

As an example:

- Big law firms: A brief trial with a small number of stakeholders may not suffice. A larger pilot is typically required to showcase the scalability and impact of your solution.

- Small in-house teams: Lengthy pilots can be resource-intensive and overwhelming. Shorter, more focused pilots are often more effective.

3. What Is the Aha Moment?

The Aha Moment is when users truly feel the value of your software. Identify when this typically happens in the user journey what actions or results lead to that moment. Design your pilot to maximise the chances of participants experiencing it.

Timelines are also a consideration here. If the typical workflow is that the aha moment occurs after a series of actions that take two weeks based on your user behaviour, designing a trial for 1 week is unlikely to yield positive results. 

You will use all of this knowledge to think about the best combination of Time, Users and Cost - the evaluation variables.

The Methods of Evaluation: Trial, Proof of Concept, and Pilot

These terms are often used interchangeably but can be broadly categorised as follows:

Method	Description	Duration	Users	Cost to Vendor & Client
Trial	A short-term, limited deployment of your technology to evaluate usability and initial impressions.	< 1 week	Up to 3	Low
Proof of Concept (PoC)	A focused process to validate whether your solution is technically feasible and solves a specific problem for the client.	3 days - 1 month	Up to 10	Medium
Pilot	A comprehensive implementation of your technology to test scalability and demonstrate ROI in a controlled environment.	2 weeks - 1 year	Up to 500	High

Structuring a Successful Pilot

1. Pre-Pilot Preparation

The most important phase. Based on the above you will have already have a template pilot plan (even if this is the first client you’ve designed the pilot for) and you’re ready to refine that for this specific client.

Things to think about:

1. Understand the process beyond the pilot. Ensure you know what happens after the pilot and align on timelines. Include a pilot review meeting on the calendar from the outset.

2. Show and refine your pilot plan. Work with the client to outline milestones, metrics, and engagement strategies.

3. Set clear guidelines and expectations. Define success metrics, expected usage, and criteria for feedback.

4. Create an engagement plan (more on this below).

Example legal tech pilot plan - download the full document below.

Pilot Plan for Legal Tech.pdf

Download PDF • 52KB

2. During the Pilot

Properly dedicate time and resources to facilitating the pilot in the best possible way. Be present and supportive throughout. 

1. Collect data continuously. Gather qualitative feedback (e.g., user testimonials) and quantitative metrics (e.g., time saved, tasks completed).

2. Engage participants. Provide ongoing support and training to ensure users get the most out of the software.

3. Build the business case. Work closely with the client to consolidate feedback and results into a compelling ROI story.

3. Pilot Review Meeting

This is your opportunity to showcase all the positive findings from the evaluation.

1. Share a summary in advance. Provide a clear overview of pilot results, including successes, opportunities for development / improvement, and metrics.

2. Ensure key stakeholders are present. Make sure decision-makers are in the room to hear the results firsthand.

3. Agree on next steps. Reconfirm timelines and the path to full deployment, addressing any outstanding concerns.

4. Pilot Ends

We now need to gather final feedback and compile our report. 

• Conduct a closing survey on user experience, challenges, and recommendations.

• Collect all information gathered throughout the pilot and package this into a report that can be easily consumer by the key stakeholders.

• Talk through the report and next steps in the post pilot review meeting.

Engagement plan

Pre-Pilot Email

Prepare participants for the pilot by setting expectations, providing key resources, and ensuring technical setup is complete.

• Welcome Message: Introduction to the pilot, goals, and expected outcomes.

• Technical Requirements: Instructions on installing necessary plugins or accessing the platform.

• Pilot Schedule: Overview of key dates, including training sessions, 1:1 support, and pilot duration.

• Support Contacts: Key vendor contacts for troubleshooting and assistance.

• User Expectations: Clarification of what is expected from participants (e.g., frequency of use, feedback submission).

• Action Items: Participants to confirm installation, access, and schedule attendance for training sessions.

Example

Welcome to the [Software Name] Pilot – Key Information & Next Steps

Dear [Participant’s Name],

We’re excited to welcome you to the pilot program for [Software Name]! This initiative is designed to help [Law Firm Name] evaluate how our platform can streamline workflows and enhance efficiency. Below are key details to help you get started:

- Start Date:

- End Date:

- Key Objectives: [Brief description of what the firm aims to achieve with this pilot]  

Setup  

To ensure a smooth experience, please complete the following before the pilot begins:  

Install the [Software Name] plugin/application and ensure you have access to the platform by logging in.

Here is a link to the platform:

Training

We’ve scheduled a group training session to guide you through the platform’s key functionalities:

- Date & Time: [Insert Date & Time]  

- Location/Link: [Insert Location or Virtual Meeting Link]  

- Duration: 45 minutes (30 minutes training + 15 minutes Q&A)

If you require additional assistance, we also offer 1:1 support sessions. Please book a slot here: [Booking Link]

Help

If you need support or have any questions you can reach us at email@legaltechsoftwarename.com or on the Teams Channel.

Thank you for your participation! We look forward to working with you and making this pilot a success.

Best,

Legal Tech Founder Name

Group Training

We want to help users with a structured introduction to the platform and ensure they understand core functionality.

Check all participants have installed the software and can log in before the session. 

Structure (45 minutes):

• Introduction (5 min): Explain purpose of the session and key objectives.

• Platform Walkthrough (15 min):

  • Ensure everyone has the plugin loaded.

  • Show each section of the platform and explain its functionality.

  • Demonstrate key workflows (e.g., document upload, review, collaboration tools).

• Interactive Exercise (10 min):

  • Participants upload a sample file (use a list of recently completed firm documents).

  • Perform a basic workflow task with guidance.

• Q&A (15 min): Open discussion for questions and clarifications.

Post-Training Action Items:

• Share session recording and training materials.

• Provide quick-reference guides for core tasks.

• Provide the working exercise so participants have a step by step guide

• Encourage participants to test the platform and note any issues.

121 Sessions

121 sessions can be useful to provide personalised support to key stakeholders and ensure they are confident using the platform.

Structure:

• 20-30 minute individual sessions (scheduled as needed).

• Tailored to each user’s role and use case.

• Troubleshoot any workflow challenges.

• Gather feedback and testimonials.

Final Thoughts

Trials, Proofs of Concept, and Pilots are critical tools but they’re only effective when designed with intention. 

Orientate your efforts towards understanding the client and users needs, aligning on what success looks like and guiding users toward the magic, Aha Moment.