Can your AI vendor answer these 17 questions? Most cannot!

The Pain Point

As we’ve been at pains to describe elsewhere, hype hurts AI. Add to that a busy legal AI vendor space – some 67 “AI” products in 11 verticals by one count in 2018. The result? Most buyers of legal AI products are left confused in terms of what to ask AI vendors in order to understand what to buy.

Dollop lashings of “Robot Lawyer” articles replete stock photo of gavel wielding android and it’s no wonder some AI vendors play fast and loose, whether deliberately or by omission, with their product claims.

This article provides 17 basic questions you can use to test the knowledge of AI vendors (or experts) regarding AI and whether what they are selling / telling you is right for your need.

But first, a health warning

This article is not meant to be exhaustive, nor demonstrate how to authoritatively benchmark one tool vs. another – hopefully we can cover that in a later post! For now, these are indicative of the types of things should try to know about AI vendors to help you make the right decisions.

It’s meant to present the ideal when it comes to good AI vendors. By this we mean good AI vendors in an ideal world. As a lot of AI vendors are new to the market – and the execution of such technologies in law relatively immature – very few AI vendors are good AI vendors across all measures.

Likewise, never start with the solution, AI or otherwise. Instead, begin by identifying the people, process and problem that needs solving. Once done, work forward from there to identify solutions to build / buy. If at that point AI seems relevant, get these questions ready!

Five rules to rule them all (and in the enlightenment bind them)

To keep things organised we’ve grouped these questions into five core themes:

1. Fit: tied to the preceding point, do I need AI? Why? And is it even possible given my problem statement?
2. Delivery: who is building, training, curating and maintaining my shiny new AI (and at what / whose cost…)?
3. Compliance: auditability, integrity and explainability. We’ll explain these in due course (pun intended).
4. Tech Test: do you even AI? In other words, is this what it says it is?
5. ROI: show me the money saved, made or maintained by your AI.

We’ve also sort to explain why each question is important and the markers of good vs. bad AI vendors in terms of AI vendor responses.

01. Do we need A.I. to solve our problem? If so why?

Why ask? To validate the AI vendor has considered and understood your business challenge + whether and to what extent AI / their solution solves your actual problem vs. an presumed problem.

As we’ve covered separately, today’s AI is ANI – able to perform a single narrowly defined task – and therefore usually a point solution rather than an end-to-end solution in and of itself. Yes, it may come wrapped in other technologies – usually workflow, automation and / or analytics (+ oftentimes search) to provide something close to an end-to-end solution, but the critical point is the AI bit won’t be the cure all it’s sold to be.

Assuming you followed the above advice – identified the people, process and problem and worked with your stakeholders to generate solution ideas, of which AI is one of several – begin by asking the vendor this question. It reveals a lot!

02. Do we have sufficient quantity and quality of data to use AI? If not, how do we get there?

Why ask? A.I. tools (i.e. those using machine learning) typically require significant quantity of good quality data. If you lack sufficient data in general and / or suitable quality of data, a machine learning solution may suboptimal or entirely redundant unless the data quantity and quality is improved.

03. Do you have domain expertise in our field?

Why ask? Although there are a number of common A.I. techniques used to solve thematically similar problems, their application and execution are not one size fits all and rarely “plug and play” for all but the most basic tasks. For this reason, it’s imperative the AI vendor understands, and ideally has experience in, your domain. Doing so will enable them to tailor techniques to the nuances of your business problem.

Without it, you end up buying clever but ultimately useless solutions. This is the difference between a tailored suit and a S, M and L offering where you are between sizes.

Often with A.I. projects (or indeed any technical undertaking) there is huge value in having someone who can bridge the gap between the domain problem and the technical problem. In other words, for a legal challenge this might be someone who is a lawyer, i.e. can talk lawyer with lawyers to understand their challenge, but also technical, i.e. can translate law to tech and vice versa. This skillset is rare, often undervalued or ignored entirely. Absence of it can disconnect the business problem from the technical challenge, solving neither.

04. What is your delivery model?

Why ask? Tied to the above, A.I. solutions vary greatly in their delivery.

Some are purely self-service, i.e. requiring no additional technical configuration / development. However, that may also mean you need to train the solution to your needs, e.g. by marking up example clauses if the system is designed to extract them from sight unseen documents.

Others are heavy on professional services, i.e. technical consulting and additional developer effort to calibrate, train and iterate a set of capabilities (rather than end-to-end user configurable features) to solve your challenge.

Depending on your business challenge one or both approaches may be better.

Self service is usually better if you have a limited time frame / budget and happy to trade-off usability and immediacy over and above a perfect fit to your use case, e.g. contract extraction tools that come pre-loaded with the ability to extract common M&A Isclauses without the solution requiring additional training. This might be useful for standard M&A due diligence review, but might not be suitable for all nuances of the particular transaction (i.e. DD of a biotech acquisition is different to DD of a timber company).

Professional services delivery is generally better if you have a longer timescale, a large budget and a need to deliver very specific results, e.g. an extraction tool to identify and classify granular data from a dataset, such as to augment the extraction and population of 100s of metadata fields in a contract management system. This might be the case for a law firm building a specific use case application to on-sell to clients, or a bank process improvement project.

05. Who is training (or trained) the solution?

Why ask? Related to the last question is who trains the system? For instance, popular A.I. contract due diligence tools typically provide:

(a) Out of the Box / OOTB Extractors: machine learning models pre-trained by the AI vendor to identify and extract in a particular format specific data points; and

(b) Customisable Extractors: blank machine learning models to be trained by you using your own data to your own needs in terms of what to extract and sometimes, in what format.

This distinction – essentially what can I do OOTB vs. not OOTB – is common to most A.I. solutions. It’s crucial to understand this as it will have a direct impact on fit and compliance.

With regard to fit do we have time, resource and understanding of the technology to train and maintain the system ourselves or are we happy to rely on the AI vendor to do so?

For compliance: how much control do we have over how the system is taught to meet our need and, if it isn’t us doing the training, do we understand with what dataset, how, by whom and why the AI vendor trained the system for us?

Failing to understand this can lead to misapplied efforts at best, or at worst an inability to correct course (and explain why it went wrong) should the solution fail.

06. Do you have any human(s) in the loop?

Why ask? Unfortunately a lot of A.I. tools adopt “Wizard of Oz” techniques.

This is nothing new in technology and refers to the iconic scene in the Wizard of Oz when Dorothy, the Scarecrow, the Lion and the Tin Man finally get to meet Oz the Great and Powerful Wizard only to discover he is an elderly magician operating machinery behind the curtain to give the impression of wizardly powers when he in fact posses none.

It’s very common in consumer “A.I.” apps, including a lot of apps that claim A.I. to verify identities or expense receipts. Often those tasks are completed by a human in a low cost centre, sometimes with or without technology. Sometimes this is disclosed, sometimes not.

Some legal AI vendors do this. Sadly not all of them admit to this. Naturally this raises significant issues concerning client confidentiality and data privacy!

07. Is the system interpretable and auditable?

Why ask? The best A.I. solutions should be interpretable. By this we mean:

(a) transparent, can we answer: “how does it work?

(b) explainable, can we answer: “why did it do that?”

Continuing the previous theme, if you can understand the what, who, how and why with regards a system’s training (and decision making) then the system is more transparent.

Knowing that, plus auditing features to track those metrics, puts you in a better position to explain results. Hopefully this is obvious, but ability to explain results will help you train (depending on its delivery model) the system more effectively.

Most importantly explainability is critical to the extent the system is responsible in whole or in part for decision making or the data for decisions. Without this, should such decisions come under court or regulator scrutiny, failure to justify the approach taken in building the system or inability to defend it could be disastrous!

PWC has produced an excellent report (available here) examining these concerns in more detail – well worth a read as a follow-up to this guide.

08. Who is liable if your solution fails?

Why ask? This is a grey area and a lawyer favourite!

Market / legal practice has not settled this issue, whether by law, regulation or contracting terms. As such, it is a question to be aware of and consider against your business case.

Is the solution replacing or significantly assisting a critical function? What is the cost of that solution failing? Is the failure rate better or worse vs. the current solution? Are there are AI vendor side humans in the loop?

Ask these questions of yourself, and weigh them up against responses to similar questions asked of the AI vendor.

09. Where is our data stored? Do you share / own our training?

Why ask? This issue is twofold.

First, where is the application and its data stored? On premise installations are expensive to set-up, support and update. Cloud distributions scale, don’t require you to buy servers or staff to maintain them and are easily updated but may be impossible if a client prohibits use of cloud services, which is typically the default request for most legal clients.

Second, does the AI vendor leverage your training to offer and / or improve it’s own “pre-trained” features? In the consumer space this is commonplace via click-though Ts&Cs. This is how your Netflix learns to predict what films you like but also anyone else similar to you, i.e. by combining data about similar behaviors.

For legal work this goes against client confidentiality, ethics and data privacy concerns. However, some legal AI vendors also admit – upfront, or after probing – using user data (presumably anonymised in some way) to improve their own OOTB pre-trained models, and in some cases, making those available to other clients.

For most legal users sharing anything resembling client data with third parties is a massive no. Watch out for this one, it could put you in breach of your organisation’sobligations!

10. Why did you choose A.I. for this product?

Why ask? This is a lot like the very first question about the vendor understanding whether or not their product fits your need. Naturally, they ought to have gone through a similar exercise when designing their solution, i.e. identified the people, process and problem they sought to solve, tested assumptions and assessed all solutions… rather than simply saying let’s solve X with Y.

In this regard, A.I. is a lot like blockchain – often at worst a solution looking for a non-existent problem, or at best over-complicating something best solved by simpler solutions.

11. Can you explain how your advertised machine / deep / supervised / unsupervised learning works in general and for my use case?

Why ask? The plethora of technical terms littered throughout any discussion of A.I. make it easy for AI vendors to obfuscate the how of their product.

As is hopefully clear from the preceding, this is a blocker to your understanding of its workings, your use of it, its fit to your need, its interpretability and so on.

It is also a basic test of whether the solution actually includes any A.I. Shockingly a 2019 survey of European A.I. start-ups identified that only 40% actually use A.I.. As we’ve noted elsewhere this is partly because A.I. is a suitcase phrase open to manipulation!

12. Do you use rules and / or search based techniques? If so, when, where, how and why?

Why ask? As we’ve covered elsewhere (see our 101 on Search vs. Rules vs. ML), most “A.I.” products, legal A.I. or otherwise, are a cocktail of techniques. AI vendors using multiple techniques, or using more of one kind than another, should not be discounted.

But then isn’t the question “what’s the best combination?” Well sort of, but the better question is “what’s the best combination for my use case?” Better still, which techniques should we use, when, where, how and why?

You need to be specific in your questions, and AI vendors precise in their answers. This is because it’s horses for courses.

Certain techniques are better suited to particular tasks, or where several techniques can all achieve a similar outcome one may be preferred over others to the extent it superior in means, e.g. a rules based solution to detect a boilerplate clause is computationally cheaper to run vs. a machine learning solution but sub-optimal for non-boilerplate language.

AI vendors who go deep on detail in answer to the very first question in this article (Do we need AI to solve our problem?) should perform well here. Such AI vendors will be open at explaining which techniques they use, but most importantly which best fit your need. Understanding this will help you assess fit and differentiate products within a noisy market.

13. What are the precision, recall and F1 scores for your product?

Why ask? Buyers typically ask in general “how accurate is your product?” or slightly more specifically “how accurate is your product at doing X?”. Neither is the right question from a machine learning perspective.

Asking about the system’s precision, recall and F1 scores and how long and with how much and what type of data was required to achieve those figures are better questions to ask.

However, accuracy vs. precision vs. recall vs. F1 scores are a detailed subject – see here for our 101.

Once you’ve read up on these terms, you’ll appreciate how little AI vendors talk in these terms despite them being fundamental measures for machine learning.

Partly this is to manage expectations, but often because AI vendors haven’t rigorously tested their systems against such measures. Sometimes that’s because they use no machine learning at all. Again, not a write-off if they don’t (depending on your use case), but it does raise a trust issue!

We often ask this question, even if we don’t really care about the answer. A lot of the time the reaction from the AI vendor says it all!

14. What qualifications does your technical team possess?

Why ask? As is neatly summarised in this excellent Law Society article by Rafie Faruq, “machine learning is becoming more commoditised and easier to understand. Nonetheless, it is a mathematical discipline rooted in fields such as linear algebra, statistics and optimisation. Not understanding the equations that underpin these models means providers won’t be able to build quality machine learning products. It is therefore best to check whether the AI provider has real expertise in the firm – typically this means founders or employees that have degrees in machine learning, and advisors who are professor level in machine learning.“

15. Are clients in production or POC with this solution? If only the latter, why?

Why ask? During the heady hype filled years of 2016 – 2018, it seemed every week legal AI vendors announced a new “deal” with a law firm or large organisation. Naturally this created serious FOMO (fear of missing out), an arms race ensued and this trend increased. But… how many had a system in production, that is solving a front line business problem vs. an internal experiment or “proof of concept” (aka “POC“)?

The answer is often hard to find. AI vendors hate disclosing this data, often because fewer than suggested customers use their system in production. Often if they do it’s on a limited scale or ad hoc. This isn’t always a fault of the software, but a change management / adoption issue for the buyer.

Buyers likewise want to be seen to “have some AI” but don’t wish to share, usually because they aren’t doing as much with it as they’d hope (and as soon) but also because it’s often seen as their competitive edge.

Answering this question matters because it should tell you whether the solution is mature, adoptable and actually does what it says on the tin. It should also ideally shed some light on how you might pre-empt and stay ahead of adoption and change management issues, for instance if the AI vendor can advise you on lessons learnt by other clients or their own deployments.

If the AI vendor can evidence referenceable clients using their solution in production then great. If not, tread with caution unless you are happy to trial a potentially immature solution (and / or the AI vendor’s outsourced quality assurance and testing hub – this is unfortunately not uncommon).

16. What ROI are clients seeing? How do they / did you measure this?

Why ask? Related to the above, POCs don’t generate ROI other than to validate or eliminate a potential solution (or perhaps to iterate it towards success).

Solutions in (or very likely to be in) production generating value, reducing cost and / or enabling new opportunities should be a given, or why else promote the solution (other than to make a fast buck or to test the market)? Granted if the company is new, production clients will take time – so bear that in mind and weigh it up in the context of your discussion and need.

To the extent a AI vendor can demonstrate an ROI, they should be able to explain how it was measured and against what, e.g. against the business as usual practice, a competitor product (hard / impossible for a AI vendor to do the latter). Without this you are back to the realms of incomplete data and inviting a misunderstanding that might jeopardise your project and by extension your own reputation within your organisation!

17. How much training / time was required for that ROI?.

Why ask? As described above, depending on the delivery style (self service vs. professional services) and features (OOTB vs. non-OOTB pre-trained features) you may need to invest more or less time and money to solve your need.

Whomever is training, it’s imperative you understand its quantum. If it’s you, do you have the resources and expertise internally to train the system? If not, where, when and can you even obtain that resource? If you’re not training the system, how long did it take the AI vendor to do so? Bear in mind the aforementioned issues to be addressed regarding the domain expertise quality and assurance processes.

Understanding these items will identify whether your problem is one solvable by A.I. within the confines of your resource, time and budget constraints. It’s also another good measure of how seriously the AI vendor understands the importance of this aspect to your own success. Likewise whether they can demonstrate with evidence any claims in this regard.

The more your goals are made known and aligned with the AI vendor, the better chance you will both succeed together.

Unfortunately AI vendors often forget you have to prove ROI and by extension struggle to answer this question, usually because they don’t (and sometimes can’t) collect the necessary data.

Forewarned is forearmed! Don’t believe the hype. A little learning goes a long way. Hopefully this guide will prove helpful should you be in the market for legal A.I.

The more you can read around and understand these concerns, and in general how A.I. tools work under the hood (you don’t need to code, theory = sufficient) will massively improve your AI hype detector. In doing so you will save yourself a lot of wasted efforts and build or buy what you need, not what somebody else prescribes!

Likewise, a nice way to think about what we today call AI is to think about its purpose. In most / all use cases, it’s about supplementing not supplanting humans. In the legal context this is making lawyers better not bots.

Framed this way against the limits of today’s AI (which are hopefully clearer from the above), you will realise today’s AI (and its use cases) are really all about IA: intelligence augmentation.

This will help you manage your own, and your stakeholders’, expectations, but will also help find the best AI vendor. The best AI vendors understand AI today is really about IA, i.e. making people better at their job. One day that might change, but that day is not now.

Lastly, when dealing with AI vendors observe the Goldilocks Rule – make sure the porridge is neither too hot nor too cold when it comes to their claims and responses!

If you have any questions or disagree with the above let us know!

Any / all of the below. Unfortunately, these have been – and continue to be – somewhat unavoidable, especially with regard to hyperbolic language.

That’s because explaining how A.I. systems work is not especially sexy… at least from a marketing perspective.

Special mention also goes to claims re X% accuracy or Y% time / cost saving. Rarely, if ever, are these claims backed up with properly benchmarked data (hence the detail above re these questions).

But at the cost of repeating ourselves, this is in part because AI vendors don’t often complete this type of exercise. If they haven’t done so, neither will it be easy for them to obtain equivalent ROI data from clients to the extent their clients have done such testing. This is usually due to clients perceiving their use of A.I. as their competitive advantage, which in legal is a bit of a none point as everyone is using the same handful of tools.

That said, these aren’t insurmountable obstacles, neither do they excuse lazy language and a failure to try harder and respect buyer intelligence.

Educate don’t obfuscate!

*This one is a joke. It’s the corporate slogan of the fictional Tyrell Corporation (from the movie Blade Runner). The Tyrell Corporation manufactures bio-engineered artificial humans, known as “replicants“.