Goodbye job applications, hello dream career
Seize control of your career and design the future you deserve with LW career

Only 7% of lawyers trust ChatGPT (for now)

As ChatGPT continues to trend globally, more legal professionals are exploring how the platform, and similar artificial intelligence (AI) technologies, can improve their day-to-day and drive efficiency. But Lawyers Weekly can reveal that only a small number of lawyers actually trust the tech fully.

user iconLauren Croft 24 February 2023 NewLaw
expand image

Following the rise of ChatGPT, Lawyers Weekly asked readers how much they trust ChatGPT — or whether they do at all, with almost 60 per cent of respondents revealing that they don’t trust AI tech at all.

This comes after a noisy debate emerged over ChatGPT and similar. On the one hand, platforms like ChatGPT are a “useful resource” for boutique firms and BigLaw firms alike and will require a focus on key skills and a rethink on legal education, even if the new tech can’t replace lawyers entirely (at least not yet).

This has been seen in the recent uptake of ChatGPT in the Clayton Utz environmental, social, and governance (ESG) practice, as well as Allen & Overy recently partnering with a chatbot lawyer, “Harvey”. This extends past BigLaw, as numerous sole general counsel have said that AI tech can help scale their workload and improve their workload. 

However, there are also fears that the rise of AI could mean the beginning of the end of lawyers — even if it should be used cautiously in courts and demands new workplace policies, and despite the fact that ChatGPT can be used to cheat on law school exams. Billable hours have also been revealed to be at further risk with the development of AI tech.

To read Lawyers Weekly’s full series of ChatGPT stories, click here.

Following this coverage, Lawyers Weekly asked its audience, via a LinkedIn poll, how much they trust ChatGPT (and other similar AI tools) to perform certain legal functions, after a number of legal practitioners voiced concerns about the bot, with some saying ChatGPT is “no different to Wikipedia”.

At the time of closing, the poll results were as follows:


According to the poll, 57 per cent of lawyers do not trust ChatGPT and similar AI tech at all, with 7 per cent stating they trust the tech a lot and 36 per cent somewhat trusting ChatGPT. 

The poll conducted was, of course, not a scientific study. But it offers an interesting snapshot of lawyers’ trust in ChatGPT and AI technologies.

The fact that so few lawyers trust ChatGPT is “not necessarily surprising”, opined Nick Mann, founder and principal of Polaris Lawyers.

“As with any new technology, there can be a certain amount of scepticism or hesitation about relying on it for critical tasks, such as providing legal advice to clients. Further, the legal profession has traditionally been slow to adopt new technologies, in part due to concerns about accuracy, confidentiality, and ethical considerations.

“Lawyers are trained to exercise independent professional judgement and to carefully consider the specific circumstances of each case or issue, and there may be concerns that Chat GPT or other AI technologies may not be able to fully replicate that level of judgement. In addition, some lawyers may not fully understand how ChatGPT or other AI technologies work or may be concerned about the potential risks associated with their use, such as inaccuracies, confidentiality breaches, or legal liability,” he explained.  

“That being said, there are also lawyers who are enthusiastic about the potential of AI technologies to assist them in their work, such as in conducting legal research, drafting documents, or managing caseloads. As the technology continues to develop and become more widely accepted, it is likely that more lawyers will begin to trust and rely on AI technologies like Chat GPT, provided that they are used appropriately and with appropriate safeguards in place to ensure accuracy, confidentiality, and ethical considerations.”

However, this could be further in the future than some are currently predicting, according to Gilbert + Tobin technology and digital partner Andrew Hii, who said that ChatGPT itself states it is not designed to provide legal advice.

“You shouldn’t trust something that expressly states it is not to be trusted! Our professional duties require us to act competently and diligently and this requires us to bring our legal knowledge and experience to the work we do. A lawyer cannot solely rely on ChatGPT (or other similar services) to do this for them,” he added.   

“Of course, ChatGPT has a role as a research tool, in the same way that Google search or Wikipedia are commonly used by lawyers. However, these kinds of tools are best used as an initial starting point for research, as none of them [has] yet come anywhere close to replacing the role of primary sources and legal textbooks.

“I have found ChatGPT to be quite good at providing surface-level responses, although the accuracy and correctness of its answers [are] very inconsistent. Also, client confidential information should not be shared with ChatGPT, so its role, at this time, in document review or preparation is very limited.”

However, View Legal founder Matthew Burgess argued that any drawbacks with AI technologies aren’t due to fundamental flaws with the technology — but rather “largely due to operator error”.

The reality is whether it be automated precedents, matter filing, research tasks, timesheet completion, continuing education, e-discovery or a range of other areas, machine learning and AI is ubiquitous in the legal space already,” he told Lawyers Weekly.

“However, likely in a similar manner to the recent research out of the US — a significant number of those in the legal industry are probably unaware of the role being played already. For example, according to the US data, only around 30 per cent of adults are able to correctly answer the role AI plays in areas such as wearable fitness trackers, product recommendations when shopping online, email spam filtering, and music playlist suggestions.”

Moreover, there are a few key reasons why lawyers may be more dismissive of AI and similar technologies.

“Unfortunately, given I believe the legal profession generally has enormous potential to add value to our community, my sense is there are three key reasons for the (arguably naively arrogant) dismissing of AI, namely that as leading legal thinker Richard Susskind, has observed ‘it’s very difficult to tell a roomful of millionaires that their business model is wrong,’ Mr Burgess outlined.

“A generation ago, in the Innovator’s Dilemma, Clayton Christensen set out in detail why disruptive innovations (such as AI in the legal arena) are treated with contempt by the incumbents.

“Perhaps though, the risks to lawyers of dismissing AI are best captured by the (in)famous Mark Twain quote: ‘The best swordsman in the world does not need to fear the second best swordsman. The person for him to be afraid of is some ignorant antagonist who has never had a sword in his hand before, does not do the thing he ought to do and therefore the expert is not prepared for him, does not do what he ought to do and often the expert is then caught out and is ended on the spot’.”

And despite the concerns and potential drawbacks, Mr Burgess maintained that there’s no reason for lawyers to be wary moving forward.  

“My sense is that other than in relation to confidentiality, there are few, if any, reasons to be cautious of AI. Without oversimplifying the platforms, they are, in essence, more sophisticated search engines, that at least in the short term, can provide material efficiency and effectiveness [sic] productivity gains for every member of a law firm,” he said.

“That said, as is getting increasing attention — how these gains impact on lawyers’ selling time is a significant issue for any firm wedded to timesheets. Although not necessarily directly referring to time billing, there is arguably a reason George Bernard Shaw once said that ‘the professions are conspiracies against the laity’ — and thus the resistance to AI by lawyers is predictable.”

Lauren Croft

Lauren Croft

Lauren is a journalist at Lawyers Weekly and graduated with a Bachelor of Journalism from Macleay College. Prior to joining Lawyers Weekly, she worked as a trade journalist for media and travel industry publications and Travel Weekly. Originally born in England, Lauren enjoys trying new bars and restaurants, attending music festivals and travelling. She is also a keen snowboarder and pre-pandemic, spent a season living in a French ski resort.

You need to be a member to post comments. Become a member for free today!