joshua browder – Techdirt (original) (raw)
DoNotPay Has To Pay, After FTC Dings It For Lying About Its Non-Existent AI Lawyer
from the robot-lawyer-malfunctioned-when-confronted-with-the-ftc dept
Remember “DoNotPay”? They were the company, run by Joshua Browder, claiming to be the “world’s first robot lawyer.” There were all sorts of sketchy things going on, some of which dated back to “DoNotPay’s” earliest days. But things really came to a head last year when legal investigator extraordinaire, Kathryn Tewson, started digging in and finding an awful lot of questionable things going on.
In May of last year, we noted that the FTC had put out a notice which we interpreted as directly targeted at DoNotPay’s myriad false and misleading claims. It looks like we were right.
On Wednesday, the FTC announced actions taken against five different companies making misleading claims about AI, including DoNotPay.
An FTC complaint claims U.K.-based DoNotPay told people its online subscription service acts as “the world’s first robot lawyer” and an “AI lawyer” by using a chatbot to prepare “ironclad” documents for the U.S. legal system. The complaint also says DoNotPay told small businesses its service could check their websites for law violations and help them avoid significant legal fees. In fact, according to the complaint, DoNotPay’s service didn’t live up to the hype. You’ll have 30 days to comment on a proposed settlement between FTC and DoNotPay, which requires DoNotPay to stop misleading people, pay $193,000, and tell certain subscribers about the case.
The details of the complaint are pretty damning.
Respondent described the Service as “the world’s first robot lawyer” and an “AI lawyer” capable of performing legal services such as drafting “ironclad” demand letters, contracts, complaints for small claims court, challenging speeding tickets, and appealing parking tickets. Through a chatbot, subscribers would submit prompts or queries to an AI “robot lawyer” that purportedly operated like a human lawyer, including by applying the relevant laws to subscribers’ particular legal and factual situations; relying on legal expertise and knowledge to avoid potential complications, such as statutes of limitations, compensation limits, and jurisdiction, when generating legal demand letters or initiating cases in small claims court; and detecting legal violations on subscribers’ business websites and providing advice about how to fix them. In fact, the DoNotPay Service was not designed to operate like a human lawyer in these respects.
The complaint confirms basically everything that Tewson reported, and based on the timing of the investigation, it sure looks like the FTC was inspired by Tewson’s work.
They also point out that, beyond just misrepresenting the whole “AI lawyer” thing, Browder and DoNotPay inflated a user-generated comment on a site for kids that the LA Times ran as an LA Times review of the service. Really.
The donotpay.com website has prominently featured a quote that purports to come from The Los Angeles Times newspaper and states, “What this robot lawyer can do is astonishingly similar – if not more – to what human lawyers do.” Compl. Exh. F.
In fact, the foregoing quote derives from a high-schooler’s opinion piece in the Los Angeles Times’ High School Insider website, a user-generated content platform for young people.
I dunno, but it sure feels like Joshua Browder has a problem with being honest.
The complaint goes on to detail a whole bunch of claims DoNotPay made about legal services its system could offer, including suggesting it was specialized in legal expertise and could help you avoid having to hire a lawyer. But, the FTC notes:
DoNotPay did not test whether the Service’s law-related features operated like a human lawyer. DoNotPay has developed the Service based on technologies that included a natural language processing model for recognizing statistical relationships between words, chatbot software for conversing with users, and an Application Programming Interface (“API”) with OpenAI’s ChatGPT. None of the Service’s technologies has been trained on a comprehensive and current corpus of federal and state laws, regulations, and judicial decisions or on the application of those laws to fact patterns. DoNotPay employees have not tested the quality and accuracy of the legal documents and advice generated by most of the Service’s law-related features. DoNotPay has not employed attorneys and has not retained attorneys, let alone attorneys with the relevant legal expertise, to test the quality and accuracy of the Service’s law-related features.
That all seems a bit important? Maybe?
Then there’s the “diagnostic feature” that DoNotPay offered, where it said it would scan your business website and see if you violated any federal or state laws based on your email address. But, as the FTC notes:
DoNotPay’s website diagnostic tool did not, in fact, analyze a consumer’s small business website for hundreds of federal and state law violations based solely on an email address.
Also, apparently the State Bar of California had investigated DoNotPay in late 2021 and accused it of unauthorized practice of law. According to the complaint, Browder agreed to change their marketing behavior:
On or about January 21, 2022, the Chief Executive Officer for DoNotPay promised the California Bar that DoNotPay would no longer represent that the DoNotPay Service was the world’s first robot lawyer
Take a wild guess how that went?
Nevertheless, DoNotPay continued to make this representation.
On June 15, 2023, the California Bar sent a cease-and-desist notice alleging the DoNotPay Service amounted to engagement in the unauthorized practice of law. On the same day, Respondent affirmed that it had removed: a) all mentions of “robot lawyer” from the DoNotPay website and related social media; b) all products on the DoNotPay website that allowed consumers to generate legal documents; and c) any mention of “sue anyone” from the DoNotPay website. Notwithstanding these representations to the California Bar, the DoNotPay website and social media account continued to promote the Service as the “World’s First Robot Lawyer” and advertise “sue anyone” claims
Still, having to pay 193,000seemslikesmallpotatoesforacompanythathasraisedover193,000 seems like small potatoes for a company that has raised over 193,000seemslikesmallpotatoesforacompanythathasraisedover25 million from some big names like Andreessen Horowitz and DST Global (oh and convict Sam Bankman-Fried).
The company does have to alert a bunch of users to this settlement as well which might reasonably cause customers to think twice about doing business with such a problematic company.
But also, the settlement includes a consent order, which says DoNotPay cannot keep misrepresenting its products. And this is often how companies get into serious trouble with the FTC. First they do something bad and get a slap on the wrist that contains a consent decree. And then they violate the consent decree. At that point the FTC can come down on them hard.
Anyway, the company is no longer pitching itself as an AI robot lawyer, but rather your “AI consumer champion.”
Would you trust a company that just got rung up for lying to the public by the FTC about its AI capabilities to be your “AI consumer champion”? I wouldn’t.
Filed Under: ai, ai lawyer, ftc, joshua browder, kathryn tewson, robot lawyer
Companies: donotpay
Can A Robot Lawyer Defend Itself Against Class Action Lawsuit For Unauthorized Practice Of Law
from the questions-questions dept
We were already expecting a lawsuit to be filed against DoNotPay, the massively hyped up company that promises an “AI lawyer” despite all evidence suggesting it’s nothing of the sort. Investigator and paralegal (and Techdirt guest author and podcast guest) Kathryn Tewson had already filed for pre-action discovery in New York, in the expectation of filing a consumer rights case against the company.
However, some others have also jumped in, with a class action complaint being filed in state court in California (first covered by Courthouse News). The full complaint is worth reading.
Defendant DoNotPay claims to be the “world’s first robot lawyer” that can help people with a range of legal issues, from drafting powers of attorney, to creating divorce settlement agreements, or filing suit in small claims court.
Unfortunately for its customers, DoNotPay is not actually a robot, a lawyer, nor a law firm. DoNotPay does not have a law degree, is not barred in any jurisdiction, and is not supervised by any lawyer.
DoNotPay is merely a website with a repository of—unfortunately, substandard— legal documents that at best fills in a legal adlib based on information input by customers.
This is precisely why the practice of law is regulated in every state in the nation. Individuals seeking legal services most often do not fully understand the law or the implications of the legal documents or processes that they are looking to DoNotPay for help with.
The key claim is that DoNotPay is engaged in the unauthorized practice of law. And, of course, this is mostly on CEO/founder Joshua Browder and his greatly exaggerated marketing claims. Of course, when Tewson confronted him on this, he told her “the robot lawyer stuff is a controversial marketing term, but I would (sic) get to wound up over it.”
Yeah, but the thing is, people relying on you for legal services might (reasonably?) get “wound up over it” if the legal services they receive make life worse for themselves. The complaint highlights just how hard the company has leaned into these claims about being a “robot lawyer.”
Yeah, I’m going to have to say that this is probably not a good look if you’re then going to claim in court that your “robot lawyer” is not actually doing legal stuff. The complaint also anticipates Browder’s usual response to critics. As we’ve noted, he has a habit of insisting that it’s all nothing important, and it’s just “greedy lawyers” who are scared that he’s disrupting their business.
The complaint pre-buts that argument:
Not surprisingly, DoNotPay has been publicly called out for practicing law without a license—most recently in relation to a stunt in which it sought to actively represent a client in court using AI. In response, DoNotPay’s CEO deflects, blaming “greedy lawyers” for getting in the way….
Sadly, DoNotPay misses the point. Providing legal services to the public, without being a lawyer or even supervised by a lawyer is reckless and dangerous. And it has real world consequences for customers it hurts.
The complaint then highlights some of the problems users of DoNotPay have faced while relying on the service:
One customer, who posted an online review, used DoNotPay’s legal services to dispute two parking tickets. According to his account, his fines actually increased because DoNotPay failed to respond to the ticket summons. The customer then cancelled his account, but DoNotPay continued to charge a subscription fee.
DoNotPay’s service then reversed another customer’s arguments in her parking ticket dispute. Where she had intended to argue she was not at fault, DoNotPay’s services instead admitted fault, and the customer had to pay a resulting $114 fine.
Those are based on online reviews, but the complaint also details the named plaintiff in this case, Jonathan Faridian, and his experience:
Plaintiff Faridian believed he was purchasing legal documents and services that would be fit for use from a lawyer that was competent to provide them. Unfortunately, Faridian did not receive that.
The services DoNotPay provided to Faridian were not provided by a law firm, lawyer, or by a person supervised by a lawyer or firm.
The services DoNotPay provided Faridian were substandard and poorly done.
For example, the demand letters DoNotPay drafted for him, and which were to be delivered to the opposing party, never even made it to his intended recipient. Rather, the letters were ultimately returned undelivered to Faridian’s home. Upon opening one of the letters, Faridian found it to be an otherwise-blank piece of paper with his name printed on it. As a result of this delay, his claims may be time-barred.
Other documents Faridian purchased from DoNotPay were so poorly or inaccurately drafted that he could not even use them. For example, Faridian requested an agency agreement for an online marketing business he wished to start. Upon reviewing the agency agreement drafted by DoNotPay, he noted that the language did not seem to apply to his business. Even the names of relevant parties were printed inaccurately. Faridian was ultimately unable to use this document in his business project. In the end, Faridian would not have paid to use DoNotPay’s services had he known that DoNotPay was not actually a lawyer.
Yikes. Perhaps not a surprise after what Tewson had found, but, still. Sending a blank piece of paper with just his name on it, and not even delivering it properly?
DoNotPay gave Courthouse News a statement that seems typical of its responses to these kinds of allegations… once again attacking the lawyers.
“The named plaintiff has submitted dozens of cases and seen significant success with our products,” the company said. “The case is being filed by a lawyer that has personally made hundreds of millions from class actions, so it’s not surprising that he would accuse an AI of ‘unauthorized practice of law.’ Once we respond in court, this will be cleared up.”
It is true that Jay Edelson is a well known class action lawyer, who has somewhat famously sued a wide variety of Silicon Valley tech companies. I would argue that not all of his lawsuits are necessarily well targeted, but plenty of them are legit, and he’s generally not messing around when he sues. In other words, this may not be the kind of thing that Browder wouldn’t get “wound up over” but… he probably should.
Filed Under: ai lawyer, california, class action, jay edelson, joshua browder, robot lawyer, unauthorized practice of law, upl
Companies: donotpay
FTC Fires Warning Shot At ‘AI’ Companies Like DoNotPay: Do Not Oversell Your AI
from the you-may-need-to-pay-the-ftc dept
Over the last few months we’ve had a few articles highlighting the pretty serious questions raised regarding how much of DoNotPay’s (“the world’s first robot lawyer”) marketing is pure bullshit and nonsense. It’s not surprising that there might be a bit of puffery from a startup, but DoNotPay’s claims are so outlandish, and its CEO, Joshua Browder seems so allergic to just telling the truth, that it’s increasingly looking like DoNotPay is not just puffing up its claims, but more or less making them up wholesale, in a manner that is fraudulent to consumers who are paying it a monthly subscription fee of up to $18.
Browder, for his part, refuses to serious address any of these allegations, continuing to double down on his “I’m a martyr schtick” in which he pretends that the concerns and complaints raised are simply from bad lawyers and paralegals who are worried about AI taking their jobs. Nothing can be further from the truth. As we discussed with investigator and paralegal, Kathryn Tewson, we both believe strongly in the potential for legal technologies (including AI) to have a tremendous, potentially transformational, ability to improve everyday access to justice for people who can’t currently afford lawyers. But, if DoNotPay is simply scamming people out of $18/month, making promises it can’t deliver, changing its terms of service to avoid transparency and scrutiny, failing to shut down accounts after promising to do so, and other such things, it raises questions not about “taking away lawyer jobs,” but about defrauding the public.
Even the fact that Browder not only lied about making a charitable donation (which he’d already misrepresented as being more generous than it was), but then forged the date on the eventual receipt to mislead people into thinking he had made the donation, raises serious questions about the ethics and honesty of the company’s CEO.
This week, Browder continued his “woe is me, the persecuted entrepreneur, with big bad lawyers out to get me,” by appearing on the a16z podcast.
He continues to just make shit up in that podcast, but given that a16z was the lead investor in DoNotPay’s seed round, and appears to have continued pumping more money into it, I guess it’s not much of a surprise that the host fails to actually push back or challenge Browder on any of this. I’m actually a fan of the podcast, and think the host often asks pretty thoughtful questions in other episodes. But not here. From the very top, she trots out the bullshit marketing line that DoNotPay lets you “sue anyone at the press of a button.”
Remember, Browder insisted that after Kathryn Tewson exposed his “AI lawyer” stuff was bullshit, that the company would remove claims about using the lawyer to go to court and file lawsuits? Of course, those claims are still on the website, and the podcast host repeated them. While she does bring up Tewson’s claims, she doesn’t do so in detail, and lets Browder trot out his well-practiced, but wholly misleading, lines about how the complaints are actually from lawyers who are worried about their jobs.
Of course, in the very same breath, he insists (as he’s done a bunch of times) that “lawyers won’t get out of bed” for the kinds of services DoNotPay claims to provide (whether or not they actually do provide those services remains a pretty open question).
Either way, last week, the FTC put out a very interesting notice to companies to “keep your AI claims in check.” While it’s likely that the notice is directed at many of the new AI companies appearing every other day or so, it reads almost as if it’s addressed directly to Josh Browder and his bullshit claims.
It calls out specific things, all of which seem likely to implicate DoNotPay and Browder’s overly zealous marketing (i.e., bullshit):
When you talk about AI in your advertising, the FTC may be wondering, among other things:
Are you exaggerating what your AI product can do? Or even claiming it can do something beyond the current capability of any AI or automated technology? For example, we’re not yet living in the realm of science fiction, where computers can generally make trustworthy predictions of human behavior. Your performance claims would be deceptive if they lack scientific support or if they apply only to certain types of users or under certain conditions.
Are you promising that your AI product does something better than a non-AI product? It’s not uncommon for advertisers to say that some new-fangled technology makes their product better – perhaps to justify a higher price or influence labor decisions. You need adequate proof for that kind of comparative claim, too, and if such proof is impossible to get, then don’t make the claim.
Are you aware of the risks? You need to know about the reasonably foreseeable risks and impact of your AI product before putting it on the market. If something goes wrong – maybe it fails or yields biased results – you can’t just blame a third-party developer of the technology. And you can’t say you’re not responsible because that technology is a “black box” you can’t understand or didn’t know how to test.
Does the product actually use AI at all? If you think you can get away with baseless claims that your product is AI-enabled, think again. In an investigation, FTC technologists and others can look under the hood and analyze other materials to see if what’s inside matches up with your claims. Before labeling your product as AI-powered, note also that merely using an AI tool in the development process is not the same as a product having AI in it.
If the FTC comes knocking on Browder’s door, he’d do well to (1) hire a real lawyer, not his pretend AI lawyer, and (2) start being honest about what his company can actually do. But, I fear that instead he’ll just go on yet another rant claiming that the FTC is just trying to “protect” the “lawyer’s guild” that he claims is out to get him. Browder is good at marketing stunts, and he seems to now be turning the fact that everyone’s calling out his claims as fraudulent into a marketing tool (“oh, woe is me, the persecuted entrepreneur,”) but the FTC isn’t going to like that very much.
And, honestly, even as his lead investor may have incentives to fluff up one of their startups with uncritical marketing nonsense, a16z should realize that if the FTC is going to come down on one of their companies, encouraging them to further misrepresent themselves is not a very good look.
Filed Under: ai, ftc, joshua browder, lies, marketing, puffery
Companies: a16z, donotpay
Techdirt Podcast Episode 344: The DoNotPay Story, With Kathryn Tewson
from the straight-from-the-source dept
If you’ve been reading Techdirt recently, you probably know all about supposed “AI Lawyer” service DoNotPay and the tireless investigation of the company undertaken by Kathryn Tewson, who has written a couple of Techdirt posts about the saga. This week, Kathryn joins us on the podcast for a long and entertaining discussion about the entire story (so far).
Follow the Techdirt Podcast on Soundcloud, subscribe via Apple Podcasts or Spotify, or grab the RSS feed. You can also keep up with all the latest episodes right here on Techdirt.
Filed Under: ai lawyer, joshua browder, kathryn tewson, law, podcast
Companies: donotpay
Kathryn Tewson Invites DoNotPay To Use Its AI Lawyer In Court… To Address Her Petition For Pre-Action Discovery For A Consumer Rights Claim
from the note-to-self:-never-conduct-fraud-that-kathryn-tewson-finds-out-about dept
Over the last few weeks, you may have noticed that the world’s most tenacious paralegal, Kathryn Tewson, has been carefully dismantling claim after claim after claim from the company DoNotPay and its CEO Joshua Browder. I won’t rehash all of her discoveries, but many of them called into question Browder’s apparent tendency to massively overstate what his company can do, even as he would then seek to quietly cover his tracks every time Tewson pointed out another bit of nonsense.
Browder made the incredibly unwise decision (he probably should have checked with a real lawyer rather than his fake AI lawyer) to appear on Bob Ambrogi’s always excellent podcast where he continued to just totally misrepresent a ton of things, some of which could get him into serious legal difficulty. As an aside, I saw some people claim that Ambrogi let Browder off easily each time he misrepresented things, but that wasn’t my impression at all. He kept bringing up more and more problems (sometimes citing us at Techdirt), and just let Browder hang himself with his own words. Words that I think he may come to regret.
I won’t go through the entire podcast, but I will note that Browder’s attack on Tewson is just blatantly false. He claims that DoNotPay delivered its first document to her, and then “the system” thought she was entering fake data, and therefore, for the next two filings she made, she got the notice she’d have to wait hours. Except… that’s not what happened. As anyone who read Kathryn’s article here knows, while Kathryn did request three documents, Browder got the order wrong. It was the first two that the company said would be delivered hours later. It was the third one that was delivered immediately (though it was terrible).
Also, Browder keeps insisting over and over again that Tewson is just mad because she was banned, except anyone following Tewson knows that’s false. She’s mad that Browder is misleading the public and selling a legal service that doesn’t do what it claims. He also keeps falsely saying that Tewson claimed that DoNotPay was really sending data off to “an army of people, typing out every letter manually.” He made that claim a few weeks ago too, but Tewson never said nor even implied that. Instead, she’s noted that the timing delays make it seem like DoNotPay is simply taking basic templates, a la LegalZoom, and having someone fill in the details from the submitted users, which would explain the delay. By exaggerating and misrepresenting Tewson’s claims, Browder gets to frame them as ridiculous, rather than addressing her real concerns.
Well, now he may have to address her real concerns. On Monday, Tewson filed a petition with the NY Supreme Court asking for an order for Browder and DoNotPay to preserve evidence, while also seeking pre-action discovery, as she plans to file a consumer rights suit, arguing that DoNotPay is fundamentally a fraud. The petition, written by J. Remy Green, is… worth reading. It’s quite impressive.
This action seeks pre-action discovery preliminary to a consumer rights suit over, at its core, a 36dollarfraud.Respondentsappeartohaveliedtoconsumersandarepretendingtohavecuttingedgelegaltechnology,alltoscamthemoutofabout36 dollar fraud. Respondents appear to have lied to consumers and are pretending to have cutting edge legal technology, all to scam them out of about 36dollarfraud.Respondentsappeartohaveliedtoconsumersandarepretendingtohavecuttingedgelegaltechnology,alltoscamthemoutofabout36 a person.
The petition lays out that DoNotPay is advertising a bunch of legal services that there is little indication it can actually provide, and calls out the similarities to Theranos.
This episode smacks of nothing so much as the Theranos fraud.
Theranos, like DoNotPay, was built on a noble idea. It was designed to cut through medical testing using cutting edge technology. Theranos was built on the claim it had a unique technology for blood testing that made it cheaper, more accessible, and less painful for patients. It drew investments from across the ideological spectrum and was valued at over $10 Billion at its peak.
But Theranos never actually had that technology.
Instead, once cornered and forced to deliver a product, Theranos dressed up an existing Seimens testing machine and ran it using too little blood for valid results.
So too here: By all appearances, Respondents are dressing up an old-fashioned document wizard and calling it a “Robot Lawyer.” Certainly that’s what Respondents did with the one document Petitioner was able to get before Mr. Browder personally began re-writing the DoNotPay terms of service to basically say “TELL NOTHING TO KATHRYN TEWSON, SHE IS BANNED FOR LIFE.”
The petition notes that it’s seeking pre-action discovery so that the upcoming suit is more accurate, and allowing Browder to show at least some evidence as to whether or not DoNotPay actually uses any AI at all.
Before bringing a consumer class action alleging this is all a house of cards, while Petitioner believes she is able to plausibly allege the fraud, Petitioner would like to be able to allege the details with specificity. And she would like to give Respondents a chance to show that at least somewhere in their start up, something was running on Artificial Intelligence.
Much of the petition is basically a rewriting of the article Tewson wrote for us and various Twitter threads she’s posted on the subject.
There is some amusement in the filing, mainly in the footnotes. In one message that Browder sent to Tewson, published in the petition, Browder claims the evidence that Tewson is using his platform for inauthentic reasons is that she made up the name “James Joyce.”
In relation to the authentic usage, I messaged you on the 24th to tell you that and reactivated your account but you should keep it real to stop our systems flagging you. Who is James Joyce? You are clearly operating in bad faith by creating fake names and “generating a TON of” (your words) of fake cases.
In the Ambrogi interview, Browder also calls Joyce a “fictional” character. James Joyce was a real person. He wrote fictional characters, but he, himself, is not fictional. In one of the footnotes, this aside is called out: “While not legally actionable, later statements demonstrate the literary offense that Mr. Browder genuinely appears not to know who James Joyce is.”
But the really fun footnote is towards the end, where Tewson notes that she consents to Browder using his robot lawyer to defend himself in this case, and practically dares him to do so:
For what it is worth, Petitioner does and will consent to any application Respondents make to use their “Robot Lawyer” in these proceedings. And she submits that a failure to make such an application should weigh heavily in the Court’s evaluation of whether DoNotPay actually has such a product.
So, perhaps Browder finally will get to test his robot lawyer in court after all…
Filed Under: ai lawyer, consumer rights, discovery, fraud, joshua browder, kathryn tewson, robot lawyer
Companies: donotpay
Here’s The Article We Didn’t Run Back In 2017 About DoNotPay
from the pulled-off-the-spike dept
So, over the last few weeks, we’ve written a bunch of articles about DoNotPay, highlighting some pretty significant questions about the company, its CEO, and the services it offers. To date, the CEO of the company, Josh Browder, has not responded particularly well to the concerns people are raising, and is acting like someone trying to hide things, rather than address the underlying issues.
Last night, Kathryn Tewson, who has been at the forefront of uncovering all sorts of sketchy behavior by Browder and DoNotPay, published yet another expose, highlighting how some of his earliest claims about how many people were using the tool to contest parking tickets in New York City and London didn’t seem like they could be accurate. It’s a wild ride.
And… it’s also a wild ride that we probably had a story on over five years ago… but did not publish. Back in the fall of 2017, Lawyerist’s founder Sam Glover reached out to me, saying he had gotten excited about the concept of DoNotPay, but when he dug into the details, nothing seemed to add up. He thought there might be a Techdirt story in all of it. Eventually, he put me in touch with David Colarusso, a lawyer and data scientist who was also investigating DoNotPay, initially for Lawyerist, and had reached out to Browder to try to better understand the details. Browder responded with some thinly veiled legal threats if Colarusso dug further (after first promising to supply him the necessary data to confirm the data), which definitely was a red flag. Glover also was unsure if the story was right for Lawyerist, and suggested it was a better story for Techdirt.
Colarusso worried that the implied legal threats might bias him in any article he had written, so first offered to hand the story off to us entirely to build on his research, but eventually felt that it was wrong to be bullied and sent over the draft of a story with a bunch of initial notes to us regarding Browder’s response to Colarusso, including detailing where Browder challenged some of Colarusso’s claims.
We went back and forth over this for a little while and, eventually, chose not to publish it. While we did feel the story was interesting, and we have a history of calling out techdudes making bullshit claims, we eventually felt there just wasn’t enough information to confirm things one way or the other, in large part due to Browder’s blustery responses to Colarusso. Even though the article admits that, and notes that Browder claimed to have the data to support the claims, but was refusing to share it with Colarusso, it felt like we needed a little more to be comfortable publishing it.
Also, at the time, DoNotPay appeared to be a side project of a college student, not a high profile startup funded by some of the biggest VC and angel investors in the world. That has now changed. And, combined with the many other highly questionable claims from Browder recently, and the additional data turned up by Tewson, Colarusso reached out to wonder if it made sense to publish the story now, with an intro like this one, to highlight how these issues were always present with the operation (things you’d think a giant VC firm like Andreessen Horowitz would have done due diligence on?!?).
In retrospect, it might have made sense to publish the story back then, though, again at the time it wasn’t part of a larger tapestry of questionable behavior, nor was it a big venture-backed startup, rather it was a noteworthy (somewhat hyped up) side project of a college student.
Anyway, we should note that after Colarusso wrote this unpublished article, he did become the lab director of the Legal Innovation and Technology lab at Suffolk University Law School, which, in some ways, is in an adjacent space to DoNotPay in that it helps digitize court forms to improve access to courts. This happened after this issue, and really just shows Colarusso’s general interest in this arena, but we wanted to post that disclaimer in the name of transparency.
So, here is the article that Colarusso wrote for us over five years ago, complete with the original notes interspersed in the piece where he highlights some issues and concerns, and leading off with the email he sent with it describing some of his thoughts. The only edits to the original were (1) to correct small typos (2) to insert some paragraph breaks for readability and (3) to remove someone’s name who was involved in the original discussion over what to do with this piece (4) to remove a short paragraph that Colarusso had included in the intro note regarding comments Browder made to Colarusso that possibly revealed sensitive information about Browder that we felt it was improper to publish. Finally, some of the links in the original piece no longer work. Some can be found via the Wayback machine, but for now we’ve chosen not to include those links. In retrospect, things might have been different if we had, in fact, published the article at the time.
Below you’ll find the draft article I put together on DoNotPay, plus a few notes to fill in recent developments.
[….]
He’s already admitted to some minor puffery with his original numbers, telling me that his initial 86,000 appeals claim is off by 10-20%. So my guess for worst case scenario here is that he overestimated his original numbers and built everything on that, and he can’t admit that he made a mistake. The thing is, even with half the numbers reported he would have likely received similar coverage.
Anywho, here’s what I have. Why don’t you give it a look and we can decide how to move forward, with a co-authored piece or you going off in your own direction. Having sat with this for a week, I have to admit I don’t like the idea of him bullying me off the story. That being said, I look forward to hearing your thoughts. Note: our CMS encloses footnotes in double parentheticals, and I’ve added notes to you in brackets.
=======
“Extraordinary claims require extraordinary evidence.” -Carl Sagan
Last month DoNotPay, the free “robot lawyer,” announced that you could sue Equifax by talking with it’s chatbot. The bot’s creator, Joshua Browder hopes his “product will replace lawyers, and, with enough success, bankrupt Equifax.” Browder’s bluster has earned him and DoNotPay a good deal of press. The story of DoNotPay is compelling, an 18-year-old student in the UK builds a tool to fight parking tickets, saving Britons £2 million in just four months. Browder expands the bot to fight tickets in New York City, overturning 160,000 tickets. Then he adds help for the newly evicted and refugees. This year’s big news? DoNotPay now helps with 1,000 areas of law, plus you can build your own. Behind the bluster, however, there is a hint of something extraordinary, the promise that someone has figured out how to use technology to help close the justice gap.
Like many in legal tech, I find Browder’s story inspiring. I teach law students how to build their own interactive flowcharts (chatbots), and DoNotPay has been a go-to example of such a product in the wild. After the sue Equifax feature launch, however, I was struck by what appeared to be, at best, a mismatch between hype and reality and, at worst, a breach of duty. I looked back on Browder’s earlier claims, and I realized I did not know the numbers well enough to put them in perspective. Implicit in the reporting was the idea that Browder was somehow leveraging technology to do something extraordinary. I wanted to know how extraordinary. Unfortunately, when asked to provide data to validate and place his original claims in perspective, Browder demurred.
DoNotPay’s first big splash came in late 2015. In December Browder was claiming to have saved Britons £2 million in just four months, citing 86,000 appeals and 30,000 overturned fines since its launch in late August. According to Browder, during this period DoNotPay was helping challenge parking tickets across all of the U.K. though in our conversation he was unsure if Scottish or Irish users would have been able to use the system. When asked about the scale of his claims, Browder explained that such numbers are just a drop in the bucket for the total number of U.K. tickets. When asked about the source of the 86,000 count, Browder explained it was not a count of appeals filed but rather a measure of completed interactions with the bot. That is, during the period from August to December the bot generated roughly 86,000 documents which could have been submitted as part of a challenge. The estimate of 30,000 overturned tickets was based on a user poll Browder conducted where he found the win percentage of users and applied this to the 86,000 number. When asked about the relationship between completed documents and actual challenges he estimated that 10 to 20% were not actually submitted. This nuance was not accounted for in his original claims and is the result of subsequent analysis on the part of Browder.
It is important to note that these appeals documents were appeals in the colloquial sense. That is, they represented initial challenges, not the formal appeals reported in most official statistics. Appealing a ticket is a multistage process. ((When referencing parking tickets, unless otherwise noted, it should be assumed that I am discussing penalty charge notices. See infra FN6.)) A win according to Browder was any challenge that fails to end with a person paying a ticket. This includes tickets that were canceled before the formal appeals process. Consequently, one cannot directly compare Browder’s numbers to data such as this from the London Councils. In Parking appeals statistics 2015-16, the councils reference 17,192 successful appeals for fiscal year 2016. Assuming a steady rate of tickets across the year, that equates to roughly 5,700 tickets for the four months covered by Browder’s claim.
When asked, Browder estimated that of the 86,000 documents cited 14.6% were for tickets in London. If the win rate was consistent across jurisdictions, we can assume this means that 14.6% of the 30,000 wins (roughly 4,400) occurred in London as well. Remember, however, we cannot compare these 5,700 and 4,400 counts. Browder is NOT claiming that DoNotPay handled around 80% of the winning appeals from London. The councils’ numbers do not include informal challenges that resolved pre appeal and so measure something different from DoNotPay’s wins which are larger than its number of successful formal appeals. The question is how much larger. ((It was the ratio of DoNotPay wins to successful official appeals that was first brought to my attention as something worthy of further examination. Jason Velez, another participant in the legal chatbot space, had found the London Councils’ parking statistics and was not sure what to make of them. Consequently, he shared them with Lawyerist.))
In 2010 and 2011, the rate of challenges across the whole of the U.K. was roughly 25%. ((This number comes from both the Civil parking enforcement statistics 2009/10 and a report from the car insurer Switcover looking at 2010 and 2011 data.)). We know the number of tickets issued in London during FY2016 was roughly 3.6 million. The adjusted four month equivalent comes in at roughly 1.2 million. Consequently, we can assume that about 300,000 tickets (25%) were challenged. 14.6% of 86,000 is about 12,600. This would be roughly 4% of all challenges. Is that a drop in the bucket? I do not know. Is it reasonable that only 14.6% of DoNotPay’s appeals came from London? I do not know. However, if this number were larger the same would be true for the percentage of challenges handled by DoNotPay, adding to the size of our drop. In 2010, London was responsible for about 56% of all on-street parking tickets across England. ((See Civil parking enforcement statistics 2009/10 and XLS tables (citing the number of on-street penalty charge notices for London as 4,023,000 and the number for all of England as 7,140,000).)) Given the relative size of Scotland, Ireland, and Wales, coupled with the question around DoNotPay’s operation in Scotland and Ireland, it seems safe to say the majority of DoNotPay’s users resided in England, and if London made up a similar proportion of England’s tickets in late 2015, it becomes reasonable to ask why there was such a relatively small percentage of DoNotPay users in London esp. given this is where Browder was based. ((England’s population accounts for more than 80% of the U.K.’s. Browder was clear to point out that his definition of London would likely differ from others as he “set a radius from central London and included every postcode within that radius, including places like Heathrow.”))
When asked why this was the case, Browder speculated that many of DoNotPay’s users might have been outside of London due to the patterns of press coverage (national and university related), and that perhaps they might involve tickets not in the government statistics either because they involved university tickets or private parking tickets (i.e., parking charge notices as opposed to council-imposed tickets, known as penalty charge notices). ((For a description of the differences between parking charge notices and penalty charge notices, as well as fixed penalty notices, see Appealing against a parking ticket. Note: Browder did not respond to my question regarding DoNotPay’s ability to fight fixed penalty notices which was prompted by his suggestion that DoNotPay numbers should include a consideration of anything other than penalty charge notices.)) In December 2015, at the time of the claims discussed here, it was reported that DoNotPay did not handle private parking tickets. See e.g., this Daily Mail article, from which the claims discussed here were drawn, stating that “[t]he appeals were all against council-imposed fines – but [Browder] plans to expand his website to cover private car parks ‘in the near future’.” When asked to explain this discrepancy, Browder noted that despite DoNotPay being focused on council-imposed tickets his users likely did not understand the difference between private and public enforcement. Consequently, he claims that users tried and succeeded to use DoNotPay to challenge private tickets given that many of the defenses applied to both.
It is worth noting that the analysis found here operates largely under the assumption DoNotPay’s services were that reported at the time of the 86,000 tickets claim (i.e., DoNotPay handled only penalty charge notices). This is justified in part by Browder’s own analysis. Browder claims to have written a program to check the status of a given ticket. This program works by submitting a ticket number to the government and payment systems and noting the response. For example, paid ticket numbers return different responses than unpaid. By noting the response for a given ticket he claims to be able to ascertain if a challenge was submitted. This program is what Browder claimed to use in determining what portion of the 86,000 tickets were actually submitted, and since it would only work with government issued tickets, we know that the earlier statement about 10 to 20% of challenges not being submitted is equivalent to 80 or 90% of them being penalty charge notices. That is, at least 80% of the 86,000 tickets were government issued tickets.
[this paragraph above is now disputed by Browder. So it can’t stand as is. Originally, I was led to believe his status checking program worked only with government-issued tickets, as described above. However, when I presented Browder with this interpretation to double check he claimed that it also worked with private parties (e.g., those issuing Parking Charge Notices). Since such a program would have to make special accommodations for private tickets, querying a different system, I pointed out that he should have access to the specific number of private vs public tickets. Browder responded to a request for these numbers by stating that I was drastically underestimating the number of private tickets (presumably disputing the logic presented above). He then claimed either that I was conducting my research on the dime of my full-time employer, namely the taxpayers of Massachusetts, or alternatively doing my work after hours such that I was sleep deprived and so doing my employer a disservice. Both of these were based on the timing of some of our emails. If the former, he demanded to know why taxpayer dollars were being used to harass a minor. He did not provide the number of private tickets contained within the 86,000 appeals number, and this is when we cut off contact as this was not the first time he had made a veiled threat and it was clear he was not acting in good faith. The first threat involved a peculiar reading of the disclaimer on my personal webpage, described below. Also, the private parking claim was an eleventh hour claim made only after many prior discussions in which it never came up. It actually came up after I had drafted much of this document and was looking to double check my facts. So I never had time to figure out how it affected the rest of the piece. That being said, my preliminary research put the number of private parking tickets at about half the number of public tickets though I didn’t find any really solid numbers.]
Unlike official appeals, I was unable to easily find aggregated numbers for parking challenges across all of the London Councils. For the small fraction of data I could easily find, the average weighted increase in challenges for FY2016 over FY2015 was less than three percent. ((These included Ealing (1% increase), Hackney 2015/16 and 2014/15 (1% decrease), Newham 2015/16 and 2013/14 (2% increase), Tower Hamlets (5% increase), and Westminster 2015/16 (1% decrease). The overall weighted average, accounting for the relative volumes of tickets, came out to be just shy of 3%, with the majority of tickets arising from either Westminster or Hackney. By way of methodology, I performed a single rudimentary Google search for each of the London Councils’ annual parking reports, varying each search by the council’s name. I took down data when: (1) I could find a report; and (2) that report contained sufficient information to determine the rate of challenges. Such a small sample is almost certainly not representative, but it need not be for the purpose at hand.)) That is, there was no dramatic uptick in challenges across the fraction of councils for which I found data. This is consistent with the “just a drop in the bucket” interpretation or the idea that DoNotPay simply replaced other methods of challenge. That is, the idea that DoNotPay users were just people who would have otherwise used the government system but for learning of DoNotPay. A news article referencing DoNotPay outranked the official gov.uk tool for challenging tickets in at least one of the Google searches I conducted about how to pay a ticket in London. So this seems possible.
This, however, is not the same as saving Britons £2 million. Admittedly, “shifting the source of challenges,” lacks a certain appeal as a headline. Depending on the number of DoNotPay users in the sample councils, however, this data is also consistent with DoNotPay driving an increase in challenges. The problem is that on its own it fails to be definitive either way. Browder called the replacement interpretation a “nasty opinion,” explaining that without access to all of the authorities’ numbers the trend of challenges was at best speculative. ((This came about as a reaction to my characterization that given the data I had available, DoNotPay didn’t seem to move the needle. In fairness, I should have added the qualifier much.)) I agree that the fraction of authority data cited here is insufficient to confidently make claims about challenge trends and effect size. That after all, is my point. If we had fine-grain data for DoNotPay challenges we could drill down into individual council numbers. For his part, Browder claimed to have enough information to know the replacement scenario was not true, and offered to put me in touch with users who could provide testimonials. He did not, however, offer to provide the breakdown of DoNotPay users for specific councils.
The rate of challenges went down in two of the five councils for which I found information, including the one with the highest volume of tickets. Westminster Council ascribed their drop, which started a year earlier in 2015/14, to improved procedures on their part, but it is also consistent with users finding DoNotPay first and then giving up after interacting with a frustrating user interface. If this were the case, DoNotPay would have actually cost people money. To be clear, I am being intentionally provocative. We have no way of knowing what actually happened absent more data because we do not know where DoNotPay users were making their challenges. I mean only to underline the point that it is possible both for Browder’s public numbers to be true and for us to take away the wrong lessons in the absence of more data. Are DoNotPay’s challenges a drop in the bucket, only shifting the source of challenges, the massive mobilization of previously unengaged drivers, or a mild distraction obfuscating government tools for challenging tickets? Given the currently available public data, one can not say because Browder has not provided sufficient detail to test competing theories. This is the problem.
Browder claims to have the data needed to help answer these questions. He says that he has a record of the ticket number for every appeals document created during the period in question, and the fact that he was able to answer my question about the percentage of appeals from London in only a few hours suggests that he can easily access information about the locations and dates of those appeals. Browder repeatedly expressed frustration over having to continually defend his claims. I suggested that if he was to provide a list of the ticket numbers, dates, and locations we would work to audit his claims and publish our results. Unique ticket numbers would remove ambiguity from his claims allowing for an easy assessment of how many tickets were public, private, or other. Coupled with location and date, they would provide a means for direct validation against council records. In our correspondence, Browder would repeatedly reply to a question by effectively explaining that this was all very complicated and that first one needed to account for this or that nuance. The sharing of detailed challenge data would cut through most such complications.
At the suggestion of sharing his data, Browder first expressed concern over his users’ confidential information. I suggested we could address this concern through a formal agreement granting access for the limited purpose of validating his claims. At this point, Browder expressed the belief that although the legal issues around sharing his data could be solved there was a larger issue of user trust and that he felt sharing his user data would be a violation of this trust. Upon further communication, he explained that he had in fact shared his fine-grained data with trusted parties but that he did not trust Lawyerist enough to share such data.
I then asked if he could put us in touch with one of these trusted parties as they may have answered the questions we still had. At first, Browder failed to answer this request directly. Instead he stated that he was working with the BBC and that they would have a piece based on his data coming out shortly. I asked for his contact there. This sparked an exchange of several emails in which Browder initially ignored the request, opting instead to question the ethics and integrity of those working at Lawyerist. ((Browder began asking questions about Lawyerist’s expected revenue for this piece and language in the disclaimer of my personal website where I state that I don’t take money to write about or feature material on either my personal website or blog. Apparently, the fact that I link to writing for which I am paid on my website caused him to think this line was misleading. He then explained that he would consider sharing contact info for those who had seen his data only after his “concerns surrounding ethics and integrity [were] satisfied.” This was followed by the suggestion that I was being paid specifically to write something with a “defamatory angle.” [eventually he put me in contact with a fellow Stanford student he believed to be an unbiased party. He, however, was unable to provide any helpful information and it was my interactions with him that led to the conversations with Browder that prompted us to cut off ties. Again, if you’d like a copy of the emails, I can provide them.]))
I am writing this post in the hope that either: (1) one of Browder’s trusted parties will step forward with a detailed analysis of his data, not just a collection of testimonials; or (2) some institution Browder trusts will step forward and offer their services to conduct a detailed audit and that Browder will avail himself of such an offer. For what it is worth, I do not believe Mr. Browder was engaged in some premeditated act of deception, and my hope is that his reticence to share the data necessary to fully assess his claims is simply a failure to recognize the burden of proof for an extraordinary claim lies with the one promoting it.
The details matter when talking about technology tools aimed at addressing access to justice because practitioners must understand the boundary of reasonable expectations for such tools. Only armed with such understanding can they be maximally used in the service of justice. I want to live in a world where a website built by an 18-year-old can save people millions in a matter of months while shrinking the access to justice gap. ((I teach law students how to build chatbots, and I’m the author of an open source markup language designed for use by attorneys. If we live in such a world, it means Browder’s success can be emulated, and that is something I want to believe, but I know enough to question strongly those things I want to be true.)) But wanting does not make it so. “Extraordinary claims require extraordinary evidence.”
Filed Under: articles, data, joshua browder, parking tickets, research, threats
Companies: donotpay
DoNotPay Promotes Itself As Helping You Get Out Of Subscriptions, But Keeps Charging Customers After Telling Them Their Own Accounts Are Closed
from the well,-look-at-that dept
We’ve been writing a bunch lately about DoNotPay, the massively hyped up “AI lawyer” run by Stanford dropout* Joshua Browder. Again, the company has received a ton of publicity regarding its “robot lawyer,” often from some of the publicity stunts that Browder pulls. Again, I think the underlying concept of using technology to help people solve problems is a good one. And that can include helping them to get better access to useful information that was, historically, kept behind expensive legal gates.
But throughout this saga, it’s becoming clearer and clearer that DoNotPay is smoke and mirrors, and very little is legitimate. Even Browder’s publicity stunts appear to be nonsense.
Still, as we keep pointing out, some of the fundamental services it claims to provide sound like they could be useful: for example, helping you cancel subscription services that companies don’t want you to cancel. We all know about these experiences. Companies make it easy to sign up, but impossible to cancel. DoNotPay is frequently lauded for helping users cancel such subscriptions. It brags about this service on its website.
That page has a long list of ridiculously poorly written “articles” that read as if they were all generated for maximum search engine optimization on how to cancel various services. Who knows if these articles were auto-generated with AI. Kathryn Tewson asked Joshua that question and he never answered.
Anyhoo, if your big claims on your website are how you can use DoNotPay to “fight corporations,” “beat bureaucracy,” and “find hidden money,” you’d think that your own service would make it easy to cancel your subscription without runaround and bureaucracy, and not keep charging people long after they’d been promised the subscription was canceled.
You’d think.
Over on Twitter, Sasha Perigo told the story of how she noticed that DoNotPay had been charging her $3/month for years last summer, despite her never remembering that she had signed up (though she admits it’s entirely possible she signed up and just forgot about it).
After emailing DoNotPay, customer support person “Quinn” told Sasha that her account had been cancelled. That email was on July 20th last summer, noting that her account would run through the end of the last monthly payment, on August 12th.
Except… Sasha just checked and found that DoNotPay continued to bill her $3 every month since then.
Kathryn Tewson leapt into action, tweeted out Sasha’s thread, and told Browder to refund Sasha’s money.
Now, Browder has Tewson blocked on Twitter, but it appears that either he or someone else on staff is monitoring her account, because 27 minutes after Tewson tweeted that out, Sasha noted that DoNotPay DM’d her and said it had cancelled her account and refunded her money. Of course, it also claimed that the mistake was that she had two accounts, except (as Sasha correctly notes) this makes little sense, as she was only charged once per month…
Also, it turns out that Sasha is not the only one. In her replies, Eve Kenneally pointed out that they have been trying to get DoNotPay to cancel for two years without success. Hilariously, Eve’s tweet shares a screenshot of an email from “Stacy” at DoNotPay saying “We’ll get this sorted out for you in no time.” And that was literally from February 1st 2021. Exactly two years ago.
Sasha DM’d DoNotPay about Eve’s account, and magically that was finally cancelled as well, with DoNotPay dubiously claiming that it hadn’t been able to find Eve’s account until Sasha called it to their attention. Then, they “sorted” Eve’s account within mere minutes.
Meanwhile, I’ll note that while the subscription fee for DoNotPay that Sasha and others were paying was 3/month(anumberlowenoughtomiss),ifyoutrytosignupnow,itsays3/month (a number low enough to miss), if you try to sign up now, it says 3/month(anumberlowenoughtomiss),ifyoutrytosignupnow,itsays36 every two months (which seems like a weird way to bill).
It sounds like DoNotPay jacked its prices way up (though, oddly, sticking to the 36number,astheoriginalpricingwas36 number, as the original pricing was 36number,astheoriginalpricingwas36 per year. A TechCrunch article from a year and a half ago says that the price was $36 every three months, as opposed to every two months today). I guess once you’ve raised $27.7 million dollars, and your tools are as flimsy as Tewson discovered last week, the best you can do to try to show your investors (including Sam Bankman-Fried and his bankrupt firm Alameda Research) that you’re growing the business is jack up prices and try to sucker in more users, and then make it hard to cancel.
But, really, you kinda have to ask yourself: how good can Browder’s “AI” service be at canceling other services when it can’t even cancel people’s own DoNotPay accounts? Do you really want to pay $36 every couple of months to find out?
* Earlier on Tuesday, Kathryn Tewson called out the fact that Browder both claimed to have a degree from Stanford in Computer Science and to have dropped out. After she tweeted about this, even though Browder blocks her, his LinkedIn magically changed to say that he had dropped out (though, oddly, it still said he had a B.S. degree from Stanford for a little while and then was updated a second time to just note that CS was his “major”).
Filed Under: ai, canceling subscriptions, joshua browder, subscriptions
Companies: donotpay
DoNotPay’s CEO Appears To Modify Donation Receipt After Being Called Out On Unfulfilled Promise
from the donotpay...-for-the-promises-that-you-made? dept
We’ve written a few stories lately about DoNotPay, the “robot lawyer” service whose gimmick of an automated AI-driven tool that would help users deal with challenges like getting out of parking tickets or cancelling subscription services that are difficult to get out of sounds like a really enticing idea. But there have long been questions about the service. While we’ve seen a bunch of truly impressive AI-generation tools in the last year or so, for years many companies claiming to offer AI-powered services often seemed to be doing little more than finding someone to hack together a complicated spreadsheet that the marketing folks would labels as “artificial intelligence.” It’s unclear how sophisticated DoNotPay’s technology actually is, though as guest poster Kathryn Tewson discovered last week, it sure seemed sketchy.
Kathryn, a paralegal with a preternatural skill at dismantling legal bullshit from people who pretend to understand the ins and outs of the law, sought to test the service’s ability to craft legal documents, and found that the whole thing raised a lot more questions than it answered with weird, potentially problematic language, questionable promises, and just the fact that out of multiple tries, the only document she actually received appeared to be produced by little more than legal madlibs, filling in a template. Furthermore, with the more “sophisticated” documents she requested, she was told they would take hours to send over, which seems strange for a robot lawyer. Of course, as her writeup got more attention, rather than deliver those documents, DoNotPay’s CEO, Joshua Browder, announced that he was shutting down these more sophisticated legal offerings, claiming that they were a “distraction.”
He claimed that he did this after various state bars suggested that his marketing stunt to have a lawyer argue in court while his “AI” whispered into the lawyer’s ear via an AirPod might result in him going to jail. Browder then made the rounds in the press implying that the criticism was from lawyers who were worried DoNotPay was going to cut into their business. In that interview he claims the pushback on his nonsense publicity stunts was “from lawyers,” but that because “there’s not a lawyer who will get out of bed for a $500 refund,” the company will instead focus on that area of business “so that they don’t come after us…”
But, the concerns from Kathryn and others are not about it cutting into the legal profession. I mean, personally, I’d love to see technology disrupt the legal business. It’s a business that could use quite a lot of disruption. The problem is that Browder’s propensity for publicity stunts means it often appears he’s vastly exaggerating what his company can do, and that’s a real concern when he’s advertising it for people involved in serious legal matters, like trying to navigate the immigration system (which, yes, was another offering from the company, which generated a lot of publicity but raised serious concerns from actual lawyers about what could go wrong).
Over the last few days, however, Kathryn keeps turning up more and more questionable behavior by Browder that is making him look like a naive, inexperienced kid pretending to run a serious company, rather than the CEO of a sophisticated “robot lawyer” company that has raised millions of dollars from sophisticated investors (and Sam Bankman-Fried).
Last week, for example, she found that almost immediately after a conversation with her in which she noted that she had not violated the company’s terms of service in running her test documents, DoNotPay’s terms of service were changed to say you were no longer allowed to create “test” documents that were not part of an authentic dispute. That was both oddly specific, and oddly… stupid. Who would ever want to use a Robot Lawyer you couldn’t first test to make sure it works well?
Over the weekend, things got even dumber. Kathryn noticed an earlier publicity stunt from Browder (who seems to spend more time thinking up dumb publicity stunts than making sure his robot lawyer actually works). He had promised to buy up medical debt for every retweet or follow of one of his tweets.
For what it’s worth, it appears he just deleted this tweet, and the rest of this article may explain why. First off, we should note that he’s correct: medical debt is a scourge. We’ve discussed how medical billing is a complete economic scam. For a variety of reasons, the US healthcare space is simply designed to siphon away every penny someone has by the time they die (sometimes hastened by that same medical system). It’s… not great. So, hey, I appreciate efforts to forgive medical debt (though I’d appreciate efforts to fix the underlying system more).
But Kathryn noticed that despite the promise to “post receipts” there were no such receipts published:
After discussing the related issue of how medical debt is often sold for pennies on the dollar, meaning that he could appear to be a lot more generous than he was in reality, Browder jumped into the conversation to claim that he absolutely did make the donation in question for 500,whichhelaterclaimedboughtup500, which he later claimed bought up 500,whichhelaterclaimedboughtup50,000 worth of debt.
In that (also since deleted!) tweet, Browder presents a receipt from the non-profit RIP Medical Debt (which was created for this kind of purchase-and-forgiveness of medical debt), showing that he paid $500, allegedly on December 2nd of last year.
But (again, never try lying to Kathryn, who seems to be the living embodiment of Natasha Lyonne’s character in Poker Face), Kathryn noticed that something was a little odd in the receipt: while the font of the dates matched the font of the rest of the notice, they did not line up properly in the image, suggesting that he might have photoshopped the date. In an amazing bit of sleuthing, Kathryn highlighted how the dates were posted a little below the line where they should be. It’s something you could really only see if you inserted guidelines and zoomed in close:
She then purchased some debt herself just to see how the email receipt shows up, and found that on her own donation, the dates lined up perfectly with the guidelines:
At the very least, it’s pretty strong circumstantial evidence that the dates on Browder’s screenshots were faked.
But then Kathryn took it up on a notch. She reached out to RIP Medical Debt and asked about Browder’s donation. RIP Medical Debt confirmed to her that Browder’s donation was not made on December 2nd, but rather it was made on January 29th, at 12:36 am EST (Kathryn shared the email with me so I can confirm RIP Medical Debt’s statement on this). 12:36 am EST was exactly four minutes after Kathryn originally tweeted her concern as to whether or not Browder ever actually did buy up the debt he promised.
Four minutes.
He posted the screenshot 17 minutes later (which would be enough time to sloppily edit the receipt to change the date).
Browder (who appears to have then gone back and deleted all of the tweets mentioned in this article) did complain about how people were “criticizing a donation.” Except no one is criticizing the donation. The donation is great. Make more of them, Josh.
What Kathryn was criticizing was how you used the claim of paying off medical debt as a publicity stunt when it was unclear that you had actually followed through and which, it now seems clear, you only followed through on months afterwards, and four minutes after Kathryn called it out. And then it appears that you fudged the date to hide that fact. Also, the fact that since you can buy medical debt for pennies on the dollar, you can appear to be way more generous than you were actually being, especially since your original tweet did not promise to pay $10 for each retweet or follow, which would have been more significant.
This would appear to be extremely questionable behavior, and not the kind of behavior that makes one say “yes, I’m going to trust this company to help me resolve legal disputes.”
Perhaps Browder’s next project should be building the “world’s first AI CEO” to replace himself. At this point, I’m not sure it could be much worse or less trustworthy than the human currently in that position. Or, hell, maybe he should ask his “AI lawyer” what it thinks of all this. I decided to ask ChatGPT what it thinks and got a pretty good answer:
And modifying a date to make it appear that you did the thing you promised is also not a good look according to ChatGPT:
Not bad, ChatGPT. Not bad at all.
Again, I wish that DoNotPay actually could do much of what it claims to do. It sounds like it could be a really useful service, one that we would actually like to see more widely implemented. But the antics and shenanigans over the last few months should raise serious concerns about why anyone would trust the company with literally anything. Browder’s seeming unwillingness to be truthful in his discussions on all of these things does not bode well.
Filed Under: ai lawyer, donations, joshua browder, medical debt, modified receipts, publicity stunts, robot lawyer, trust
Companies: donotpay, rip medical debt
AI Lawyer Has A Sad: Bans People From Testing Its Lawyering After Being Mocked
from the DoNotTest-DoNotPay dept
Well, a lot has happened since I first started looking into the “World’s First Robot Lawyer,” from DoNotPay. First, Joshua Browder, DoNotPay’s CEO, reached out to me via direct message (DM) and told me he would get me access to my documents by 2 PM the next day – Tuesday, January 24th – saying that the delay was caused by my account being locked for “inauthentic activity,” a term he did not explain or define. Then, Josh claimed he was going to pull out of the industry entirely, canceling his courtroom stunt and saying he would disable all the legal tools on DoNotPay.com. He said he was doing it because it was a distraction, but the fact that he cited exactly the same two documents that I was waiting to receive seemed like a hell of a coincidence.
But plus ça change, plus c’est la même fucking chose, as the poet says. Here’s what hasn’t changed: I still don’t have my documents, and Josh still hasn’t answered the questions I asked him like he said I would.
In his direct message, Josh said he would be willing to answer any questions I asked in good faith. I took him at his word, and responded to the email he sent me announcing his pullout with the following four questions:
- Can you describe for me the process DoNotPay used to identify the relevant law for a demand letter? (Cf. “Based on your location, DoNotPay will generate a formal demand letter on your behalf with the most relevant state legislation regarding defamation,” from here: https://web.archive.org/web/20230127063724/https://donotpay.com/learn/cease-and-desist-order/)
- Were humans involved in the generation of any client documents described by anything under your “Legal Tools” section? I don’t mean the creation of the templates, etc., I mean in the production of a document based on client responses to prompts.
- Are the articles in the “Learn” section of your website written by ChatGPT or equivalent, or by humans?
- Who signed the subpoena for the officer in the traffic case that was referenced in your now-deleted tweet?
I asked all these questions in good faith, and for good reasons. Josh represents — over and over and over and over again — that DoNotPay features a robot lawyer with artificial intelligence, going so far as to say that it uses AI instead of “human knowledge.” The sole document I got was one that didn’t make any kind of promise of customization or that it would contain “the most” relevant legislation for anything; the ones that did require that kind of analysis were the ones that got hung up in multi-hour deadlines and never ultimately delivered. Given how much weight he puts on these claims, I think it’s fair to interrogate and test them.
The articles, or “blog posts” as Josh calls them, are a slightly different situation. There are a TON of them, and they are all published without dates or bylines. But many of those articles have significant errors, both legal and factual, and if someone relied upon them, they could be in big trouble. And while I didn’t actually expect him to answer the question about the subpoena, he opened the door by bragging about it in the first place, as far as I’m concerned.
(Only two sentences in this screenshot above are completely accurate. I’m not going to represent to you which two they are, because I am not a lawyer.)
This email, to my great disappointment, went completely ignored.
It wasn’t because he was too busy taking down all his bots, either. On Thursday night, I logged back into the site to check, and discovered that all the prompts I had accessed before were still available, save the two that Josh mentioned specifically in his tweet — the defamation demand letter and the divorce settlement. But even those were still being advertised; every “blog post” on the site has a signup teaser in it advertising access to the site’s legal and other services with no sign that they’re inaccessible until after DoNotPay has your money. One particularly egregious blog post that advertised “free legal advice” to those looking for help navigating the immigration process to become American citizens finally pushed me over the edge, and so I pinged Josh to remind him that I was still waiting for my documents and an answer to my questions, let that sit for a bit, and then started another thread about all the ways he was being less than forthright with the truth. In the course of writing that thread, I discovered that I was suddenly banned from the site; not only could I not log in, but any attempts to do so gave me an error message that read merely “something went wrong.”
It took Josh less than an hour to get back to me via DM, informing me that my money had been completely refunded and complaining that I was lying about the “bots” still being up, although he later admitted that only 7 bots had come down in the previous 36 hours (out of an advertised 1,000):
When I told him I had tested them myself and even generated new documents and cases, he demanded “Was your usage authentic?” I responded “It certainly complied with every provision of the Terms of Service.” At this point, Josh disappeared from the conversation.
After this pause had stretched out for a while, something was kind of nagging at me. I went back and looked at that question and answer again and thought “what is it about the Terms of Service that suddenly had him needing to leave immediately?”
By sheer coincidence, someone — not me (at least I don’t think so) — had archived the DoNotPay Terms of Service just about exactly when I started tweeting my thread, so I know exactly what they said then.
It also meant that I could spot exactly what had changed between that time and this one, a mere two hours later: Josh added a clause to the TOS prohibiting users from testing the website prior to using it in a live dispute.
If you can’t see that, right after I told him I was not violating his terms, he appeared to add this to his terms:
You represent that any dispute or request submitted is an authentic problem you are having. You are responsible for any damages to DoNotPay or others from fake, inauthentic or test disputes.
So now “test” disputes violate the terms?
This change was made after he banned me, without any warning. He claims he told me to “keep it real,” but he absolutely did not, and his claim that I “triggered his anti-spam” by making 10 or 15 new cases seems to run against his site’s promises that one can create an “unlimited number of documents.”
He didn’t answer my questions outside of saying “no the letters aren’t being hand typed out and no we didn’t write them,” which… didn’t answer my questions in the slightest. And then he blocked me.
So there you go. Joshua Browder, CEO of DoNotPay.com, would rather block me, ban my account, retcon his terms of service to disallow any test usage at all, and claim to pull out of the “Legal Services” industry that his site is PLASTERED with branding for, rather than show me the two documents I generated and tried to buy.
I wonder what he doesn’t want me to see?
Filed Under: ai, ai lawyer, joshua browder, robot lawyer, terms, testing, unauthorized practice of law
Companies: donotpay
If The AI Lawyer You Built Can’t Keep You Out Of Jail, Maybe It’s Time To Hire A Real Lawyer
from the a-fool-for-a-lawyer dept
So, we’ve written a few times about DoNotPay, the supposedly AI-powered “robot lawyer” that was initially designed to help you contest parking tickets but then expanded to helping (usefully) with a bunch of consumer annoyances, like cancelling accounts, obtaining owed refunds, and the like. But it’s also got some shadiness in its past, like the time it wanted to automate sharing of streaming account credential cookies. And, while there have long been pretty serious questions about what things you should trust it to do, a couple years ago the company raised a round of venture capital from such trustworthy investors as (checks notes) Sam Bankman-Fried, along with a16z, and DST Global, which has a bit of an interesting history.
Anyway, since then it has ramped up some of its attempts to generate publicity via very questionable stunts. Earlier this month it announced that its “AI lawyer” would represent clients in traffic court via an earbud, and then went even further by promising a very publicity-stuntalicious $1 million to any lawyer who would use its AI lawyer via an earbud to argue in front of the Supreme Court. Leaving aside that such tech isn’t allowed in the Supreme Court, at one point CEO Josh Browder said that they would effectively abuse the rules the court has in place for making courts “accessible” to get the ear buds in. Which is, really not great.
All of these publicity stunts brought with them some scrutiny. And, as we posted this week, Kathryn Tewson (who is one of the most astute legal commentators out there) decided to test out many of its “legal” services. She found them extremely lacking. And maybe that’s fine in low stakes situations like parking tickets or consumer refunds and complaints. But it could do real harm to people relying on the service for larger things. Personally, the fact that it seemed to be encouraging people to make questionable defamation threats seemed pretty problematic to me.
On Wednesday morning, Browder announced that the company was (1) ditching the plans to be used in court and (2) stepping back from some of those more harrowing legal jobs (coincidentally, the very ones that Kathryn had tested out). Of course, rather than admit that the company shouldn’t be doing those things and was not qualified to be doing them, Browder did the techbro equivalent of claiming that it was all a “distraction” with very little usage. Oh, and also, that state bars were threatening him with potential jail time for the courtroom antics:
I guess Browder didn’t trust his AI lawyer whispering in his AirPods to keep him out of jail for the unauthorized practice of law. And if he’s unwilling to eat his own dogfood that way, perhaps he shouldn’t be offering it to other people.
The “distraction” claim is not all that believable, honestly:
If they were a distraction, they were a self-created distraction, in which Browder used all this to generate publicity for his not-ready-for-prime-time “legal” services that people with real problems might have otherwise come to rely on.
I actually do think that there is some interesting ideas in the underlying concept behind this company. Using smarter tools to help automate the horrible processes that large companies and governments put people through to get them to give up and not exercise their rights is a good idea. But it feels like someone should sit Browder down and explain to him the value of actually delivering a good and useful product over hyping up something that can’t do what it promises.
Filed Under: ai lawyer, joshua browder, robot lawyer
Companies: donotpay