Experiment 2's skip rate missed the authors' own cutoff

The message below is a constructed example, written so one kind of forwarded sentence is visible. No study group sent that exact text in this article.

Someone in a study group forwards this: “Berkeley just proved that 10 minutes with an AI tool means you lose the ability to persist. This week, nobody opens an assistant while doing the homework.” The sender did not attach the paper. Before you reply, open the study itself and write three lines about the experiment the authors treat as the cleaner one. Do not rewrite the week’s study rules from the forwarded line.

That line points at a Berkeley News story dated 9 October 2026. The news page says a research group found that about 10 minutes of relying on an AI tool reduces the ability to focus and to persist at hard tasks. It also says 1,222 people took part in the randomized trials, that the first experiment had 354 people doing fractions, that a larger experiment had 667, and that a reading experiment had 201. That is how the news page tells it. The paper, posted on arXiv as 2604.04721, tells experiment 2 more tightly than the news page does.

Three numbers from the second experiment

Experiment 2 recruited 667 people who were in the United States, through Prolific, a platform that pays people to complete online surveys. Each person received 3.40 US dollars. The authors estimate the session at about 15 minutes. That sum is pay for participants in the United States. It is not the price of a piece of software, in Vietnam or anywhere else.

Everyone worked on fractions. First came three one-step problems, with no assistant. Then one group received a sidebar with GPT-5, a model from OpenAI, already loaded with the problem and the solution, so asking for the answer returned the solution. The other group had no such assistant. Then the assistant was turned off, and both groups worked the same three final problems. A participant could press skip. Because they had been told that a wrong answer would not cost them money, the authors use the skip button as a sign of not continuing, separate from getting the problem right or wrong.

After removing people who failed an attention check, and people who did not get at least one of the first three easy problems right, 585 people remained: 308 who had had the assistant, and 277 who had not. The exclusion rates were close, 11.7% and 12.9%. The authors include this to address a flaw in experiment 1. In that first experiment, the way people were removed could leave, inside the assistant group, people who could not do basic fractions but had submitted an answer the machine supplied.

On the three final problems in experiment 2, the group that had had the assistant averaged 0.71 correct. The group that had not used one averaged 0.77. The authors report that this difference passes the threshold they use when they call a result statistically significant (p = 0.020). The skip rate was 0.10 compared with 0.07. For those skip numbers across the whole group, the authors write that the difference did not reach that threshold (p = 0.239).

Berkeley News describes this larger group as getting the problems wrong, or giving up entirely, once the assistant was taken away, while the group without the assistant persisted and succeeded more at the end. That sentence is how the news page tells it. In the paper, the “gave up” part for the whole group in experiment 2 is not enough to support a conclusion. The authors do keep the finding that the group did worse on accuracy right after the assistant was turned off.

Experiment 1, with 307 people left after the exclusion, showed a correct rate of 0.57 compared with 0.73, and a skip rate of 0.20 compared with 0.11. The authors themselves are the ones who attach the warning about how that sample was trimmed. The figure 354 on the news page is the number recruited at the start, before the exclusion the authors later treat as not clean.

Asking for the answer is not the same as asking for one step

At the end of experiment 2, people in the assistant group reported how they had used it. 189 people, that is 61%, said they mainly asked for the answer. 82 people, 27%, said they asked for a hint or an explanation. 37 people, 12%, said they did not use it. The authors compared those reports with the chat logs for the group that said it had not used the assistant. Most of that group barely sent a message.

On the three final problems, people who said they asked for the answer averaged 0.65 correct, compared with 0.77 for the group that had no assistant. Their skip rate was 0.13 compared with 0.07, and the authors report that this difference passes the threshold (p = 0.033). People who asked for hints averaged 0.76 correct, close to the group with no assistant.

The authors say plainly that this cut is a cross-section of people who were already in the assistant group. It shows where the gap sits. It does not prove that asking for the answer caused the skipping, because people chose how to use the tool. They were not randomly assigned to “you may only ask for the answer” or “you may only ask for a hint.”

Experiment 3 moved to reading comprehension, using items taken from free SAT practice material online. The SAT is a standardized test used in college admissions in the United States. These were not questions from a real exam. The sample that remained was 168 people. The group that had had the assistant averaged 0.76 correct, compared with 0.89, and skipped at 0.08 compared with 0.01. In this experiment the authors report that both differences pass the threshold. An answer given in under five seconds was also counted as a skip, because there was not enough time to read the passage. So a greater tendency to stop shows clearly on this reading task, and does not show clearly enough when the whole fractions group from experiment 2 is taken together.

The paper also states a limit near the opening. The authors advise readers not to take the results as broad evidence that an AI assistant reduces persistence, and the ability to work unaided, on every kind of task. The summary written for the public says that if the effect piled up across many days, that would be a concern. That is something they did not measure. The session stopped right after the assistant was turned off.

What to send back to the group

Three lines are enough for a reply.

You can write this: the cleaner experiment measures the moment the assistant is turned off, with adults in the United States working laboratory fractions. It does not measure a week of your own class. The whole group that had the assistant scored lower on the three final problems, 0.71 compared with 0.77. The skips for that whole group, 0.10 compared with 0.07, are a difference the authors do not yet treat as large enough to conclude from. The higher skip rate sits with the people who said they asked for the answer, and the paper does not treat that split as evidence of cause and effect.

If you study by asking the assistant for the finished answer, the change this week is narrower than a ban. The next time you work a problem, ask for one step, then write the following line yourself after the assistant panel is closed. The sentence on the news page is not enough to forbid every 10 minutes with an assistant open. Neither the news page nor the paper measured your study schedule.

If the message says the study proved that Vietnamese students lost persistence, or that the students in your own class did, that sentence is not in these two sources. The sample is participants in the United States.

Sources

1. Liu, Christian, and colleagues — AI Assistance Reduces Persistence and Hurts Independent Performance, arXiv:2604.04721 — the design of the three experiments, the samples after exclusion, the correct rates and skip rates in experiments 2 and 3, the share who said they asked for the answer, and the limit that the results are not evidence for every kind of task.
2. Berkeley News, 9 October 2026 — Using AI for just 10 minutes erodes your ability to persist at hard things — the university news story, with the 10-minute sentence, the total of 1,222 people, and the account of the experiments as 354, then 667, then 201.