Academic Review  ·  Productivity

Generative AI and Workplace Productivity

What emerging research reveals about productivity gains, differences between workers and the conditions that determine whether AI improves real performance.

Read the Review
Introduction
Unlike the future-of-work literature I reviewed previously, which draws on decades of labour market data, the productivity research on generative AI specifically is young, the earliest rigorous controlled studies date to 2023. I think that youth is worth naming plainly before summarising what the studies actually found.

What makes this body of work valuable despite its youth is its method. Rather than surveying opinion or extrapolating from technical capability, the strongest studies in this area are randomised controlled experiments: real workers, given a real task, randomly assigned to complete it with or without AI assistance, with output measured against consistent quality criteria. This is a meaningfully stronger evidentiary basis than the exposure estimates discussed in my previous review, because it measures what actually happened rather than what plausibly could happen.

This review covers four of the most methodologically careful studies published to date, on professional writing, customer support, software development and management consulting, and draws out what I consider the most robust and recurring finding across all four: that generative AI's productivity effect is not uniform across workers, and that the pattern of who benefits most carries real implications for how organisations should think about deploying these tools.

I want to flag one limitation before I begin, in the interest of the accuracy this review is committed to: several of these studies remain working papers or have been published relatively recently, meaning the peer review and replication process that eventually strengthens confidence in economic findings is still underway. I have represented the figures reported by the original authors as accurately as I can, and I would encourage readers to treat early-stage empirical findings, however careful the method, with the appropriate provisional caution that any single study, however well designed, warrants before a wider body of replication exists.

01. Writing Tasks: Faster, and Judged Better
Noy and Zhang's 2023 study, published in Science, remains one of the cleanest experimental demonstrations of generative AI's effect on a specific, common category of professional work: mid-level writing tasks.

The study recruited several hundred college-educated professionals and assigned them writing tasks representative of real occupational duties, press releases, short reports, analyses, with half the participants given access to ChatGPT and half working unaided. The AI-assisted group completed their tasks in substantially less time, and independent evaluators, blind to which group produced which piece of writing, rated the AI-assisted output as higher quality on average (Noy and Zhang, 2023). This combination, faster and judged better, rather than faster at the expense of quality, is the finding that made the study particularly notable, since a great deal of prior scepticism about AI-assisted work assumed a straightforward speed-versus-quality trade-off that this experiment did not find.

The detail I find most important, and the one I think is most often dropped when this study is summarised in secondary reporting, is that the productivity gain was not evenly distributed. Participants who scored lower on the task in the unassisted condition, in other words, the weaker writers in the sample, showed the largest gains from AI assistance, while the strongest writers in the sample showed a much smaller benefit. Noy and Zhang describe this explicitly as a compression of the quality distribution: AI assistance narrowed the gap between weaker and stronger performers rather than lifting everyone by a uniform amount.

02. Customer Support: The Novice-to-Expert Pattern Confirmed
Brynjolfsson, Li and Raymond's study of a large customer support operation is, in my view, the single most important paper in this review, because it observed the same pattern as Noy and Zhang under real operational conditions rather than a laboratory task.

The researchers studied the staggered rollout of a generative AI assistant to several thousand customer support agents at a software firm, using the tool's uneven deployment schedule as a natural experiment. They found that access to the AI assistant increased the number of issues resolved per hour, with the effect concentrated overwhelmingly among newer and lower-skilled agents, while the most experienced, highest-performing agents saw little to no measurable benefit, in some specifications, effectively none (Brynjolfsson, Li and Raymond, 2023). The authors interpret this as the AI system effectively codifying and disseminating the tacit knowledge and best practices that experienced agents had already accumulated through experience, giving less experienced workers faster access to expertise that previously took time on the job to acquire, while offering little additional benefit to agents who already possessed it.

The paper also reported effects on customer sentiment and agent retention that moved in a positive direction alongside the productivity gains, and, importantly for the "compression" pattern first identified by Noy and Zhang, found that the gap in performance between the least and most experienced agents narrowed substantially following the tool's introduction. Two independent studies, in different work settings, using different methods, converging on the same underlying pattern is a considerably stronger basis for confidence than either study would provide alone.

Central Insight

The most consistent finding across this literature is not "AI makes people more productive." It is that AI's productivity effect is heterogeneous, and specifically that it tends to raise the floor more than it raises the ceiling, helping less experienced or lower-performing workers close the gap with more experienced ones, while offering diminishing or negligible returns to those already performing at a high level. Any organisational plan built on an assumed uniform productivity uplift across a workforce is building on a pattern the evidence does not actually support.

03. Software Development: A Speed Gain With a Different Shape
Peng et al.'s study of GitHub Copilot offers a useful contrast, because the task, writing code to a defined specification, is more structured and objectively verifiable than either writing or customer conversation, and the study design reflects that.

The researchers recruited software developers and randomly assigned them to complete a defined coding task, implementing an HTTP server in JavaScript, either with or without access to GitHub Copilot, an AI coding assistant. Developers using Copilot completed the task substantially faster than the control group, with the paper reporting a completion-time reduction in the range of roughly half the time taken by the unassisted group, without a corresponding degradation in code quality by the criteria the study used (Peng et al., 2023). This is a somewhat different shape of finding from the writing and customer support studies: rather than measuring the redistribution of performance across a skill spectrum, the coding study primarily measured a time saving on a well-specified, verifiable task.

I think the coding study is a useful reminder that the "compression" pattern found elsewhere in this literature is not a universal law of AI-assisted work, it appears to depend on the nature of the task, and specifically on whether the task rewards tacit, experience-accumulated judgement of the kind the customer support and writing studies were measuring, versus a more procedural, specification-driven task where the main constraint is speed of correct execution rather than accumulated expertise.

04. The Jagged Frontier: Where AI Helps and Where It Quietly Hurts
Dell'Acqua et al.'s study of management consultants at Boston Consulting Group is, I think, the most important corrective in this entire literature, because it is the study that most directly documents AI making performance worse under specific, identifiable conditions.

The researchers assigned several hundred BCG consultants a range of realistic consulting tasks and randomly gave a subset access to GPT-4. For tasks the researchers classified as within the AI's capability frontier, tasks the underlying model was actually good at, consultants using the tool completed more tasks, completed them faster, and produced work rated more highly by evaluators than the unassisted group. But for a specific subset of tasks the researchers deliberately designed to sit just outside the model's reliable capability, tasks requiring the kind of business judgement the model was more prone to getting subtly wrong, consultants using AI performed measurably worse than the unassisted control group, and were more likely to accept an incorrect AI-generated answer than to catch the error themselves (Dell'Acqua et al., 2023).

The authors coined the phrase "jagged technological frontier" to describe this pattern, the observation that AI capability is not evenly distributed across task types in a way that maps cleanly onto human intuitions about task difficulty, meaning workers cannot always accurately judge, in the moment, whether a given task sits inside or outside the model's reliable zone of competence. This finding is, in my reading, the single most important piece of evidence in this entire review for why unmonitored, ungoverned AI adoption is a genuinely different proposition from monitored, governed adoption, the same tool that reliably improves performance on one task can reliably degrade it on a superficially similar one, and the worker using it may have no clear signal, in the moment, of which case they are in.

05. Where I Think This Evidence Base Is Still Thin
I want to push back on how neatly this literature can be summarised, because I think the tidiness of the "compression" narrative slightly outruns the actual breadth of evidence behind it.

All four studies I have discussed measure short-term, task-level productivity, minutes or hours of work, observed over weeks or months. None of them, to my knowledge, has yet tracked what happens to skill development, career trajectories, or long-run organisational capability over a period of years. There is a reasonable hypothesis, which I have not seen rigorously tested, that if less experienced workers routinely lean on AI assistance rather than developing judgement through the harder, slower route of direct experience, the short-run productivity gain identified in these studies could coexist with a long-run erosion of the very expertise the AI tools are currently substituting for. I raise this not because I have evidence it is happening, but because I have not seen evidence ruling it out either, and I think a review committed to representing this field honestly should say so.

I would also note that three of the four studies discussed here were conducted in white-collar, English-language, professional-services-adjacent settings, customer support, consulting, general knowledge work, and generalising confidently to manual, technical or non-English-language work environments would be running ahead of what this specific evidence base can support.

Conclusion
The controlled evidence to date supports a real productivity effect from generative AI on several categories of professional work, but a highly uneven one, both across workers and across task types within the same job.

Writing, customer support, coding and consulting studies each found meaningful gains, with the most consistent and best-evidenced pattern being that less experienced or lower-performing workers benefit disproportionately, narrowing rather than widening performance gaps. The BCG consulting study's "jagged frontier" finding is, in my view, the most consequential result in this body of work for practical deployment decisions: the same tool that reliably improves performance on one task can reliably degrade it on a task that looks similar to the worker using it but sits outside the model's genuine competence.

The practical implication I draw from this evidence, and the one I think matters most for organisations acting on it, is that productivity gains from generative AI are real but conditional, conditional on the task, conditional on the worker's existing skill level, and conditional on some form of oversight or training that helps workers recognise when they have moved from inside the reliable frontier to outside it. Deploying these tools without attending to that condition is deploying against the specific finding this literature has been most consistent about.

References
01

Noy, S. and Zhang, W. (2023) "Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence," Science, 381(6654), pp. 187–192.

View source
02

Brynjolfsson, E., Li, D. and Raymond, L. (2023) "Generative AI at Work," National Bureau of Economic Research Working Paper, No. 31161.

View source
03

Peng, S., Kalliamvakou, E., Cihon, P. and Demirer, M. (2023) "The Impact of AI on Developer Productivity: Evidence from GitHub Copilot," arXiv preprint, arXiv:2302.06590.

View source
04

Dell'Acqua, F., McFowland, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F. and Lakhani, K. R. (2023) "Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality," Harvard Business School Working Paper, No. 24-013.

View source
Scroll