9 Strengths and weaknesses of line-by-line completion
You’ve now used inline completion, comment-driven completion, and Gander. Before we move on to agents in the next section, it’s worth stepping back and asking: when is the line-by-line approach the appropriate choice?
I wrote about this at length in a post called “Mitigating the risks of AI coding tools with judicious use for research tasks”. The short version: being judicious with AI coding tools mitigates most of the risks that concern researchers — wasted time, electricity use, cybersecurity, and data privacy. By judicious I mean using a tool in a targeted way for a specific task where you already know it works well, and carefully preparing the context and prompt rather than just throwing AI at a poorly defined problem.
Line-by-line completion is, by construction, judicious. It operates on one line or one small block at a time. The “context” it needs is just your open script (or, with Gander, your R environment). There’s no autonomous multi-step plan for it to wander off course on. If it gets something wrong, you see it immediately and reject the suggestion.
That’s the strength. It’s also the weakness: line-by-line completion can’t hold a whole analysis in its head. It won’t notice that the model you fit three scripts ago used a different family than the one you’re fitting now. It can’t restructure a project, write a report, or decide whether a negative binomial or a Poisson GLM is the right call. For learning a new method, a more conversational setup can work better — LLM tutors have been used to accelerate method development in routine analytical workflows (Le et al. 2025).
Another benefit of line-by-line editing is it forces you to have a clear workflow and your ‘prompts’ are short (just the code in your file). This lets you use a cheaper, smaller model. Cheaper models use less energy, and a surgical approach — giving an assistant access to fewer files and less context — creates fewer security risks too (more on that in Section 4). So the same discipline that saves you time also tends to save electricity and reduce your security risks.
The leading alternative to line-by-line editing is Agents, which we’re about to meet.
Think of one task in the last week where you used AI to help you, it could be coding, writing or something else. Did you end up spending more time fixing or rewriting what it gave you than if you’d written it yourself?
A well-known study by METR found that professional software developers thought they’d go faster using AI tools, but on a set of real tasks they actually went slower. I think this problem is probably worse for researchers than for software developers. Developers have a comparatively easy way to verify success — the app either works or it doesn’t. Research is harder to verify, because by definition you’re doing something that hasn’t been done before; some fields take decades to confirm a result. That makes it easy to burn time “fixing up slop,” correcting mistakes, or rewriting an answer into your own style (if you’re tired of seeing the same “on the other hand” LLM sentence structure showing up in blogs and papers everywhere, you’re not alone).
The trick is to experiment just enough to know at what points in your workflow AI genuinely helps, then put the hard mental work in up front — on your prompts and your context — rather than hoping a bigger model will compensate for a vague ask. I find it’s best to start this process with simpler tools, like line-by-line completion, before moving onto more complex tools, like agents.