How to design long running browser tasks with Roamkiln

Theo Brandt, Nia Okafor & Lena Sato
Engineering
8 min read

TL;DR: Tasks that run for hours fail in different ways than tasks that run for seconds. This guide covers the patterns we use to keep long browser jobs on track: checkpoints, small steps and clear stopping points.
A short task can simply start again when something goes wrong. A task that walks through hundreds of pages cannot. Losing an hour of progress to one slow page is the most common complaint we hear about long jobs.
Break the work into steps
The first rule is to make each step small enough to retry on its own. A step should do one thing, record what it did, and leave the browser in a known state for the next step.
Save a checkpoint after every step, not only at the end.
Keep page state in the session so a retry does not need to sign in again.
Set a time limit per step instead of one limit for the whole job.
Resuming after a failure
When a step fails, call session.resume(checkpoint) instead of starting over. Roamkiln restores the cookies and storage saved with that checkpoint, and the agent picks up where it stopped.
If the same step keeps failing, stop the job and surface the page it was on. A clear failure with a replay is easier to fix than a job that retries forever.
Watching a job while it runs
Long jobs are easier to trust when you can see them. Live view and step logs show which page the agent is on and how far it has come, without waiting for the run to finish.
Talk to us about Roamkiln for your team
Get started with sessions for your agents
Get in touch
Keep reading

Introducing Vaults in Roamkiln Functions
Mara Velde & Jonah Price
Engineering

What is Agent Passport?
Ilse Moreau
Engineering

Roamkiln and Keystone: shared sign in for browser agents across every app your team uses
Roamkiln Team
Company

What is Kiln Script?
Mara Velde
Engineering
View all blog posts
