Lighthouse says pass and the search report says fail? Lab runs and real-visitor data answer different questions

Two kinds of number

Lab data is one page load, run by a tool under fixed conditions: one simulated device, one network, one place, usually an empty cache. Field data is what real visitors experienced, collected over time and summarised. Google's Core Web Vitals thresholds are written for field data: LCP of 2.5 seconds or less, INP of 200 milliseconds or less and CLS of 0.1 or less, each at the 75th percentile and reported separately for mobile and desktop. A page passes only when all three meet their targets at that percentile. INP replaced First Input Delay as a Core Web Vital in March 2024, and it needs real interactions, so Lighthouse cannot measure it; Total Blocking Time is the lab stand-in.

• Lab: repeatable, good for finding causes and testing a fix before release.
• Field: representative, good for deciding what matters and for the verdict.

Why they disagree

web.dev lists the usual reasons. A lab run uses one device and connection, while real traffic spans many. Lab tests start with an empty cache, while returning visitors have files stored. In the field, the browser stops looking for a larger element once the visitor scrolls or interacts, so the LCP element can differ. Field layout shift counts shifts across the whole life of the page, while a lab test usually captures only the load. Personalised or late-inserted content such as ads can shift a real page in ways a generic test never sees. So a lab pass does not prove that visitors are happy, and a field fail does not mean the lab is wrong.

• Good lab, poor field: the lab profile may not match your visitors, or shifts happen after load.
• Poor lab, good field: field data covers only people who managed to load the page.
• Run lab tests several times: Lighthouse documents run-to-run variation and says the median of five runs is about twice as stable as a single run.

What the real-visitor reports can and cannot tell you

The reports are slow to move and sometimes absent. They use a trailing 28-day window, so a fix released today is blended with weeks of earlier visits. A page needs enough visits to appear; PageSpeed Insights falls back to data for the whole site, and shows nothing at all when even that is too small. Search Console groups similar pages and applies one result to the group, evaluates mobile and desktop separately and lists only indexed pages. A new or low-traffic page may therefore have no field verdict, which is an unknown, not a pass or a fail.

• Do not read silence as success.
• Allow weeks before judging a change against field data.
• Find out which group a page sits in before assuming a result is about that page.

Agreeing a fair pass before work starts

Use lab data to define the acceptance test and field data to track the outcome. A fair lab test names the page, the metric, the device and network profile, the number of runs and the statistic, for example the median of five runs on each of two profiles. It also names what must not get worse and which elements must still appear. A field target needs a date no earlier than the window allows and cannot be a condition of payment, because it depends on your visitors.

• Write down the threshold for the chosen metric and how many runs.
• Name a second profile that must not worsen.
• Note any known gap between lab and field and say how you will watch it.

How the paid jobs use this

Our one-page job (posted test price from £495, untested) is accepted on the agreed lab runs; real-visitor figures are shared when they have updated but do not decide payment. Our monthly service (posted test price £295 a month, untested) re-measures up to five named pages each month and after up to four releases a month that you tell us about, reports the median and all runs, and opens a fix or explains what only you can change; it has no round-the-clock cover or response-time guarantee. Neither promises ranking, traffic or sales. Send the page address and the report you were given in your first enquiry, not credentials.