What Should You Do When Two Tools Disagree About a Number?

What should you do when two tools disagree about a number?

First work out whether the difference is structural or broken, because those need opposite responses. A structural difference is permanent, explainable, and should be documented once. A broken pipeline is a bug you fix. Most people treat every mismatch as the second kind, spend a day hunting, and find the first kind.

This matters more now that so much reporting is automated. When a person builds a report by hand they notice the oddity and think about it. When a workflow assembles it at six in the morning, nobody thinks about anything, and the client reads two numbers that contradict each other over breakfast.

I build these pipelines for clients, and reconciliation is the part I spend the most design effort on. Pulling data is easy. Deciding what to do when the sources disagree is the actual work.

Why do Search Console and your analytics never match?

Because they are measuring different things, and Google says so explicitly in its own documentation. That documentation is worth reading once properly, because it turns a mystery into a list of known behaviours.

Google states that to protect user privacy the Performance report does not show all data, and gives the example that it omits some queries that are searched a very small number of times. It also says some processing of the source data might cause stats to differ from other sources, giving the elimination of duplicates as an example.

Then there is the one that catches almost everybody. Google notes that some tools, such as Google Analytics, track traffic only from users who have enabled JavaScript in their browser. Search Console is not counting page views in a browser at all. One is a record of what Google served, the other is a record of what a script executed. Those will never converge, and they are not supposed to.

Which differences are structural and which are bugs?

Structural differences are stable and directional. The same source is consistently higher by roughly the same proportion, week after week, and you can name the mechanism. A bug is sudden, or inconsistent in direction, or affects one segment and not others.

The test I apply is whether the gap holds across time. Pull thirty days rather than one, put both numbers side by side, and look at the ratio. A steady ratio with a known cause is a documented difference. A ratio that was steady and jumped on a Tuesday is an implementation change, and the question becomes what changed that day.

Aggregation is a structural difference people mistake for a bug constantly. Google's documentation notes that chart totals can sometimes differ from table totals, and says this is usually due to differences in aggregation, giving property versus page as the example. That is not an error in your pipeline. It is two correct answers to two slightly different questions.

How much disagreement is acceptable?

You have to decide, write it down, and then stop relitigating it. I am not going to hand you a percentage, because I have no source for an industry tolerance and I am not going to invent one. What I can tell you is that having an explicit band is what separates a reliable report from an anxious one.

Pick a band per metric pair based on your own observed history, not on a rule of thumb. Watch the two numbers for a month, see what normal looks like on your site, and set the threshold outside that. The threshold exists to trigger investigation, so set it where investigation would actually be useful.

Then make the automation enforce it. A pipeline that silently accepts any difference is not reporting, it is transcription. A pipeline that flags anything outside the band is doing the job a person would do if they were paying attention, which is the only reason to automate it in the first place.

What does time zone do to a daily report?

It shifts your day, and on daily reporting that is enough to break a comparison entirely. Google's documentation states that in all views except the 24 hour view, the Performance report tracks and labels daily data in Pacific Time.

Think about what that means if your client is in Bengaluru or London and your analytics property is set to local time. The two tools are not reporting the same Tuesday. They are reporting overlapping windows with a substantial offset, and the discrepancy will be largest on the days where traffic is least evenly distributed.

This is a genuinely common cause of a mismatch that looks like a tracking failure. It is also trivially fixable once you know: compare weekly or monthly totals rather than daily ones, or convert deliberately, and note the convention on the report itself so nobody rediscovers it every quarter.

Why does a number change after you already sent the report?

Because the data was not finished when you pulled it, and both platforms tell you so. Google says collected data for the Performance report is usually available in 2 to 3 days, and that there can be a lag between when the numbers are calculated and when they are visible.

The analytics side has its own window. Google's documentation states that in Google Analytics 4, events are processed if they arrive up to 72 hours late. So a report generated on the first of the month for the month just ended is reading a picture that is still developing on both sides.

The fix is a scheduling decision rather than a technical one. Run monthly reports several days into the following month, not on the first, and if a client insists on the first, label the figures as provisional on the report itself. A number that was always labelled provisional and then moved is fine. A number presented as final that moves costs you credibility you will not get back cheaply.

How should an automated report handle all this?

By carrying its own provenance. Every figure in an automated report should be able to answer three questions without a human being asked: which tool it came from, what window it covers, and when it was pulled. If a number cannot answer those, it should not be in the report.

In practice that means the workflow writes a small metadata record alongside every fetch, and the report renders a footer from that record rather than from a hardcoded string. It costs almost nothing to build and it eliminates the entire category of conversation where two people disagree about which version of a dashboard they are looking at.

Beyond that, pick one system of record per question and say so. Impressions and queries come from Search Console. On site behaviour comes from your analytics. Pipeline comes from the CRM. When the same concept exists in two systems, one of them wins by written decision, and the other appears as context rather than as a competing answer. I have made this argument at length for attribution on a small B2B team, and it applies identically here.

What do you tell a client who noticed?

The truth, immediately, in one short paragraph. Name the mechanism, say which number you consider authoritative for that question, and say what you will do about it. Clients handle known limitations well. What they handle badly is discovering that you did not know.

Have the explanation written down before you need it, because you will need it more than once and you do not want to be composing it under pressure. A short document that lists each metric, its source, its known quirks, and its authoritative counterpart is the single highest value artefact in a reporting engagement.

It is also worth checking the boring causes before you reach for the interesting ones. Bot and crawler traffic distorts one side of a comparison and not the other, which I covered in how AI bots skew site analytics, and a filter change upstream will do the same.

What should you do next?

Take the two numbers you most often compare, pull thirty days of both, and look at whether the ratio between them is steady. If it is, write one sentence explaining the mechanism and put that sentence into your report template permanently. If it is not, you have a real bug and now you know where to look.

Then check the scheduling. If any report runs before the data behind it has settled, move it, and label anything you cannot move as provisional. Both of those changes take minutes and they remove most of the reconciliation conversations you are currently having. If you build client reports on a schedule, the same discipline applies to your monthly reporting view.

And if you have an automated report that keeps producing numbers you cannot defend, reach out. Working out which source should win is usually a one afternoon problem and it stays solved.