Your contractor has 4.5 stars. About half the time, that rating doesn't tell the whole story.

The criticism, fairly stated: The report leans on a single 0–100 "trust score," but never publishes the formula, the input weights, or why the "Trusted" line sits at 70 rather than 65 or 75. Every headline — "41% are Trusted," "16.5% of roofers clear the bar" — is a direct function of choices readers can't see. An unexplained composite that produces dramatic numbers deserves skepticism until the machinery is open.

Our answer: This is a legitimate gap and we are closing it. For this edition the score combines three families of signal — (1) review volume and consistency, (2) business verification and active operating status, and (3) available business-history signals — with review volume and consistency deliberately weighted most heavily. We have not yet published the exact coefficients or the 70 threshold's derivation. That is a shortcoming of a v1 benchmark, not a defensible choice. Disclosure we are adding: a public methodology appendix with the component weights, the score distribution, and a sensitivity table showing how the "% Trusted" figure moves as the threshold slides from 60 to 80. Until that appendix is out, treat the 70 cutoff as one reasonable line on a continuum, not a bright natural boundary — and read the distribution (most contractors cluster in a wide middle) as the durable finding, rather than any single side of the cutoff.

The criticism, fairly stated: Review volume is the heaviest input. That makes several headline findings partly definitional. "29% are effectively invisible" and "the review desert" aren't discoveries about trustworthiness — they're restatements of the input. And "stars lie" compares a Google rating against a score that itself ingests Google reviews, so the two are correlated by construction.

Our answer: This is the sharpest fair hit in the report, and we want to be precise about where it lands and where it doesn't.

Where it lands: the "invisible contractor" finding is, by design, a coverage statement. A contractor with two reviews is not necessarily bad — they are unmeasurable, and our score reflects low confidence, not proven low quality. We should not, and going forward will not, let the word "trust" blur that. Disclosure we are adding: we will relabel the low-volume cohort as "low-transparency / low-evidence" rather than implying poor performance, and state plainly that absence of a track record is a reason to verify independently, not proof of a bad contractor — the same standard we hold Google's stars to.

Where it doesn't land: the score is not only volume. Two contractors with identical review counts can score differently on consistency, verification, and operating history. And the star-vs-score comparison isn't pure circularity: the score down-weights a high average that rests on a thin or erratic sample, which is exactly the information a raw star average discards.

The honest one-line version: our score measures how much verifiable evidence of accountability exists, weighted toward review track record. It is a strong proxy for trust, not a measurement of workmanship, and it is partly correlated with the very stars it critiques.

The criticism, fairly stated: The dataset is the population of Google-listable businesses, not the population of contractors. Excellent operators who run on word-of-mouth, referrals, or trade-association channels and never cultivate a Google profile are either absent or land in the "invisible/low-trust" bucket. So the report may be measuring digital-marketing behavior as much as trustworthiness.

Our answer: Correct, and we should not have used "every active contractor in Ontario" without this qualifier. Our frame is every active, publicly-listable Ontario home-service business we could identify from public sources, which skews toward businesses that maintain an online presence. A genuinely great referral-only contractor can score low here purely for lack of a public footprint. Disclosure we are adding: an explicit coverage statement in the methodology, and a reframe of low-evidence scores as "we cannot verify," not "we verified them as untrustworthy." This also sharpens the homeowner takeaway: if you can't find a public record for someone, that's a prompt to ask for references, licensing, and proof of insurance directly.

The criticism, fairly stated: Trust in the trades has many public signals — reviews on other platforms, BBB accreditation and complaint history, ECRA/ESA and provincial licensing, WSIB coverage, small-claims and Consumer Protection Ontario actions. Using Google alone imports Google's specific weaknesses (fake-review susceptibility, no licensing awareness) and omits the records that most directly indicate whether a contractor has actually harmed customers.

Our answer: Partly fair. Google is the largest review source in the score — it's the most complete and most consulted public source — but it isn't the only one: the score also draws on reviews from other major platforms, BBB records, WSIB records, and the provincial licensing registers where a licence applies (ESA for electrical contractors, TSSA for gas and fuel work, and HCRA for builders of new homes). Small-claims and consumer-protection records are not yet included, so read this as a review-and-verification index, not a compliance-and-litigation index.

The criticism, fairly stated: The publisher scores the exact market it monetizes. A report whose central message is "the free signal everyone uses is broken, and the thing we sell is the fix" has an obvious incentive to make the market look as untrustworthy as possible. Lowering the "Trusted" threshold, weighting toward a metric that produces alarming scarcity, and headlining "checking reviews won't protect you" all point the same commercially convenient direction.

Our answer: The conflict is real and we won't wave it away. We can't prove our own good faith by assertion, so instead we've tried to make the analysis falsifiable and reproducible by outsiders: fixed, stated denominators; a documented snapshot date; sample sizes alongside every percentage; and (per Q1) a forthcoming weights-and-threshold appendix so a critic can recompute our headline under different assumptions. Two structural points partly blunt the incentive claim: we score every contractor whether or not they engage with us, and we don't sell placement or rank — so the report doesn't advantage any paying customer. But the deepest mitigation is the threshold-sensitivity disclosure: if our "41% are Trusted" figure only looks the way it does because of where we drew the line, an open sensitivity table lets you catch us doing it. We would rather you check that than take our independence on trust.

The criticism, fairly stated: East York, North York, Scarborough, Etobicoke, and "York" are not suburbs of Toronto — they are Toronto, amalgamated into the single City of Toronto in 1998. The ranking treats them as separate cities (East York 20th, "Toronto" 32nd, North York 46th, Scarborough 36th, Etobicoke 31st) based on Google's neighborhood labels. So the flagship "big-city surprise" is substantially an artifact of splitting one city by postal-area labels: the "Toronto" row is really just the downtown/old-Toronto core, and one of its own districts (East York) ranks 20th province-wide. The same problem applies to Ottawa, whose Nepean, Kanata, Orléans, Gloucester and Stittsville districts are listed separately, and to Vaughan (Woodbridge, Concord) and Hamilton (Stoney Creek).

Our answer: This is the most important factual correction and we accept it. The city field reflects the locality label on each business's public listing, not municipal boundaries, so amalgamated Toronto and Ottawa are each spread across several rows. The "Toronto 32nd" figure is the old-Toronto core, not the City of Toronto as a whole; aggregating all Toronto districts produces a composite of 64.3 across 2,637 contractors, slightly lower than the "Toronto" row alone. Corrections we are making: (1) a clear note that "Toronto," "North York," "Scarborough," "Etobicoke," "East York," and "York" are districts of one municipality, likewise the Ottawa districts; (2) a supplementary ranking computed on true amalgamated-municipality boundaries; and (3) softening the "big-city surprise" claim to what the data actually supports — variation within the GTA and between listing-labeled areas — rather than a clean city-beats-suburb story. The genuinely robust point survives: trust varies widely across Ontario geography and is not concentrated in the largest urban labels.

The criticism, fairly stated: The 40-contractor floor still lets small samples (Innisfil 40, Trenton 40, Bradford 43, Thornhill 43) sit on a ranked ladder against markets ten to twenty times larger, reported to a tenth of a point. With samples that small the confidence interval on an average trust score easily spans several rank positions, so most of the ordering — especially "1st," "70th," and adjacent ties — is noise dressed as precision.

Our answer: Fair, and partly self-inflicted by presenting a strict 1–70 ordering. A 40–90 contractor sample supports a band ("upper third / middle / lower third"), not a precise rank, and certainly not decimal-point separation between neighbors. Disclosure we are adding: confidence intervals (or standard errors) on every city average, a caution that cities within roughly a few points of each other are statistically indistinguishable, and a shift in how we present the extremes — "among the lowest-scoring markets" rather than a definitive "least trustworthy city in Ontario." The superlatives on the smallest samples are the least defensible sentences in the report and we are dialing them back.

The criticism, fairly stated: Ranks 3 through 20 span roughly 68.7 down to 66.7 — two points across eighteen positions, often decimals apart. Treating a 0.1–0.5 point gap as a rank difference implies a precision the data doesn't have, and invites readers to over-read "my city beat yours."

Our answer: Agreed. The rank ladder is more precise-looking than the underlying data warrants. The honest reading is tiers, not ranks: there is a real, several-point spread between the top tier and the bottom tier, but adjacent positions inside a tier are effectively ties. Disclosure we are adding: we will group cities into a small number of statistically-separated tiers as the primary presentation and demote the exact 1–70 ordering to a reference appendix with the precision caveat attached. The defensible claim is the spread between top and bottom; the indefensible one is any specific adjacent-rank bragging right.

The criticism, fairly stated: For a trust index, the omission of the two most concrete trust signals — is this contractor licensed for the trade, and are they insured — is conspicuous. It leaves the score resting on softer proxies.

Our answer: We agree these are the strongest signals. Licence and WSIB records from the provincial registers feed the score where they apply (see Q4), but this Index does not report province-wide licensing or insurance rates: a single provincial rate would mix trades that need a licence with trades that don't. Licence coverage by trade, measured against each regulator's own register, is published separately in our licensing data release.

The criticism, fairly stated: A trade sitting far below the rest is a classic signature of a threshold/weighting artifact rather than a real-world fact. Roofing work is project-based and episodic: customers hire for it rarely, so even excellent firms accumulate reviews slowly compared to a plumber doing hundreds of small jobs a year. If the score is volume-weighted, it will structurally disadvantage low-frequency trades regardless of quality — which means a low "% Trusted" may be measuring review cadence, not trustworthiness.

Our answer: A fair caution, and the finding we'd most ask readers not to over-read. Part of the effect is likely structural: project-based trades generate fewer, lumpier reviews per firm than high-frequency service trades, so a review-weighted score can disadvantage them even where quality is high. The roofing result should not be read as "few trustworthy roofers exist." The residual, defensible point is narrower but still useful: on big-ticket jobs, look past the star average to the volume of reviews and the credentials behind it. That's a statement about evidence, which the data does support.

The criticism, fairly stated: The report says review volume is weighted heavily, yet drywall simultaneously has an above-median "% Trusted" (50.4%) and the highest "% with <5 reviews" (40.5%), on a small sample (262). Those two facts sit awkwardly together and suggest either a small-sample quirk or that the weighting doesn't behave as described.

Our answer: A fair catch, and worth explaining rather than glossing. Two things are happening: drywall is one of our smaller trade samples, so its rates are noisier; and the score isn't volume alone — verification and consistency can lift a contractor with a short but clean, consistent record above the line even with modest review counts. Still, the juxtaposition shows our current model can send mixed signals, and it reinforces why the weights need to be public (Q1) so readers can see exactly how a thin-but-consistent record is treated. Disclosure we are adding: we'll flag drywall's smaller sample and use this specific pair of numbers as the worked example in the published-weights appendix.

The criticism, fairly stated: Everything reflects a single capture on/before November 7, 2026. Reviews lag real performance, businesses change hands, and a one-day snapshot can't distinguish a firm on the way up from one on the way down. Periodic publication means figures age between updates.

Our answer: Correct on all counts, and inherent to any point-in-time index. We disclose the vintage prominently precisely so no one over-reads month-to-month precision into it, and we regularly rescore our platform and update this Index to stay current and accurate. Two honest caveats we're stating more clearly: individual contractors can move between our bands as their public record updates, so the Index is a snapshot for research and comparison, not a live verdict on any single business; and review lag means the score trails real-world changes by however long it takes customers to post. The population-level findings (distribution shape, trade and geographic spread) are far more stable across a snapshot than any individual score, and those are what we intend the report to be cited for.

The criticism, fairly stated: A model is only a trust measure if a low score actually predicts bad outcomes. Without a validation study against ground truth — consumer-protection complaints, small-claims judgments, licensing revocations — the score is an internally-consistent index that has never been shown to track the thing it's named after.

Our answer: This is the most important thing we can't yet claim, and we won't pretend otherwise. For 2026 the score is a construct-based estimate built from signals that plausibly relate to accountability; it has not been back-tested against an external ground-truth outcome set. That is a genuine limitation of this Index. What we're committing to: a validation study in a future edition that tests whether lower ProScore bands are associated with higher rates of documented complaints or adverse licensing actions, published whether or not the result flatters the model. Until that exists, the accurate framing — which we'll state in the methodology — is that ProScore measures the strength and consistency of a contractor's public accountability record, and is a tool for comparison, not a guarantee or a verdict on any individual business.

The criticism, fairly stated: The word "trust" and the alarmist framing imply a measurement of whether a contractor will do right by you. What's actually measured is the density and consistency of a public paper trail. The gap between the label and the substance is where a skeptic — or a competitor — will attack hardest.

Our answer: Fair, and it's the throughline behind several answers above. We're tightening the language so the claim matches the substance: ProScore measures verifiable evidence of accountability, weighted toward review track record and verification. It is a strong, comparative proxy for trust — not a character judgment, not a workmanship rating, and not a prediction about your specific job. The headline advice we stand behind without hedging is the modest one: a star rating is a starting point, not a decision; look past the average to the volume, recency, and independent verification behind it — most of all on the biggest jobs. That's a claim the data genuinely supports, and it's the one we want cited.