One canonical lead, two portals, no polling

Two portals that agree on nothing

Australian real-estate enquiries arrive from two dominant listing portals. One offers a subscription-based webhook feed. The other historically arrives as email, parsed into structured fields. They disagree about field names, about how a person's name is split, about what identifies an enquiry, and about whether you are told when something goes wrong.

The naive integration writes each portal straight into the CRM with its own field mapping. That works until you need to ask a question that spans both — how many enquiries this week, which source converts, is anything missing — at which point every report has to know about every portal, forever.

So the engine normalises on the way in. Both providers produce one canonical Lead, and the CRM only ever sees that. Adding a third portal means writing a provider, not touching the CRM mapping, the reporting or anything downstream.

Capability-driven providers, not a special case per portal

Each provider declares what it can do rather than being hardcoded into the pipeline: webhook delivery, polling, or metadata lookup. The engine reads those capabilities and drives each source accordingly.

That distinction is what allowed the migration from polling to webhooks to be a configuration change rather than a rewrite. Polling remained available as a fallback long after webhooks went live, because the safe way to retire a working mechanism is to stop using it while it can still be turned back on.

The CRM side is deliberately the opposite: one concrete implementation rather than an abstraction over many. A generic CRM layer would have been speculative work for a system with exactly one CRM, and abstractions built before a second case exists tend to fit neither.

The hard part is detecting silence

A webhook integration fails in a way polling does not. If a poll breaks, the next poll errors and you find out. If webhook delivery stops, nothing happens — and nothing happening looks exactly like a quiet week for enquiries.

The engine subscribes to a permanent heartbeat alongside the real enquiry subscription. The heartbeat fires on a known cadence regardless of whether any buyer submits anything, so its absence is unambiguous evidence that delivery has stalled rather than that the market is slow. A liveness endpoint exposes the answer for monitoring.

This is the piece most integrations skip, because it costs real design effort to detect a condition that has not happened yet. It is also the only reason anyone would notice a stall before the client did.

Multi-tenant from the first line

Every resource is scoped to a tenant — credentials, connection state, configuration, cache, links and the leads themselves — with the CRM portal identifier as the tenant key. That was true before there was a second tenant, because retrofitting tenancy onto a single-tenant schema means touching every query and every index at once.

Failure handling follows the same principle. Messages that cannot be processed are capped rather than retried forever, surfaced for an operator, and replayable once the underlying fault is fixed — so a bad payload is a thing someone looks at, not an infinite loop and not a silently dropped enquiry.

Why the usual approach fails, from having fixed one

The common way to get portal enquiries into a CRM is an email parser: the portal sends a notification, a parser reads it, a connector creates the contact. It is cheap and it works until it does not.

On a separate engagement I found one of these that had been running for months writing the literal words "Name", "Email" and "Phone" into every record. Both parse templates had been built with the tokens on the field labels rather than the values, so the parser had been told that the part which varies between emails is the word "Name:". It did exactly what it was configured to do, on every lead, with no queue backing up and no error rate to watch. Sales saw contacts arriving and assumed the feed worked.

That failure has no surface, which is the entire argument for building the ingestion properly: normalise on the way in, verify by reading the record at the far end, and make silence detectable rather than assuming absence means nothing happened.

What I would tell a client considering this

Almost every lead-source integration is sold as a connector and delivered as a field mapping. The field mapping is genuinely the easy part, and it is not where these systems fail.

They fail when a portal changes a field name, when delivery stops silently, when the same enquiry arrives twice and creates two contacts, or when a second office is added and the whole thing turns out to have assumed there was one. Those are the four things this engine was designed around, and they are worth asking about before buying any integration.