Match rules determine how source profiles are matched together into unified profiles during the identity resolution process. The number of match rules you create, and how you arrange criteria within the match rules, can raise or lower the effectiveness (consolidation rate) of your ruleset. Understanding how match rules and match criteria operate gives you the ability to optimize your rulesets to achieve the optimal level of profile consolidation to suit your business needs.
It’s easy to look at consolidation rates and profile unification numbers in your Identity Resolution Summary stats and assume your goal is to achieve the highest possible rates of consolidation—but that assumption would be wrong.
Your goal is to create match rules that achieve the “perfect” level of consolidation for your business needs. Too much consolidation can lead to combining records for unrelated customers, while too little can leave customers frustrated by missing information or multiple profiles. Either case can erode the most critical element of any business: the trust your customers have in your business. But how can you achieve the elusive perfect balance with your match rules? It starts with understanding the levers represented by the match rules and match rule criteria in your identity resolution rulesets.
The Three Levers in Every Ruleset
1. Match Rules: Each match rule is an opportunity for records to be matched. Records need to match the criteria of only one match rule to get matched by the ruleset. The more match rules you create, the more opportunities you give your records to be matched. Adding more rules to a ruleset raises the ruleset’s consolidation rate.
2. Match Criteria: Within each match rule, the criteria determine how precise a particular match rule is. Any two records must match all criteria within a match rule to be matched. The more criteria you include, the harder it is for any two records to match. Adding more criteria to a match rule lowers the ruleset’s consolidation rate. Match methods are useful to fine-tune your match criteria, but we won’t discuss them in detail here.
3. Reconciliation Rules: Reconciliation rules are applied after match rules to determine how to select values when only one can be stored in a unified profile, such as last name. Reconciliation rules don’t apply to fields such as contact information, where multiple values are stored in a separate object, such as Contact Point Email. These rules are applied last, after your records have been consolidated according to your match rules, and don’t impact your consolidation rate.
Let’s Look at an Example
Let’s consider how we can arrange these 4 criteria into match rules to most effectively match a set of profiles.
- Fuzzy – Medium Precision First Name
- Exact Normalized Last Name
- Exact Normalized Phone Number
- Exact Normalized Email Address
In this exercise, we’ll look at source data that includes 8 source Individual profiles from 5 data sources. Each individual’s name has been entered differently into different systems, and they don’t always use the same phone number or email address.
Our goal is to arrange our criteria into match rules that effectively sort these 8 source profiles into 2 Unified Individual profiles. However, your business needs might prioritize precision of matching over consolidation rate, in which consolidating into just 2 unified profiles would be overconsolidation. The flexibility Data Cloud provides in arranging match rules and match criteria lets you aim for either outcome.
Scenario 1: One Match Rule (3 criteria)
In this scenario, records with a similar name and the same phone number are matched using one rule with these criteria:
Match Rule Criteria:
- Fuzzy – Medium Precision First Name
- AND Exact Normalized Last Name
- AND Exact Normalized Phone Number
Some records are matched into unified individual profiles, but some potential matches are skipped because they contain a different phone number even though the name is the same.
The reconciliation rule is set to Most Recently Updated for all fields, so where only one value can be selected for a field, such as for name, the record is selected.
Note that in the last record, Thomas’ Johnson’s phone number ends in 2134, whereas all phone numbers associated with Tommy Johnson’s profile end in 1234. Is there a typo in Thomas’ phone number? If Thomas’ phone number were changed to 202-555-1234, his record would be matched with Tommy’s.
Scenario 2: One Match Rule (3 criteria)
In this scenario, records with a similar name and the same email address are matched by one rule with these three criteria:
Match Rule Criteria:
- Fuzzy – Medium Precision First Name
- AND Exact Normalized Last Name
- AND Exact Normalized Email Address
Some records are matched into unified individual profiles, but some potential matches are skipped because they contain a different email address.
The reconciliation rule is set to Most Recently Updated for all fields, so where only one value can be selected for a field, such as for name, the record is selected. If “Source Priority” or “Most Frequently Occurring” were selected, the version of the name saved into the unified profile might be different. For example, if the reconciliation rule was changed from “Most Recently Updated” to “Most Frequently Occurring”, the first name for Tommy Johnson’s unified profile would change to Tom.
Scenario 3: One Match Rule (4 criteria)
In this scenario, records with a similar name AND the same phone number are matched AND the same email address are matched. One match rules has all four criteria:
Match Rule Criteria:
- Fuzzy – Medium Precision First Name
- AND Exact Normalized Last Name
- AND Exact Normalized Phone Number
- AND Exact Normalized Email Address
Adding more criteria to a single match rule makes the rule more strict. Because records must match all 4 criteria within the match rule to be unified, very few records are matched and the consolidation rate is low. Matches made from many criteria are very precise, but records may be undermatched.
In some cases, it might be desirable to use stricter matching to produce fewer unified profiles, as shown here, rather than attempt to refine them into fewer profiles. This setup represents a more cautious approach to profile consolidation to prevent overmatching.
Scenario 4: Two Match Rules (3 criteria each)
In this scenario, records are matched if they have a similar name AND the same phone number are matched, OR if they have a similar name AND the same email address. The rules look like this:
Match Rule 1 Criteria:
- Fuzzy – Medium Precision First Name
- AND Exact Normalized Last Name
- AND Exact Normalized Phone Number
Match Rule 2 Criteria:
- Fuzzy – Medium Precision First Name
- AND Exact Normalized Last Name
- AND Exact Normalized Email Address
Adding more match rules allows more options and increases the consolidation rate. The two different match rules work together, allowing all of these records to be matched into just 2 unified profiles.
Other Overconsolidation Risks
The Address + Phone Number Rule Trap
When setting up match rules, it can be tempting to try to streamline them by leaving out fields like First Name or Last Name. For example, it’s possible to match records based on data points like physical address and phone number without considering names, like this:
Match Rule Criteria:
- Normalized Exact Address
- Normalized Exact Phone Number
However, it’s common for business contacts to share an address (the office address) and phone number (a single incoming line). Matching on just contact points can result in overconsolidation of some profiles, causing your business contacts to become inappropriately overconsolidated even when they’re really profiles for different people. Including names in match rules when matching based on Individual makes them less risky.
The Default Values Trap
Matching on fields that contain default values is another common cause of overconsolidation. This problem originates with your source data rather than your match rules, but it’s important to consider the possibility of default values in fields you’re using in your match rules.
This typically happens when a source system you’re ingesting data from assigns a default value to fields when a custom value isn’t set. For example, suppose a customer record system assigns the first name “John” and last name “Smith” to any record that doesn’t contain first and last names. Sounds like a handy way to signal to your reps that a customer’s real name isn’t known, right? But if your match rules consolidate records based on first and last name, you have a problem! Suddenly you’re consolidating every “John Smith” in the system into one gigantic unified profile. Superbig unified profiles (unified profiles built from more than 50,000 source profiles) are tossed out during matching, causing processing errors and missing data. Moderately large unified profiles, on the other hand, can just combine source profiles into one big ball of nonsense.
You can also accidentally overconsolidate records into gigantic unified profiles if you allow match criteria to match on blanks for fields that are frequently left empty.
In either case, it’s critical that your data is groomed to ensure accuracy and completeness before it’s even processed by your ruleset.
Conclusion
How you structure your match rules and match rule criteria determines how many source profiles are consolidated into unified profiles. Adding too many match rules or rules based on data that isn’t unique can lead to high rates of consolidation. Depending on your business’s use case, you might risk overconsolidating your profiles by matching similar profiles that actually represent different individuals. But on the flip side, making match rules too strict by adding excess criteria could make it difficult to match many profiles at all.
Identity resolution allows up to 10 match rules in a ruleset, each of which can contain 10 match criteria. You probably don’t need, and shouldn’t use, more than a couple match rules in a ruleset, or more than a couple criteria in a rule. The key to striking the right balance between too strict and too loose lies in creating the right rules with the right criteria for your data and your business needs.
Written by
