Deduplication Agent, an AI-assisted duplicate resolution for enterprise asset records.
Updated: Oct 12
Overview
Large enterprise organizations manage hundreds of thousands, even millions, of physical assets across factories, warehouses, plants, and field operations.
Over time, duplicate asset records become unavoidable.
A pump might be registered twice because different technicians entered it separately. An imported spreadsheet may recreate assets that already exist. Different business units may use different naming conventions for the same equipment.
Although duplicates seem harmless at first, they slowly corrupt the entire asset ecosystem.
Instead of one reliable asset history, organizations end up with fragmented maintenance records, duplicate work orders, conflicting inspections, and disconnected integrations with external ERP systems.
Cleaning these duplicates manually is extremely expensive.
Our goal was to design an AI-powered Deduplication Agent that could detect duplicate assets, explain why they were duplicates, and help users merge them safely, without losing trust or damaging enterprise data.
The Challenge
Most customers already knew duplicate assets existed. The real problem wasn't detection.
It was resolution. Users had very little confidence merging records because every merge carried significant operational risk.
A single incorrect merge could:
Remove maintenance history
Disconnect work orders
Break preventive maintenance schedules
Lose external ERP mappings
Create compliance issues
Because of this fear, duplicate records accumulated for years. Instead of fixing the problem, teams simply learned to work around it.
The challenge became:
How might we design an AI system that users trust enough to perform irreversible actions on critical enterprise data?
Understanding Users
We collaborated with industry experts and professionals with deep operational experience to better understand how enterprises handle duplicate asset records in day-to-day workflows.
Despite differences in organizations and processes, the challenges and workflows were remarkably consistent.
Existing workflow
Teams typically:
Exported assets into Excel.
Compared records manually.
Investigated linked work orders.
Contacted technicians for confirmation.
Deleted one record.
Updated references manually.
A single duplicate could take several minutes to validate. Across thousands of records, cleanup became a massive operational burden.
Major Pain Points
No confidence in duplicate detection
Even when records looked identical, users wanted evidence before accepting they referred to the same physical asset.
High fear of accidental deletion
Deleting the wrong record could permanently disconnect years of maintenance history.
Relationships mattered more than attributes
Users cared less about which description survived.
They cared about:
Work Orders
Maintenance Plans
Spare Parts
Inspections
ERP IDs
Sensor connections
Those relationships represented years of operational history.
Manual comparison was exhausting
Users constantly switched between multiple screens comparing fields line by line.
This created significant cognitive load.
Design Principles
From our research we established four design principles.
AI should explain, not decide
Recommendations needed to feel transparent.
Users shouldn't wonder why something was flagged.
Humans own irreversible decisions
The AI could recommend.
The user always confirmed.
Relationships are more valuable than fields
Field conflicts are easy.
Broken relationships are expensive.
The interface should prioritize protecting linked records.
Reduce comparison effort
Users shouldn't compare twenty fields manually.
The system should surface meaningful differences automatically.
Designing the Experience
Step 1
Configure the Scan
Instead of hiding configuration inside settings, users begin by selecting:
Business Object
Matching Attributes
Similarity Rules
This gives complete visibility into how the AI will evaluate duplicates. The goal was making the scan predictable before it even begins.
** The visuals have been recreated or modified for confidentiality purposes. They are intended to represent the design approach and process, but may differ from the original product and used for interview purposes.
** The visuals have been recreated or modified for confidentiality purposes. They are intended to represent the design approach and process, but may differ from the original product and used for interview purposes.
** The visuals have been recreated or modified for confidentiality purposes. They are intended to represent the design approach and process, but may differ from the original product and used for interview purposes.
Step 2
AI Scan
Once configured, the AI scans records asynchronously. Instead of displaying a generic loading spinner, we visualized scan progress. This gave users confidence that the system was actively analyzing records rather than simply waiting.
The scan also summarized:
Total records analyzed
Duplicate groups identified
Confidence distribution
** The visuals have been recreated or modified for confidentiality purposes. They are intended to represent the design approach and process, but may differ from the original product and used for interview purposes.
Step 3
Review AI Recommendations
This became the heart of the experience. Rather than presenting a confidence score alone, every duplicate group included explainable reasoning.
For example:
These assets match on: • Serial Number• Manufacturer• Installation Location Differ only in: • Installation Date
Users could immediately understand why the AI made its recommendation. This dramatically reduced the need to inspect every attribute manually.
Each group clearly identified:
Primary Record
Duplicate Record
Matching Evidence
Differing Fields
Possible actions remained intentionally simple:
Ignore
Delete
Merge
Nothing happened automatically.
Step 4
Guided Merge Wizard
The merge flow became a multi-step wizard. Instead of overwhelming users with every decision simultaneously, we separated the process into manageable steps.
Resolve Field Conflicts
Whenever records contained conflicting values, users selected which value should survive.
Preserve Relationships
This was the most critical step.
The wizard automatically reattached:
Work Orders
Maintenance Schedules
Attachments
Asset Relationships
ERP IDs
External References
Instead of hiding this behavior, we surfaced it clearly so users understood exactly what would happen.
Step 5
Final Preview
Before executing the merge, users reviewed the final surviving asset.
This preview showed:
Final attribute values
Relationships retained
Associations transferred
Duplicate record removal
Only after reviewing this screen could users confirm the merge. This made the destructive action intentional rather than accidental.
From Concept to a Safer, Faster Workflow
This case study represents the initial design direction for the Deduplication Agent. Transforming this concept into a production-ready solution requires multiple rounds of team discussions, design iterations, usability testing, and collaboration with Product, Engineering, and Applied AI teams. The goal is to create a reliable workflow that helps users identify duplicate asset records, understand why they are considered duplicates, and merge them confidently without losing critical information. By replacing time-consuming manual comparisons and record-by-record updates with AI-assisted recommendations and a guided merge process, the platform aims to significantly reduce the time and human effort required for deduplication.
Expected Impact
Since this is an initial design draft rather than a launched product with measured results, the following outcomes represent the expected impact of the solution and will need to be validated through usability testing and real-world implementation.
Success metric | Expected outcome |
Time to resolve duplicates | Reduce the time spent identifying, reviewing, and merging duplicate records. |
Manual effort | Minimize repetitive comparisons and manual record updates through AI-assisted recommendations. |
Data integrity | Preserve work orders, maintenance schedules, external IDs, and related asset information during merges. |
Decision confidence | Help users understand AI recommendations and make informed merge decisions. |
Operational efficiency | Enable asset management teams to resolve duplicate records more efficiently and consistently. |
Thank you for taking the time to read my case study. This document is intended solely for interview and portfolio presentation purposes. Some details have been modified or simplified to respect confidentiality and NDA requirements.
