Remove Duplicate Lines
Deduplicate a list, keep the order, and decide what counts as a duplicate.
Two lines that look identical often are not. One has a trailing space, another differs by a capital letter, a third is the same entry typed twice with different spacing. A deduplicator that compares raw strings keeps all three. This tool lets you say what counts as the same — case, surrounding whitespace, blank lines — before it compares anything, and can also be inverted to show only the entries that appear more than once.
How it works
Paste your list
One entry per line: keywords, emails, IDs, log lines, anything.
Set the comparison rules
Whitespace and case are ignored or respected as you choose.
Run it
The first occurrence of each entry is kept, in its original order.
Deduplication is a definition problem before it is a task
The comparison rule decides the answer. Alice, alice and Alice are three distinct strings and one person, so a tool that compares raw text reports no duplicates at all on a list full of them. That is why the options here come before the button rather than after it.
Trailing whitespace is the most common invisible difference. It survives copy-and-paste from spreadsheets, arrives with every line of a CSV export that pads its columns, and shows up in anything that was hand-edited. Leading spaces come from indented lists and pasted code. Both are stripped by default here, because a list where they matter is rarer than a list where they are noise.
Case is the opposite default. Email addresses are effectively case-insensitive in the part that matters, and so are most keyword lists — but identifiers, hashes, API keys and passwords are not, and silently merging a3F with a3f would corrupt the data. So the box starts unticked, and you turn it on when you know the list tolerates it.
Order, and what the inverted view is for
Entries are returned in the order they first appeared, not the order they were last seen and not sorted. That preserves any meaning the sequence carried — a priority list, a chronological log, a manually curated ranking — which alphabetical output would destroy. Sorting is available as a checkbox for the cases where the order was never meaningful.
The inverted option answers a different question: not "what is the clean list" but "what went wrong". Turning it on returns only the entries that occurred more than once, which is how you find the double-booked slot in a schedule, the customer entered twice under the same email, or the keyword you accidentally targeted on two pages.
The counters under the result exist for the same reason. Knowing that 4,000 lines produced 3,998 unique entries tells you the file was basically clean; knowing it produced 1,200 tells you something upstream is duplicating records, and that the deduplication here is a symptom fix rather than a solution.