Remove duplicate lines — and see what was removed
Paste a list, pick your options, and get a clean unique list plus a line-by-line report of every duplicate that was dropped.
🔒 100% client-side · zero server upload
What was removed
Malaysian rules & edge cases
Order matters. This tool keeps the first occurrence of every line, so your original ordering survives. Turn on sorting only when order does not matter.
| Option | What it does | Typical Malaysian case |
|---|---|---|
| Ignore capitals | "Apple" and "apple" count as the same line | Customer names typed by different staff |
| Trim spaces | Leading/trailing spaces are ignored when comparing | Numbers pasted out of Excel or WhatsApp |
| Remove empty lines | Blank lines are dropped and reported | Lists exported from WhatsApp chats |
| Sort results | A–Z or Z–A using locale-aware sorting | Building a supplier or price list |
Windows line endings (CRLF) from Excel and Notepad are handled automatically, and a trailing newline at the end of the file does not create a phantom empty line.
Before and after
Before
Apple
apple
Banana
Banana
Apple
DurianAfter
Apple
apple
Banana
Banana
DurianHow it works
- Paste your list into the INPUT box, or press "Load sample data" to see a Malaysian example.
- Tick the comparison options you need: ignore capitals, trim spaces, drop blank lines, or sort.
- Press Run. The OUTPUT box holds the clean list, and the report below lists every line that was removed and why.
FAQ
Does this upload my list to a server?
No. The whole tool is static JavaScript. You can open your browser Network tab and confirm that pressing Run sends zero requests, which matters when the list contains customer phone numbers or emails.
Which duplicate is kept?
The first occurrence in your original order. If you need the last one instead, reverse the list before pasting, or sort first and then deduplicate.
Can it handle 100,000 lines?
Yes, up to roughly 200,000 short lines on a normal laptop. Beyond that the browser tab may slow down, which is a browser memory limit rather than a server timeout.
Does it work for Malay or Chinese text?
Yes. Comparison is character based and sorting uses locale-aware collation, so Malay words, accented characters and Chinese lines are compared correctly.