Remove Duplicate Lines Online - Free Tool

Frequently asked questions

Which copy of a duplicate is kept?

The first occurrence, with later repeats removed, so the original ordering of the surviving lines is preserved. That matters when the sequence carries meaning, such as a log or a priority list.

Is the comparison case-sensitive?

Lines are compared exactly, so 'Apple' and 'apple' count as different entries. Normalise case first if you want them treated as the same — a common need with email addresses and usernames.

Does trailing whitespace affect matching?

Yes, an invisible trailing space makes two otherwise identical lines distinct. If duplicates are not being removed as expected, that is almost always the cause — trim the list first.

Pro tips

Trim whitespace before deduplicating. A trailing space is invisible and makes an otherwise identical line survive as a separate entry.
Decide whether case matters for your data before running it — collapsing case on identifiers can merge two genuinely different records.
Sort before deduplicating when you want to eyeball the result; identical entries end up adjacent and the removals are easy to verify.
Use natural ordering for anything with numbers in it, or item10 files before item2 and version lists come out nonsensical.
Keep the original list until you have checked the count. A dedupe that removed far more than you expected usually means case or whitespace handling was not what you assumed.

About Line Sorter & Deduplicator

Sorts alphabetically or by length, strips duplicates and blank lines, trims stray whitespace, and reverses or shuffles order, with case sensitivity as a toggle. The usual reason to reach for it is a list pasted out of a spreadsheet or a log.

Paste a list to strip out duplicate and empty lines, trim whitespace and optionally sort what remains, with a case-insensitive mode. The list is processed entirely in your browser.

Paste a list and this tool reorders it any way you need: alphabetically A→Z or Z→A, by line length, reversed, or randomly shuffled. In the same pass it can remove duplicate lines, drop empty lines and trim stray whitespace, with an optional case-insensitive mode so 'Apple' and 'apple' are treated as the same entry.

It is the fast way to clean a keyword list, tidy a CSV column, dedupe an email or URL list, or randomise names for a draw. Sorting uses natural ordering, so 'item2' comes before 'item10' rather than after it.

Sorting by raw character codes is not alphabetical order, and the difference shows immediately. Byte order puts every capital letter before every lowercase one, so Zebra lands before apple — correct by code point and wrong by any human expectation. Proper alphabetical ordering is locale-aware, and the right answer genuinely differs by language: Swedish places å, ä and ö after z, while German files them alongside a and o. There is no single universal ordering to fall back on.

Deduplication needs the same care about what counts as identical. Whether entries differing only in case are duplicates depends entirely on the data — for email addresses usually yes, for case-sensitive identifiers usually no. Trailing whitespace is the quieter culprit: two lines that look the same but differ by one invisible space are distinct strings, and both survive a dedupe that would otherwise have caught them.

Common use cases

Cleaning a mailing list before importing it
Removing repeated entries from a log or export
Deduplicating a list of URLs or IDs collected from several sources

How it comparesSpreadsheets can do all of this through Remove Duplicates and a sort, across several dialogs and a column selection. Command-line sort and uniq are faster still if the data is already in a file and you remember that uniq only collapses adjacent lines. Pasting a list is quicker than either when it came out of an email.