Remove Duplicate Lines
Remove duplicate lines — keep first/last/only-unique, find only the duplicates, trim-compare, sort, and add occurrence counts. Live as you type.
About this ToolHow it works, benefits & use casesTap to collapse
This Remove Duplicate Lines tool is a full uniq-style deduper that works live as you type. Paste a list and it strips repeated lines, with control over which copy to keep: the first occurrence, the last occurrence, or only lines that appear exactly once. You can also flip it around to output just the duplicated lines. Comparison can be case-sensitive or not, can trim leading and trailing whitespace before matching, and can ignore blank lines entirely. The output can be left in its original order or sorted A→Z or Z→A, optionally trimmed, and optionally prefixed with each line's occurrence count (like uniq -c). Live stat cards show total lines, lines kept, and lines removed, and a "Most duplicated" panel ranks the values that repeat most. The result downloads as a .txt file, and the input is captured in a shareable URL. It all runs in your browser with nothing uploaded.
How to Use
- 1In "Options", set "Keep" to first occurrence, last occurrence, or only-unique lines, and pick a sort order.
- 2Toggle comparison settings: case sensitive, trim before compare, ignore blank lines, only duplicates, trim output, and show counts.
- 3Paste your list into the input box, or click "Load example" — the deduplicated result appears live below.
- 4Read the stat cards (total, kept, removed) and the "Most duplicated" panel to see what repeated most.
- 5Click "Remove duplicates" to confirm, then copy or download the result as a .txt file.
Key Benefits
- Keep the first, the last, or only lines that occur exactly once
- "Only duplicates" mode to extract just the repeated lines
- Case-sensitive or case-insensitive matching with optional trim-before-compare
- Ignore blank lines so empty rows do not skew the result
- Sort output A→Z or Z→A and optionally trim each emitted line
- Occurrence counts prefixed per line, like uniq -c
- Live total / kept / removed stats and a most-duplicated ranking; shareable URL
Common Use Cases
- Cleaning a copied list of emails, IDs, or URLs down to unique entries
- Finding which values repeat in a log or export with "Only duplicates"
- Deduplicating keywords or tags before importing them somewhere
- Counting how many times each line appears with the show-counts option
- Merging two pasted lists and removing the overlap in one pass
Options
Ignore leading/trailing spaces
Output just the repeated lines
Prefix each line with its count
0 characters · live
Enter text above to see the deduplicated result.
Was this tool helpful?
Share Your Experience
Help others discover this tool!
Related tools
- Console.log RemoverRemove console statements from your JavaScript code
- UUID GeneratorGenerate v4 UUIDs
- Sort LinesSort text lines alphabetically or numerically
- Whitespace RemoverRemove extra whitespace from text
- Word CounterCount words, characters, sentences, and reading time
- String ReverserReverse any text string
Keep first removes later repeats and keeps the earliest copy of each line; keep last keeps the final copy and removes earlier ones; and only-unique keeps just the lines that appear exactly once, discarding every line that has any duplicate. Choose based on whether order or true uniqueness matters more for your list.
Instead of removing repeats, "Only duplicates" outputs one copy of each line that appears more than once and drops everything that is unique. It is the inverse of deduplication and is useful for finding which values are repeated in a dataset rather than which are distinct.