CentOS Linux Server Guide

Displaying unique file contents in CentOS Linux

Uniq command is used to eliminate duplicated lines in a file.

This command is performed only in respect to adjacent lines only.
Normally, This command is used after sort command, the sort command will sort all the file contents into indexed lines; this include some of duplicated data. Uniq command is then called to remove the duplicated lines
Uniq command does not manipulate/ edit the original file. This results of the output is shown to your computer screen only. If wish to save the results, you simply need to redirect them to a file
You can also use -d option to display the duplicate lines of a file

Command and Options:

uniq [options] filename

uniq command by default (without using any options) is to remove any duplicate lines only.

Option:

Options Description

-- only display unique lines

-d only display duplicate lines

-c summarized the tota number of lines of the duplication of a file

-w specify the number of characters to compare in the lines.

Examples:

The the following example will remove any duplicate lines in mail.log

uniq mail.log

The following example will remove any duplicate lines of A.log and redirect the output (save) to B.log

uniq A.log B.log

The following example will count and display the number of times of A.log duplication

uniq -c A.log

The following example will sort the field 2 to filed 10 of A.log

sort +2 -11 A.log

The following example, the sort command will merge the files of A.log and C.log by sorting them together

sort A.log C.log

Back to Managing Files and Folders in CentOS Linux

All trademarks are the property of their respective owners.