Linux伊甸园论坛 » Shell编程 » 如何删去重复行并保持顺序不变？

TUDOU01 发表于 2007-8-11 10:36

如何删去重复行并保持顺序不变？

文本如下：
I'm a student.
You are a teacher.
He is a farmer.
He is a farmer.
He is a farmer.
We are workers.
将重复行找到，并删去只留一行，并且文本顺序不能改变，如何用sed实现?

[[i] 本帖最后由 dearvoid 于 2007-8-16 09:19 编辑 [/i]]

li-jiahuan 发表于 2007-8-11 11:45

sed可以做行比较吗

Roc.Ken 发表于 2007-8-11 11:56

用 uniq 比较方便:
[code]
cax@ibm:~/Desktop> cat a
I'm a student.
You are a teacher.
He is a farmer.
He is a farmer.
He is a farmer.
We are workers.
cax@ibm:~/Desktop> cat  a | sort
He is a farmer.
He is a farmer.
He is a farmer.
I'm a student.
We are workers.
You are a teacher.
cax@ibm:~/Desktop> cat  a | sort | uniq
He is a farmer.
I'm a student.
We are workers.
You are a teacher.
[/code]

li-jiahuan 发表于 2007-8-11 12:23

LZ要求顺序不变

li-jiahuan 发表于 2007-8-11 12:26

如果重复行挨在一起， 不用sort就可以了

TUDOU01 发表于 2007-8-11 14:05

看来用awk比较方便！

li-jiahuan 发表于 2007-8-11 15:42

怎么个方便法

dearvoid 发表于 2007-8-13 09:25

[quote]原帖由 [i]TUDOU01[/i] 于 2007-8-11 10:36 发表 [url=http://www.linuxeden.com/forum/redirect.php?goto=findpost&pid=580710&ptid=158021][img]http://www.linuxeden.com/forum/images/common/back.gif[/img][/url]
文本如下：
I'm a student.
You are a teacher.
He is a farmer.
He is a farmer.
He is a farmer.
We are workers.
将重复行找到，并删去只留一行，并且文本顺序不能改变，如何用sed实现? [/quote]
用 sed 有点儿难度 :(

dearvoid 发表于 2007-8-13 09:27

[quote]原帖由 [i]li-jiahuan[/i] 于 2007-8-11 15:42 发表 [url=http://www.linuxeden.com/forum/redirect.php?goto=findpost&pid=580735&ptid=158021][img]http://www.linuxeden.com/forum/images/common/back.gif[/img][/url]
怎么个方便法 [/quote]
用 awk 这么个方便法: :)
[quote][font=Courier][color=blue]-(dearvoid@LinuxEden:Forum)-(~/tmp)-
[30959 0][/color] [color=red]; cat file
[/color]I'm a student.
You are a teacher.
He is a farmer.
He is a farmer.
He is a farmer.
We are workers.
[color=blue]-(dearvoid@LinuxEden:Forum)-(~/tmp)-
[30959 0][/color] [color=red]; awk '{ if (!seen[$0]++) { print $0; } }' file
[/color]I'm a student.
You are a teacher.
He is a farmer.
We are workers.
[color=blue]-(dearvoid@LinuxEden:Forum)-(~/tmp)-
[30959 0][/color] [color=red]; bye
[/color][/font][/quote]

[[i] 本帖最后由 dearvoid 于 2007-8-22 12:26 编辑 [/i]]

li-jiahuan 发表于 2007-8-13 16:29

又是这种伎俩：）

dearvoid 发表于 2007-8-13 17:17

小伎俩, 大智慧 :)

[[i] 本帖最后由 dearvoid 于 2007-8-13 17:44 编辑 [/i]]

TUDOU01 发表于 2007-8-15 20:02

不错！不错！又学一技！

dearvoid 发表于 2007-8-15 21:20

[quote]原帖由 [i]TUDOU01[/i] 于 2007-8-11 14:05 发表 [url=http://www.linuxeden.com/forum/redirect.php?goto=findpost&pid=580730&ptid=158021][img]http://www.linuxeden.com/forum/images/common/back.gif[/img][/url]
看来用awk比较方便！ [/quote]
看似这句话是随便说的？:)

li-jiahuan 发表于 2007-8-15 22:56

忍了很久，还是要唤出我最喜欢的Perl。。。
学dearvoid的awk
[code]
$ cat foo
I'm a student.
You are a teacher.
He is a farmer.
He is a farmer.
He is a farmer.
We are workers.

$ perl -lne ' print unless $seen{$_}++ ' foo
I'm a student.
You are a teacher.
He is a farmer.
We are workers.

[/code]

[[i] 本帖最后由 li-jiahuan 于 2007-8-15 22:58 编辑 [/i]]

dearvoid 发表于 2007-8-16 09:12

perl 无所不能，完全可以替代 shell、grep、sed、awk 等一系列 tools，并且一定程度上能替代 C :)

li-jiahuan 发表于 2007-8-16 09:16

“perl”只不过是个虚拟机
“Perl”是一门编程语言
简单点
perl不过是用来跑Perl的。。。:lol

Ai.Game 发表于 2007-8-16 13:03

[quote]原帖由 [i]li-jiahuan[/i] 于 2007-8-16 09:16 发表 [url=http://www.linuxeden.com/forum/redirect.php?goto=findpost&pid=581105&ptid=158021][img]http://www.linuxeden.com/forum/images/common/back.gif[/img][/url]
“perl”只不过是个虚拟机
“Perl”是一门编程语言
简单点
perl不过是用来跑Perl的。。。:lol [/quote]
perl 到底是用来跑地。。。。

还是用来用的，此perl 就是此perl?:'( :'( :'( :'( :'( :'( :'( :o :o :o :o

li-jiahuan 发表于 2007-8-16 13:42

呵， 这帖子被我转移话题了。。。

更简单地
我们写的是Perl代码， 由perl编译后运行

zhuomingliang 发表于 2007-8-16 19:29

哎呀:Q :Q :Q

dearvoid 发表于 2007-8-17 22:46

[quote]原帖由 [i]zhuomingliang[/i] 于 2007-8-16 19:29 发表 [url=http://www.linuxeden.com/forum/redirect.php?goto=findpost&pid=581196&ptid=158021][img]http://www.linuxeden.com/forum/images/common/back.gif[/img][/url]
哎呀:Q :Q :Q [/quote]
此‘哎呀’何意？:)

查看完整版本: 如何删去重复行并保持顺序不变？