Several methods of python list deduplication (array)

1. Method 1

code show as below	copy code
ids = [1,2,3,3,4,2,3,4,5,6,1] news_ids = [] for id in ids: if id not in news_ids: news_ids.append(id) print news_ids

The ideas seem relatively clear and simple, and the previous order can also be maintained.

2. Method 2

Processing through the set method

code show as below	copy code
ids = [1,4,3,3,4,2,3,4,5,6,1] ids = list(set(ids))

It is relatively simple to process, using the collection method set for processing, but the results will not retain the previous order.

3. Method 3

Processing using lambda anonymous functions and reduce functions

code show as below	copy code
ids = [1,4,3,3,4,2,3,4,5,6,1] func = lambda x,y:x if y in x else x + [y] reduce(func, [[], ] + ids)

4. Method 4

Using the itertools module

code show as below	copy code
import itertools ids = [1,4,3,3,4,2,3,4,5,6,1] ids.sort() it = itertools.groupby(ids) for k, g in it: print k

5. Unable to maintain the original order

code show as below	copy code
liebiao=set(liebiao)

Six, while traversal to remove duplication

code show as below	copy code
def delRepeat(liebiao): for x in liebiao: while liebiao.count(x)>1: del liebiao[liebiao.index(x)] return liebiao