Multiple methods of python list deduplication

List deduplication in python

How to quickly de-duplicate the list? Will the original order be changed after de-duplication?
The order will change after deduplication

set deduplication

#列表去重改变原列表的顺序了
l1 = [1,4,4,2,3,4,5,6,1]
l2 = list(set(l1))
print(l2)    # [1, 2, 3, 4, 5, 6]

However, the index in the list can be used to ensure that the order after deduplication remains unchanged.

l1 = [1,4,4,2,3,4,5,6,1]
l2 = list(set(l1))
l2.sort(key=l1.index)
print(l2)      # [1, 4, 2, 3, 5, 6]

itertools.groupby

'''
遇到问题没人解答？小编创建了一个Python学习交流QQ群：778463939
寻找有志同道合的小伙伴，互帮互助,群里还有不错的视频学习教程和PDF电子书！
'''
import itertools
l1 = [1,4,4,2,3,4,5,6,1]
l1.sort()
l = []
it = itertools.groupby(l1)
for k,g in it:
    l.append(k)
print(l)      # [1, 2, 3, 4, 5, 6]

fromkeys

l1 = [1,4,4,2,3,4,5,6,1]
t = list({
    
    }.fromkeys(l1).keys())
# 解决顺序问题
t.sort(key=l1.index)
print(t)         # [1, 4, 2, 3, 5, 6]

By deleting the index

'''
遇到问题没人解答？小编创建了一个Python学习交流QQ群：778463939
寻找有志同道合的小伙伴，互帮互助,群里还有不错的视频学习教程和PDF电子书！
'''
l1 = [1,4,4,2,3,4,5,6,1]
t = l1[:]
for i in l1:
    while t.count(i) >1:
          del t[t.index(i)]
# 解决顺序问题
t.sort(key=l1.index)
print(t)    # [1, 4, 2, 3, 5, 6]

Removal does not change the order

Create a new list[]

l1 = [1,4,4,2,3,4,5,6,1]
new_l1 = []
for i in l1:
    if i not in new_l1:
        new_l1.append(i)
print(new_l1)    # [1, 4, 2, 3, 5, 6]

reduce method

from functools import reduce
l1 = [1,4,4,2,3,4,5,6,1]
func = lambda x,y:x if y in x else x + [y]
print(reduce(func,[[],]+l1))   # [1, 4, 2, 3, 5, 6]