There are several main ways to quickly deduplicate lists in daily work. I hope they can help you.
1. The set method in python removes duplicates, but it will change the original list to order.
l1 = [1,4,4,2,3,4,5,6,1]
l2 = list(set(l1))
print(l2) # [1, 2, 3, 4, 5, 6]
2. Ensure that the order after deduplication remains unchanged through the index method in the list
l1 = [1,4,4,2,3,4,5,6,1]
l2 = list(set(l1))
l2.sort(key=l1.index)
print(l2) # [1, 4, 2, 3, 5, 6]
itertools.groupby
3. Use the import itertools function
l1 = [1,4,4,2,3,4,5,6,1]
l1.sort()
l = []
it = itertools.groupby(l1)
for k,g in it:
l.append(k)
print(l) # [1, 2, 3, 4, 5, 6]
4.Python dictionary fromkeys() method
l1 = [1,4,4,2,3,4,5,6,1]
t = list({
}.fromkeys(l1).keys())
5. By deleting the index
l1 = [1,4,4,2,3,4,5,6,1]
t = l1[:]
for i in l1:
while t.count(i) >1:
del t[t.index(i)]
# 解决顺序问题
t.sort(key=l1.index)
print(t) # [1, 4, 2, 3, 5, 6]
But if you want to keep the original order unchanged, you can try the following method:
1. Loop through and create a new list[]
l1 = [1,4,4,2,3,4,5,6,1]
new_l1 = []
for i in l1:
if i not in new_l1:
new_l1.append(i)
print(new_l1) # [1, 4, 2, 3, 5, 6]
2.reduce method
from functools import reduce
l1 = [1,4,4,2,3,4,5,6,1]
func = lambda x,y:x if y in x else x + [y]
print(reduce(func,[[],]+l1)) # [1, 4, 2, 3, 5, 6]