Seven ways to achieve array deduplication

7 ways to achieve array deduplication

1. Method 1: Using the splice method of two-layer loop + array

The array elements are compared one by one through a two-level loop, and then duplicate elements are deleted through the splice method. This method cannot deduplicate NaN because when comparing NaN !== NaN.

let arr = [1, 2, 2, 'abc', 'abc', true, true, false, false, undefined, undefined, NaN, NaN]
function removeDuplicate(arr) {
  let len = arr.length
  for (let i = 0; i < len; i++) {
    for (let j = i + 1; j < len; j++) {
      if (arr[i] === arr[j]) {
        arr.splice(j, 1)
        len--
      }
    }
  }
  return arr
}
removeDuplicate(arr)
console.log(arr) // [ 1, 2, 'abc', true, false, undefined, NaN, NaN ]

2. Method 2: Use Set()+Array.from()

• SetObject: is a collection of values ​​whose elements you can iterate in order of insertion . The elements in the Set will only appear once , that is, the elements in the Set are unique .
• Array.from()Method: Creates a new, shallow copy of an array instance from an array-like or iterable object .

function removeDuplicate(arr) {
  // return [...new Set(arr)]
  return Array.from(new Set(arr))
}
 // [ 1, 2, 'abc', true, false, undefined, NaN ]

3. Method 3: Using the indexOf method of the array

Create a new empty array, traverse the array that needs to be deduplicated, and store the array elements into the new array. Before storing, determine whether the array already contains the current element, and if not, store it. This method also cannot remove NaNduplicates

• indexOf()Method: Returns the index of the first occurrence of the specified value in the String object that calls it, and fromIndexsearches from . If the value is not found, -1 is returned

function removeDuplicate(arr) {
  let newArr = []
  arr.map(item => {
    if (newArr.indexOf(item) === -1) {
      newArr.push(item)
    }
  })
  return newArr
}
console.log(removeDuplicate(arr)) // [ 1, 2, 'abc', true, false, undefined, NaN, NaN ]

4. Method 4: Use the includes method of the array

The logic of this method is similar to the indexOf method to remove duplicates, except that the includes method is used to determine whether duplicate elements are included.

• includes()Method: Used to determine whether an array contains a specified value. Depending on the situation, if it does, it will be returned true, otherwise it will be returned false.

function removeDuplicate(arr) {
  let newArr = []
  arr.map(item => {
    if (!newArr.includes(item)) {
      newArr.push(item)
    }
  })
  return newArr
} // [ 1, 2, 'abc', true, false, undefined, NaN ]

Note : Why includesit is possible to detect inclusion in an array NaNinvolves includesthe underlying implementation. The following figure includesshows part of the code implemented. When judging whether an element is included, sameValueZero方法comparison will be called. If so NaN, it will be isNaN()converted.

For specific implementation, please refer to: developer.mozilla.org/zh-CN/docs/…

Simple test includes()to NaNjudge:

简单测试includes()对NaN的判断：
const testArr = [1, 'a', NaN]
console.log(testArr.includes(NaN)) // true

5. Method 5: Use array filter()+indexOf()

The filter method will store the elements that meet the conditions into a new array and make a judgment based on the indexOf method.

• filter()Method: Creates a new array containing all elements of the test implemented by the provided function .

function removeDuplicate(arr) {
  return arr.filter((item, index) => {
    return arr.indexOf(item) === index
  })
}

const result = removeDuplicate(arr)
console.log(result) // [ 1, 2, 'abc', true, false, undefined ]

Note : The output result here NaNis not included because indexOf()NaN cannot be judged, that is, arr.indexOf(item) === indexthe return result is false. The test is as follows:

const testArr = [1, 'a', NaN]
console.log(testArr.indexOf(NaN)) // -1

6. Use Map()

The Map object is a data structure provided by JavaScript. The structure is in the form of key-value pairs. The array elements are stored as the keys of the map, and then combined with the has()and set()method to determine whether the keys are repeated.

• MapObject: used to save key-value pairs and remember the original insertion order of keys. Any value (object or primitive) can be used as a key or a value.

function removeDuplicate(arr) {
  const map = new Map()
  const newArr = []

  arr.forEach(item => {
    if (!map.has(item)) { // has()用于判断map是否包为item的属性值
      map.set(item, true) // 使用set()将item设置到map中，并设置其属性值为true
      newArr.push(item)
    }
  })

  return newArr
}
-----------------或者-------------------------------------------------------
function removeDuplicate(arr) {
  let map = new Map()
  arr.map(item => {
    if (!map.has(item)) map.set(item)
  })
  return [...map.keys()]
}

const result = removeDuplicate(arr)
console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN ]

Note : Use Map()can also be used to NaNremove duplicates. The reason is that when Map is judged, it is considered NaNto be NaNequal to and , and all other values ​​​​are ===judged to be equal based on the result of the operator.

7. Utilization objects

The implementation idea Map()is almost the same, mainly taking advantage of the non-repeatable property name of the object.

function removeDuplicate(arr) {
  let obj = {}
  arr.map(item => {
    if (!obj[item]) {
      obj[item] = true
    }
  })
  return Object.keys(obj)
}

To learn more development knowledge, please pay attention to the CRMEB open source mall system