Google Apps Scripts: Remove Duplicate Values from a list.
Removing duplicate values from a single dimensional array is pretty straight forward. You can use the ‘Set’ which stores unique value types or ‘indexOf’ with ‘filter’ operation. We will outline the ‘Set’ and ‘indexOf’ in a different article.
When it comes to removing duplicate values from two dimensional lists then that requires a more nuanced solution. We will use a small trick by serialising the data values and using it as an index to a list. The list ‘foundValues’ will keep track if the values have been seen or not before. If the value is new then we will insert it into a ‘uniqeList’ array that will be returned by the method ‘removeDupsInList()’.
Our function takes a single parameter ‘valueList’ that is a two dimensional array. With this solution you can have more than just the two dimensional list, but we will only discuss a two dimensional array for brevity.
To start we will iterate each array row and serialize all the columns to a string:
var currRowAsStr = JSON.stringify(valueList[currRow]);
Using the newly created ‘currRowAsStr’ variable, we are going to keep track of the serialized values in an array:
foundValues[currRowAsStr] = true;
If the next row that was serialized already exists, then we can skip that row of columns by checking the array value to be ‘true’. If the row already exists then continue to the next row:
if ( foundValues[currRowAsStr] ) {
continue;
};
If the row does not yet exists, we insert the row as a value to a ‘uniqueList’ array:
uniqueList.push( valueList[currRow] );
Putting it all together, the Full code to the method ‘removeDupsInList(valueList)’:
function removeDupsInList(valueList) {
var uniqueList = [];
var foundValues = {};
var valueListLen = valueList.length;
for(var currRow = 0; currRow < valueListLen; currRow++) {
var currRowAsStr = JSON.stringify(valueList[currRow]);
if ( foundValues[currRowAsStr] ) {
continue;
};
uniqueList.push( valueList[currRow] );
foundValues[currRowAsStr] = true;
}
return(uniqueList);
};