Identifying Duplicate Values in an Array using Mule 4 DataWeave 2.0

Hi Guys,

We might have seen how to remove duplicates using distinctBy function of DW 2.0

But what if we have a scenario where we need to know what are the values that are repeated more than once?

In that case, I have written a small function to identify the duplicate values that are present in an array.

Just to explain about the function I have written below :

• distinctVal is a variable which returns distinct Values by removing duplicates
• duplicates is a variable which iterates distinctVal data which we got in the above step over original data and checks if the value is repeated more than once.

Let's consider an array which has values  [  "a" , "b" ,"a" ,"c" ,"d", "e" ,"c" ,"f" ]

In above array, we see that values "a" and "c" are repeated more than once and we have to find that using DataWeave code. Let's do it!

Dataweave code:

%dw 2.0

output application/json  

var record =     [    "a" , "b" ,"a" ,"c" ,"d", "e" ,"c" ,"f" ]

 var distinctVal = record distinctBy $

 var duplicates = (distinctVal map(key,value) -> {

	count : if(sizeOf((record map $ == key) filter $ ) > 1) key else null

}) filter $.count !=null 

---

{

	distincValues : distinctVal,

	

	duplicatesAre : duplicates.count

}

Output :

{

  "distincValues": [

    "a",

    "b",

    "c",

    "d",

    "e",

    "f"

  ],

  "duplicatesAre": [

    "a",

    "c"

  ]

}

So here :

"distinctValues" are the final array of values by removing the values which are repeated

"duplicatesAre" will display the values that are repeated more than once.

In this way, you can find the duplicate values in an array using DataWeave 2.0

Happy Learning

Yours

Sravan Lingam