How to remove all duplicates from a List in Java 8?

1. Overview

In this article, you'll explore the different ways to clean up and remove duplicates from the list and ArrayList.

Let us see the example programs using plain java and java 8 stream api lambda expressions.

2. Removing Duplicates Using Plain Java

A simple way is to remove the duplicates to clean up the list using List.contains() method. Create a new list and Pass every value of the original list to the contains() method on a new list. Returns false if the current value is not present in the new list and add the value to the new list. if returns false means that value is already present in the list and skip the value-adding to the new list.

Example:

import java.util.ArrayList;
import java.util.List;

public class RemoveDuplicatesContains {

	public static void main(String[] args) {

		// Creating a new list
		List<String> originalList = new ArrayList<>();
		
		// Adding duplicate values
		originalList.add("A");
		originalList.add("B");
		originalList.add("C");
		originalList.add("C");
		originalList.add("B");
		originalList.add("A");
		
		// printing the original list
		System.out.println("Original list values : "+originalList);
		
		// created a new list to add only unique values
		List<String> newList = new ArrayList<>();
		
		// filtering the duplicates from the 
		originalList.forEach(eachValue -> {
			if(!newList.contains(eachValue)) {
				newList.add(eachValue);
			}
		});
		
		// printing the cleaned list without duplicatesa
		System.out.println("newlist values are : "+newList);

	}
}

Output:

Original list values : [A, B, C, C, B, A]
newlist values are : [A, B, C]

3. Removing Duplicates Using LinkedHashSet

Next, use LinkedHashSet to remove the duplicates and preserve the order as in the original list. LinkedHashSet is a collection api Set interface. Set interface implementation such as HashSet or LinkedHashSet. But, many developers use HashSet rather than LinkedHashSet.

Example:

import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;

public class RemoveDuplicatesLinkedHashSet {

	public static void main(String[] args) {

		// Creating a new list
		List<String> originalList = new ArrayList<>();

		// Adding duplicate values
		originalList.add("A");
		originalList.add("B");
		originalList.add("C");
		originalList.add("C");
		originalList.add("B");
		originalList.add("A");

		// printing the original list
		System.out.println("Original list values : " + originalList);

		// Creating linkedhashset object
		Set<String> linkedSet = new LinkedHashSet<>();

		// adding list values to set
		linkedSet.addAll(originalList);

		// removing all values from list
		originalList.clear();

		// add all values from set to list.
		originalList.addAll(linkedSet);

		// printing the cleaned list without duplicates
		System.out.println("originalList values ater removing duplicates  : " + originalList);

	}
}

Output:

Original list values : [A, B, C, C, B, A]
originalList values ater removing duplicates  : [A, B, C]

4. Remove Duplicates From a List Using Java 8 Lambdas

Let us look at the new JDK 8 lambda expressions and Stream api's distinct() method to remove duplicates.

distinct() method internally calls equals() method on each value and filters the duplicates objects.

In our example, we are adding Strings to the list so equals() method of String class will be called. If you are passing the Custom objects then you need to override the equals() method in the custom class as per needed.

import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;

public class RemoveDuplicatesJava8Distinct {

	public static void main(String[] args) {

		// Creating a new list
		List<String> originalList = new ArrayList<>();

		// Adding duplicate values
		originalList.add("A");
		originalList.add("B");
		originalList.add("C");
		originalList.add("C");
		originalList.add("B");
		originalList.add("A");

		// printing the original list
		System.out.println("Original list values : " + originalList);

		// Java 8 Stream API - distinct() method used
		originalList = originalList.stream().distinct().collect(Collectors.toList());

		// printing the cleaned list without duplicates
		System.out.println("Removed duplicates with java 8 api : " + originalList);

	}
}

Output:

Original list values : [A, B, C, C, B, A]
Removed duplicates with java 8 api : [A, B, C]

5. Conclusion

In this article, you've seen the 3 different ways to remove all duplicates from ArrayList with examples programs.

All are over GitHub.

distinct example