Java 8 Stream - Distinct By Property Example

1. Overview

In this tutorial, you'll learn How to get the distinct values from collection using java 8 stream api distinct() method.

Read more on 

Java 8 Stream API

In other words, how to remove the duplicates from list or collection using java 8 streams. This is a common tasks to avoid duplicates in the list. After java 8 roll out, it has become simple filtering using functional programming language.

Java 8 stream api is added with a unique distinct() method to remove the duplicate objects from stream.

distinct() is an intermediate operation that means it returns Stream<T> as output.

Next, let us jump into examples programs using Strings and Custom objects.

2. Java 8 distinct() example - Strings Values in List

First, create a simple list with duplicate string values. Now, we want to get only the unique values from it.

For this, we need to convert the list into stream using stream() method and next call distinct() method. Here, this distinct() method eliminates the duplicate string values.

Finally, invoke the Collectors.toList() method to take the distinct values into List.

import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;

public class DistinctStringExample {
	public static void main(String[] args) {
		// create a list with string values
		List<String> strings = new ArrayList<>();
		
		// adding values to list 
		strings.add("ABC");
		strings.add("XYZ");
		strings.add("ABC");
		strings.add("MNO");
		strings.add("ABC");
		strings.add("MNO");
		strings.add("PQR");
		
		// Getting the distinct values from stream using distinct() method
		List<String> uniqueStrings = strings.stream().distinct().collect(Collectors.toList());
		
		//printing the values
		System.out.println("Original list : "+strings);
		System.out.println("Unique values list : "+uniqueStrings);
	}
}

Output:

Original list : [ABC, XYZ, ABC, MNO, ABC, MNO, PQR]
Unique values list : [ABC, XYZ, MNO, PQR]

3. Java 8 distinct() example - By Custom Object Property

In the above program, we've seen with the simple strings. But in the real time, you will be adding the real objects such as Employee, Trade or Customer objects.

Let us create a Customer class with id, name and phone number.  Next, add 5 Customer objects to the List with duplicate id values.

Finally, use our custom logic will  get only the distinct by id field using Function Functional interface.

Customer.java

package com.javaprogramto.java8.streams.distinct;

public class Customer {

	private int id;
	private String name;
	private long phonenumber;
	
	public Customer(int id, String name, long phonenumber) {
		super();
		this.id = id;
		this.name = name;
		this.phonenumber = phonenumber;
	}

	public int getId() {
		return id;
	}

	public void setId(int id) {
		this.id = id;
	}

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	public long getPhonenumber() {
		return phonenumber;
	}

	public void setPhonenumber(long phonenumber) {
		this.phonenumber = phonenumber;
	}

	@Override
	public String toString() {
		return "Customer [id=" + id + ", name=" + name + ", phonenumber=" + phonenumber + "]";
	}
	
}

Distinct by Property Example:

package com.javaprogramto.java8.streams.distinct;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;

public class DistinctByCustomPropertyExample {

	// predicate to filter the duplicates by the given key extractor.
	public static <T> Predicate<T> distinctByKey(Function<? super T, Object> keyExtractor) {
		Map<Object, Boolean> uniqueMap = new ConcurrentHashMap<>();
		return t -> uniqueMap.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
	}

	public static void main(String[] args) {

		// creating customer objects with repeated id's 100, 101
		Customer customer1 = new Customer(100, "Jhon", 675000000l);
		Customer customer2 = new Customer(101, "Peter", 675000001l);
		Customer customer3 = new Customer(100, "Paul", 675000002l);
		Customer customer4 = new Customer(102, "Noel", 675000003l);
		Customer customer5 = new Customer(101, "Nup", 675000004l);

		// created a list to store the customer objects
		List<Customer> customers = new ArrayList<>();

		// adding customer objects
		customers.add(customer1);
		customers.add(customer2);
		customers.add(customer3);
		customers.add(customer4);
		customers.add(customer5);

		List<Customer> distinctElements = customers.stream().filter(distinctByKey(cust -> cust.getId()))
				.collect(Collectors.toList());

		System.out.println("customers size : " + customers.size());
		System.out.println("Distinct customers size : " + distinctElements.size());

	}

}

Output:

customers size : 5 Distinct customers size : 3

In the above program, the core is the distinctByKey() method which does the job removing the duplicates.

This is the advantage of core functional programming language.

4. Java 8 distinct toMap() example - Distinct values collecting into Map

In the above example, we have stored the output into List with Customer objects. But, we want to store it into map with id as key and customer object as value.

Use Collectors.toMap() method to remove the duplicates and collect into map.

package com.javaprogramto.java8.streams.distinct;

import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

public class DistinctByMapExample {

	public static void main(String[] args) {

		// creating customer objects with repeated id's 100, 101
		Customer customer1 = new Customer(100, "Jhon", 675000000l);
		Customer customer2 = new Customer(101, "Peter", 675000001l);
		Customer customer3 = new Customer(100, "Paul", 675000002l);
		Customer customer4 = new Customer(102, "Noel", 675000003l);
		Customer customer5 = new Customer(101, "Nup", 675000004l);

		// created a list to store the customer objects
		List<Customer> customers = Arrays.asList(customer1, customer2, customer3, customer4, customer5);

		// removing the duplicates and collecting into map id as key, customer as value
		Map<Integer, Customer> mapIdCustomer = customers.stream()
				.collect(Collectors.toMap(Customer::getId, c -> c, (c1, c2) -> c1));

		// printing the map
		System.out.println("Final map after eliminating the duplicates - "+mapIdCustomer);

	}

}

Output:

Final map after eliminating the duplicates - 
{100=Customer [id=100, name=Jhon, phonenumber=675000000],
 101=Customer [id=101, name=Peter, phonenumber=675000001], 
 102=Customer [id=102, name=Noel, phonenumber=675000003]}

5. Conclusion

In this short article, you've seen how to get the distinct values from collection and store them into List and Map.

GitHub

DistinctByCustomPropertyExample.java

DistinctByMapExample.java

DistinctStringExample.java

Ref

How to get the distinct value from Array?

Java 8 forEach Examples