Erlang Central

Extracting Unique Elements From a List

From ErlangCentral Wiki



You want to eliminate duplicate values from a list.


There are several possible solutions for this problem. Here are some of these:

Using lists:usort

The lists module contains a wealth of list processing functionality. One possible solution to this problem is to use the lists:usort function, which takes a list and returns a sorted copy of the original list, with all duplicates removed:

1> UL = [1,2,8,7,8,10,3,12,3,99,188,3,2,1,3,5,15,72].
2> lists:usort(UL).

Using the sets Module

Erlang standard libraries includes a module, sets, with a variety of functions related to generating, creating, and manipulating mathematical sets.

10> Set = sets:from_list(UL).
11> sets:to_list(Set).

Note that sets:to_list(sets:from_list(L)) produces an unreliably arranged list.

Using a General Balanced Tree Set (gb_set)

Erlang's standard libraries includes an implementation of Professor Arne Andersson's General Balanced Trees. These structures are more costly than sorting lists for small sets, but this is a much more efficient implementation when working with large sets of data.

The gb_set:from_list function will produce an ordered set of elements (dropping duplicates). The set can then be extracted back to a list for other use:

3> GBSet = gb_sets:from_list(UL).
4> gb_sets:to_list(GBSet).