Erlang Central

Mnesia Table Fragmentation

From ErlangCentral Wiki



This HOWTO describes how you make Fragmented Mnesia tables and how to use them.


Let's say I have to make a book library index application. There's a table I use to record all the available library books. The record structure is as below. Due to the high volume of data, I want this table to be fragmented in a single Erlang node. If you want to make this fragmented table distributed, you may refer to tutorial on making distributed table. All the rest of the work related to table fragmentation remains same.

Sample Fragmented Table

I need this table to be disk_copies. Other modes also operate the same way.

-record(book_info, {isbn, name, author, keywords, category, description}).

Start an Erlang node

Our example node foo@example has the default disc storage path set to the directory in the current directory.

 erl -sname foo

The directory can be overridden by using -mnesia dir '"/path/of/your/preference"' ' when starting the node.

Let create a disk based schema by running,


Create the Fragmented table with 20 table fragments

In this example all 20 fragments are in the same Erlang/Mnesia node. Also the fragments are disc_copies.

   [{frag_properties, [{node_pool, [node()]}, {n_fragments, 20}, {n_disc_copies, 1}]},
   {index, [name, keywords, category]},
   {attributes, record_info(fields, book_info)}]),

Data operations

In order to be able to access a record in a fragmented table, Mnesia must determine to which fragment the actual record belongs. This is done by the mnesia_frag module, which implements the mnesia_access callback behaviour. Wrap standard Mnesia operation functions inside the function and pass to mnesia:activity/4 with callback module mnesia_frag.

Add records

Create a function which calls mnesia:write/3.

 AddFun = fun() ->
           mnesia:write(book_info, Record, write)

Now call that function inside mnesia:activity/4 as below.

 ok = mnesia:activity(transaction, AddFun, [], mnesia_frag)

Notice that I have used the activity access context as "transaction". Transaction makes sure that the operation is all successfull or all fail (atomic). AccessContext I can use are,

 {transaction, Retries} 
 {sync_transaction, Retries} 

For example if you want to do above activity in dirty mode, you can write,

 ok = mnesia:activity(async_dirty, AddFun, [], mnesia_frag)
 Refer to mnesia:activity/4 documentation for more info.

Select records with limit

As an example let's select books by Author "steve" with 10 books limit. Remember 10 is not a hard limit. Create a function with mnesia:select/4 function.

 MatchHead = #book_info{author = "steve", _ = '_'},
 Guard = [],
 Result = ['$_'],
 MatchSpec = [{MatchHead, Guard, Result}],
 SelFun = fun() ->
           mnesia:select(book_info, MatchSpec, 10, read)

Now call that function inside mnesia:activity/4 as below.

 Result = mnesia:activity(transaction, SelFun, [], mnesia_frag)
 Result -> '$end_of_table' | {[Objects], Cont} | transaction abort

In a fragmented table, if it returns {[Objects], Cont} and the number of objects returned is less than the number of objects expected (10), you need to recursively run mnesia:select/1 with the return Cont (continuation) until you get the expected number of results or '$end_of_table'.

 SelFun2 = fun() ->
 Result2 = mnesia:activity(transaction, SelFun2, [], mnesia_frag)
 Result2 -> '$end_of_table' | {[Objects], Cont} | transaction abort

List of fragmented tables

To obtain the list of tables, use mnesia:table_info/2 with the option frag_names:

 mnesia:activity(async_dirty, mnesia:table_info/2, [store, frag_names], mnesia_frag).

That's it! Now you know how to write a basic Mnesia fragmented tables program.