Erlang Central

Mnesia Table Fragmentation

Revision as of 06:10, 27 October 2007 by Erlangonxen (Talk | contribs)



This HOWTO describes how you make Fragmented Mnesia tables and how to use them.


Let's say I have a make book library index application. There's a table I use to record all the available library book. The record structure is as below. Due to the high volume of data, I want this table to be fragmented in a single Erlang node. If you want to make this fragmented table distributed, you may refer to tutorial on making distributed table. All the rest of the work related to table framentation remains same.

Sample Fragmented Table

I need this table to be disk_copies. Other modes also operates the same way.

-record(book_info, {isbn, name, author, keywords, category, description}).

Start a Erlang node

Our example node have a default disc storage paths set to the directory in the current directory.

The directory can be overriden by using -mnesia dir '"/path/of/your/preference"' ' when starting the node.

Let create a disk based schema by running,


Create the Fragmented table with 20 table fragments

In this example all the 20 fragments are in the same Erlang/Mnesia node. Also the fragments are disc_copies.

   [{frag_properties, [{node_pool, [node()]}, {n_fragments, 20}, {n_disc_copies, 1}]},
   {index, [name, keywords, category]},
   {attributes, record_info(fields, book_info)}]),

Data operations

In order to be able to access a record in a fragmented table, Mnesia must determine to which fragment the actual record belongs. This is done by the mnesia_frag module, which implements the mnesia_access callback behaviour. Wrap standard Mnesia operation functions inside the function and pass to mnesia:activity/4 with callback module mnesia_frag.

Add records

Create a function with mnesia:write/3 function.

 AddFun = fun() ->
           mnesia:write(book_info, Record, write)

Now call that function in inside mnesia:activity/4 as below.

 ok = mnesia:activity(transaction, AddFun, [], mnesia_frag)

Notice that I have used the activity access context as transaction. Transaction makes sure that the operation is all successfull or all fail (atomic). AccessContext I can use are,

 {transaction, Retries} 
 {sync_transaction, Retries} 

For example if you want to do above activity in dirty mode, you can write,

 ok = mnesia:activity(async_dirty, AddFun, [], mnesia_frag)
 Refer to mnesia:activity/4 documentation for more info.

Select records with limit

Select books by Author "steve" with 10 books limit. Remember 10 is not a hard limit. Create a function with mnesia:select/4 function.

 MatchHead = #book_info{author = "steve", _ = '_'},
 Guard = [],
 Result = ['$_'],
 MatchSpec = [{MatchHead, Guard, Result}],
 SelFun = fun() ->
           mnesia:select(book_info, MatchSpec, 10, read)

Now call that function in inside mnesia:activity/4 as below.

 Result = mnesia:activity(transaction, AddFun, [], mnesia_frag)
 Result -> '$end_of_table' | {[Objects], Cont} | transaction abort

In fragmented table, if it returns {[Objects], Cont} and the number of objects return is less than the number of expected objects (10), you need to run recursivly mnesia:select/1 with the return Cont (continuation) until you get the expected number of results or '$end_of_table'.

 SelFun2 = fun() ->
 Result2 = mnesia:activity(transaction, AddFun, [], mnesia_frag)
 Result2 -> '$end_of_table' | {[Objects], Cont} | transaction abort

That's it. Now you know how to write your basic Mnesia fragmented tables program.