MDHIM - Multi-Dimensional Hashing Indexing Middleware
MDHIM is a parallel key/value store framework written in MPI.
Unlike other big data solutions, MDHIM has been designed for an HPC
environment and to take advantage of high speed networks.
Leveldb: http://code.google.com/p/leveldb/ Mysqldb: http://www.mysql.com/ Mysqldb C Connector: http://dev.mysql.com/downloads/connector/c/ An MPI distribution that supports MPI_THREAD_MULTIPLE well (this excludes OpenMPI)
- Modify Makefile.cfg to point to your leveldb and MPI installations
- Type: make
- If all went well, you have the library in src/libmdhim.a
- cd tests
- Type: make
- If all went well, you have all the tests compiled
With mysql you must remove the manifest file before running mdhim with a new set of paramters.
Typical setup to run file: ./mdhimtst -ftest.txt -q -t5 -d3 -r0~2 -w1 -p ./
Typical batch file setup:
put single_insert.txt get single_insertget.txt del single_insertdel.txt
Batch command file setup:
For put get del bput bget bdel [command] [#of items in file(if necessary)] [file containg key and value (if necessary)]
For nput ngetn (random numbers are generated for this function) [#number of records to put/get] [key size (only applicapble for byte/strng)] [value size (nput only)] [size correctance number]
For flush (only command is used to flush)
Parameters:
-f (file with batch commands)
-d (Type of DB to use: levelDB=1 mysql=3)
-t (Type of keys: int=1, longInt=2, float=3, double=4, longDouble=5, string=6, byte=7)
-p (path where DB will be created)
-n (Name of DataBase file or directory)
-b (MLOG_CRIT=1, MLOG_DBG=2)
-a (DB store append mode. By default records with same key are overwritten. This flag turns on the option to append to existing values.
-w This flag turns on the option to either allow or deny threads to do command based on if it is dividiable by the modlus of the modulus number
-r ~ This flag turns on the option to either allow or deny threads to do command based on if the rank falls inclusively inbetween the rank ranges. NOTE: You must use the '' inbetween the numbers. Example: -r02
-q<0|1> (Quiet mode, default is verbose) 1=write out to log file