Skip to content

This package generates people, credit cards, bank accounts and transactions data for dev or data science purposes. Returns python dicts or csv strings to be printed or stored.

License

Notifications You must be signed in to change notification settings

nandoabreu/mass-data-generator

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

48 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

mass-data-generator

Mass data generator for dev pruposes. This module generates csv data with as many records as needed, creating people, credit cards, bank accounts and/or bank transactions fake data to be used in developing, data science or any purposes.

Created data are invented and random, but some (credit cards) are generated using real validation algorithm.

When generating person data, the namefake.com API is used. Main module may request a proportional number of names and birthdays and mix queried data in order to increase performance, reducing requests to the external RESTful service.

Note: if you system is configured with ulimits, you may need to generate larger sets in more than one execution; with my cpu limit set to 180, I generate 1M people per time.

 
 

README Map

Run from command line

From the project's root directory, run the following command and follow instructions to
generate any number of registries and save them to a csv file:

python -m datagenerator

Run from python console

from datagenerator.Person import Person
for i in range(3):
    p = Person()
    p.create()
    if i == 0: print(','.join(p.info().keys()))
    print(p.csv())

from datagenerator.CreditCard import CreditCard
cc = CreditCard(person=p)
cc.info()
cc.csv()

from datagenerator.BankAccount import BankAccount
acc = BankAccount(person=p); acc.info()

from datagenerator.BankTransaction import BankTransaction
t = BankTransaction(account=acc, value=1000.00, description='Opening deposit'); t.info()
t = BankTransaction(account=acc, value=-100.00, description='Withdrawl'); t.csv()

Automatic tests

Python tests are available using unittest:

python -m unittest tests/test_*

Logs

Logs are recorded in the 'logs' directory in the project's root.
Please see docs/logs if you wish to access samples of the generated logs.

Documentation

See docs for documentation. Or run:

python -c "import datagenerator; help(datagenerator)"
python -c "import datagenerator.Person; help(datagenerator.Person)"
python -c "import datagenerator.CreditCard; help(datagenerator.CreditCard)"
python -c "import datagenerator.BankAccount; help(datagenerator.BankAccount)"
python -c "import datagenerator.BankTransaction; help(datagenerator.BankTransaction)"

To do

  • Add automatic tests.
  • Automatic rotation of log files.
  • Log classes other than Person.
  • Document main module.

About

This package generates people, credit cards, bank accounts and transactions data for dev or data science purposes. Returns python dicts or csv strings to be printed or stored.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages