The purpose of this tool is:
- to transliterate and generate possible usernames out of a full names list that may include names written in multiple (non-English) languages, common problem occurring from scraped employee name lists (e.g. from Linkedin).
- to transliterate a wordlist that may include words/phrases written in multiple (non-English) languages, common problem occurring from spidering web sites to generate wordlists, e.g., with CeWL.
Romanization feature currently supports: Greek, Hindi, Spanish, French, Polish, Danish, Portuguese and Hungarian. Looking for people to contribute language classes, check how it's done and contribute!
https://www.youtube.com/watch?v=550S6oAYfDo
Use the following command to transliterate a list with words/phrases in multiple (non-English) languages:
python3 BabelStrike.py -f wordlist.txt -r
Note: Words that are too short will probably cause many language identification missmatches. If the result is inaccurate and you know which language should be used to transliterate from, provide it with --language (-l)
python3 BabelStrike.py -f wordlist.txt -r -l spanish
Use the following command to transliterate and generate possible usernames out of a full names list that may include names written in multiple (non-English) languages
python3 BabelStrike.py -f full_names.txt -r -c -a
The rules can be automatically aplied to the reversed version of the full name as well, by using [-a].
| {firstname}{lastname} | {f}{l} | {lastname}{f} | {f}{la} | {firstname} |
| {firstname}.{lastname} | {f}.{l} | {lastname}.{f} | {f}.{la} | {lastname} |
| {firstname}_{lastname} | {f}_{l} | {lastname}_{f} | {f}_{la} | |
| {firstname}-{lastname} | {f}-{l} | {lastname}-{f} | {f}-{la} | |
| {firstname} {lastname} | {f} {l} | {lastname} {f} | {f} {la} | |
| {f}{lastname} | {fi}{lastname} | {lastname}{fi} | {la}{f} | |
| {f}.{lastname} | {fi}.{lastname} | {lastname}.{fi} | {la}.{f} | |
| {f}_{lastname} | {fi}_{lastname} | {lastname}_{fi} | {la}_{f} | |
| {f}-{lastname} | {fi}-{lastname} | {lastname}-{fi} | {la}-{f} | |
| {f} {lastname} | {fi} {lastname} | {lastname} {fi} | {la} {f} |
| {firstname}{middle}{lastname} | {f}{m}{l} | {lastname}{middle}{f} | {f}{m}{l} | |
| {firstname}.{middle}.{lastname} | {f}.{m}.{l} | {lastname}.{middle}.{f} | {f}.{m}.{l} | |
| {firstname}_{middle}_{lastname} | {f}_{m}_{l} | {lastname}_{middle}_{f} | {f}_{m}_{l} | |
| {firstname}-{middle}-{lastname} | {f}-{m}-{l} | {lastname}-{middle}-{f} | {f}-{m}-{l} | |
| {firstname} {middle} {lastname} | {f} {m} {l} | {lastname} {middle} {f} | {f} {m} {l} | |
| {f}{middle}{lastname} | {fi}{middle}{lastname} | {lastname}{middle}{fi} | {firstname} | |
| {f}.{middle}.{lastname} | {fi}.{middle}.{lastname} | {lastname}.{middle}.{fi} | {middle} | |
| {f}_{middle}_{lastname} | {fi}_{middle}_{lastname} | {lastname}_{middle}_{fi} | {lastname} | |
| {f}-{middle}-{lastname} | {fi}-{middle}-{lastname} | {lastname}-{middle}-{fi} | ||
| {f} {middle} {lastname} | {fi} {middle} {lastname} | {lastname} {middle} {fi} |
pip3 install -r requirements.txt
In order to be able to process certain languages, you may need to install additional fonts on your machine. Here's for example how to install Hindi fonts:
- Download the fonts from here: https://fonts.google.com/specimen/Hind.
- Unzip and move the fonts in
~/.local/share/fonts(if the directory does not exist then create it).
If you open a new terminal now the characters will be represented properly (this was tested on kali linux).
babelstrike.py [-h] -f FILE [-r] [-c] [-a] [-d DOMAIN] [-u] [-q]
options:
-h, --help show this help message and exit
-f FILE, --file FILE File to process.
-r, --romanization Transliterate names to the latin alphabet.
-c, --convertion Perform name-to-username convertions.
-a, --auto-reverse Perform name-to-username convertion patterns against the reversed version of each name as well.
-d DOMAIN, --domain DOMAIN
Comma seperated list of domains to add as prefix to each generated username (e.g. EVILCORP\scott.henderson).
-u, --update Pull the latest version from the original repo.
-q, --quiet Do not print the banner on startup.
In order for the Romanization feature to be accurate, I decided to use custom character substitution maps for each language preferably made by native speakers. I'm looking for some cool people around the world to create such maps that are basically a Python dictionary.
If you want to contribute a language Class all you have to do is:
- Find an official Romanization standard for your language's alphabet (e.g. in Wikipedia),
- Copy a language Class file from the language_classes folder to use as a template (I suggest you use Greek.py),
- Edit the filename and the Class name to represent your language,
- Edit the char_substitution_map dictionary and create the character substitution map (Important: Don't change the name of the dictionary),
- Map lowercase letters only,
- Take in consideration double or triple letter sounds that may be transliterated in a single character of the Latin alphabet,
- Take in consideration accented characters (e.g. à, è, ì, ò, ù),
- When a letter has more than one transliteration equivalents, use a list to include all of them (BabelStrike will handle all variations). Example:
# In Greek, the letter 'υ' may be transliterated as 'y' or 'u'. # This is how it should be declared in the character mapping dictionary: char_substitution_map = { 'ά' : 'a', 'έ' : 'e', 'υ' : ['y','u'] } - Save the new class in the language_classes folder named appropriately.