Skip to content

This is a simple tool for splitting a document into sentences and words. Also, you can find out the frequency of token appearance.

License

Notifications You must be signed in to change notification settings

sagnik-sudo/Text-to-Sentence-Tokenizer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Text to Sentence Tokenizer

Description

This is a simple tool for splitting a document into sentences.

Steps to start

  1. Enable Virtual Environment

    virtualenv venv

    source venv/bin/activate

  2. Install dependencies

    pip install -r requirements.txt

  3. Run the tool

    uvicorn main:app --reload --port 8080

  4. Open the browser and go to http://localhost:8080/tokenizer

You are good to go!

Prerequisites

  1. If you are using Windows

  2. If you are using Linux

Maintainer

Name: Sagnik Das

Email: [email protected]

Suggestions are welcome

For suggestions and contributions, please visit here

If you like my work, please star it on here

About

This is a simple tool for splitting a document into sentences and words. Also, you can find out the frequency of token appearance.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages