Skip to content

Rust natural language processing model with a focus on mapping back to source and "layerable" recognizers

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT
Notifications You must be signed in to change notification settings

old-storyai/layered-nlp

Repository files navigation

Layered NLP

Incrementally build up recognizers over an abstract token that combine to create multiple possible interpretations.

Key features:

  • Abstract over token type to support "rich" tokens like we have at Story.ai.
  • May generate multiple interpretations of the same token span.
  • Produces a set of ranges over the input token list with different attributes, for example:

Layering

The key idea here is to enable starting from a bunch of vague tags and slowly building meaning up through incrementally adding information that builds on itself.

Simplification: Money = '$' + Number

    $   123   .     00
                    ╰Natural
              ╰Punct
        ╰Natural
        ╰Amt(Decimal)╯
    ╰Money($/£, Num)─╯

Simplification:

  • Location(NYC) = 'New' + 'York' + 'City'
  • Location(AMS) = 'Amsterdam'
  • Address(Person, Location) = Person + Verb('live') + Predicate('in') + Location
    I     live      in      New York City
                                     ╰Noun
                                ╰Noun
                            ╰Adj
                    ╰Predicate
          ╰Verb
    ╰Noun
    ╰Person(Self)
                            ╰──Location─╯
    ╰────Address(Person, Location)─────╯

MIT licensed APACHE licensed

About

Rust natural language processing model with a focus on mapping back to source and "layerable" recognizers

Resources

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Stars

Watchers

Forks

Packages

No packages published

Languages