Skip to content
This repository has been archived by the owner on Aug 9, 2020. It is now read-only.
/ RBNF Public archive

This project's lifetime has ended. The successor is https://github.com/thautwarm/frontend-for-free which is WIP. You can check lark-parser project which is a good alt.

License

Notifications You must be signed in to change notification settings

thautwarm/RBNF

Repository files navigation

RBNF: Parser Generator for Context Sensitive Grammars

Build Status PyPI version MIT License

Tutorials

See Jupyter Notebook.

Preview

  • rbnf.zero: solution to processing complex texts. Similar to re.

    See source file Poly.rbnf.

    # parse polynomials from text.
    import rbnf.zero as ze
    ze_exp = ze.compile("import poly.[*]", use='Poly')
    print(ze_exp.match("2x^2 + 3 + 4 - 7 x^5 + 4 x + 5 x ^2 - x + 7 x ^ 11").result)

    Then we got output [(0, 7), (1, 3), (2, 7), (5, -7), (11, 7)].

  • CLI tool chains:

    λ rbnf.exe --help
    Available commands:
     cc
         rbnf source code compiler.
    
     - filename(positional or keyword arg)        : input source file
     - output(positional or keyword arg) = None   : output file name. default to be ...
     - name(positional or keyword arg) = 'unname' : name of language
    
     run
         You can apply immediate tests on your parser.
         P.S: use `--opt` option takes longer starting time.
    
     - filename(positional or keyword arg)    : python file generated by `rbnf` command, or rbnf sour file
     - opt(positional or keyword arg) = False : optimize switch
    
    
    
    λ printf "ignore [space]\nspace := R'\s+'\nterm := R'[^\s\(\)]+'\nsexpr ::= '(' sexpr* ')' | term" > lisp.rbnf
    λ rbnf run lisp
    
    type `:i` to switch between python mode and parsing mode.
    The last result of parsing is stored as symbol `res`.
    
    runbnf> (add 1 (add 2))
    
    sexpr[
          Tokenizer(name='auto_const', value='(', lineno=0, colno=0)
          sexpr[
                Tokenizer(name='term', value='add', lineno=0, colno=1)
          ]
          sexpr[
                Tokenizer(name='term', value='1', lineno=0, colno=5)
          ]
          sexpr[
                 Tokenizer(name='auto_const', value='(', lineno=0, colno=7)
                 sexpr[
                       Tokenizer(name='term', value='add', lineno=0, colno=8)
                 ]
                 sexpr[
                       Tokenizer(name='term', value='2', lineno=0, colno=12)
                 ]
                 Tokenizer(name='auto_const', value=')', lineno=0, colno=13)
          ]
          Tokenizer(name='auto_const', value=')', lineno=0, colno=14)
    ]
    
    runbnf> exit
    
    λ rbnf cc lisp.bnf -output lisp.py
    ...

Related

RBNF is leveraged to handle parsing tasks in some projects. Feel free to check them to get a better knowledge:

  • Yet Another Python Python:

    A Python bytecode compiler to provide custom extended syntaxes and optimization passes.
    This provides a parser for extended Python(compatible to Python 3.6.x-3.8.0).

    {"a" : a, "b": {"c": c}} = {"a": 1, "b": {"c": 2, **other2}, **others1}
    assert a == 1 and c == 2
  • Reley

    A haskell-like language implemented in Python. This provides a parser for a part of haskell.

      import operator (add, eq)
      import functools (reduce)
      import toolz (curry)
      import reley.prelude ((+))
    
      infix 5 (==)
      infix 0 ($)
      (==) = curry eq
      ($) a b = a b
      (+) = curry add
    
    
      m_sum lst = if lst == [] then 0
              else destruct lst
              where
                  destruct (a, b) = a + m_sum(b)
  • ReFining:

    A type checker.

    python cli.py
    
    reF> let x: 'a = 1 in
          fn x: 'a -> x ;;
    =>  (int`2 => int`2)
    
  • RMalt:

    A concise demo of intepreter implementation.

    python test.py
    
    malt> let (%+) = (l, r) -> (l + r) % r;;
    malt> let f = (a, b) -> a  + b;;
    malt> f 1 2;;
    =>  3
    malt> f 1
          ;;
    =>  Fn(b)<id=3061211909288>
    malt> infix %+ 5;;
    =>  5
    malt> let (%+) = (l, r) -> (l % r) + r;;
    malt> 5 %+ 6;;
    =>  11
    
    

EDSL in Python

To be continue.

RBNF's compiler(rbnf -> python or others) relies on bootstrap.

Actually, in current version, bootstrap of RBNF is implemented in EDSL.

See rbnf bootstrap in Python.

A snippet could be previewed here.

rbnf = Language("RBNF")

...

@rbnf
class Primitive(Parser):

    @classmethod
    def bnf(cls):
        # @formatter:off
        return optimize(
                 C('(') + Or @ "or_" + C(')')
               | C('[') + Or @ "optional" + C(']')
               | Name @ "name"
               | Str  @ "str")
        # @formatter:on

     @classmethod
    def rewrite(cls, state: State):
        get = state.ctx.get
        or_: Parser = get('or_')
        optional: Parser = get('optional')
        ...

...

Other backend

To be continue.

About

This project's lifetime has ended. The successor is https://github.com/thautwarm/frontend-for-free which is WIP. You can check lark-parser project which is a good alt.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published