Command Processing

[flatcap]

When the user enters a command, the string is parsed into chunks using MoreArgs and parse_extract_token().

Some current refactoring work, #4173, made me wonder if we could do better, using regexes (and Prex).

This might amount to nothing, but...

We'll need:

• List of commands and their arguments
• Syntax for each command
• Regex for each command

[flatcap]

Command	Arguments
account-hook	regex command
alias	[ -group name ...] key address [ , address ...]
alternates	[ -group name ...] regex [ regex ...]
alternative_order	mime-type [ /mime-subtype ] [ mime-type [ /mime-subtype ] ...]
append-hook	regex "shell-command"
attachments	{ + | - } disposition mime-type [ mime-type ...]
	?
auto_view	mime-type [ /mime-subtype ] [ mime-type [ /mime-subtype ] ...]
bind	map [ ,map ...] key function
cd	directory
charset-hook	alias charset
close-hook	regex "shell-command"
color	object [ attribute ...] foreground background
	{ header | body } [ attribute ...] foreground background regex
	index [ attribute ...] foreground background [ pattern ]
crypt-hook	regex keyid
echo	"message"
exec	function [ function ...]
fcc-hook	pattern mailbox
fcc-save-hook	pattern mailbox
finish	none
folder-hook	regex command
group	[ -group name ...] { -rx regex ... | -addr address ... }
hdr_order	header [ header ...]
iconv-hook	charset local-charset
ifdef	symbol "config-command [args...]"
ifndef	symbol "config-command [args...]"
ignore	string [ string ...]
index-format-hook	name [!]pattern format-string
lists	[ -group name ] regex [ regex ...]
lua	lua-cmds
lua-source	filename
macro	menu [ ,menu ...] key sequence [ description ]
mailboxes	mailbox [ mailbox ...]
mailto_allow	{ * | header-field ... }
mbox-hook	[ -noregex ] regex mailbox
message-hook	pattern command
mime_lookup	mime-type [ /mime-subtype ] [ mime-type [ /mime-subtype ] ...]
mono	object attribute
	{ header | body } attribute regex
	index-object attribute pattern
my_hdr	string
named-mailboxes	description mailbox [ description mailbox ...]
nospam	{ * | regex }
open-hook	regex "shell-command"
push	string
reply-hook	pattern command
reset	variable [ variable ...]
save-hook	pattern mailbox
score	pattern value
send-hook	pattern command
send2-hook	pattern command
setenv	{ ?variable | variable value }
set	{ [ no | inv | & | ? ] variable | variable=value } [...]
shutdown-hook	command
sidebar_pin	mailbox [ mailbox ...]
sidebar_unpin	{ * | mailbox ... }
source	filename
spam	regex format
startup-hook	command
subjectrx	regex replacement
subscribe-to	imap-folder-uri
subscribe	[ -group name ...] regex [ regex ...]
tag-formats	tag format-string { tag format-string ...}
tag-transforms	tag transformed-string { tag transformed-string ...}
timeout-hook	command
toggle	variable [ variable ...]
unalias	[ -group name ...] { * | key ... }
unalternates	[ -group name ...] { * | regex ... }
unalternative_order	{ * | [ mime-type [ /mime-subtype ] ...]}
unattachments	{ + | - } disposition mime-type [ mime-type ...]
	*
unauto_view	{ * | [ mime-type [ /mime-subtype ] ...]}
unbind	{ * | map [ ,map ...] } [ key ]
uncolor	{ index | header | body } { * | pattern ... }
ungroup	[ -group name ...] { * | -rx regex ... | -addr address ... }
unhdr_order	{ * | header ... }
unhook	{ * | hook-type }
unignore	{ * | string ... }
unlists	[ -group name ...] { * | regex ... }
unmacro	{ * | map [ ,map ...] } [ key ]
unmailboxes	{ * | mailbox ... }
unmailto_allow	{ * | header-field ... }
unmime_lookup	{ * | [ mime-type [ /mime-subtype ] ...]}
unmono	{ index-object | header | body } { * | pattern ... }
unmy_hdr	{ * | field ... }
unscore	{ * | pattern ... }
unsetenv	variable
unset	variable [ variable ...]
unsubjectrx	{ * | regex }
unsubscribe-from	imap-folder-uri
unsubscribe	[ -group name ...] { * | regex ... }
version	none

[roccoblues]

Coming up with regexes will be hard (or they will be useless) for some of the commands. For example echo can take:

Command	Result
echo foo	foo
echo foo#bar	foo
echo foo # bar	foo
echo "foo bar"	foo bar
echo foo\ bar	foo bar
echo foo\#bar	foo#bar
echo "foo # bar"	foo # bar
echo "foo # bar" # comment	foo # bar
echo foo"bar	foobar
echo foo\"bar	foo"bar

[flatcap]

Yeah, scary.
Perhaps a full command syntax is a bit ambitious :-)

[ossilator]

the canonical way to specify a grammar is using BNF or one of its descendants.
from that also follows that parsing grammars with "flat" regexes is a bit of a dead end, because you really want the hierarchy/recursion.

context-sensitive grammars are "fun", because the tokenizer needs to be interwoven with the parser. but this small-scale parser does that anyway.

i don't see any major problem with parsing quoted/escaped strings as tokens.
with clear layering, it's not even a problem for nested quoting (cf. regexes and shell commands). however, it's a bit of a no-win when it comes to UX, because either the user has to consistently double-escape everything (i think that's the status quo), or one gives up on clear layering to simplify the common case, at the expense of having more special rules. a somewhat middle ground is to have something like c++'s raw strings, i.e. flexible choice of the "quotes", which is of course somewhat verbose "at the ends". perl's qr// and qx// operators are a less powerful version to that.

[flatcap]

Yeah, I had EBNF in mind for the syntax.
(though I don't have much experience using it)

the tokenizer needs to be interwoven with the parser

hehe, we've cracked open the black box, a little.
We have a few unit tests, they're mostly trivial.

Would you be interested in getting your hands dirty?

[ossilator]

Would you be interested in getting your hands dirty?

not actively - way too long todo list, as always. 😇
but if someone has specific questions about how to approach a particular matter, go ahead. parsers and quoting are a minor specialty of mine ...