main: introduce --_makeTagEntryReflection-<LANG> option to filter tags #3027
Conversation
|
@rickalex21, could you try this pull request?
I will merge this pull request but not in soon because this pull request includes one critical bug. |
masatake
force-pushed
the
makeTagEntryReflection
branch
from
e166632
to
d34d67c
Compare
|
Without this pull request: With this pull request: I repeated the test. With this pull request, ctags runs slower but it is acceptable impatct. |
masatake
changed the title
masatake
force-pushed
the
makeTagEntryReflection
branch
from
d34d67c
to
5831145
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #3027 +/- ##
==========================================
- Coverage 85.02% 84.93% -0.10%
==========================================
Files 228 200 -28
Lines 53901 47953 -5948
==========================================
- Hits 45832 40731 -5101
+ Misses 8069 7222 -847 ☔ View full report in Codecov by Sentry. |
|
I have make a PR #3029 to resolve this. |
|
@leleliu008 , thank you! |
|
@masatake I did follow your steps but I can't seem to get the same output as you. This is the output that I'm getting: Command ctags/ctags --options=mymd.ctags --fields=+'{extras}' -o - input.mdOutput I created a dummy markdown-test page here and the myTags generated and files are here. Ideally, I would like to do Another thing I was thinking about is that, how would one deal with same name files like README.md? For In that situation you would have to go based on the folder
python.md ## Linux
You can install python in linux via a package manager or source code. |
|
Looks strange. Could you try the following command line? I expect you will get: |
|
I tested it with my markdown repo and it looks like that works when I do the tag like this: :tag linux@markdowndir/code/python.mdIs there a way where I can just do: :tag linux pythonOR :tag python linux@masatake This does look the same as yours. I did get this message also Here is my output: |
Thank you for trying. Now, we confirmed that my change works expectedly. You had to add I don't know well about vim. So I cannot help you with the usage of vim. tags file: We call the field where Consider "Linux" is in "foo/bar/baz.md".
Instead of choosing some of them, you can write down the algorithm for making the extended names. |
I'm not sure what I need to revise I used the files that you provided input.md and mymd.ctags? The I don't think these conflict with anything unless I use them in the command line?
Sorry that I was not explicit. What I was trying to say is how can I get these tags?
I edited the tags file to show the expected result: Out of these files: code/python.md ## Introduction
Python is an interpreted high-level general-purpose programming language.
## Getting Started
In this section we will cover the following topics.
* Installing
* Configuring
* Hello World
## Installing
Python usually comes pre-installed on most systems but you will need the latest version
of python. There are many ways to install the latest version.code/lua.md ## Introduction
Lua is is a lightweight, high-level, multi-paradigm programming language designed
primarily for embedded use in applications.
## Getting Started
In this section we will cover the following topics.
* Installing
* Configuring
* Hello World
## Installing
To get started with Lua, visit the [Lua Download Page](https://www.lua.org/download.html)
and follow the instructions or see the options below.I don't know why it captured this: I think it had something to do with the frontmatter in the file which I removed. ---
title: Python
draft: false
date: 2021-05-16T13:15:31-05:00
tags: ['code','python']
menu:
main:
parent: 'code'
---Thanks for your help, I appreciate it. |
Is this written in the standard markdown syntax? I found it. |
No it's not markdown, it's it's frontmatter language yaml. Special information(meta data) used by static site generators like hugo and how you mentioned jekyll. Pandoc, vuepress and many other's use it. This is extra information that is needed to create html documents. Your default markdown.ctags ignores it because it does not match the patterns.
You found it that's a good example. This is an example template: ---
title: "{{ replace .Name "-" " " | title }}"
date: {{ .Date }}
draft: false
tags: ['code','{{.Name}}']
menu:
main:
parent: 'code'
---Then this would be created: ---
title: "Example"
date: 2021-05-17T23:20:21-05:00
draft: false
tags: ['code','example']
menu:
main:
parent: 'code'
---
## Introduction
Markdown starts here.
I agree the command I already have another tags file that I create that captures the frontmatter (the 'code' and 'example' tags). frontmatter.ctags |
|
Thank you. Before introducing If I can merge your frontmatter.ctags to our ctags source tree, you can remove it from your "~/.ctags.d". |
|
I found frontmatter has two notations: json and yaml. |
You want to use my frontmatter.ctags in your repo? I'm ok with that but that file is not optimized, do you think you can optimize it to make it faster? My *.ctags files are not as fast as yours.
Toml, yaml, and JSON. Toml and yaml are more popular for front matter. |
Yes, I would like to introduce your parser to ctags. In the future, I will rewrite the Frontmatter parser as a subparser of YAML. However, the detailed parser implementation is not important here. |
It is not true. The markdown.ctags couldn't ignore effectively. I opened #3031. With the change, the markdown parser can skip the frontmatter area. You may want to extract "tags:" in the frontmatter area. Lets' find the way to extract them after solving #3020. |
They are loaded automatically when ctags starts. I cannot say much without reading the files. With this line, we cannot predict whether Markdown parser runs or the frontmatter parser runs on a fie having .md as suffix. |
|
Here is the new mymd.ctags: tags output:
|
I think it's up to the users to decide what they want to capture from a front matter. ---
title: Computer Services
pricing:
enable : true
title : Our Plans
pricing_table :
# pricing table loop
- name : Free
price : $99
unit : month
services:
- 1GB Disk Space
- 10 Email Account
- Script Installer
- 1 GB Storage
- 10 GB Bandwidth
- 24/7 Tech Support
button:
label : "Signup"
link : "#"
---
## Welcome
Hello, welcome to our pricing plans...Look at this
You're right, what I meant is that it's not captured by markdown.ctags because it's not valid This is in yaml: ---
title: "Testing"
date: 2021-05-18T10:12:21-05:00
# How about a comment here? Don't confuse me with markdown.
draft: false
tags: ['code','testing']
menu:
main:
parent: 'code'
---
## Testing
This is a test.This is in json: {
"title": "I'm in Json",
"date": "2021-05-18T15:12:21.000Z",
"draft": false,
"tags": [
"code",
"testing"
],
"menu": {
"main": {
"parent": "code"
}
}
}
## Testing
This is a test.This is in toml: +++
title = 'Testing in Toml'
date = 2021-05-18T15:12:21.000Z
draft = false
tags = [ "code", "testing" ]
#comment here, don't confuse me with markdown
[menu.main]
parent = "code"
+++
## Testing
This is a test.I confirm that toml, json, and yaml work in my single.html template with this: {{ .Params.tags }}Output:
The files are here. frontmatter.ctags - This captures tags in front matter. For example if you had this front pandoc.md ---
title: "Pandoc"
date: 2021-03-07T22:30:18-05:00
draft: false
tags: ['code','tutorials','pandoc']
menu:
main:
parent: "code"
---It would output this in your tags file. tags-fm The command to run this is like this: ctags --fields=+n -f /some/path/content/en/.tags-fm --languages=frontmatter -R /some/path/content/en/md.ctags - This should be taken out of ~/.ctags.d/ to avoid conflicts. This was created because web.ctags - At the time I wanted to tag my files html,css,scss, and js with comments. Not sure Let's say you want to tag 'main' in all your files. // main.js main function
function main(){This would produce a tags file like this: The command to generate the tags is this: ctags --fields=+n -f /some/path/.tags --languages=web -R /some/path/layouts -R /some/path/assets/sass -R /some/path/staticThe other files I deleted I don't need anymore. The file default.md which I deleted. At the time I wanted to add pandoc markdown with extension .pdc
The problem as to why this command was was not working
I think you meant NONE.
It looks good but I see 3 issues with this.
I don't know too much about programming masatake but maybe it's worth taking a look at a The downside to that is that you're limited by how fast a program can create a syntax tree to Every time ctags runs, the entire project is parsed again. I'm not sure if there is a work around to If you want to use my frontmatter.ctags it's right here. Thanks |
|
Too many topics are included in this discussion. I cannot handle everything at once.
Here is the new mymd.ctags. |
|
@masatake Thanks, it looks like that will work but there are a few issues.
This is my output: |
Add
Add ctags(1) man page explains both options. |
|
|
|
@masatake I'm still getting relative paths, but that's ok because it doesn't matter because all:
../ctags/ctags --options=NONE --exclude=Makefile --tag-relative=no --quiet --options=mymd.ctags -R --fields=+'{extras}' |
|
I'm sorry. I read ctags(1) of Exuberant Ctags. What we should read is that of Universal Ctags. |
|
@masatake Thanks, that works. 👍 |
masatake
force-pushed
the
makeTagEntryReflection
branch
from
5831145
to
8f5385f
Compare
|
I would like to add more code and documents. |
|
It will take more time for merging this pull request.
|
masatake
force-pushed
the
makeTagEntryReflection
branch
from
8f5385f
to
70ceb9b
Compare
| @@ -29,8 +29,6 @@ extern langType getSubparserLanguage (subparser *s); | |||
|
|
|||
| /* A base parser doesn't have to call the following three functions. | |||
There was a problem hiding this comment.
The commet must be updated.
| @@ -348,6 +349,14 @@ static void cppInitCommon(langType clientLang, | |||
| : clientLang) & CORK_SYMTAB)) | |||
| ? makeMacroTable () | |||
| : NULL; | |||
|
|
|||
| if (Cpp.lang != Cpp.clientLang | |||
There was a problem hiding this comment.
Comments are needed.
Performance evaluation is needed for C input.
| @@ -381,6 +390,14 @@ static void cppClearMacroInUse (cppMacroInfo **pM) | |||
|
|
|||
| extern void cppTerminate (void) | |||
| { | |||
| if (Cpp.lang != Cpp.clientLang | |||
| @@ -377,6 +377,8 @@ static optionDescription LongOptionDescription [] = { | |||
| {1,1," Define new extra for <LANG>. --extras-<LANG>=+{name} enables it."}, | |||
| {1,1," --_fielddef-<LANG>=<name>,<description>"}, | |||
| {1,1," Define new field for <LANG>."}, | |||
| {1,1," --_makeTagEnryReflection-<LANG>={{ optscript-code }}"}, | |||
There was a problem hiding this comment.
s/makeTagEnry/makeTagEntry/
| @@ -159,4 +159,28 @@ const char ctagsCommonPrelude []= | |||
| " } forall\n" | |||
| " pop\n" | |||
| "} __bddef\n" | |||
| "\n" | |||
| "(string end:string _ENDWITH boolean)\n" | |||
| "/_endwith {\n" | |||
There was a problem hiding this comment.
_strendwith may be better name.
masatake
force-pushed
the
makeTagEntryReflection
branch
from
70ceb9b
to
2ece0af
Compare
parsers/rpmspec.c
Outdated
| @@ -385,10 +386,17 @@ extern parserDefinition* RpmSpecParser (void) | |||
| "rpm-spec", /* the mode name in Emacs */ | |||
| NULL }; | |||
| parserDefinition* const def = parserNew ("RpmSpec"); | |||
|
|
|||
| static parserDependency dependencies [] = { | |||
There was a problem hiding this comment.
The need for change must be explained in the commit log.
Signed-off-by: Masatake YAMATO <[email protected]>
Signed-off-by: Masatake YAMATO <[email protected]>
Signed-off-by: Masatake YAMATO <[email protected]>
Signed-off-by: Masatake YAMATO <[email protected]>
TODO:
- write about the new option in docs/optlib.rst.
- add --_makeTagEntryNotification-<LANG>={{...}}.
Signed-off-by: Masatake YAMATO <[email protected]>
masatake
force-pushed
the
makeTagEntryReflection
branch
from
2ece0af
to
9e59df0
Compare
See #3020.
The new option
--_makeTagEntryReflection-<LANG>allows users to add optscript code which ctags runs when making a tag.In the optscript code, you can modify the tag, make an extra tag derived from the tag, and/or make the tag not being printed.
Let's see examples.
input.md:
mymd.ctags
the command session:
The input file name is appended to the names of original tags.
The script hides the original tags.
If you don't want to hide them, remove the line
. _markplaceholderin mymd.ctags.TODO:
--_makeTagEntryNotification-<LANG>option and support it in optlib2c (?, should I do this when we really need it?).