Notes on porting a manuscript from Scrivener

[instancezero]

I'm moving my WIP from Scrivener to NW with this process:

• Select a scene in Scrivener
• Copy it to LibreOffice Writer, save the document
• Use pandoc to convert the ODT to MD
• Run a little script to tweak the pandoc output and convert it to NW's dialect of MD
• Create a scene in NW
• Open the scene and copy the scene header
• Import the MD text
• Paste the scene header back in
• Copy comments from Scrivener and add as % comments near the paragraph they refer to

My first suggestion is a simple one: instead of the "this will overwrite yes/no" dialog, present a dialog that offers Replace/Append/Cancel as options, thus eliminating the copy/paste of the scene header.

It would also be helpful in the file selection dialog defaulted to all file formats (.txt, .md, and .nwd) with options for a specific format in the drop-down.

I miss the ability to highlight and annotate text in Scrivener. Actually, the requirement to highlight text is a little irritating; often I just want to add a notation at a specific point. My primary use case is to quickly add some feedback from a critique group in real time. Would you consider adding an inline comment capability? I'd suggest:

[note]some MD free note here[/note]

Although I'm sure that others value the ability to attach the comment to some text. Maybe something like

Jane found it peculiar [note/ over-use of peculiar here] that...

for a stand-alone comment, and

[note "her" is ambiguous here]he thought it was hers[/note]

for a comment that applies to a segment of text. No MD allowed in the comment itself, of course.

As a bonus it would be cool if those notes got exported as comments in an ODT file.

[vkbo]

I'm moving my WIP from Scrivener to NW with this process:
[...]
My first suggestion is a simple one: instead of the "this will overwrite yes/no" dialog, present a dialog that offers Replace/Append/Cancel as options, thus eliminating the copy/paste of the scene header.

The current import feature isn't well suited for repeated imports, so I tend to prefer to do it in bulk.

novelWriter has a split document feature that can split by any header level you want. I have used this to import large public domain texts that I use for stress testing novelWriter. Including the full King James bible, Moby Dick and A Tale of Two Cites. It is still a bit of work, but it is quite manageable. It's a lot easier than to import many small clips.

Anyway, Replace/Append/Cancel should be doable. It is fairly straightforward.

It would also be helpful in the file selection dialog defaulted to all file formats (.txt, .md, and .nwd) with options for a specific format in the drop-down.

Sure, that can be managed.

I miss the ability to highlight and annotate text in Scrivener. Actually, the requirement to highlight text is a little irritating; often I just want to add a notation at a specific point. My primary use case is to quickly add some feedback from a critique group in real time. Would you consider adding an inline comment capability?

There is already a feature idea for a general note comment (not inline) here: #1133.

Highlighting has been implemented (#705, #1715) and will be available in the 2.4 release (currently in beta). However, highlighting doesn't come with a note capability.

I would very much like to add such a feature, but since this is not rich text, it is a little tricky to allow free form text inside tags and parse them reliably with just simple regex. I haven't come up with a good implementation idea yet, but I'm very open to suggestions.

Perhaps we could add them in a similar way to footnotes (#342). The idea there is to add a marker to put in the text, and a way to specify the text that goes into the footnote as a separate text block. Perhaps something similar to special comments, so we can avoid adding more ways to format additional text.

The idea is to let a [fn:<ref>] or [footnote:<ref>] tag indicate where the footnote is inserted, and the <ref> points to the corresponding text. The text can be placed anywhere, so the logical placement would be next to the paragraph where it is defined. This is close enough to be easy to write and edit, and doesn't matter for exports since the text is indexed in advance, so any such reference can be known before tokenizing the text.

My immediate thought is that an inline note feature can easily be implemented in a similar way with [note:<ref>].

Outputting each of them to ODT would be easy enough to manage.

How does this sound?

[instancezero]

I was thinking for the import/export use case, the "pre-set format" would be something like ISO8601, which is also why I'd make it a specific use bit of metadata. Those of us with custom calendars could keep doing what we're doing and ignore that use case.

As a user I have no problem with your release cycle. I'm used to projects where point releases are a year apart or more.

As a developer, especially with a) my more rudimentary Python skills and b) resisting my inclination to think "these tokens should be refactored as a dict", it's hard to contribute because you're actively doing a lot of work on the code and have already overhauled tokenizer.py.

Just yesterday I realized I should stop trying to muck with the code and get back to my [curse words redacted] manuscript. I wrote a blog post about getting suckered into coding holes. Now I'm happy to be leaving my contributions at the conceptual level!

[vkbo]

I was thinking for the import/export use case, the "pre-set format" would be something like ISO8601, which is also why I'd make it a specific use bit of metadata. Those of us with custom calendars could keep doing what we're doing and ignore that use case.

In my social media profile I have "ISO 8601 Enthusiast" listed, so I am definitely a fan!

However, since I mostly write SciFi, I don't only deal with calendars tailored to the orbital properties of Earth. Hence I don't want to lock down format. I think letting the user chose between a few sort functions could solve most of it. Like alphabetical, and positive/negative numerical, etc. ISO8601 is sortable by plain alphabetical/numerical sorting, so it's a good format and would still work.

As a user I have no problem with your release cycle. I'm used to projects where point releases are a year apart or more.

novelWriter is still in the feature-adding part of its life cycle. The main reason I manage to retain a good level of stability is that I currently have 99.51% test coverage, with multiple overlapping tests. It lets me know immediately if I break anything. There are certain things not covered, like some GUI features like drag and drop. These are notoriously difficult to test with the Qt framework, and is quite buggy because Qt too don't test it properly. I also use high quality linting tools to assist me, which tend to catch type inconsistencies before they become bugs. I had a lot more type-related bugs early on.

As a developer, especially with a) my more rudimentary Python skills and b) resisting my inclination to think "these tokens should be refactored as a dict", it's hard to contribute because you're actively doing a lot of work on the code and have already overhauled tokenizer.py.

Heh. When I first started this project, Python dicts weren't ordered, so that wasn't possible. Unless you meant the individual tokens in the list. Using the tuple is more efficient, but less readable and a little more error prone. Although the type checking tool lets me know if I make a mistake.

The project structure is also still stored in memory as a list preserving the order, and a dict for the data. The double storage is too closely tied to many features that I can't be bothered rewriting it, and it has worked for years and is pretty solid.

I am constantly updating the code when Python changes, and when I learn new things about how to optimally use Python. So this project is keeping me more on my toes than what I do at my day job, which is also mostly writing Python code.

Just yesterday I realized I should stop trying to muck with the code and get back to my [curse words redacted] manuscript. I wrote a blog post about getting suckered into coding holes. Now I'm happy to be leaving my contributions at the conceptual level!

I read that earlier today, yeah. I too need to step away from the code some times to do some actual writing!

[instancezero]

I think I was ranting just last week about SF where humans land on a brand new planet and the occupants talk about events that were "x years ago" and I'm screaming "how long is one of their years, anyway?" Not to mention that if humans got there at near-light speed their own timeline is completely different from Earths anyway. Relativistic physics is one big reason why my work is set in the near future on Earth!

I'm not aware of timeline tool that accounts for time-space dilation so I'm assuming that people who want to use a third party tool are going to be forced into an Earth calendar. Even then ISO8601 can accommodate "Stardate 5503" and crowbar in a fractional part. My guess is that the main application would be for something like historical fiction where nailing the dates would be important for a credible story.

[vkbo]

I'm a physicist by education, so Einstein doesn't scare me. 😄

One of my favourite authors, Alastair Reynolds, who's also a physicist, handles near light speed and relativistic time tracking quite well and makes it a part of the plot. I like his approach, although I don't need to do that in any of my current stories.

Anyway, I mostly track time in my writing for my own sake, for internal consistency.

[instancezero]

My background is physics too. No fear but lots of respect. ;) I'll look up the work of Mr. Reynolds.