Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates for PO files #14601

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Updates for PO files #14601

wants to merge 3 commits into from

Conversation

RestorerZ
Copy link
Contributor

The following has been done to correct a possible unexpected processing of PO
files described in issue #14490 :

  • All existing PO files were reformatted to textwidth 79;
  • The modeline vim:tw=79:ts=8:ft=po: was added to the PO files;
  • In makefiles, the --width=79 parameter has been set for gettext tools;
  • An $(PS) $(PSFLAGS) Set-ItemProperty -Path '$@' -Name LastWriteTime -Value (Get-Date -Format 'o')
    command was added to Make_mvc.mak for guaranteed timestamp updates.

Other:

  • Added the option to Makefile to convert files using the "msgconv" tool;
  • When creating a template file "vim.pot", the values in the
    "Project-Id-Version" and "Report-Msgid-Bugs-To" fields of the file header are
    automatically set.

@RestorerZ
Copy link
Contributor Author

Timestamps of PO files after commit:

$ ls --time-style=full-iso -l *.po
-rw-r--r-- 1 Admin 197121 112644 2024-04-20 14:02:51.774302800 +0300 af.po
-rw-r--r-- 1 Admin 197121 288999 2024-04-20 14:02:51.774302800 +0300 ca.po
-rw-r--r-- 1 Admin 197121 107811 2024-04-20 14:04:46.112465000 +0300 cs.cp1250.po
-rw-r--r-- 1 Admin 197121 107803 2024-04-20 14:02:51.774302800 +0300 cs.po
-rw-r--r-- 1 Admin 197121 178938 2024-04-20 14:02:51.774302800 +0300 da.po
-rw-r--r-- 1 Admin 197121 301825 2024-04-20 14:02:51.774302800 +0300 de.po
-rw-r--r-- 1 Admin 197121  14954 2024-04-20 14:02:51.774302800 +0300 en_GB.po
-rw-r--r-- 1 Admin 197121 223252 2024-04-20 14:02:51.774302800 +0300 eo.po
-rw-r--r-- 1 Admin 197121 293130 2024-04-20 14:02:51.774302800 +0300 es.po
-rw-r--r-- 1 Admin 197121 272131 2024-04-20 14:02:51.774302800 +0300 fi.po
-rw-r--r-- 1 Admin 197121 225578 2024-04-20 14:02:51.774302800 +0300 fr.po
-rw-r--r-- 1 Admin 197121 254909 2024-04-20 14:02:51.774302800 +0300 ga.po
-rw-r--r-- 1 Admin 197121 154444 2024-04-20 14:02:51.774302800 +0300 hu.po
-rw-r--r-- 1 Admin 197121 294468 2024-04-20 14:02:51.774302800 +0300 it.po
-rw-r--r-- 1 Admin 197121 289136 2024-04-20 14:04:46.742465900 +0300 ja.euc-jp.po
-rw-r--r-- 1 Admin 197121 330101 2024-04-20 14:02:51.774302800 +0300 ja.po
-rw-r--r-- 1 Admin 197121 289366 2024-04-20 14:04:48.912468900 +0300 ja.sjis.po
-rw-r--r-- 1 Admin 197121 164008 2024-04-20 14:04:49.532469800 +0300 ko.po
-rw-r--r-- 1 Admin 197121 181354 2024-04-20 14:02:51.774302800 +0300 ko.UTF-8.po
-rw-r--r-- 1 Admin 197121   9073 2024-04-20 14:02:51.774302800 +0300 lv.po
-rw-r--r-- 1 Admin 197121 144753 2024-04-20 14:02:51.774302800 +0300 nb.po
-rw-r--r-- 1 Admin 197121 105886 2024-04-20 14:02:51.774302800 +0300 nl.po
-rw-r--r-- 1 Admin 197121 144753 2024-04-20 14:02:51.774302800 +0300 no.po
-rw-r--r-- 1 Admin 197121 168842 2024-04-20 14:04:50.802471600 +0300 pl.cp1250.po
-rw-r--r-- 1 Admin 197121 168834 2024-04-20 14:02:51.774302800 +0300 pl.po
-rw-r--r-- 1 Admin 197121 178999 2024-04-20 14:04:50.202470700 +0300 pl.UTF-8.po
-rw-r--r-- 1 Admin 197121 183699 2024-04-20 14:02:51.774302800 +0300 pt_BR.po
-rw-r--r-- 1 Admin 197121 381605 2024-04-20 14:04:51.542472600 +0300 ru.cp1251.po
-rw-r--r-- 1 Admin 197121 514897 2024-04-20 14:02:51.774302800 +0300 ru.po
-rw-r--r-- 1 Admin 197121 143845 2024-04-20 14:04:52.132473400 +0300 sk.cp1250.po
-rw-r--r-- 1 Admin 197121 143837 2024-04-20 14:02:51.774302800 +0300 sk.po
-rw-r--r-- 1 Admin 197121 376927 2024-04-20 14:02:51.774302800 +0300 sr.po
-rw-r--r-- 1 Admin 197121 145328 2024-04-20 14:02:51.774302800 +0300 sv.po
-rw-r--r-- 1 Admin 197121 303448 2024-04-20 14:02:51.774302800 +0300 tr.po
-rw-r--r-- 1 Admin 197121 296322 2024-04-20 14:04:52.802474400 +0300 uk.cp1251.po
-rw-r--r-- 1 Admin 197121 371604 2024-04-20 14:02:51.774302800 +0300 uk.po
-rw-r--r-- 1 Admin 197121 139744 2024-04-20 14:02:51.774302800 +0300 vi.po
-rw-r--r-- 1 Admin 197121 238203 2024-04-20 14:04:53.992476000 +0300 zh_CN.cp936.po
-rw-r--r-- 1 Admin 197121 238154 2024-04-20 14:04:53.452475300 +0300 zh_CN.po
-rw-r--r-- 1 Admin 197121 260121 2024-04-20 14:02:51.774302800 +0300 zh_CN.UTF-8.po
-rw-r--r-- 1 Admin 197121 111722 2024-04-20 14:04:54.582476800 +0300 zh_TW.po
-rw-r--r-- 1 Admin 197121 120882 2024-04-20 14:02:51.774302800 +0300 zh_TW.UTF-8.po

@@ -65,14 +65,18 @@ PO_INPUTLIST = \

first_time: $(PO_INPUTLIST) $(PO_VIM_INPUTLIST)
$(VIM) -u NONE --not-a-term -S tojavascript.vim $(LANGUAGE).pot $(PO_VIM_INPUTLIST)
$(XGETTEXT) --default-domain=$(LANGUAGE) \
$(XGETTEXT) --width=79 --package-name="Vim" --package-version="9.1" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with adding --package-name="Vim", but please leave out --package-version="9.1" and msgid-bugs-address. I think bugs should better be adressed by the relevant maintainers, and package-version will just be another version number that needs to be addressed on a new release

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes indeed, about the version number I overlooked that, that's another place to remember to change when a new version comes out. I'll clean that up.

As for the "msgid-bugs-address" field, this is exactly what was done for developers. Here's what the documentation says:

‘--msgid-bugs-address=email@address’

Set the reporting address for msgid bugs. This is the email address or URL to which the translators shall report bugs in the untranslated strings:

https://www.gnu.org/software/gettext/manual/gettext.html#index-_002d_002dmsgid_002dbugs_002daddress_002c-xgettext-option

That's why I made it auto-complete. I think that, probably, it is not necessary to clean it up.

@@ -123,22 +123,24 @@ ja.sjis.po: ja.po
@$(MAKE) -nologo -f Make_mvc.mak sjiscorr
-$(RM) $@
!IF EXIST ("$(GETTEXT_PATH)\msgconv.exe")
"$(GETTEXT_PATH)\msgconv.exe" -t CP932 $? | .\sjiscorr.exe > $@
"$(GETTEXT_PATH)\msgconv.exe" -w 79 -t CP932 $? | .\sjiscorr.exe > $@
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here you are using -w 79, further down you are using --width=. Please use one consisten style. Also with regards to the quoation. you are now changing from \" . I'd prefer if you don't change unrelated things, it makes reviewing harder. If you want to change quotation style, add an additional commit just for those stylistic changes please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here you are using -w 79, further down you are using --width=. Please use one consisten style.

For brevity, I wanted the line to fit into 80 columns. I'll fix it.

Also with regards to the quoation. you are now changing from \" . I'd prefer if you don't change unrelated things, it makes reviewing harder. If you want to change quotation style, add an additional commit just for those stylistic changes please.

Oh, I'm sorry! You've already talked about it once. Let me not redo it this time. I'm sorry, please, once again.

@@ -524,11 +556,13 @@ update-po: $(MOFILES:.mo=)

# Don't add a dependency here, we only want to update the .po files manually.
$(LANGUAGES):
@$(MAKE) -nologo -f Make_mvc.mak GETTEXT_PATH="$(GETTEXT_PATH)" $(PACKAGE).pot
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are you removing -nologo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, no, I didn't delete "-nologo". The "-l" switch is a synonym for "-nologo". It's just not used very often and not documented much anywhere. This way the string is shorter and fits into 80 columns.

iconv -f UTF-8 -t EUC-JP $< | \
$(SED) -e 's/charset=[uU][tT][fF]-8/charset=EUC-JP/' > $@; \
fi
$(SED) -i.bkp -e \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sed -i is not portable.

Copy link
Contributor Author

@RestorerZ RestorerZ Apr 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I once read that the "-i" switch is an extension implemented by GNU.

But now, as I checked, it is supported in both BSD and macOS. You just have to be sure to specify the extension of the backup file (you can specify an empty one).
https://man.freebsd.org/cgi/man.cgi?sed
https://www.baeldung.com/linux/gnu-bsd-stream-editor#2-the--i-option
https://stackoverflow.com/questions/5694228/sed-in-place-flag-that-works-both-on-mac-bsd-and-linux

I checked on FreeBSD and it works.

src/po/es.po Outdated
@@ -10219,3 +10199,12 @@ msgstr "nombre de la biblioteca dinámica MzScheme"
msgid "name of the MzScheme GC dynamic library"
msgstr "nombre de la biblioteca dinámica MzScheme GC"

#, fuzzy, c-format
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that and the following lines can be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I guess so. It was formatted by gettext and it must have tagged those lines and I didn't check after it. I'll delete those lines and check the other files.

@RestorerZ RestorerZ marked this pull request as draft April 21, 2024 09:55
@RestorerZ RestorerZ marked this pull request as ready for review April 21, 2024 18:30
@julio-b
Copy link
Contributor

julio-b commented Apr 22, 2024

Hi, i tested your tree and I'm not sure why but some files are still being modified. At least for me:

~/p/vim ((2a4cf5d2b...) *$%) $ git status
HEAD detached at FETCH_HEAD
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   src/po/ja.euc-jp.po
	modified:   src/po/ja.sjis.po
	modified:   src/po/ru.cp1251.po
	modified:   src/po/sk.cp1250.po
	modified:   src/po/uk.cp1251.po

@RestorerZ
Copy link
Contributor Author

@julio-b
Thanks for the feedback!
That's too bad.
Honestly, I don't have any ideas on how to fix it yet.
Hopefully @k-takata will give me some tips on where to at least start.
But in any case, I'll try to solve it somehow.

@RestorerZ
Copy link
Contributor Author

RestorerZ commented Apr 23, 2024

@julio-b
Can you please tell me how you got this?
On the host machine (WinNT 6.1 32-bit) I have no reproduction.

$ git fetch VimOrigin pull/14601/head && git checkout FETCH_HEAD && git status
From https://github.com/vim/vim
 * branch                refs/pull/14601/head -> FETCH_HEAD
Updating files: 100% (79/79), done.
Note: switching to 'FETCH_HEAD'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 2a4cf5d2b Remarks eliminated
HEAD detached at FETCH_HEAD
nothing to commit, working tree clean

What am I doing wrong?

$ ls --time-style=full-iso -l ja.po ja.euc-jp.po ja.sjis.po ru.po ru.cp1251.po sk.po sk.cp1250.po uk.po uk.cp1251.po
-rw-r--r-- 1 Admin 197121 289136 2024-04-23 15:11:05.624709000 +0300 ja.euc-jp.po
-rw-r--r-- 1 Admin 197121 330101 2024-04-23 15:11:05.644709000 +0300 ja.po
-rw-r--r-- 1 Admin 197121 289366 2024-04-23 15:11:05.694709100 +0300 ja.sjis.po
-rw-r--r-- 1 Admin 197121 381596 2024-04-23 15:11:05.910709700 +0300 ru.cp1251.po
-rw-r--r-- 1 Admin 197121 514888 2024-04-23 15:11:05.940709800 +0300 ru.po
-rw-r--r-- 1 Admin 197121 143836 2024-04-23 15:11:05.940709800 +0300 sk.cp1250.po
-rw-r--r-- 1 Admin 197121 143828 2024-04-23 15:11:05.960709800 +0300 sk.po
-rw-r--r-- 1 Admin 197121 296322 2024-04-23 15:11:06.050709900 +0300 uk.cp1251.po
-rw-r--r-- 1 Admin 197121 371604 2024-04-23 15:11:06.070709900 +0300 uk.po

@RestorerZ
Copy link
Contributor Author

@julio-b
I started Ubuntu 22 in a virtual machine today. I tweaked it a bit and tried the same thing I did on the host machine.
The result is the same.

user@user-vm:~/vim4tst$ git fetch origin pull/14601/head && git checkout FETCH_HEAD && git status
remote: Enumerating objects: 83, done.
remote: Counting objects: 100% (83/83), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 83 (delta 67), reused 80 (delta 67), pack-reused 0
Unpacking objects: 100% (83/83), 1.75 MiB | 1.50 MiB/s, done.
From https://github.com/vim/vim
 * branch                refs/pull/14601/head -> FETCH_HEAD
Note: switching to 'FETCH_HEAD'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 2a4cf5d2b Remarks eliminated
HEAD detached at FETCH_HEAD
nothing to commit, working tree clean
user@user-vm:~/vim4tst/src/po$ ls --time-style=full-iso -l ja.po ja.euc-jp.po ja.sjis.po ru.po ru.cp1251.po sk.po sk.cp1250.po uk.po uk.cp1251.po
-rw-rw-r-- 1 user user 289136 2024-04-23 22:06:12.680003354 +0300 ja.euc-jp.po
-rw-rw-r-- 1 user user 330101 2024-04-23 22:06:12.680003354 +0300 ja.po
-rw-rw-r-- 1 user user 289366 2024-04-23 22:06:12.680003354 +0300 ja.sjis.po
-rw-rw-r-- 1 user user 381596 2024-04-23 22:06:12.692003527 +0300 ru.cp1251.po
-rw-rw-r-- 1 user user 514888 2024-04-23 22:06:12.696003587 +0300 ru.po
-rw-rw-r-- 1 user user 143836 2024-04-23 22:06:12.696003587 +0300 sk.cp1250.po
-rw-rw-r-- 1 user user 143828 2024-04-23 22:06:12.696003587 +0300 sk.po
-rw-rw-r-- 1 user user 296322 2024-04-23 22:06:12.700003645 +0300 uk.cp1251.po
-rw-rw-r-- 1 user user 371604 2024-04-23 22:06:12.700003645 +0300 uk.po

@julio-b
Copy link
Contributor

julio-b commented Apr 23, 2024

@julio-b Can you please tell me how you got this? On the host machine (WinNT 6.1 32-bit) I have no reproduction.

$ git fetch VimOrigin pull/14601/head && git checkout FETCH_HEAD && git status

@RestorerZ you need to compile vim after getting these commits. Just a checkout will not modify the po files, obviously.

Anyway, I think trying to predict the wrap behavior is not going to solve the problem for everyone. After digging around in the mailing list i discovered this message https://lists.gnu.org/archive/html/bug-gettext/2023-07/msg00026.html which explains the situation. (This thread is about msgattrib, but the same thing applies for msgconv and the whole gettext suite).

Also it appears that these files have been problematic since forever, forcing Bram to add workarounds like this:

vim/Makefile

Lines 335 to 343 in ea99903

# Make sure ja.sjis.po is newer than ja.po to avoid it being regenerated.
# Same for cs.cp1250.po, pl.cp1250.po and sk.cp1250.po.
touch dist/$(VIMRTDIR)/src/po/ja.sjis.po
touch dist/$(VIMRTDIR)/src/po/cs.cp1250.po
touch dist/$(VIMRTDIR)/src/po/pl.cp1250.po
touch dist/$(VIMRTDIR)/src/po/sk.cp1250.po
touch dist/$(VIMRTDIR)/src/po/zh_CN.cp936.po
touch dist/$(VIMRTDIR)/src/po/ru.cp1251.po
touch dist/$(VIMRTDIR)/src/po/uk.cp1251.po

@RestorerZ
Copy link
Contributor Author

@RestorerZ you need to compile vim after getting these commits. Just a checkout will not modify the po files, obviously.

Yeah, just updated “gettext” in Ubuntu now and will try to compile.

Anyway, I think trying to predict the wrap behavior is not going to solve the problem for everyone. After digging around in the mailing list i discovered this message https://lists.gnu.org/archive/html/bug-gettext/2023-07/msg00026.html which explains the situation. (This thread is about msgattrib, but the same thing applies for msgconv and the whole gettext suite).

Also it appears that these files have been problematic since forever, forcing Bram to add workarounds like this:

@julio-b
Look, that changes everything then.
Thank you very much for this research!
I'll see what I can do then and if anything can be done at all.
Thanks!

@RestorerZ RestorerZ marked this pull request as draft May 19, 2024 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants