Python 2.7.13 not included

[prensing]

The python2 packages do not have 2.7.13, which has been out for quite a while.

[prensing]

Actually, I am behind the times. Python2 is up to 2.7.15.

[TheBigBear]

@prensing any chance you would care to write the PR to catch up on this progress in python 2.x land ...?

[prensing]

OK, I can do that. I am converting it to a loop, similar to git.sls.

Does anyone know why the SLS version are like "2.7.12150"? Where does the "150" come from? I don't see that in any of the Python files.

[TheBigBear]

@prensing check the version number windows detects and stores about python. Highlight python in add/remove programs and double check the 'version' number it detects ( usually stored in a uninstall registry key ) It is what salt winrpeo-ng uses and reads to check that version is what is expected and will only detect a clean install if those versions match.

[prensing]

See #1472 for the updates for Python2.

I did not do Python3 yet; need some advice. The newer Python 3 installers (.MSI) appear to include a dozen separate install packages (eg one for the Python executables, one for the library, etc). Is there some way to link them all to the same Salt package?

Also, since I played with it on my machine, I now have 3 different versions installed and can't seem to find the correct un-install. Need to research that.

[TheBigBear]

@twangboy I still don't install or use python 3 anywhere myself. So I have not come across this myself. Do you have any experience and insights to share with @prensing or know someone that could give the required feedback to @prensing ? He has done a great job, I think, finally making a jijnja template for the 2.x versions of python.

[TheBigBear]

@prensing I did a ver. 3 install, but it only installed one version and not multiple products for me though? Well, actually one launcher and one python 3. Did you install several and now have parallel installations of multiple ver 3.x versions? Or are you asking about fine-tuning what it installs on one version in terms of install option choices, such as 'Documentation, pip, tcl/tk and idle, python test suite, py launcher, for all users', not sure I understand your actual question?

[prensing]

@TheBigBear Sort of yes to both questions. I was testing an updated Py3 SLS (which I did not send over). The versions before 3.4 (if I remember) installed similar to Py2. For Py 3.5 and later, it installed what looks like multiple "packages":

Python 3.7.0 Core Interpreter (64-bit):
3.7.150.0
Python 3.7.0 Development Libraries (64-bit):
3.7.150.0
Python 3.7.0 Documentation (64-bit):
3.7.150.0
Python 3.7.0 Executables (64-bit):
3.7.150.0
Python 3.7.0 Standard Library (64-bit):
3.7.150.0
Python 3.7.0 Test Suite (64-bit):
3.7.150.0
Python 3.7.0 Utility Scripts (64-bit):
3.7.150.0
Python Launcher:
3.5.6063.0

(Ignore the Py launcher exact version...). Also I now have a version of 3.5, 3.6 and 3.7 on my machine. Maybe I just need to get the version numbers correct? However, when I go on the actual machine, I don't have Python 3 in the "Apps" control panel.

Note that I just used the install method that was already in the file for 3.5.2.

[twangboy]

You can just pass the name of the MSI used to install python. The pkg.remove function will add the /X option which will tell it to uninstall. Pass the same flags used to install. There are many examples in the repo.

For example: https://github.com/saltstack/salt-winrepo-ng/blob/master/adobeshockwaveplayer.sls

[twangboy]

Py3.5 changed something in their installer. It does seem to install multiple separate packages. They all show up as their own entries in the registry, yet only a single item shows up in add/remove programs. Looks like they're doing some msi voodoo that other msi installers aren't doing. We probably need to figure out how to handle this in Salt.

[twangboy]

There may be an issue with the 3.5 installer. Even though you install for All Users, the registry entry that displays in Add/Remove Programs is actually found in: HKEY_CURRENT_USER\Software\Classes\Installer\Dependencies\CPython-3.5

3.6 seems to behave the same way. That means the single Python entry as shown in Add/Remove programs won't show up in pkg.list_pkgs. Instead, you'll get entries for all the individual packages. I'm not sure how to handle this.

[twangboy]

There's also an entry under HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Uninstall\{5d57524f-af24-49a7-b90b-92138880481e}, which is still NOT AllUsers.

[TheBigBear]

@prensing OK, whilst Twangboy has looked into it, I also installed '3.7.0' using the settings as back in 3.5.x ( seen also at end of this comment ) and now I see what you mean.
But in windows add/remove it only installs two, the launcher and python itself, but salt detects 11 newly installed pkgs???

BUT at least on the surface of it the pkg.install and pkg.remove do install and remove cleanly. So not sure this is an actual problem, but have not looked deeper into whether it is indeed installed for whole system for every user, or only for salt service user etc....

this is the '3.7.0' sls definition I used:

python3_x64:
  '3.7.150.0':
    full_name: 'Python 3.7.0 (64-bit)'
    installer: 'https://www.python.org/ftp/python/3.7.0/python-3.7.0-amd64.exe'
    install_flags: '/quiet InstallAllUsers=1'
    uninstaller: 'https://www.python.org/ftp/python/3.7.0/python-3.7.0-amd64.exe'
    uninstall_flags: '/uninstall'
    msiexec: False
    locale: en_US
    reboot: False

and this is the cmd line log of an install and subsequent uninstall/remove of it.

C:\salt>salt-call pkg.remove python3_x64

local:
    ----------
    Python 3.7.0 Core Interpreter (64-bit):
        ----------
        new:
            3.7.150.0
        old:
    Python 3.7.0 Development Libraries (64-bit):
        ----------
        new:
            3.7.150.0
        old:
    Python 3.7.0 Documentation (64-bit):
        ----------
        new:
            3.7.150.0
        old:
    Python 3.7.0 Executables (64-bit):
        ----------
        new:
            3.7.150.0
        old:
    Python 3.7.0 Standard Library (64-bit):
        ----------
        new:
            3.7.150.0
        old:
    Python 3.7.0 Tcl/Tk Support (64-bit):
        ----------
        new:
            3.7.150.0
        old:
    Python 3.7.0 Test Suite (64-bit):
        ----------
        new:
            3.7.150.0
        old:
    Python 3.7.0 Utility Scripts (64-bit):
        ----------
        new:
            3.7.150.0
        old:
    Python 3.7.0 pip Bootstrap (64-bit):
        ----------
        new:
            3.7.150.0
        old:
    Python Launcher:
        ----------
        new:
            3.7.6386.0
        old:
    python3_x64:
        ----------
        install status:
            success

and the subsequent removal of it.

C:\salt>salt-call pkg.remove python3_x64

local:
    ----------
    python3_x64:
        ----------
        current:
            not installed

All looks 'OK' to me? Although I have to admit the "11" new entries do look a bit over the top and confusing, not sure why salt winrepo-ng reads "11" of them and not just two like windows itself in add/remove?

[twangboy]

Looks like we need to revisit the pkg.list_pkgs to make it more accurate.

[TheBigBear]

@damon-atkins do you have any experience and/or thoughts on this newer pkg.list_pkgs output, or is your work on NG winrepo-ng not run into this?

[twangboy]

I'm thinking we need something more along the lines of this: https://social.technet.microsoft.com/Forums/windows/en-US/d913471a-d7fb-448d-869b-da9025dcc943/where-does-addremove-programs-get-its-information-from-in-the-registry?forum=w7itprogeneral

[damon-atkins]

The issues is the python installer puts some of its registry entries in the current user space and some in the machine space. i.e its broken. They are not going to change Python2.

I looked at the GUI for the installer in 3.7 and its also broken. e.g. Install Launcher for everyone yet it defaults to installing python under the user.

The next step would be to try lots of command line installer options.
https://docs.python.org/3/using/windows.html

We can not support software installed under the user, for example the registry is not split between 32bit and 64 bit under the user. And also access issue, as the installer would need to run as the user and not SYSTEM/Admin. Looked into and put it into the too hard basket.

2018.3 includes a tool to dump the software, and it can also dump all users.
https://docs.saltstack.com/en/latest/topics/releases/2018.3.0.html#new-utils-module-salt-utils-pkg-win

[TheBigBear]

@prensing would you mind adding the ver 3.x newer pkgs in the much neater and shorter jinja version, like you did for version 2.x?
I think on the windows level it all works as expected and installs and removes the right versions.
At the moment it is just a mostly 'cosmetic' difference in winrepo-ng that reports 11 pkgs instead of just 2 of them ..?
@twangboy and @damon-atkins is that 'ok' to ask @prensing to proceed with adding the python ver. 3.x newer installer ones? Or do you think this should wait?

[damon-atkins]

No problem with PY3 as long as the option force the install into the correct location of the registry and it installs for everyone. I would start with the latest version of PY3 and only support the older ones if someone requests it.

e.g.
python-3.7.0.exe /quiet InstallAllUsers=1 CompileAll=1

The 11 new entries sort of suggest they have fix where the registry entries have been place when installing for all users. I would have expect the same number for the remove however. Try /uninstall /log c:\temp\remove.txt would not have thought your would need to add InstallAllUsers=1 for removal but maybe you do.

You can also try
c:\salt\bin\python.exe c:\salt\bin\lib\site-packages\salt\utils\pkg\win.py detail system+user
before and after see what happens
e.g.
"<username>@<doman_realm>;microsoft corporation\\\\microsoft onedrive\\\\soft":

"<username>@<doman_realm>;python software foundation\\\\python 3.5.1 (32-bit)\\\\soft":