Issue 31148: Can we get an MSI installer for something past 3.4.4? (original) (raw)
Created on 2017-08-08 16:38 by D Gentry, last changed 2022-04-11 14:58 by admin. This issue is now closed.
Messages (17)
Author: D Gentry (D Gentry)
Date: 2017-08-08 16:38
I've noticed that the only installers that are able to successfully complete on my windows 7 x64 system are the MSI installers.
Unfortunately, python.org hasn't released an MSI version since 3.4.4 it seems.
Is it possible to get an MSI installer for one of the more recent versions?
Author: R. David Murray (r.david.murray) * 
Date: 2017-08-08 17:44
The short answer is no. We no longer use the MSI installer.
Perhaps the windows experts will be interested in exploring why you can't use the current installers, since they work for most people.
Author: Steve Dower (steve.dower) *
Date: 2017-08-08 20:57
Looking at your log, you clearly have some sort of configuration blocking installers or else corruption somewhere on your system. There really isn't enough information available for me to be able to tell.
Perhaps you have a virus scanner running that is blocking the installer?
Author: D Gentry (D Gentry)
Date: 2017-08-09 01:33
If that's the best thing that you can suggest, it's obvious why my request is being redirected.
That is unfortunate that you have decided to opt out of using the MSI installer, as that seems to be the solution as far as I can see.
I've enjoyed working with Python for years and was hoping for a new MSI installer to be released eventually, but hearing that you have decided to discontinue porting for any future release for MSI is definitely not what I was expecting.
I guess I'll wait to see if your Windows guys have anything to add but save it to say that I have already tried disabling any anti virus software, checked my user permission as well as the folder permissions, and attempted installing via administrator as well, all with no difference in the result. Meanwhile, I have been able to uninstall python 2.7 and re-install it without issue (at the time I was investigating a possible path conflict as the issue) and even installed 3.4.4rc1, as that is the most current version available in MSI format.
Your exe files all fail to execute properly, and ends up throwing the same non-descript error in a log that offers very little insight into the root of the problem, yet you tell me the problem is with my system when I have investigated the alternatives and the only issue I have found is you don't have a MSI version offered any more.
Are you expecting users to compile the source code into a MSI installer, if that's what they require?
If that's what it's going to take, I'll do it myself, get what I need done and move on from python after this project.
On Tue, Aug 8, 2017 at 2:57 PM, Steve Dower <report@bugs.python.org> wrote:
Changes by Steve Dower <steve.dower@python.org>:
type: crash -> behavior
Python tracker <report@bugs.python.org> <http://bugs.python.org/issue31148>
Author: Paul Moore (paul.moore) *
Date: 2017-08-09 12:10
To be clear, Steve is our main Windows developer, and specifically the person who developed the Windows installers we now use. They work perfectly for many people, including myself, so there certainly isn't a general issue with them. I myself routinely install Python with the new installers on Windows 7 x64 systems, so it's definitely something specific to your system.
Is your PC running a "corporate" build that might include security restrictions or other permission limitations that might be affecting the behaviour of the installer?
As a workaround, Python is installable using nuget, from https://www.nuget.org/packages/python. I don't know much myself about nuget, but basically "nuget install python" will create a local copy of Python in the current directory that you can use without needing to be installed.
Author: D Gentry (D Gentry)
Date: 2017-08-10 07:54
Ok, it sounds like as good a solution as I can expect.
No, I don't have a corporate build but I think my registry developed issues some time in the past and windows locked it down to prevent exe from installing.
It's been a while but that is the only thing I can come up with that would be why it won't work.
Your exe isn't the only one that gives me issues but its usually a false positive error with other installs and everything works as expected.
I have tried every suggestion I have found online and nothing seems to improve the issue.
If the problem is the registry issue that I suspect, then nothing short of rebuilding the entire system from initial install would solve the problem.
I have communicated with Microsoft, who's answer was to upgrade to Windows 10.
I have two systems with python installed, both with windows 7 x64, the older system won't install without an MSI where as the new one would.
The closest thing I can figure is there is some permission handling that is done differently via MSI vs executable installs and this is causing the issue, as all the documentation online seems to agree that it is a permissions related error.
I can't fix my system if the problem that I suspect is the culprit so my only option is to build the MSI of your installer.
It definitely seems like that will be more successful than continuing to be directed and redirected to perform the same tasks that have already been completed to no avail.
From what I have been reading online, its simple enough with Visual Studio as long as you have access to the source code.
Thanks for enlightening me to the most of your abilities.
It seems pretty clear that I on my own on this one.
On Wed, Aug 9, 2017 at 6:10 AM, Paul Moore <report@bugs.python.org> wrote:
Paul Moore added the comment:
To be clear, Steve is our main Windows developer, and specifically the person who developed the Windows installers we now use. They work perfectly for many people, including myself, so there certainly isn't a general issue with them. I myself routinely install Python with the new installers on Windows 7 x64 systems, so it's definitely something specific to your system.
Is your PC running a "corporate" build that might include security restrictions or other permission limitations that might be affecting the behaviour of the installer?
As a workaround, Python is installable using nuget, from https://www.nuget.org/packages/python. I don't know much myself about nuget, but basically "nuget install python" will create a local copy of Python in the current directory that you can use without needing to be installed.
Python tracker <report@bugs.python.org> <http://bugs.python.org/issue31148>
Author: Steve Dower (steve.dower) *
Date: 2017-08-10 13:24
Okay, if that's the way you want to go. Be aware that the old MSI code is gone though, so rebuilding it will not be trivial. Though if you do, there are a few people who would find it more convenient, so you may be able to enlist help maintaining it (we didn't get any offers of help, just requests for free work, so you may not get it either...)
If you're having trouble with a range of installers, there's a process you can run to reset your security policy, which is what I'd suggest doing. I'd have to find the details for you, but it basically resets permissions on everything. Doesn't delete any files, but may help with this problem. Let me know if you'd like me to find the details (requires Windows Pro IIRC, at least pre-Win 10).
Author: R. David Murray (r.david.murray) *
Date: 2017-08-10 18:00
Steve, when we changed installers was that when we also fixed the security/permissions problems with the install dir? If permissions are the issue the OP's problem may have nothing to do with it not being msi.
Author: Steve Dower (steve.dower) *
Date: 2017-08-10 20:40
when we changed installers was that when we also fixed the security/permissions problems with the install dir
Yes, but the permissions issue here isn't the install directory - it is probably the TEMP directory or some other system restriction. It's basically impossible to tell from the logs, and I don't even know where to start asking for configuration settings to see what may be wrong.
Author: D Gentry (D Gentry)
Date: 2017-08-12 03:00
If you would like to keep track of others who would find it useful, I'd be happy to contribute the msi to the community once its completed.
Also, any information or good references that you could recommend that might aid in producing the installer would be appreciated.
Steve mentioned the temp directory as possibly being part of the problem and that would make sense as its only when installer make use of this that the issues seem to occur so I will definitely be looking at that as a possible root cause for the failure.
Steve also mentioned that it's not as easy to produce an installer as it may seem.I was wondering if you could elaborate some on what you mean.
On a scale of one to ten, with ten being the most difficult, how difficult of a task would you say it is to create this as it seems you have some experience doing this in the past and this will be my first attempt, so I would like to know what I'm getting into before I start.
From the Microsoft article on producing MSI installers, https://msdn.microsoft.com/en-us/library/aa266427(v=vs.60).aspx#vehowbuildinginstallerpackagefileanchor1, it seems fairly simple as long as you have Visual Studio.
Is there something I'm not seeing or understanding in the process?
Again, if you know of any references that might help educate me or direct me through the process, these would be helpful.
I already have a couple strategies I'm thinking of to do this but I would like to see what someone with some experience has to say regarding the issue.
-David Gentry
On Thu, Aug 10, 2017 at 2:40 PM, Steve Dower <report@bugs.python.org> wrote:
Steve Dower added the comment:
when we changed installers was that when we also fixed the security/permissions problems with the install dir
Yes, but the permissions issue here isn't the install directory - it is probably the TEMP directory or some other system restriction. It's basically impossible to tell from the logs, and I don't even know where to start asking for configuration settings to see what may be wrong.
Python tracker <report@bugs.python.org> <http://bugs.python.org/issue31148>
Author: Steve Dower (steve.dower) *
Date: 2017-08-14 18:55
The general requirements of an installer are:
- runs without any preinstalled prerequisites
- does not install anything permanent until the user says so (Visual Studio breaks this rule :( )
- installs all required prerequisites
- install on all supported operating systems
- allow users to [de]select optional components
- correctly uninstall previous compatible versions
- correctly do not uninstall previous identical versions
- update user environment settings (registry, environment, etc.)
- create Start Menu shortcuts
- create file associations
- create context menus
- install files into customizable location with correct security settings
- correctly roll back when installation fails, including restoring previously uninstalled components and attempting to execute installed tools that may not be installed correctly
- attempt to execute just-installed components that may not be installed correctly (e.g. ensurepip, compileall, trigger environment variable reload)
- detect and abort when installation is going to fail, and explain to the user how to solve the problem
- provide live progress feedback and information about what operations are occurring
- provide detailed log files to help offer user support post-installation, and help users find those logs
- provide registration information so the operating system can help users modify or remove the installation
- allow users to modify, repair and upgrade the installation without corrupting it (or any others) or forcing them to remove/reinstall
- allow users to choose whether to install just for themselves or for the entire machine, and request correct permissions based on this choice
- allow administrators to script an entire installation and have it run silently and/or with progress (and cancellation) only
- allow users to minimize download size by deselecting optional components
- allow users to obtain all optional components to allow full installation without public internet access
- install without executing arbitrary third-party code that may have been maliciously placed in a user's download directory
- automatically include new files added by developers who have not explicitly modified the installer
These are all the features of our current installer, and nearly all of them work for basically every user. Many of these are literally not achievable with just an MSI, or require significantly more complicated commands to be run by the end user, rather than providing options in a user interface.
If you're still interested in giving it a go, visit http://wixtoolset.org/ and have a look at their tools. These are the gold-standard right now for building MSIs (and Burn bundles, which is what we currently have).
Author: D Gentry (D Gentry)
Date: 2017-08-16 02:16
Thank you for the information. That will definitely be helpful.
Also, thank you for being so detailed with your explanation.
Even though it will be difficult and may take more time than I was initially intending, I believe its within the realm of possibility.
Therefore, I will be trying my hand at crafting the MSI, though like you said earlier, it's not nearly as easy as the msdn article made it appear.
The toolset you offered should help out a lot.
Thank you for all of your expert advice.
I have one last question before I begin on my journey.
You mentioned "burn bundles" at the end of the message. Are these something I would have to build myself or is there a repository where I can download the files?
On Mon, Aug 14, 2017 at 12:55 PM, Steve Dower <report@bugs.python.org> wrote:
Steve Dower added the comment:
The general requirements of an installer are:
- runs without any preinstalled prerequisites
- does not install anything permanent until the user says so (Visual Studio breaks this rule :( )
- installs all required prerequisites
- install on all supported operating systems
- allow users to [de]select optional components
- correctly uninstall previous compatible versions
- correctly do not uninstall previous identical versions
- update user environment settings (registry, environment, etc.)
- create Start Menu shortcuts
- create file associations
- create context menus
- install files into customizable location with correct security settings
- correctly roll back when installation fails, including restoring previously uninstalled components and attempting to execute installed tools that may not be installed correctly
- attempt to execute just-installed components that may not be installed correctly (e.g. ensurepip, compileall, trigger environment variable reload)
- detect and abort when installation is going to fail, and explain to the user how to solve the problem
- provide live progress feedback and information about what operations are occurring
- provide detailed log files to help offer user support post-installation, and help users find those logs
- provide registration information so the operating system can help users modify or remove the installation
- allow users to modify, repair and upgrade the installation without corrupting it (or any others) or forcing them to remove/reinstall
- allow users to choose whether to install just for themselves or for the entire machine, and request correct permissions based on this choice
- allow administrators to script an entire installation and have it run silently and/or with progress (and cancellation) only
- allow users to minimize download size by deselecting optional components
- allow users to obtain all optional components to allow full installation without public internet access
- install without executing arbitrary third-party code that may have been maliciously placed in a user's download directory
- automatically include new files added by developers who have not explicitly modified the installer
These are all the features of our current installer, and nearly all of them work for basically every user. Many of these are literally not achievable with just an MSI, or require significantly more complicated commands to be run by the end user, rather than providing options in a user interface.
If you're still interested in giving it a go, visit http://wixtoolset.org/ and have a look at their tools. These are the gold-standard right now for building MSIs (and Burn bundles, which is what we currently have).
Python tracker <report@bugs.python.org> <http://bugs.python.org/issue31148>
Author: Steve Dower (steve.dower) *
Date: 2017-08-16 04:12
Burn bundles are part of Wix, but it's the part you don't like :)
If you just want a single MSI, you won't want a bundle. But you will have to leave out nearly half of the features I listed because they simply are not possible without it.
Author: D Gentry (D Gentry)
Date: 2017-08-17 03:33
So basically what your saying is that the MSI that I compile will only work for my specific situation as I'm not building it with a bundle?
Hmm, that kind of sucks.
Disregarding the difficulty of manually converting from source to MSI, if I was to choose to produce a version that included all the features you listed so it was able to be considered a full MSI release so others could benefit as well, would it be possible to get a bundle, even if was only for some of the older releases?
I have a strategy I am considering employing to solve the problem where the more data I can get, the better it will all come together.
Basically, I have a 5x raspberry pi 3 neural network unit that I am planning on getting here within the next couple of weeks to experiment with machine learning. Considering the situation, I believe this may be a good task to apply a convolution neural net to solve.
The problem is essentially converting the raw source code into the corresponding MSI, via Wix. If I use the old release's source and MSI repositories as models, I believe it should be possible to train the system to do the conversion for future versions as well. The more data I have to feed the system, such as the bundles you mentioned, would definitely help increase the accuracy of the algorithm more quickly, but I'm not sure if this would be something Python.org would be willing to share. Even without the bundles, I think it will still be achievable. It just may take the system longer to attain sufficient accuracy as it would have to extrapolate more from the raw data.
Is this an idea that intrigues you even slightly? Even if its a long shot, I think the benefits would be worth the attempt and the vast majority of the investment of time and resources would be my responsibilty. After the fact, should the algorithm succeed, I would be happy to share all my data with python.org regarding the experiment so you could begin providing MSI releases again with minimal effort on your part.
Should you feel like there is something that would inherently and irrefutably inhibit the process, making this a completely hopeless endeavor, I will resign myself to producing the one-time use, stripped down version you were describing previously.
I just hate investing a bunch of time into a project and the code I create is basically useless after I use it once, but I do respect your opinion therefore wanted to submit the proposal for your consideration before diving down the rabbit hole.
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
On Tue, Aug 15, 2017 at 10:12 PM, Steve Dower <report@bugs.python.org> wrote:
Steve Dower added the comment:
Burn bundles are part of Wix, but it's the part you don't like :)
If you just want a single MSI, you won't want a bundle. But you will have to leave out nearly half of the features I listed because they simply are not possible without it.
Python tracker <report@bugs.python.org> <http://bugs.python.org/issue31148>
Author: Steve Dower (steve.dower) *
Date: 2017-08-17 15:01
The idea interests me from a theoretical point of view, but then I did my PhD in that field. I don't believe it's a practical solution, if only because you would need to manually generate the result in order to validate before releasing, and that's the step you're going to avoid.
I don't think there's anything left to pursue here. Good luck with your efforts.
Author: Gregory Szorc (indygreg) *
Date: 2018-01-06 20:01
I was going to update some CI that tests all supported versions of Python and pins the Python 3 versions. That CI was previously using the MSI installers for Python 3.4. To my surprise, the MSIs stopped being generated after the 3.4.4 release! https://www.python.org/ftp/python/3.4.4/ has MSIs (and MacOS .pkg files). But all subsequent releases (e.g. https://www.python.org/ftp/python/3.4.7/) do not.
I'm OK with the decision to abandon MSIs and move to the .exe installer for newer Python releases (like 3.5 and 3.6). However, ceasing to produce the MSIs (and MacOS .pkg files for that matter) midway through 3.4's support cycle feels wrong to me. 3.4.4 was the last release with binary distributions for Windows and MacOS. 3.4.5 and newer releases have fixes to security issues. That means people using the binary distributions of 3.4 on these platforms are stuck on 3.4.4 and are thus vulnerable to known security issues. That's... not great.
I think the MSI (and .pkg) distributions should be restored for 3.4. Furthermore, I would encourage Python to adopt the practice that distribution mechanisms aren't modified during a Python version's support cycle. Someone will inevitably rely on any distribution format that is published. And taking it away in a support cycle will orphan those users on an old, buggy release.
Author: Steve Dower (steve.dower) *
Date: 2018-01-07 00:20
It is standard procedure for us to stop producing binary releases when a version switches to security-only mode. The burden on our volunteers would become overwhelming otherwise.
If you'd like to propose a change to this policy, start by posting on python-list, and then if there is general support someone there will bring the conversation to the development team. Resurrecting old bugs is not an efficient way to request this kind of change.
History
Date
User
Action
Args
2022-04-11 14:58:49
admin
set
github: 75331
2021-12-04 14:57:11
zach.ware
set
messages: -
2021-12-04 14:56:56
zach.ware
set
messages: -
2021-12-04 10:57:33
macoptimizer
set
messages: +
2021-12-04 05:00:02
macoptimizer
set
nosy: + macoptimizer
messages: +
2018-01-07 00:20:36
steve.dower
set
messages: +
2018-01-06 20:01:32
indygreg
set
nosy: + indygreg
messages: +
2017-08-17 15:01:48
steve.dower
set
messages: +
2017-08-17 03:33:22
D Gentry
set
messages: +
2017-08-16 04:12:12
steve.dower
set
messages: +
2017-08-16 02:16:46
D Gentry
set
messages: +
2017-08-14 18:55:32
steve.dower
set
messages: +
2017-08-12 03:00:10
D Gentry
set
messages: +
2017-08-10 20:40:05
steve.dower
set
messages: +
2017-08-10 18:00:08
r.david.murray
set
messages: +
2017-08-10 13:24:11
steve.dower
set
status: open -> closed
resolution: works for me
messages: +
stage: resolved
2017-08-10 07:54:27
D Gentry
set
messages: +
2017-08-09 12:10:31
paul.moore
set
messages: +
2017-08-09 01:33:27
D Gentry
set
messages: +
2017-08-08 20:57:56
steve.dower
set
type: crash -> behavior
2017-08-08 20:57:48
steve.dower
set
messages: +
2017-08-08 17:44:19
r.david.murray
set
nosy: + paul.moore, tim.golden, r.david.murray, zach.ware, steve.dower
messages: +
components: + Windows
2017-08-08 16:38:31
D Gentry
create