On 4/21/2012 8:53 PM, Brett Cannon wrote:     imp.cache_from_source() (and thus also       imp.source_from_cache()) has special semantics compared to how       os.path.join() works. For instance, if you look at test_imp you       will notice it tries to use the same path separator as is the       farthest right in the path it is given::       
        
      
      
        
            self.assertEqual(imp.cache_from_source('\\foo\\bar/baz/qux.py', True), '\\foo\\bar\\baz/__pycache__/qux.{}.pyc'.format(self.tag))
        

        
        
But if you do the same basic operation using ntpath, you           will notice it simply doesn't care::
        

        
        
          
  >>>             ntpath.join(ntpath.split('a\\b/c/d.py')[0], '__pycache__',             'd.cpython-32.pyc')
          
  'a\\b/c\\__pycache__\\d.cpython-32.pyc
        
        

        
        
Basically imp.cache_from_source() goes to a bunch of effort           to reuse the farthest right separator when there is an           alternative separator *before* and path splitting is done. But           if you look at ntpath.join(), it doesn't even attempt that           much effort.
          

          
          
Now that we can reuse os.path.join() (directly for             source_from_cache(), indirectly through easy algorithmic             copying in cache_from_source()) do we want to keep the             "special" semantics, or can I change it to match what ntpath             would do when there can be more than one path separator on             an OS (i.e. not do anything special)?
          
        
      
         
    Is there an issue here with importing from zip files, which use /     separator, versus importing from the file system, which on Windows     can use either / or \ ?  I don't know if imp.cache_from_source cares     or is aware, but it is the only thing I can think of that might have     an impact on such semantics.  (Well, the other is command line     usage, but I don't think you are dealing with command lines at that     point.)

Right now zipimport doesn't even support __pycache__ (I think). Besides, zipimport already does a string substitution of os.altsep with os.sep (see Modules/zipimport.c:90 amongst other places) so it also doesn't care in the end.
">

(original) (raw)



On Sun, Apr 22, 2012 at 01:44, Glenn Linderman <v+python@g.nevcal.com> wrote:

On 4/21/2012 8:53 PM, Brett Cannon wrote:
imp.cache\_from\_source() (and thus also imp.source\_from\_cache()) has special semantics compared to how os.path.join() works. For instance, if you look at test\_imp you will notice it tries to use the same path separator as is the farthest right in the path it is given::

self.assertEqual(imp.cache\_from\_source('\\\\foo\\\\bar/baz/qux.py', True), '\\\\foo\\\\bar\\\\baz/\_\_pycache\_\_/qux.{}.pyc'.format(self.tag))

But if you do the same basic operation using ntpath, you will notice it simply doesn't care::

>>> ntpath.join(ntpath.split('a\\\\b/c/d.py')\[0\], '\_\_pycache\_\_', 'd.cpython-32.pyc')
'a\\\\b/c\\\\\_\_pycache\_\_\\\\d.cpython-32.pyc

Basically imp.cache\_from\_source() goes to a bunch of effort to reuse the farthest right separator when there is an alternative separator \*before\* and path splitting is done. But if you look at ntpath.join(), it doesn't even attempt that much effort.

Now that we can reuse os.path.join() (directly for source\_from\_cache(), indirectly through easy algorithmic copying in cache\_from\_source()) do we want to keep the "special" semantics, or can I change it to match what ntpath would do when there can be more than one path separator on an OS (i.e. not do anything special)?

Is there an issue here with importing from zip files, which use / separator, versus importing from the file system, which on Windows can use either / or \\ ? I don't know if imp.cache\_from\_source cares or is aware, but it is the only thing I can think of that might have an impact on such semantics. (Well, the other is command line usage, but I don't think you are dealing with command lines at that point.)

Right now zipimport doesn't even support \_\_pycache\_\_ (I think). Besides, zipimport already does a string substitution of os.altsep with os.sep (see Modules/zipimport.c:90 amongst other places) so it also doesn't care in the end.