9894 Deliver python3 modules 5571 Provide 64-bit python modules

Review Request #1236 — Created Oct. 14, 2018 and submitted

Information
citrus
illumos-gate
master
5571, 9894, 9904
8115aaa...
Reviewers
general 
9894 Deliver python3 modules 

5571 Provide 64-bit python modules

9904 Split python modules out into separate packages


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#!/bin/sh

py2test()
{
        echo "************* $1"

        echo '              32-bit'
        ISALIST=i386 python2 -c "$2"
        echo '              64-bit'
        ISALIST=amd64 python2 -c "$2"
}

py3test()
{
        echo "************* $1"

        python3 -c "$2"
}

zfsmods="allow dataset groupspace holds table unallow userspace util"

echo "***"
echo "*** python2"
echo "***"
python2 --version

py2test libbe_py 'import libbe_py; print len(libbe_py.beList())'
py2test solaris.misc 'import solaris.misc; print solaris.misc.isglobalzone()'
for z in $zfsmods; do
        py2test zfs.$z "import zfs.$z"
done
ISALIST=i386 pfexec python2 /usr/lib/zfs/pyzfs.py userspace rpool
ISALIST=amd64 pfexec python2 /usr/lib/zfs/pyzfs.py userspace rpool

echo "***"
echo "*** python3"
echo "***"
python3 --version

py3test libbe_py 'import libbe_py; print(len(libbe_py.beList()))'
py3test solaris.misc 'import solaris.misc; print(solaris.misc.isglobalzone())'
for z in $zfsmods; do
        py3test zfs.$z "import zfs.$z"
done
pfexec python3 /usr/lib/zfs/pyzfs.py userspace rpool









bloody% ./pytest
***
*** python2
***
Python 2.7.15
************* libbe_py
              32-bit
2
              64-bit
2
************* solaris.misc
              32-bit
1
              64-bit
1
************* zfs.allow
              32-bit
              64-bit
************* zfs.dataset
              32-bit
              64-bit
************* zfs.groupspace
              32-bit
              64-bit
************* zfs.holds
              32-bit
              64-bit
************* zfs.table
              32-bit
              64-bit
************* zfs.unallow
              32-bit
              64-bit
************* zfs.userspace
              32-bit
              64-bit
************* zfs.util
              32-bit
              64-bit
TYPE        NAME  USED  QUOTA
POSIX User  root   10K   none
TYPE        NAME  USED  QUOTA
POSIX User  root   10K   none
***
*** python3
***
Python 3.5.6
************* libbe_py
2
************* solaris.misc
1
************* zfs.allow
************* zfs.dataset
************* zfs.groupspace
************* zfs.holds
************* zfs.table
************* zfs.unallow
************* zfs.userspace
************* zfs.util
TYPE        NAME  USED  QUOTA
POSIX User  root   10K   none





Also ran four nightly builds to cover the variations of BUILDPY2='#' and BUILDPY3='#' in the environment.


Tested building with both python versions then ONU-ing to a machine without python3. That resulted in this:




bloody% pkg list -a system/library/python/\*
NAME (PUBLISHER)                                  VERSION                    IFO
system/library/python/libbe-27                    0.5.11-151027.0            i--
system/library/python/libbe-35                    0.5.11-151027.0            ---
system/library/python/solaris-27                  0.5.11-151027.0            i--
system/library/python/solaris-35                  0.5.11-151027.0            ---
system/library/python/zfs-27                      0.5.11-151027.0            i--
system/library/python/zfs-35                      0.5.11-151027.0            ---





and installing python3 correctly pulled in the new packages:




bloody% pfexec pkg install -v python-35
           Packages to install:         4

Changed packages:
omnios
  runtime/python-35
    None -> 3.5.6-151027.0
  system/library/python/libbe-35
    None -> 0.5.11-151027.0
  system/library/python/solaris-35
    None -> 0.5.11-151027.0
  system/library/python/zfs-35
    None -> 0.5.11-151027.0





and removal was prevented unless they were removed as a set:




bloody% pfexec pkg uninstall python-35
Creating Plan (Solver setup): /
pkg uninstall: Unable to remove 'runtime/python-35@3.5.6-151027.0' due to the following packages that depend on it:
  system/library/python/libbe-35@0.5.11-151027.0
  system/library/python/solaris-35@0.5.11-151027.0
  system/library/python/zfs-35@0.5.11-151027.0

bloody% pfexec pkg uninstall system/library/python/libbe-35
... lots of output but including
Package runtime/python-35 must be uninstalled before the requested operation can be performed.
  Reject:  pkg://omnios/runtime/python-35@3.5.6-151027.0

bloody% pfexec pkg uninstall python-35 system/library/python/\*-35
            Packages to remove:  4
alp
alp 
  1. Generally I like it, however I'd like to discuss the following. In its current form after integration illumos-gate basic packages will unconditionally require both python2.7 and python3.5. It feels wrong. Can we do something like extracting python-27-specific parts in *-27 packages and python-35-specific parts in *-35 packages and conditionally depend on  these packages, based on which python (2.7 or 3.5) is present?
    1. Actually, I deliberately did not add a dependency on a python-3x package to
      
those system packages. The packages will include the python3 modules but only
      
continue to require python-27. In a couple of years' time we will probably
      
want to restructure this to drop python2 entirely but this felt like the
      
best approach for now.
    2. But dependencies are automatically detected on pkgdepend resolve. Modules are liniked to libpython3.5m.so.1.0 . 
      

      $ ldd proto/root_i386/usr/lib/python3.5/vendor-packages/libbe_py.so  |grep python
      
        libpython3.5m.so.1.0 =>  /usr/lib/64/libpython3.5m.so.1.0
      

      So, we have 
      
depend fmri=pkg:/runtime/python-35@3.5.6-2018.0.0.0 type=require
      
in install/beadm and every updated/new OI/OmniOS CE installation will likely have both pythons installed.
      

      This is why I adviced to move python modules in specific PYVER packages (library/python/libbe-27, library/python/libbe-35 and so on) and depend on them in basic packages (libbe and so on) in a way like 
      

      depend fmri=library/python/libbe-27 predicate=runtime/python-27 type=conditional
      
depend fmri=library/python/libbe-35 predicate=runtime/python-35 type=conditional
    3. Ah, I see. For some reason I thought that automatic dependency resolution was disabled in gate.
      
Yes, your approach seems ideal - I'll incorporate it.
  2. 
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    3. 




  

 
















 



 

  

   




citrus




   

   

  

 



 















 





 

  

   




alp




   

   







  

 



 

  


   

   

    
 
alp


    
   


  

  

  



    




  1. 
 
    
  
    
   
    I've just succesfully built it on OI. Looks fine.
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    2. 















  2. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    3. 




  

 
















 



 

  

   




citrus




   

   

  

 



 















 





 

  

   




alp




   

   







  

 



 

  


   

   

    
 
alp


    
   


  

  

  



    




  1. 
 
    
  
    
   
    Ship It!
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    2. 















  2. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    3. 




  

 
















 





 

  

   




citrus




   

   





  

 



 

  


   

   

    
 
citrus


    
   


  

  

  



    




  1. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    2. 

















  2. 
 
    
  
  
    
  
 
    
  
    
   
    
    
     
      usr/src/pkg/Makefile
      


       (Diff revisions 2 - 3)


      
     
    
    
       
   
    
     
    

    
     
    

       
  
    
 
    

   
    
    
    




    
    Typo - I'll fix it
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    3. 




  3. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    4. 




  

 
















 



 

  

   




citrus




   

   

  

 



 















 





 

  

   




tsoome




   

   







  

 



 

  


   

   

    
 
tsoome


    
   


  

  

  



    




  1. 
 
    
  
    
   
    Ship It!
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    2. 















  2. 
 
    
  
    
   
    
  
    
 
    

 
    
  


    
 
    
 
      

  

 
    


    

 
    

    3. 




  

 
















 



 

  

   




citrus




   

   

  

 



 

  


   

   

    
 Review request changed

    
   


  

  

  






 

  Status:
  

  Closed (submitted)

  
 







 


 










  

 









     

    

   

  

  

   
   Loading...