On 01/15/2014 08:35 PM, Ryan Smith-Roberts wrote:
    
                               On Wed, Jan 15, 2014 at 7:57 PM, Ryan             Smith-Roberts <rmsr@lab.net>             wrote:
                                            
socket.getservbyname(servicename[, protocolname])
                
->
                
socket.getservbyname(servicename,                   protocolname=None)
                                        

            
            
Here is a more complicated example, since the above               does technically have an alternative fix:
            
              
            
            
sockobj.sendmsg(buffers[, ancdata[, flags[, address]]])
            
->
            
sockobj.sendmsg(buffers, ancdata=None, flags=0,               address=None)
            
                                    
    I feel like Ryan didn't sufficiently explain the problem.� I'll     elaborate.
    
    
    For functions implemented in Python, it's always true that:
    
          
  • a parameter that is optional always has a default value, and
    •       
  • this default value is visible to Python and is a Python value.
    •     
    The default value of every parameter is part of the function's     signature--you can see them with inspect.signature() or     inspect.getfullargspec().
    
    A corollary of this: for every function implemented in Python, you     can construct a call to it where you fill in every optional value     with its published default value, and this is exactly equivalent to     calling it without specifying those arguments:
    
def foo(a=any_python_value): ...
      
      sig = inspect.signature(foo)
      defaults = [p.default for p in sig.parameters.values()]
      foo(*defaults) == foo()
    
    Assuming neither foo nor "any_python_value" have side effects, those     two calls to foo() will be exactly the same in every way.
    
    
    But!� Builtin functions implemented in C commonly have optional     parameters whose default value is not only opaque to Python, it's     not renderable as a Python value.� That default value is NULL.�     Consider the first two paragraphs of SHA1_new() in     Modules/sha1module.c:
    
static PyObject *
      SHA1_new(PyObject *self, PyObject *args, PyObject *kwdict)
      {
      ��� static char *kwlist[] = {"string", NULL};
      ��� SHA1object *new;
      ��� PyObject *data_obj = NULL;
      ��� Py_buffer buf;
      
      ��� if (!PyArg_ParseTupleAndKeywords(args, kwdict, "|O:new",       kwlist,
      ������������������������������������ &data_obj)) {
      ������� return NULL;
      ��� }
      ...
    
    The "string" parameter is optional.� Its value, if specified, is     written to "data_obj".� "data_obj" has a default value of NULL.�     There is no Python value you could pass in to this function that     would result in "data_obj" being (redundantly) set to NULL.� In     Python SHA1_new is known as "_sha1.sha1".� And:
    
sig = inspect.signature(_sha1.sha1)
      defaults = [p.default for p in sig.parameters.values()]
      _sha1.sha1(*defaults) == _sha1.sha1()
    
    There's no value we could put in the signature for _sha1.sha1 that     would behave exactly the same as that NULL.
    
    The ultimate goal of Argument Clinic is to provide introspection     information for builtins.� But we can't provide a default value to     Python for the "string" parameter to _sha1.sha1() without changing     the semantics of the function.� We're stuck.
    
    Ryan's question, then, is "can we change a function like this so it     accepts None?"� My initial reaction is "no".� That might be okay for     _sha1.sha1(), but it'd be best to avoid.
    
    In the specific case of SHA1_new's "string" parameter, we could lie     and claim that the default value is b''.� Internally we could still     use NULL as a default and get away with it.� But this is only a     happy coincidence.� Many (most?) functions like this won't have a     clever     Python value we can trick you with.
    
    What else could we do?� We could add a special value, let's call it     sys.NULL, whose specific semantics are "turns into NULL when passed     into builtins".� This would solve the problem but it's really,     really awful.
    
    The only other option I can see: don't convert SHA1_new() to use     Argument Clinic, and don't provide introspection information for it     either.
    
    Can you, gentle reader, suggest a better option?
    
    
    /arry
    
    p.s. Ryan's function signatures above suggest that he's converting     code from using PyArg_ParseTuple into using     PyArg_ParseTupleAndKeywords.� I don't think he's *actually* doing     that, and if I saw that in patches submitted to me I would ask that     it be fixed.
  ">

(original) (raw)

Well, I think these are�mostly artifacts from old times, and usually passing None *should* be the same as omitting the argument. But check each case!

On Wednesday, January 15, 2014, Larry Hastings <larry@hastings.org> wrote:


On 01/15/2014 08:35 PM, Ryan Smith-Roberts wrote:
On Wed, Jan 15, 2014 at 7:57 PM, Ryan Smith-Roberts <rmsr@lab.net> wrote:
socket.getservbyname(servicename\[, protocolname\])
->
socket.getservbyname(servicename, protocolname=None)

Here is a more complicated example, since the above does technically have an alternative fix:

sockobj.sendmsg(buffers\[, ancdata\[, flags\[, address\]\]\])
->
sockobj.sendmsg(buffers, ancdata=None, flags=0, address=None)

I feel like Ryan didn't sufficiently explain the problem.� I'll elaborate.


For functions implemented in Python, it's always true that:
  • a parameter that is optional always has a default value, and
  • this default value is visible to Python and is a Python value.
The default value of every parameter is part of the function's signature--you can see them with inspect.signature() or inspect.getfullargspec().

A corollary of this: for every function implemented in Python, you can construct a call to it where you fill in every optional value with its published default value, and this is exactly equivalent to calling it without specifying those arguments:
def foo(a=any\_python\_value): ...

sig = inspect.signature(foo)
defaults = \[p.default for p in sig.parameters.values()\]
foo(\*defaults) == foo()
Assuming neither foo nor "any\_python\_value" have side effects, those two calls to foo() will be exactly the same in every way.


But!� Builtin functions implemented in C commonly have optional parameters whose default value is not only opaque to Python, it's not renderable as a Python value.� That default value is NULL.� Consider the first two paragraphs of SHA1\_new() in Modules/sha1module.c:
static PyObject \*
SHA1\_new(PyObject \*self, PyObject \*args, PyObject \*kwdict)
{
��� static char \*kwlist\[\] = {"string", NULL};
��� SHA1object \*new;
��� PyObject \*data\_obj = NULL;
��� Py\_buffer buf;

��� if (!PyArg\_ParseTupleAndKeywords(args, kwdict, "|O:new", kwlist,
������������������������������������ &data\_obj)) {
������� return NULL;
��� }
...
The "string" parameter is optional.� Its value, if specified, is written to "data\_obj".� "data\_obj" has a default value of NULL.� There is no Python value you could pass in to this function that would result in "data\_obj" being (redundantly) set to NULL.� In Python SHA1\_new is known as "\_sha1.sha1".� And:
sig = inspect.signature(\_sha1.sha1)
defaults = \[p.default for p in sig.parameters.values()\]
\_sha1.sha1(\*defaults) == \_sha1.sha1()
There's no value we could put in the signature for \_sha1.sha1 that would behave exactly the same as that NULL.

The ultimate goal of Argument Clinic is to provide introspection information for builtins.� But we can't provide a default value to Python for the "string" parameter to \_sha1.sha1() without changing the semantics of the function.� We're stuck.

Ryan's question, then, is "can we change a function like this so it accepts None?"� My initial reaction is "no".� That might be okay for \_sha1.sha1(), but it'd be best to avoid.

In the specific case of SHA1\_new's "string" parameter, we could lie and claim that the default value is b''.� Internally we could still use NULL as a default and get away with it.� But this is only a happy coincidence.� Many (most?) functions like this won't have a clever Python value we can trick you with.

What else could we do?� We could add a special value, let's call it sys.NULL, whose specific semantics are "turns into NULL when passed into builtins".� This would solve the problem but it's really, really awful.

The only other option I can see: don't convert SHA1\_new() to use Argument Clinic, and don't provide introspection information for it either.

Can you, gentle reader, suggest a better option?


/arry

p.s. Ryan's function signatures above suggest that he's converting code from using PyArg\_ParseTuple into using PyArg\_ParseTupleAndKeywords.� I don't think he's \*actually\* doing that, and if I saw that in patches submitted to me I would ask that it be fixed.


--
--Guido van Rossum (on iPad)