Python Tricks : Fun With args and kwargs

Python Tricks: Fun With *args and **kwargs

I once pair-programmed with a smart Pythonista who would exclaim “argh!” and “kwargh!” every time he typed out a function definition with optional or keyword parameters. We got along great otherwise, I guess that’s what programming in academia does to people eventually.

Now, while easily mocked, *args and **kwargs parameters are nevertheless a highly useful feature in Python. And understanding their potency will make you a more effective developer.

So what are *args and **kwargs parameters used for? They allow a function to accept optional arguments, so you can create flexible APIs in your modules and classes:

In [2]: def foo(required, *args, **kwargs):
   ...:     print(required)
   ...:     if args:
   ...:         print(args)
   ...:     if kwargs:
   ...:         print(kwargs)

The above function requires at least one argument called "required, " but it can accept extra positional and keyword arguments as well.

If we call the function with additional arguments, args will collect extra positional arguments as a tuple because the parameter name has a * prefix.

Likewise, kwargs will collect extra keyword arguments as a dictionary because the parameter name has ** prefix. Both args and kwargs can be empty if no extra arguments are passed to the function.

As we call the function with various combinations of arguments, you’ll see how Python collects them inside the args and kwargs parameters according to whether they’re positional or keyword arguments:

In [2]: foo()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-c19b6d9633cf> in <module>
----> 1 foo()

TypeError: foo() missing 1 required positional argument: 'required'

In [3]: foo('hello')
hello

In [4]: foo('hello', 1, 2, 3)
hello
(1, 2, 3)

In [5]: foo('hello', 1, 2, 3, key1='value', key2=999)
hello
(1, 2, 3)
{
    
    'key1': 'value', 'key2': 999}

I want to make it clear that calling the parameters args and kwargs is simply a naming convention. The previous

Example would work just as well if you called them *parms and **argv. The actual syntax is just the asterisk(*) or double asterisk(**), respectively.

However, I recommend that you stick with the accepted naming convention to avoid confusion. (And to get a chance to yell “argh!” and “kwargh!” every once in a while.)

Forwarding Optional or Keyword Arguments

It’s possible to pass optional or keyword paramteters from one function to another. You can do so by using the argument-unpacking operators * and ** when calling the function you want to forward arguments to.

This also gives you an opportunity to modify the arguments before you pass them along. Here’s an example:

In [6]: def foo(x, *args, **kwargs):
   ...:     kwargs['name'] = 'Alice'
   ...:     new_args = args + ('extra', )
   ...:     bar(x, *new_args, **kwargs)

This technique can be useful for subclassing and writing wrapper functions. For example, you can use it to extend the behavior of a parent class without having to replicate the full signature of its constructor in the child class. This can be quite convenient if you’re working with an API that might change outside of your control:

In [7]: class Car:
   ...:     def __init__(self, color, mileage):
   ...:         self.color = color
   ...:         self.mileage = mileage
   ...: 

In [8]: class AlwaysBlueCar(Car):
   ...:     def __init__(self, *args, **kwargs):
   ...:         super().__init__(*args, **kwargs)
   ...:         self.color = 'blue'

In [9]: AlwaysBlueCar('green', 48392).color
Out[9]: 'blue

The AlwaysBlueCar constructor simply passes on all arguments to its parent class and then overrides an internal attribute. This means if the parent class constructor changes, there’s a good chance that AlwaysBlueCar would still function as intended.

The downside here is that the AlwaysBlueCar constructor now has a rather unhelpful signature–we don’t know what arguments it expects without looking up the parent class.

Typically you wouldn’t use this technique with your own class hierarchies. The more likely scenario would be that you’ll want to modify or override behavior in some external class which you don’t control.

But this is always dangerous territory, so best be careful(or you might soon have yet another reason to scream “argh!”).

One more scenario where this technique is potentially helpful is writing wrapper funcitons such as decorators. There you typically also want to accept arbitrary arguments to be passed through to the wrapped function.

And, if we can do it without having to copy and paste the original function’s signature, that might be more maintainable:

import functools
def trace(f):
    ...:     @functools.wraps(f)
    ...:     def decorated_function(*args, **kwargs):
    ...:         print(f, args, kwargs)
    ...:         result = f(*args, **kwargs)
    ...:         print(result)
    ...:     return decorated_function
    
In [11]: @trace
    ...: def greet(greeting, name):
    ...:     return '{}, {}!'.format(greeting, name)

In [14]: greet('Hello', 'Bob')
<function greet at 0x7fefa69db700> ('Hello', 'Bob') {
    
    }
Hello, Bob!

With techniques like this one, it’s sometimes difficult to balance the idea of making your code explicit enough and yet adhere to the Don’t Repeat Yourself(DRY) principle.

This will always be a tough choice to make. If you can get a second opinion from a colleague, I’d encourage you to ask for one.