wrappers

Reusables – Part 2: Wrappers

Spice up your code with wrappers! In Python, a wrapper, also known as a decorator, is simply encapsulating a function within other functions.

@wrapper
def my_func(a, b=2):
    print(b)

@meta_decorator(foo="bar")
def my_other_func(**kwargs):
    print(kwargs)

In Reusables, all the wrappers take arguments, aka meta decorators, so you will at minimum have to end them with parens ()s. Let’s take a look at the one I use the most first.

time_it

import reusables

@reusables.time_it()
def test_func():
    import time, random
    time.sleep(random.randint(2, 5))

test_func()
# Function 'test_func' took a total of 5.000145769345911 seconds

time_it documentation
The message by default is printed. However it can also be sent to a log with a customized message. If log=True it will log to the Reusables logger, however you can also specify either a logger by name (string) or pass in a logger object directly.

reusables.add_stream_handler('reusables')

@reusables.time_it(log=True, message="{seconds:.2f} seconds")
def test_time(length):
    time.sleep(length)
    return "slept {0}".format(length)

result = test_time(5)
# 2016-11-09 16:59:39,935 - reusables.wrappers  INFO      5.01 seconds

print(result)
# slept 5

It’s also possible to capture time taken in a list.

@reusables.time_it(log='no_log', append=my_times)
def test_func():
    import time, random
    length = random.random()
    time.sleep(length)
    
for _ in range(10):
    test_func()
    
my_times
# [0.4791555858872698, 0.8963652232890809, 0.05607090172793505, 
# 0.03099917658380491,0.6567622821214627, 0.4333975642063024, 
# 0.21456404424395714, 0.5723555061358638, 0.0734819056269771, 
# 0.13208268856499217]

unique

The oldest wrapper in the Reusables library forces the output of a function to be unique, or else it will raise an exception or if specified, return an alternative output.

import reusables
import random


@reusables.unique(max_retries=100)
def poor_uuid():
    return random.randint(0, 10)


print([poor_uuid() for _ in range(10)])
# [8, 9, 6, 3, 0, 7, 2, 5, 4, 10]

print([poor_uuid() for _ in range(1000)])
# Exception: No result was unique

unique documentation

lock_it

A very simple wrapper to use while threading. Makes sure a function is only being run once at a time.

import reusables
import time


def func_one(_):
    time.sleep(5)


@reusables.lock_it()
def func_two(_):
    time.sleep(5)


@reusables.time_it(message="test_1 took {0:.2f} seconds")
def test_1():
    reusables.run_in_pool(func_one, (1, 2, 3), threaded=True)


@reusables.time_it(message="test_2 took {0:.2f} seconds")
def test_2():
    reusables.run_in_pool(func_two, (1, 2, 3), threaded=True)


test_1()
test_2()

# test_1 took 5.04 seconds
# test_2 took 15.07 seconds

log_exception

It’s good practice to catch and explicitly log exceptions, but sometimes it’s just easier to let it fail naturally at any point and log it for later refinement or debugging.

@reusables.log_exception()
def test():
    raise Exception("Bad")

# 2016-12-26 12:38:01,381 - reusables   ERROR  Exception in test - Bad
# Traceback (most recent call last):
#     File "<input>", line 1, in <module>
#     File "reusables\wrappers.py", line 200, in wrapper
#     raise err
# Exception: Bad

queue_it

Add the result of the function to the specified queue instead of returning it normally.

import reusables
import queue

my_queue = queue.Queue()


@reusables.queue_it(my_queue)
def func(a):
    return a


func(10)

print(my_queue.get())
# 10

New in 0.9

catch_it

Catch specified exceptions and return an alternative output or send it to an exception handler.

def handle_error(exception, func, *args, **kwargs):
    print(f"{func.__name__} raised {exception} when called with {args}")

@reusables.catch_it(handler=handle_error)
def will_raise(message="Hello"):
    raise Exception(message)


will_raise("Oh no!")
# will_raise raised Oh no! when called with ('Oh no!',)

retry_it

Once is never enough, keep trying until it works!

@reusables.retry_it(exceptions=(Exception, ), tries=10, wait=1)
def may_fail(dont_do=[]):
    dont_do.append("x")
    if len(dont_do) < 6:
        raise OSError("So silly")
    print("Much success!")

may_fail()
# Much success!

 

This post is a follow-up of Reusables – Part 1: Overview and File Management.

Python Decorators

Often times in python there comes a need to have multiple functions act in a similar manner. It could be anything from making sure that similar functions output the right type of result, they all log when they are called or all report exceptions in the same way.

decorators An easy and repeatable way to accomplish this is . They look like:

@decorator
def my_function(print_string="Hello World"):
    print(print_string)

my_function()
# Hello World

Decorators are simply wrapping a function with another function. In other words, it is exactly the same as doing:

def my_function(print_string="Hello World"):
    print(print_string)

decorator(my_function)("My message")
# My message

So what does a decorator look like?

Decorator Template

from functools import wraps

def decorator(func):
    """This is an example decorators"""
    @wraps(func)
    def container(*args, **kwargs):
        # perform actions before running contained function
        output = func(*args, **kwargs)
        # actions to run after contained function
        return output
    return container

Running the code above does nothing extra currently. It is showing how a decorator runs another function within itself.

@decorator
def my_function():
    print "Hello World"

my_function()
# "Hello World"

Line by line breakdown

def decorator(func):

The name of the decorator function, which takes the wrapped function as its single argument.

@wraps(func)

Wraps is a built-in method that replaces the decorators name and docstring with that of the wrapped function, the section after the line by line breakdown explains why this is necessary.

def container(*args, **kwargs):

This inner function collects the parameters and keyword parameters that are going to be passed to the original function. This allows the decorator access to incoming arguments to verify or modify before the function is ever run.

output = func(*args, **kwargs)

This runs the function with the original arguments and captures the output. It is also possible to return a completely different result. It is more common to check or modify the output, like if you wanted to make sure everything returned was an integer.

return output

Don’t forget to actually return the original function’s output (or a custom one). Otherwise the function will simply return None.

return container

The container function is the actual function being called,  hence why *args, **kwargs are passed to it. It is necessary to return it from the outside decorator so it can be called.

The importance of Wraps

We need to incorporate wraps so that the function name and docstring appear to be from the wrapped function, and not those of the wrapper itself.

@decorator
def my_func():
    """Example function"""
    return "Hello"

@decorator_no_wrap
def second_func():
    """Some awesome docstring"""
    return "World"

help(my_func)
# Help on function my_func:

# my_func()
#    Example function

help(second_func)
# Help on function container:

# container(*args, **kwargs)

It is possible, though more work to accomplish the same thing yourself.

def decorator(func):
    """This is an example decorators"""
    def container(*args, **kwargs):
        return func(*args, **kwargs)
    container.__name__ = func.__name__
    container.__doc__ = func.__doc__
    return container

Useful Example

Now lets turn it into something useful. In this example we will make sure that the function returns the expected type of result. Otherwise it will raise an exception for us so there are not hidden complications down the road.

from functools import wraps

def isint(func):
    """ 
    This decorator will make sure the resulting value 
    is a integer or else it will raise an exception.
    """
    @wraps(func)
    def container(*args, **kwargs):
        output = func(*args, **kwargs)
        if not isinstance(output, int):
            raise TypeError("function did not return integer")
        return output
    return container

@isint
def add(num1, num2):
    """Add two numbers together and return the results"""
    return num1 + num2

print(add(1, 2))
# 3

print(add("this", "that"))
# Type Error: function did not return integer

Regular decorators are already called on execution, so you do not need to add ()s after their name, such as @isint. However, if the decorator accepts arguments, aka a meta-decorator, it will require () even if nothing additional is passed to it.

Meta-decorators

Passing arguments to a decorator turns it into a Meta-decorator. To pass these arguments in, it requires either another function wrapped around the decorator or turn the entire thing into a class.

from functools import wraps

def istype(instance_type=int):
    def decorator(func):
        """ 
        This decorator will make sure the resulting value is the 
        type specified or else it will raise an exception. 
        """
        @wraps(func)
        def container(*args, **kwargs):
            output = func(*args, **kwargs)
            if not isinstance(output, instance_type):
                raise TypeError("function did not return proper type")
            return output
        return container
    return decorator

@istype()
def add(num1, num2):
    """Add two numbers together and return the results"""
    return num1 + num2

@istype(str)
def reverse(forward_string):
    """Reverse and return incoming string"""
    return forward_string[::-1]


print(add(1, 2))
# 3

print(reverse("Hello"))
# "olleH"

print(add("this", "that"))
# Type Error: function did not return proper type

Remember running a decorator is equivalent to:

decorator(my_function)("My message")

Running a meta-decorator adds an additional layer.

reversed_string = istype(str)(reverse)("Reverse Me")

Hence why @decorator doesn’t require to be called when put above a function, but @istype() does.

You can also create this meta-decorator as a class instead of another function.

class IsType: # Instead of 'def istype'

    def __init__(self, inc_type=int):
        self.inc_type = inc_type

    def __call__(self, func): # Replaces 'def decorator(func)'
        @wraps(func)
        def container(*args, **kwargs):
            output = func(*args, **kwargs)
            if not isinstance(output, self.inc_type):
                raise TypeError("function did not return proper type")
            return output
        return container

In functionality they are the same, but be aware they are technically different types. This will really only impact code inspectors and  those trying to manually navigate code, so it is not a huge issue, but is something to be aware of.

type(IsType)
<class 'type'>

type(istype)
<class 'function'>

 Things to keep in mind

  1. Order of operation does matter. If you have multiple decorators around a single function keep in mind they are run from top to bottom
  2. Once a function is wrapped, it cannot be run without the wrapper.
  3.  Meta-decorators require the additional parentheses, regular decorators do not.