You Should Use __all__

In the latest Python Weekly issue 607, there was a well written article by James Turk called You Don’t Need __all__ (waybackmachine backup link) which seemed to miss an important point I wanted to discuss. With no way to comment on his blog, I decided might as well turn it into a learning experience here!

Where We Agree

James makes a lot of good points throughout his article and explains how imports work well, so please do give it a quick read through! We are step in step with the mindset that you probably shouldn’t be using import * anyways. It’s messy and unclear.

However everyone writing Python code isn’t in a perfect dev environment. A lot are people just trying to run stuff quickly in notebooks to produce a report, new to Python, or frankly just like the convenience of using import *.

Where You Need __all__

Something not covered in the article is what __all__ is really handy for. It’s too limit the imports from a module. Including other imports.

Let’s say you have this code.

from json import loads
import shutil
import pathlib

def my_reverse_function(my_var: str) -> str:
    return my_var[::-1]

If you use from messy import * you aren’t only going to get my_reverse_function you will also get all the other imports from that file, including loads, shutil and pathlib.

>>> from messy import *

>>> print(globals().keys()) 
dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'loads', 'shutil', 'pathlib', 'my_reverse_function'])

This is instantly solved by adding __all__

from json import loads
import shutil
import pathlib

__all__ = ["my_reverse_function"]

def my_reverse_function(my_var: str) -> str:
    return my_var[::-1]

Notice that loads and the other imports are no longer imported.

>>> from messy import *         

>>> print(globals().keys()) 
dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'my_reverse_function'])

Name Collisions

Say you have a simple file with a custom loads function, I’ll call it

import os

def loads(item, value):
    """Load item into environment variables"""
    os.environ[item] = value

So you import it and run it and all is good.

>>> from my_stuff import * 
>>> loads("MY_VAR", "WORLD")

Then later in your notebook you want that handy reverser function, so you import everything from Then you try running loads again and suddenly it broke!

>>> from messy import *
>>> loads("MY_VAR", "WORLD")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: loads() takes 1 positional argument but 2 were given

Suddenly it’s using the loads from json imported from because there was no __all__.

Real World Example

So if you are writing a library that other people are going to use, say like python-box that other people use and some will surely misuse it in some way, and don’t want them to hurt themselves or mess up their globals, it’s good to add the __all__ .

Look at all these imports needed in that library.

import copy
import re
import warnings
from keyword import iskeyword
from os import PathLike
from typing import Any, Dict, Generator, List, Optional, Tuple, Type, Union
from inspect import signature

    from typing import Callable, Iterable, Mapping
except ImportError:
    from import Callable, Iterable, Mapping

    from IPython import get_ipython
except ImportError:
    ipython = False
    ipython = True if get_ipython() else False

import box
from box.converters import (
from box.exceptions import BoxError, BoxKeyError, BoxTypeError, BoxValueError, BoxWarning

__all__ = ["Box"]

But because it uses the __all__ it give the user a very neat import if they call this directly.

>>> from import *
>>> print(globals().keys())
dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'Box'])

If It didn’t have that, it would look like:

>>> from import *
>>> print(globals().keys())
dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'copy', 're', 'warnings', 'iskeyword', 'PathLike', 'Any', 'Dict', 'Generator', 'List', 'Optional', 'Tuple', 'Type', 'Union', 'signature', 'Callable', 'Iterable', 'Mapping', 'box', 'BOX_PARAMETERS', 'msgpack_available', 'toml_read_library', 'toml_write_library', 'yaml_available', 'BoxError', 'BoxKeyError', 'BoxTypeError', 'BoxValueError', 'BoxWarning', 'NO_DEFAULT', 'NO_NAMESPACE', 'Box'])

Defining imports in

Again, James does also touch on a great design principal that you should just declare the exports explicitly in your file. Notice I am doing from import * ? You shouldn’t be doing that anyways, rather from box import *. Because python-box is already using the technique he described, by just defining the imports they want exposed in the root

from import Box
from box.box_list import BoxList
from box.config_box import ConfigBox
from box.exceptions import BoxError, BoxKeyError
from box.from_file import box_from_file, box_from_string
from box.shorthand_box import SBox, DDBox

So if you did from box import * you would only get those specific items, even if they didn’t have the __all__ in the file.

>>> from box import * 
>>> print(globals().keys())
dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'Box', 'BoxList', 'ConfigBox', 'BoxError', 'BoxKeyError', 'box_from_file', 'SBox', 'DDBox'])

Why does it matter?

Explicit is better than implicit.
PEP 20

Because if you have code used by other people, this could save them someday without them ever realizing it. Avoiding a hidden name collision, or prevent using wrong function that’s named the same elsewhere. Just think about it, I would bet you’ve seen a lot of functions just named loads, dumps, run or similar.

Yes, it is better to avoid import *, and yes it’s better to have a well structured project, but you can’t stop other’s from using Python as they see fit. And import * is fast and valid. You shouldn’t ignore convention because it’s not convenient to your way of coding if your library is used by others.