In the latest Python Weekly issue 607, there was a well written article by James Turk called You Don’t Need
__all__ (waybackmachine backup link) which seemed to miss an important point I wanted to discuss. With no way to comment on his blog, I decided might as well turn it into a learning experience here!
James makes a lot of good points throughout his article and explains how imports work well, so please do give it a quick read through! We are step in step with the mindset that you probably shouldn’t be using
import * anyways. It’s messy and unclear.
However everyone writing Python code isn’t in a perfect dev environment. A lot are people just trying to run stuff quickly in notebooks to produce a report, new to Python, or frankly just like the convenience of using
Something not covered in the article is what
__all__ is really handy for. It’s too limit the imports from a module. Including other imports.
Let’s say you have this
from json import loads import shutil import pathlib def my_reverse_function(my_var: str) -> str: return my_var[::-1]
If you use
from messy import * you aren’t only going to get
my_reverse_function you will also get all the other imports from that file, including
>>> from messy import * >>> print(globals().keys()) dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'loads', 'shutil', 'pathlib', 'my_reverse_function'])
This is instantly solved by adding
from json import loads import shutil import pathlib __all__ = ["my_reverse_function"] def my_reverse_function(my_var: str) -> str: return my_var[::-1]
loads and the other imports are no longer imported.
>>> from messy import * >>> print(globals().keys()) dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'my_reverse_function'])
Say you have a simple file with a custom
loads function, I’ll call it
import os def loads(item, value): """Load item into environment variables""" os.environ[item] = value
So you import it and run it and all is good.
>>> from my_stuff import * >>> loads("MY_VAR", "WORLD")
Then later in your notebook you want that handy reverser function, so you import everything from
messy.py. Then you try running
loads again and suddenly it broke!
>>> from messy import * >>> loads("MY_VAR", "WORLD") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: loads() takes 1 positional argument but 2 were given
Suddenly it’s using the
json imported from
messy.py because there was no
So if you are writing a library that other people are going to use, say like python-box that other people use and some will surely misuse it in some way, and don’t want them to hurt themselves or mess up their globals, it’s good to add the
Look at all these imports needed in that library.
import copy import re import warnings from keyword import iskeyword from os import PathLike from typing import Any, Dict, Generator, List, Optional, Tuple, Type, Union from inspect import signature try: from typing import Callable, Iterable, Mapping except ImportError: from collections.abc import Callable, Iterable, Mapping try: from IPython import get_ipython except ImportError: ipython = False else: ipython = True if get_ipython() else False import box from box.converters import ( BOX_PARAMETERS, _from_json, _from_msgpack, _from_toml, _from_yaml, _to_json, _to_msgpack, _to_toml, _to_yaml, msgpack_available, toml_read_library, toml_write_library, yaml_available, ) from box.exceptions import BoxError, BoxKeyError, BoxTypeError, BoxValueError, BoxWarning __all__ = ["Box"]
But because it uses the
__all__ it give the user a very neat import if they call this directly.
>>> from box.box import * >>> print(globals().keys()) dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'Box'])
If It didn’t have that, it would look like:
>>> from box.box import * >>> print(globals().keys()) dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'copy', 're', 'warnings', 'iskeyword', 'PathLike', 'Any', 'Dict', 'Generator', 'List', 'Optional', 'Tuple', 'Type', 'Union', 'signature', 'Callable', 'Iterable', 'Mapping', 'box', 'BOX_PARAMETERS', 'msgpack_available', 'toml_read_library', 'toml_write_library', 'yaml_available', 'BoxError', 'BoxKeyError', 'BoxTypeError', 'BoxValueError', 'BoxWarning', 'NO_DEFAULT', 'NO_NAMESPACE', 'Box'])
Again, James does also touch on a great design principal that you should just declare the exports explicitly in your
__init__.py file. Notice I am doing
from box.box import * ? You shouldn’t be doing that anyways, rather
from box import *. Because
python-box is already using the technique he described, by just defining the imports they want exposed in the root
from box.box import Box from box.box_list import BoxList from box.config_box import ConfigBox from box.exceptions import BoxError, BoxKeyError from box.from_file import box_from_file, box_from_string from box.shorthand_box import SBox, DDBox
So if you did
from box import * you would only get those specific items, even if they didn’t have the
__all__ in the
>>> from box import * >>> print(globals().keys()) dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__annotations__', '__builtins__', 'Box', 'BoxList', 'ConfigBox', 'BoxError', 'BoxKeyError', 'box_from_file', 'SBox', 'DDBox'])
Explicit is better than implicit.PEP 20
Because if you have code used by other people, this could save them someday without them ever realizing it. Avoiding a hidden name collision, or prevent using wrong function that’s named the same elsewhere. Just think about it, I would bet you’ve seen a lot of functions just named
run or similar.
Yes, it is better to avoid
import *, and yes it’s better to have a well structured project, but you can’t stop other’s from using Python as they see fit. And
import * is fast and valid. You shouldn’t ignore convention because it’s not convenient to your way of coding if your library is used by others.