Table of Contents

• Vocabulary
• Directory Structure Overview
• Standalone Script: Basics
• Modules in Directories
• Packages
• Controlling Public and Private Exports with __all__
• Leading Underscore Convention in Python
• Running Scripts and Modules via File and Module Syntax
• Releasing Your Directory Structure as a Package
• Summary

Vocabulary

Before diving into the intricacies, let’s clarify some essential Python terms:

• Standalone Script: A single Python file executed directly, often containing a main() function or other logic to be run.
• Module: Any Python file (.py) containing definitions (functions, classes, or variables). Modules can be imported by other Python files.
• Package: A directory containing a special __init__.py file (can be empty) and other modules or sub-packages. Packages allow hierarchical structuring of code.
• Directory: A folder that may or may not qualify as a package depending on the presence of __init__.py.
• Relative Import: Importing modules relative to the current module’s location using . (current directory), .. (parent directory), and so on.

Directory Structure Overview

Here is a sample directory structure to demonstrate the concepts:

project_root/
    standalone.py
    module1.py
    importer1.py
    dir_as_package/
        __init__.py
        submodule1.py
        submodule2.py
    dir_as_directory/
        submodule3.py
        submodule4.py
    package_with_subpackage/
        __init__.py
        module_in_package.py
        subpackage/
            __init__.py
            module_in_subpackage.py

1. Standalone Script: Basics

File: standalone.py

# standalone.py
print("This is a standalone script.")

This script can be executed directly via:

python standalone.py

It does not import or interact with other files, making it self-contained. However, it can be enhanced by including a __main__ block for better control when executed as a script or imported as a module.

File: standalone.py with __main__

# standalone.py
def main():
    print("This is a standalone script.")

if __name__ == "__main__":
    main()

You can execute this script the same way:

python standalone.py

Here, the main() function will only run when the file is executed directly, and not if it is imported elsewhere.

Key Point

Standalone scripts are useful for simple tasks or as entry points for larger applications.

---

2. Modules in Directories

File: module1.py

# module1.py
def greet():
    return "Hello from module1!"

You can import and use this module from another file:

File: importer1.py

# importer1.py
import module1

print(module1.greet())

Run it via:

python importer1.py

Directory Without __init__.py

Files in dir_as_directory are not considered a package. Directly importing a submodule using the standard Python file hierarchy (e.g., dir_as_directory.submodule3) will fail unless the directory is explicitly added to the PYTHONPATH. However, you can manually import modules by appending the directory to sys.path:

File: importer1.py

# importer1.py
import sys
import os

# Add the directory to sys.path
sys.path.append(os.path.join(os.path.dirname(__file__), 'dir_as_directory'))

# Now, import submodules
import submodule3
import submodule4

print(submodule3.some_function())
print(submodule4.another_function())

File: dir_as_directory/submodule3.py

# submodule3.py
def some_function():
    return "Function from submodule3"

File: dir_as_directory/submodule4.py

# submodule4.py
def another_function():
    return "Function from submodule4"

Key Point

Although directories without __init__.py are not considered packages, you can still import their modules using manual additions to sys.path. This approach is shown here for completeness only and should be avoided in practice, as it can lead to maintenance challenges and fragile code structures.

---

3. Packages

File: dir_as_package/__init__.py

# __init__.py
# This file is initially empty. It signifies that the directory is a package.

File: dir_as_package/submodule1.py

# submodule1.py
def hello():
    return "Hello from submodule1!"

Importing from the Package

You can import directly from the package:

File: importer2.py

# importer2.py
from dir_as_package import submodule1

print(submodule1.hello())

Alternatively, import specific functions or classes:

from dir_as_package.submodule1 import hello

print(hello())

Importing Using Relative Imports Within a Package

Modules within the same package can use relative imports to reference each other. For example:

File: dir_as_package/submodule2.py

# submodule2.py
from .submodule1 import hello

def greet_from_submodule2():
    return f"{hello()} and greetings from submodule2!"

To use this in an importer:

File: importer2.py

# importer2.py
from dir_as_package.submodule2 import greet_from_submodule2

print(greet_from_submodule2())

Run this to see how submodule2 uses the hello function from submodule1 via a relative import.

Note that this is only possible because the dir_as_package is a package and not just a directory.

Key Point

Packages organize modules and allow hierarchical imports, improving code structure and readability.

---

4. Controlling Public and Private Exports with __all__

Python allows developers to control what symbols (functions, classes, variables) are publicly accessible when a module or package is imported using the from <module> import * syntax. This is managed through the __all__ attribute.

What __all__ Does

• Defines Public API: __all__ is a list of strings, each representing the name of a symbol (e.g., function or class) that should be exposed when import * is used.
• Restricts Wildcard Imports: Symbols not included in __all__ will not be accessible when using import *.
• Does Not Affect Explicit Imports: Direct imports like from module import symbol are unaffected by __all__.

Example with __all__

File: dir_as_package/__init__.py

# __init__.py
__all__ = ["submodule1"]

File: importer4.py

# importer4.py
from dir_as_package import *

# Accessible
from dir_as_package.submodule1 import hello
print(hello())  # Works

# Not Accessible
try:
    from dir_as_package.submodule2 import greet_from_submodule2
except ImportError:
    print("submodule2 is not accessible via wildcard import.")

Limitations of __all__

• Explicit Imports Bypass __all__: Symbols can still be imported explicitly even if they are not included in __all__.
• No Enforcement for Internal Use: __all__ only affects import * and does not prevent internal or direct access to excluded symbols.

When to Use __all__

• Use __all__ in larger packages to define a clear and controlled public API.
• Avoid excessive reliance on import * to minimize confusion.

Key Point

Understanding and adhering to these conventions ensures better readability and avoids unintended conflicts in Python projects.

---

__all__ is a useful mechanism for managing namespaces in Python, but it is not a strict enforcement tool. Explicit imports should always be preferred for clarity and maintainability.

---

Leading Underscore Convention in Python

The use of leading underscores in Python has specific implications for visibility and behavior:

• Single Leading Underscore _name: This is a convention to indicate that a variable or function is intended for internal use only. It does not enforce restrictions but signals to developers that it is not part of the public API.
• **Double Leading Underscore ** __name: This triggers name mangling in classes, where the variable name is internally rewritten to avoid accidental name clashes in subclasses.
• **Support in **import *_: Objects with a leading underscore are excluded from wildcard imports (e.g., from module import _), unless explicitly listed in the **all** attribute.

Limitations

• No True Privacy: Both single and double underscores are conventions or mechanisms for avoiding conflicts, not true access restrictions.
• Explicit Imports: Leading underscores do not prevent explicit access using direct imports or attribute references (e.g., from module import _name).

Key Point

The leading underscore convention provides a lightweight way to distinguish between public and internal elements in a module, promoting better code organization and readability while still allowing flexibility when needed.

---

5. Running Scripts and Modules via File and Module Syntax

Python scripts and modules can be executed in two primary ways: directly as a file or using the module syntax. Understanding these differences is critical for leveraging Python effectively.

Executing as a File

This approach involves running the script directly using the python command followed by the file name.

Example:

File: script.py

# script.py
def say_hello():
    print("Hello from script.py")

if __name__ == "__main__":
    say_hello()

Run the script:

python script.py

Output:

Hello from script.py

Executing as a Module

Modules can be executed using the -m flag, which treats the specified module as an entry point.

Example:

python -m dir_as_package.submodule1

This requires the module to be importable (i.e., its parent directory must be in the PYTHONPATH).

Example:

File: dir_as_package/submodule1.py

# submodule1.py
def hello():
    print("Hello from submodule1!")

if __name__ == "__main__":
    hello()

Run:

python -m dir_as_package.submodule1

Output:

Hello from submodule1!

Key Differences

1. File Syntax python script.py:

   • The script is executed as a standalone file.
   • The current working directory is used for imports.

2. Module Syntax python -m module_name:

   • The module is executed as part of a package or project.
   • Ensures proper package-reve imports are used.

---

6. Releasing Your Directory Structure as a Package

Once you have a well-structured project directory, you can make it reusable by others in three main scenarios: for local projects, for use within your organization, and for public distribution.

6.1 Local Project Reuse

If you want to reuse your package in other local projects on the same computer:

1. Ensure your directory structure includes an __init__.py file in the relevant folders to make them packages.
2. Use the absolute path to include your package in the PYTHONPATH environment variable, or install it as a local package.

Example:

Run the following command in your terminal from the project_root:

pip install .

This installs your package locally and makes it available for import in other projects.

6.2 Sharing Within Your Organization

To share the package within your company:

1. Create a Python Wheel or Source Distribution:

   python setup.py sdist bdist_wheel
   

2. Host the package on a private package index, such as Artifactory or a private PyPI server.
3. Colleagues can install the package using:

   pip install --index-url <private_index_url> your-package-name
   

Requirements:

• Ensure your setup.py file includes metadata like name, version, packages, and dependencies.
• Use a requirements.txt or pyproject.toml to manage dependencies.

6.3 Publishing Publicly

To make the package publicly available:

1. Register for an account on PyPI.
2. Ensure your setup.py file is complete, including a proper description, license, and author information.
3. Upload the package using twine:

   twine upload dist/*
   

4. Once uploaded, anyone can install your package using:

   pip install your-package-name
   

Best Practices:

• Include clear documentation and examples.
• Use semantic versioning for releases.
• Add unit tests and continuous integration to ensure quality.

Key Point

Releasing a Python package involves organizing your code, creating a setup.py file, and optionally hosting it privately or publicly. Following these steps ensures your package is easily reusable and maintainable.

---

Summary

Understanding Python’s modular structure and imports helps you:

1. Organize code effectively using modules and packages.
2. Avoid common pitfalls with absolute and relative imports.
3. Leverage Python’s flexibility to scale projects efficiently.

Use these concepts to build well-structured, maintainable Python applications.