ch6s5_WorkingWithOSandFilesystem

Python provides rich tools to interact with the **operating system**, **filesystem**, and **environment variables**.

Chapter 6: Python Standard Library

Sub-Chapter: Working with OS and Filesystem — Interacting with the System Environment

Python provides rich tools to interact with the operating system, filesystem, and environment variables.
Modules like os, pathlib, shutil, tempfile, and sys give you low- and high-level access to directories, files, permissions, and paths — all in a cross-platform way.

🧱 1. The os Module — Core System Interaction

The os module lets you perform tasks such as navigating directories, creating and removing files, and managing environment variables.

Listing Files and Directories

import os

directory = "/path/to/your/directory"
for item in os.listdir(directory):
    print(item)

Or recursively walk through all subdirectories:

for root, dirs, files in os.walk(directory):
    print("Directory:", root)
    for name in files:
        print("  File:", name)

📁 2. Creating and Removing Directories

import os

# Create a single directory
os.mkdir("new_folder")

# Create intermediate directories if needed
os.makedirs("nested/folder/structure", exist_ok=True)

# Remove an empty directory
os.rmdir("new_folder")

# Remove directories recursively
import shutil
shutil.rmtree("nested/folder")

⚠️ Always use exist_ok=True when creating directories to avoid exceptions if they already exist.

🧾 3. Working with Files

Creating and Deleting Files

# Create a file
with open("example.txt", "w") as f:
    f.write("Hello World!")

# Delete a file
import os
if os.path.exists("example.txt"):
    os.remove("example.txt")

Moving and Renaming Files

import shutil
shutil.move("file.txt", "archive/file_old.txt")

Copying Files and Directories

import shutil

# Copy a single file
shutil.copy("source.txt", "backup.txt")

# Copy entire directory tree
shutil.copytree("project_folder", "project_backup", dirs_exist_ok=True)

🧭 4. Path Handling — os.path vs. pathlib

Using os.path

import os

path = "/Users/john/docs/file.txt"
print(os.path.basename(path))   # file.txt
print(os.path.dirname(path))    # /Users/john/docs
print(os.path.exists(path))     # True or False
print(os.path.join("folder", "sub", "file.txt"))  # folder/sub/file.txt

Using pathlib (Modern and Recommended)

from pathlib import Path

p = Path("folder") / "subfolder" / "data.txt"
print(p.name)           # data.txt
print(p.parent)         # folder/subfolder
print(p.exists())       # True or False
p.write_text("Sample Data")
print(p.read_text())

✅ pathlib is object-oriented and cross-platform — ideal for modern Python projects.

⚙️ 5. Environment Variables

Accessing Environment Variables

import os

user = os.getenv("USER", "unknown")
path = os.environ.get("PATH")
print(user, path)

Setting Environment Variables

os.environ["MY_APP_MODE"] = "production"

Environment variables are commonly used for configuration, security keys, and deployment settings.

🧩 6. File Metadata and Permissions

import os
import time

info = os.stat("example.txt")
print("Size:", info.st_size, "bytes")
print("Last modified:", time.ctime(info.st_mtime))

# Change permissions
os.chmod("example.txt", 0o644)

🧰 7. Temporary Files and Directories — tempfile

The tempfile module creates temporary files and directories safely.

import tempfile

with tempfile.TemporaryDirectory() as tmpdir:
    print("Temp dir:", tmpdir)

with tempfile.NamedTemporaryFile(delete=False) as tmpfile:
    tmpfile.write(b"Temporary data")
    print("Temp file:", tmpfile.name)

Temporary files are automatically deleted when closed (unless delete=False).

🧮 8. Disk and System Information

Disk Usage

import shutil

usage = shutil.disk_usage("/")
print(f"Total: {usage.total // (2**30)} GB")
print(f"Used: {usage.used // (2**30)} GB")
print(f"Free: {usage.free // (2**30)} GB")

Detecting Platform and Paths

import sys, os

print(sys.platform)  # 'win32', 'darwin', or 'linux'
print(os.name)       # 'nt' or 'posix'
print(os.getcwd())   # Current working directory

🧠 9. Error Handling for File Operations

Always use try/except blocks to prevent crashes from file errors.

import os

try:
    os.remove("nonexistent.txt")
except FileNotFoundError:
    print("File not found.")
except PermissionError:
    print("No permission to delete file.")

🧮 10. Real-World Examples

Example 1 — Directory Cleaner

from pathlib import Path

def clean_temp(folder="temp"):
    p = Path(folder)
    for file in p.glob("*.tmp"):
        print("Removing:", file)
        file.unlink()

clean_temp()

Example 2 — Backup Copier

import shutil
from datetime import datetime

def backup_folder(src, dst_root="backups"):
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    dst = f"{dst_root}/backup_{timestamp}"
    shutil.copytree(src, dst)
    print("Backup created at:", dst)

backup_folder("project_data")

Example 3 — File Summary Reporter

from pathlib import Path

def summarize_folder(path):
    p = Path(path)
    total_size = sum(f.stat().st_size for f in p.rglob("*") if f.is_file())
    print(f"Total size: {total_size / 1024:.2f} KB")
    print(f"Files count: {len(list(p.rglob('*')))}")

summarize_folder(".")

🧭 11. UML-Style Overview of Filesystem Modules

+-----------------------------+
|           os                |
+-----------------------------+
| mkdir()  listdir()  getenv()|
| remove()  walk()   chmod()  |
+-----------------------------+

+-----------------------------+
|         pathlib             |
+-----------------------------+
| Path objects for files/dirs |
| read_text(), write_text()   |
| glob(), exists(), rename()  |
+-----------------------------+

+-----------------------------+
|          shutil             |
+-----------------------------+
| copy(), move(), rmtree()    |
| copytree(), disk_usage()    |
+-----------------------------+

+-----------------------------+
|         tempfile            |
+-----------------------------+
| TemporaryFile()             |
| TemporaryDirectory()        |
+-----------------------------+

+-----------------------------+
|           sys               |
+-----------------------------+
| platform, path, argv, exit()|
+-----------------------------+

🧾 12. Best Practices

✅ Always check file/directory existence (Path.exists()).
✅ Use with open() for safe file handling.
✅ Prefer pathlib over os.path for clarity and portability.
✅ Wrap file deletions or writes in try/except blocks.
✅ Use tempfile for safe, auto-cleaned temporary storage.
✅ Never hardcode paths — use os.path.join() or Path().

🧠 Summary

• The os module handles system-level operations.
• pathlib provides a clean, modern API for file paths.
• shutil extends copying, moving, and disk management.
• tempfile manages temporary data safely.
• sys provides runtime and platform info.

Mastering these tools gives you full control over the file system and operating environment — from simple file automation to robust, cross-platform system utilities.