Reading and Writing Files — Managing External Data

File handling is one of the most essential aspects of programming.

Chapter 4: File Handling

Sub-chapter: Reading and Writing Files — Managing External Data

File handling is one of the most essential aspects of programming.
It allows Python programs to read, write, and manage data stored on disk — such as text, logs, or structured data (CSV, JSON, etc.).

Python’s built-in file handling functions and context managers make it safe and efficient to work with files, ensuring automatic cleanup and error management.

📘 Opening and Closing Files

The open() function is used to open a file. It returns a file object that provides methods and attributes for reading, writing, and manipulating files.

Basic Syntax:

file_object = open("filename", mode)

📂 Reading Files

Read Entire File

with open("example.txt", "r") as file:
    content = file.read()
    print(content)

Read Line by Line

with open("example.txt", "r") as file:
    for line in file:
        print(line.strip())

Read All Lines into a List

with open("example.txt", "r") as file:
    lines = file.readlines()
print(lines)

✍️ Writing to Files

Write Mode ('w')

Overwrites the file or creates a new one if it doesn’t exist.

with open("output.txt", "w") as file:
    file.write("First line\n")
    file.write("Second line\n")

Append Mode ('a')

Adds content to the end of the file.

with open("output.txt", "a") as file:
    file.write("Appended line\n")

🔁 Reading and Writing Together

You can open files in combined mode using r+, w+, or a+.

with open("data.txt", "w+") as file:
    file.write("Python File Handling\n")
    file.seek(0)  # Move cursor to start
    print(file.read())

🧠 Using with Statement (Context Manager)

Using with ensures the file is automatically closed after use, even if an exception occurs.

with open("example.txt", "r") as file:
    data = file.read()
# File automatically closed here

⚙️ Working with Different File Formats

1. Reading and Writing CSV Files

import csv

# Writing to CSV
with open("data.csv", "w", newline="") as file:
    writer = csv.writer(file)
    writer.writerow(["Name", "Age", "City"])
    writer.writerow(["Alice", 30, "New York"])

# Reading from CSV
with open("data.csv", "r") as file:
    reader = csv.reader(file)
    for row in reader:
        print(row)

2. Working with JSON Files

import json

# Writing JSON
data = {"name": "Alice", "age": 30, "skills": ["Python", "AI"]}
with open("data.json", "w") as file:
    json.dump(data, file, indent=4)

# Reading JSON
with open("data.json", "r") as file:
    loaded = json.load(file)
print(loaded)

💾 Working with Binary Files

Useful for handling non-text data such as images or executables.

# Reading binary data
with open("image.png", "rb") as file:
    binary_data = file.read()

# Writing binary data
with open("copy.png", "wb") as file:
    file.write(binary_data)

🧱 Pathlib for File Operations (Modern Way)

pathlib offers a clean, object-oriented approach to working with files and directories.

from pathlib import Path

file_path = Path("example.txt")

if file_path.exists():
    print("File exists!")
    print(file_path.read_text())

file_path.write_text("Hello from Pathlib!")

⚠️ Handling File Errors

Always handle exceptions when working with files to avoid crashes.

try:
    with open("nonexistent.txt", "r") as file:
        data = file.read()
except FileNotFoundError:
    print("Error: File not found!")
except IOError:
    print("Error: Input/Output operation failed!")

🧩 Example — Simple Log Analyzer

Let’s create a mini-project that reads a log file, counts error entries, and writes a report.

error_count = 0
with open("system.log", "r") as log_file:
    for line in log_file:
        if "ERROR" in line:
            error_count += 1

with open("report.txt", "w") as report:
    report.write(f"Total ERROR entries: {error_count}\n")

print("Log analysis complete. Report saved to report.txt")

🗂️ Example — Backup Utility

This script copies file contents safely using context managers.

def backup_file(source, destination):
    try:
        with open(source, "rb") as src, open(destination, "wb") as dst:
            dst.write(src.read())
        print(f"Backup completed: {destination}")
    except FileNotFoundError:
        print("Source file not found!")
    except Exception as e:
        print(f"Unexpected error: {e}")

backup_file("data.json", "backup_data.json")

🧾 Best Practices

✅ Always use with open() — ensures automatic closing.
✅ Use absolute paths or pathlib to avoid path confusion.
✅ Handle errors with try/except for robustness.
✅ Use binary mode for non-text files.
✅ Use CSV/JSON modules for structured data.
✅ Avoid reading very large files into memory at once — iterate line-by-line.

🧠 Summary

• File handling allows Python to persist data beyond runtime.
• Modes (r, w, a, etc.) define how files are accessed.
• Context managers (with) prevent resource leaks.
• Use specialized libraries (csv, json, pathlib) for specific formats.
• Real-world applications: log analysis, backups, configuration files, and data pipelines.

By mastering file handling, you gain the ability to build robust applications that interact with the external world efficiently and safely.