Working with JSON Files in Python

Table of Contents

    JavaScript Object Notation (JSON) has become a ubiquitous data format used for configuration files, API communication, and more. Its simple structure, light weight, and built-in support in JavaScript make it a popular choice for data interchange.

    In Python, the json module provides excellent support for both encoding and decoding JSON. It is part of the standard library, making it readily available for any Python application. The json module handles translating between JSON documents and Python datatypes like dictionaries, lists, strings, numbers, booleans, and None. This makes it easy to load a JSON file from disk and work with the data in native Python structures.

    Some key benefits of working with JSON files in Python include:

    • Simple and reliable data interchange with other systems
    • Configuration files that are easy to read, edit, and parse
    • Built-in serialization of Python objects to and from JSON
    • Support for nested objects and complex data
    • Lightweight portability of data across systems
    • JSON validity checking to avoid parsing failures

    This guide will provide examples and best practices for loading, parsing, manipulating, and writing JSON files in Python. Whether you need to configure an application, communicate with an API, or store data for later use, these techniques will enable you to effectively work with JSON documents. The json module coupled with Python's file handling capabilities make JSON an accessible, versatile format for everyday data tasks.

    Loading JSON from File

    To load JSON data from a file in Python, use the json.load() method. This deserializes the JSON into a Python object.

    import json

    with open('data.json') as f: data = json.load(f)
    • json.load() takes a file object and returns the parsed JSON as a Python object
    • Use open() to open the JSON file and pass it to json.load()
    • Make sure to close the file after loading with with open() as

    Parsing JSON in Python

    The json module handles parsing JSON into Python objects. Common mappings:

    • JSON object -> Python dict
    • JSON array -> Python list
    • JSON strings -> Python string
    • JSON numbers -> Python int/float

    No need to manually parse the JSON - json.load() handles conversion.

    Writing JSON to a File

    To write a Python object as JSON to a file:

    import json

    data = { 'name': 'John', 'age': 30 }

    with open('data.json', 'w') as f: json.dump(data, f)
    • json.dump() serializes Python objects to JSON
    • Use open() in write mode to open target file
    • Pass Python object and target file object to json.dump()

    Reading a JSON File

    To simply read a JSON file's content without parsing:

    with open('data.json') as f:
    	data = f.read()
    • Use open() to open file and file.read() to read contents
    • This reads the raw JSON as a string, no parsing
    • For structured data, use json.load() instead

    Handling Large JSON Files

    For large JSON files, avoid loading the entire file into memory. Use a loop:

    import json

    with open('large.json') as f: for line in f: data = json.loads(line) # process data
    • Read and parse line-by-line instead of all at once
    • Use json.loads() to parse each line separately

    Checking JSON Validity

    To check if a JSON file contains valid JSON syntax before trying to parse it:

    import json

    try: json.loads(data) print('Valid JSON') except ValueError: print('Invalid JSON')
    • Pass JSON string to json.loads() and catch ValueError on failure
    • Good for validating JSON before loading into Python objects

    Customizing JSON Encoding

    To customize how Python objects are encoded to JSON:

    class Person:
    	def __init__(self, name, age):
    		self.name = name
    		self.age = age

    person = Person('John', 30)

    print(json.dumps(person, default=lambda o: o.__dict__))
    • Passed a default function to json.dumps()
    • Converts Person to dict before encoding
    • Useful for encoding custom classes

    Reading JSON from a Request

    To parse JSON returned from a web API request:

    import requests

    resp = requests.get('https://api.example.com/data') data = resp.json()
    • Call response.json() on requests Response to parse JSON
    • Same as loading JSON file, but from a web request

    Hope these additional pointers help you work effectively with JSON in Python! Let me know if you need any clarification or have additional questions.

    Andrew ThompsonAndrew Thompson
    I have over 10 years of experience programming in Python. I am skilled in web development with Django and Flask, data analysis with Pandas and NumPy, and scientific computing with SciPy. I am also proficient at Python automation and scripting.