Debug HTTP Requests in Python

Table of contents

• 1. Introduction

• 2. How to use httpdbg

  • Makefile:
  • requirements.txt:
  • main.py:
  • console:

• 3. Conclusion

1. Introduction

I came across a new tool on Hacker News, httpdbg. I think it will be mighty useful. It records HTTP requests made from local python app and allows you to view those in the browser at http://localhost:4909

The following is a quick guide on how to use it.

2. How to use httpdbg

Makefile:

# Makefile
install:
	python3 -m venv .venv
	.venv/bin/pip install -r requirements.txt

dev1:
	.venv/bin/python3 main.py

dev2:
	.venv/bin/pyhttpdbg -m main

test:
	.venv/bin/pyhttpdbg -m pytest tests/

requirements.txt:

# requirements.txt
requests==2.32.3
httpdbg==0.22.0
pytest==8.3.3

main.py:

# main.py
import requests
from pprint import pprint


def main():
    url1 = "http://httpbin.org/get"
    resp = requests.get(url1)
    pprint(resp.json())

    url2 = "http://httpbin.org/post"
    data = {"key1": "value1", "key2": "value2"}
    resp = requests.post(url2, data=data)
    pprint(resp.json())


if __name__ == "__main__":
    main()

console:

$ make install
$ make dev1
.venv/bin/python3 main.py
{'args': {},
 'headers': {'Accept': '*/*',
             'Accept-Encoding': 'gzip, deflate',
             'Host': 'httpbin.org',
             'User-Agent': 'python-requests/2.32.3',
             'X-Amzn-Trace-Id': 'Root=1-66f4dcf8-325cc5ec6359eeb1267fbb55'},
 'origin': 'XXX.XXX.XXX.XXX',
 'url': 'http://httpbin.org/get'}
{'args': {},
 'data': '',
 'files': {},
 'form': {'key1': 'value1', 'key2': 'value2'},
 'headers': {'Accept': '*/*',
             'Accept-Encoding': 'gzip, deflate',
             'Content-Length': '23',
             'Content-Type': 'application/x-www-form-urlencoded',
             'Host': 'httpbin.org',
             'User-Agent': 'python-requests/2.32.3',
             'X-Amzn-Trace-Id': 'Root=1-66f4dcf8-21e824d936d2b5897a9a6059'},
 'json': None,
 'origin': 'XXX.XXX.XXX.XXX',
 'url': 'http://httpbin.org/post'}
Press Enter to continue...

$ make dev2
.venv/bin/pyhttpdbg -m main
.... - - .--. -.. -... --. .... - - .--. -.. -... --. .... - - .--. -.. -... --.
  httpdbg - HTTP(S) requests available at http://localhost:4909/
.... - - .--. -.. -... --. .... - - .--. -.. -... --. .... - - .--. -.. -... --.
{'args': {},
 'headers': {'Accept': '*/*',
             'Accept-Encoding': 'gzip, deflate',
             'Host': 'httpbin.org',
             'User-Agent': 'python-requests/2.32.3',
             'X-Amzn-Trace-Id': 'Root=1-66f4dd30-306234f35b946edddddf7b7d'},
 'origin': 'XXX.XXX.XXX.XXX',
 'url': 'http://httpbin.org/get'}
{'args': {},
 'data': '',
 'files': {},
 'form': {'key1': 'value1', 'key2': 'value2'},
 'headers': {'Accept': '*/*',
             'Accept-Encoding': 'gzip, deflate',
             'Content-Length': '23',
             'Content-Type': 'application/x-www-form-urlencoded',
             'Host': 'httpbin.org',
             'User-Agent': 'python-requests/2.32.3',
             'X-Amzn-Trace-Id': 'Root=1-66f4dd31-4dad44c860167980702dddd'},
 'json': None,
 'origin': 'XXX.XXX.XXX.XXX',
 'url': 'http://httpbin.org/post'}
.... - - .--. -.. -... --. .... - - .--. -.. -... --. .... - - .--. -.. -... --.
  httpdbg - HTTP(S) requests available at http://localhost:4909/
.... - - .--. -.. -... --. .... - - .--. -.. -... --. .... - - .--. -.. -... --.
Waiting until all the requests have been loaded in the web interface.
Press Ctrl+C to quit.

3. Conclusion

I think this tool will be very useful for debugging HTTP requests in Python and in general quickly making sense of network endpoints when approaching a new codebase.