Skip to content

Instantly share code, notes, and snippets.

@tomsaleeba
Last active June 16, 2023 11:09
Show Gist options
  • Save tomsaleeba/c463550b43eb9c58d8b415523c49f70b to your computer and use it in GitHub Desktop.
Save tomsaleeba/c463550b43eb9c58d8b415523c49f70b to your computer and use it in GitHub Desktop.
Debugging HTTP traffic with mitmproxy

This will let you see the request and response headers for traffic going through.

We're going to run this as a reverse proxy, rather than a usual proxy, so you don't get completely flooded with traffic.

Start the proxy

  1. create a new VM
  2. expose port 8080 to the public internet
  3. SSH to the VM
  4. make sure you have at least python 3.6
  5. install mitmproxy with pip
    sudo -H pip3 install mitmproxy
  6. create a python script that will be run for each request. Use the script.py file in this gist.
  7. define an environmental variable with the endpoint you wish to proxy
    export THE_ENDPOINT=https://example.com:80 # change me
  8. find out your public network interface address
    ifconfig
  9. create an env var with the public IP:
    export THE_INTERFACE=11.22.33.44 # change me
  10. start mitmproxy
    mitmdump \
      --listen-host $THE_INTERFACE \
      -p 8080 \
      --mode reverse:$THE_ENDPOINT \
      --set block_global=false \
      -s /path/to/script.py

Configure your client

Now you need to configure your client. Remember, this is not a proxy server, it's a reverse proxy so just change your code/config to use the IP and port of the VM instead of your endpoint.

Then you trigger your client and you should see see the traffic in the console on the VM as mitmproxy is running in the foreground.

Example output

==================================================
GET /preheat HTTP/1.1
------------------------- request headers -------------------------
ACCEPT-ENCODING               : identity
X-REQUESTED-WITH              : XMLHttpRequest
ACCEPT                        : application/json
USER-AGENT                    : Appcelerator Titanium/7.2.0 (Nexus 5; Android API Level: 27; en-AU;)
HOST                          : example.com
CONNECTION                    : Keep-Alive
------------------------- response headers -------------------------
CONTENT-TYPE                  : application/json; charset=utf-8
CONTENT-LENGTH                : 27
CONNECTION                    : keep-alive
DATE                          : Wed, 30 Jan 2018 03:05:42 GMT
VARY                          : Accept-Encoding
X-POWERED-BY                  : up
ETAG                          : W/"1b-SWSBjMAAv99BCA8zPdG9jaVeqEs"
------------------------- body (first 100 bytes) -------------------------
some body stuff...
118.210.255.140:47127: GET https://example.com/preheat
                    << 200 OK 27b
def response(flow):
print("")
print("="*50)
print(flow.request.method + " " + flow.request.path + " " + flow.request.http_version)
print("-"*25 + " request headers " + "-"*25)
for k, v in flow.request.headers.items():
print("%-30s: %s" % (k.upper(), v))
print("-"*25 + " response headers " + "-"*25)
for k, v in flow.response.headers.items():
print("%-30s: %s" % (k.upper(), v))
print("-"*25 + " body (first 100 bytes) " + "-"*25)
print(flow.request.content[0:100])
@govindkailas
Copy link

Is there a way to use this to check what the docker API calls happening when we do a docker login and pull ?

@tomsaleeba
Copy link
Author

@govindkailas there are two options I'd try.

The first is using mitmproxy in the normal usage: as a proxy. The instructions I have above are using a reverse-proxy instead. You can look at how to set up the docker CLI to use a proxy (https://docs.docker.com/network/proxy/) and then run mitmproxy on your local machine to intercept the traffic.

Alternatively, if the docker CLI can be configured to point to custom endpoints (container registries), which I'm pretty sure it can, then you can use the reverse proxy idea from this gist as a facade for the container registry. This way is harder, I'd try the regular proxy idea first.

@govindkailas
Copy link

thanks for the details, I tried the first option,

  • added the localhost:8080 as http and https proxy to docker config as explained in the docker documentation.
  • restarted docker service
  • started mitmproxy in 8080 which is the default

I don't see any traffic being intercepted when I do a docker pull. I tried a couple of filters from the docs but that doesn't seem to work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment