Server Address with and without wireguard

[rosydawn6]

Hello,

I do some filtering in the server_connect method for which I use the FQDN coming in on the address field: https://docs.mitmproxy.org/stable/api/mitmproxy/connection.html#Server.address - The documentation states: "The server's (host, port) address tuple. The host can either be a domain or a plain IP address." (emphasis mine) I am wondering under what conditions it will be a domain or IP address?

For the following setup:
Mitmproxy: 11.0.0.dev (+3, commit 7aa41a8)
Python: 3.11.2
OpenSSL: OpenSSL 3.1.2 1 Aug 2023
Platform: Linux-6.1.0-10-cloud-arm64-aarch64-with-glibc2.36

When running with defaults: ./mitmdump --mode 'regular@6077' and issuing
curl --proxy http://10.0.0.45:6077 http://example.com

The address host in def server_connect is a fully qualified domain name.

When running the same requests using wireguard:
mitmdump --mode 'wireguard:/home/admin/.mitmproxy/wireguard.conf@51821'

the Server address host is an ip address. I am wondering if this a known condition perhaps inherent in using wireguard?

Is there another way I could obtain the FQDN in the server_connect step (before the connection is made through mitmproxy http layer)? When I review the default logging when running in wireguard mode I see the dns query value. Is there any way to propagate the dns query domain to server_connect? Or is there another day to obtain the FQDN or even the domain in server_connect?

[20:47:01.592][10.0.0.121:59293] client connect
[20:47:01.597][10.0.0.121:26998] client connect
10.0.0.121:26998: DNS QUERY (HTTPS) example.com
<< NOTIMP
10.0.0.121:59293: DNS QUERY (A) example.com
<< 93.184.216.34, 93.184.216.34, 93.184.216.34
[20:47:01.618][10.0.0.121:61140] client connect
10.0.0.121:61140: DNS QUERY (HTTPS) example.com
<< NOTIMP

Thanks for this amazing tool.

[mhils]

I am wondering under what conditions it will be a domain or IP address?

Does #6352 answer that question? Let me know if it can be made more clear.

Is there another way I could obtain the FQDN in the server_connect step (before the connection is made through mitmproxy http layer)?

Try setting https://docs.mitmproxy.org/stable/concepts-options/#connection_strategy to lazy. For TLS connections, this should give you the SNI. For HTTP, you can also return a response in the request hook before anything hits the server.

You may also want to check if the allow_hosts/ignore_hosts options already do what you want to achieve.

When I review the default logging when running in wireguard mode I see the dns query value. Is there any way to propagate the dns query domain to server_connect?

From mitmproxy's point of view those two (DNS and TCP) are independent connections. You could do this manually with an addon that stores IP address -> domain mappings, but if possible I'd go for setting connection_strategy instead.

[rosydawn6]

Thanks @mhils your updated docstring clarifies the scenario.

Good thought to try setting lazy connection_strategy for the wg use case but the server.sni was still None but your docstring clarification pretty much settles that this cannot be done with wg.

The ignore_hosts does not quite fit as I want to obtain metrics on these hosts so not fully ignore them. A separate mapping addon may solve the problem for me.

Thanks for your help

[rosydawn6]

I wanted to follow up after having done some additional testing. My use case was to obtain the hostname and use it in the event lifecycle before Http Events in wireguard mode. In regular mode the hostname is available in Connection Events but in wireguard mode the hostname is an ip address. As mitmproxy can do things like ignore_hosts based on domain name even in wireguard mode I was looking for where it obtains that information at the earlier lifecycle events.

Initially I was looking at the documentation and put the following code in the Connection Events such as:

def server_connected(self, data):
			logger.info(f"server_connected {data.client.sni}")
			logger.info(f"server_connected {data.server.sni}")

But the sni was always None at this stage of the lifecycle.

In next_layer.py the sni is obtained here:
client_hello := self._get_client_hello(context, data_client)
This value can be retrieved in a number of ways but can be done via the lifecycle event tls_clienthello. It is important to note that from here it can be used as needed/propagated but that the sni value is not yet available in the Connection Events.

def tls_clienthello(self, data: mitmproxy.tls.ClientHelloData):
		logger.info(f"clienthello: {data.client_hello.sni}")