[http-probe] Add HTTP headers to CONNECT requests

[matt-malarkey]

Issue

When sending HTTPS requests with http-probe through a proxy, using proxy_url, user defined HTTP headers are not added to the CONNECT request.

This appears to be a limitation of https://pkg.go.dev/net/http where HTTP headers are only added to CONNECT requests if you specify them when creating the Transport object.

Fix

In http.go:L457 we could add any specified HTTP headers also to the CONNECT request like so:

for _, header := range p.c.GetHeaders() {
    if header.GetName() == "Host" {
        continue
    }
    t.ProxyConnectHeader = http.Header{header.GetName(): {header.GetValue()}}
}

It seems fairly simple to implement this but I'm not sure if adding user defined headers also to the CONNECT request has any other implications.

[manugarg]

@matt-malarkey Thank you for reporting the issue and researching it.

Off the top of my head, this sounds reasonable to me. I'll think more about it.

[manugarg]

Sorry for dropping the ball on this one, but I was looking at it again (yes, I know after a year :() and realized that connect headers will likely look very different from the general request headers. It will be better to add another field called proxy_connect_header for this.

[manugarg]

#763 adds proxy_connect_headers field.