Configuration of mod_auth_gssapi w/Proxy. #302

bviviano · 2024-04-10T17:27:33Z

bviviano
Apr 10, 2024

Hello. I am having an issue configuring mod_auth_gssapi with a Proxy to a remote host and I am hoping there is something simple on my side I am missing. If this is not the best place to ask, then please let me were I should submit my question. Thanks.

I am using Red Hat Enterprise Linux 9 w/ Apache 2.4.57 and mod_auth_gssapi 1.6.3-7. My RHEL9 system is joined to our Active Directory. mod_auth_gssapi works fine for web pages (HTML and PHP) and other resources hosted directly on the Apache Server, but I can not get the configuration correct when Proxying through Apache to a remote system.

I have a web camera on a private network attached to my Apache host. I want to proxy the live stream of the camera feed through my Apache server on the public interface, but limit who can see the live stream based on group membership. A simple proxy itself is working fine using the following:

<Location /camera>
	ProxyPreserveHost On
	ProxyPass http://camera.mgt/video1s1.mjpg
	ProxyPassReverse http://camera.mgt/video1s1.mjpg

        <RequireAny>
		Require all granted
        </RequireAny>
</Location>

I have also set it up using BasicAuth with LDAP Authentication through our Active Directory Server with no problem:

<Location /camera>
	ProxyPreserveHost On
	ProxyPass http://camera.mgt/video1s1.mjpg
	ProxyPassReverse http://camera.mgt/video1s1.mjpg

        AuthType Basic
        AuthBasicProvider ldap
        AuthName "Camera"

        AuthLDAPURL "ldap://ad.example.com?uid?sub?(objectClass=*)" TLS
        AuthLDAPRemoteUserAttribute uid
        AuthLDAPGroupAttribute member memberUid
        AuthLDAPGroupAttributeIsDN on

        <RequireAny>
               Require ldap-group cn=camera,cn=Groups,dc=example,dc=com
        </RequireAny>
</Location>

However, when I try and throw mod_auth_gssapi into the mix in place of LDAP username/password authentication, I get the following error in the web browser:

400 Bad Request
Your client has issued a malformed or illegal request.

My GSSAPI setup in Apache is

<Location /camera>
	ProxyPreserveHost On
	ProxyPass http://camera.mgt/video1s1.mjpg
	ProxyPassReverse http://camera.mgt/video1s1.mjpg

        AuthName "Camera"
        AuthType GSSAPI
        GssapiAllowedMech krb5
        GssapiLocalName On
        GssapiUseSessions On
        GssapiSSLonly On
        GssapiCredStore keytab:/etc/krb5.keytab

        Session On
        SessionCookieName gssapi_session path=/private;httponly;secure;

        AuthLDAPURL "ldap://ad.example.com?uid?sub?(objectClass=*)" TLS
        AuthLDAPRemoteUserAttribute uid
        AuthLDAPGroupAttribute member memberUid
        AuthLDAPGroupAttributeIsDN on

        <RequireAny>
               Require ldap-group cn=camera,cn=Groups,dc=example,dc=com
        </RequireAny>
</Location>

I use the above GSSAPI configuration on the same Apache server for other <Location> and <Directory> directives without any issues and mod_auth_gssapi works as expected there (so it's not an issue with verifying the KRB5 ticket with the domain controller).

There is nothing in the Apache logs that points to what the problem is.

Since a simple proxy works, Proxy w/BasicAuth (using LDAP) works, but Proxy w/mod_auth_gssapi doesn't, it seems like there is something about the GSSAPI setup that is causing the problem, but I've read through all the documentation on this site, searched the web and tried all the combinations of Gssapi* settings I thought might impact my setup, with no luck.

Thanks!

simo5 · 2024-04-10T18:44:26Z

simo5
Apr 10, 2024
Maintainer

Please raise the log level to debug and post what errors you get.

At a glance a few things that can cause issues:

GssapiLocalName On
This requires users to be present in /etc/passwd (or via a nsswitch passwd service like sssd or winbind)
GssapiCredStore keytab:/etc/krb5.keytab
Generally apache does not have access to the system keytab, both the file permission and SELinux policy will block you.
SessionCookieName gssapi_session path=/private;httponly;secure;
session's won't work for /camera if you tell the browser they are valid only for /private
The whole LDAP stuff.. unclear how it would work given GSSAPI will return you a principal name not a uid to search on.

If the server is AD you should probably use something like:
```
AuthLDAPURL "ldap://ad.example.com?cn=users,dc=exmple,dc=com?userPrincipalName?sub?(objectClass=user)"
AuthLDAPGroupAttribute memberOf
AuthLDAPGroupAttributeIsDN on

...Require ldap-group cn=camera,cn=Groups,dc=example,dc=com
```
The question is what mod_authnz_ldap will use as "user" name, I am not sure it reads the REMOTE_USER variable, I have not seen any doc, in fact, that says that you can do authorization w/o also doing authentication via LDAP first.

If you can resolve the above then you may also need to provide a user to auth to LDAP, I do not think AD these days allows anonymous searches, and given authentication is not happening via LDAP you do not have an authenticated connection to query for group info on.

For authorization you may want to use another service than direct LDAP queries anyway.
You may want to look into using SSSD (or Winbind) [in which case GssapiLocalName On will work] and these services can provide the host with group information.

For authorization apache does not provide a good way to directly query the nsswitch service, but a good workaround sometimes is to simply have a cron job that does something like:
getent group camera > /etc/httpd/myauthzfile.txt and then configure mod_authz_groupfile to use that file.
This assumes you get groups from AD via SSSD or Winbind, keeping in mind both tend to disable enumeration for performance reasons these days, so read the docs on how to expose specific groups or enable full enumeration.

Of course you can also avoid SSSD/Winbind entirely and just produce a group file using a cron job and a script that periodically connects to LDAP, runs a search and dumps in a file the list of users in the group you care for (a bit of parsing would be need so using a scripting language for this operation would be needed) and remember that the user will be provide by its principal name in this case so something like [email protected] and that is what apache will query.

0 replies

bviviano · 2024-04-10T19:20:19Z

bviviano
Apr 10, 2024
Author

Thanks for responding. I will turn up debugging on the server and do more testing.

For authorization you may want to use another service than direct LDAP queries anyway.

In case I wasn't clear, I only included the details about LDAP to make it clear that the Proxy setup is working fine with

No Authorization/Authentication
LDAP Authorization/Authentiation

And that the 400 client error message ONLY happens when I enable GSSAPI via

AuthType GSSAPI

This makes me think there is something specific about mod_auth_gssapi that is upsetting the ProxyPass / ReverseProxyPass setup.
If you have any suggestions specific to mod_auth_gssapi and possible Gssapi* setting that could prevent a Proxy from working, that's what I am looking for.

GssapiLocalName On
This requires users to be present in /etc/passwd (or via a nsswitch passwd service like sssd or winbind)

Yes, we use sssd. There is nothing wrong with that setup. This Apache server correctly sets REMOTE_USER based on the Kerberos Ticket, etc when used with AuthType GSSAPI. We use the above setup to protect and control access to 30 or 40 different URL's on this and a few other Apache servers. It all works perfectly, EXCEPT for when I try to use mod_auth_gssapi with ProxyPass and ProxyReversePass directives to a remote server.

Thanks!

0 replies

bviviano · 2024-04-11T14:46:30Z

bviviano
Apr 11, 2024
Author

Ok, I've been working on this for several hours now. I've reviewed the Apache logs with LogLevel Debug with nothing is standing out as to what is going on. I am having the same problem on Linux (RHEL9) and Windows 10/11 clients, tried with Chrome, Edge and Firefox.

I've stripped my GSSAPI configuration down to just the following:

<Location /camera>
        SSLRequireSSL
        SSLOptions +StdEnvVars
        ProxyPreserveHost On
        ProxyPass http://camera.mgt/video1s1.mjpg
        ProxyPassReverse http://camera.mgt/video1s1.mjpg

        AuthType GSSAPI
        AuthName "Camera"
        GssapiAllowedMech krb5
        GssapiLocalName On
        GssapiSSLonly On
        GssapiCredStore keytab:/etc/krb5.keytab

        <RequireAny>
                Require valid-user
        </RequireAny>
</Location>

This is the extent of what's in the Apache error_log when I try and make a connection using mod_auth_gssapi

[Thu Apr 11 10:15:35.279536 2024] [proxy:debug] [pid 3158653:tid 3158694] mod_proxy.c(1520): [client IPADDR:57147] AH01143: Running scheme http handler (attempt 0)
[Thu Apr 11 10:15:35.279543 2024] [proxy_ajp:debug] [pid 3158653:tid 3158694] mod_proxy_ajp.c(795): [client IPADDR:57147] AH00894: declining URL http://camera.mgt/video1s1.mjpg
[Thu Apr 11 10:15:35.279547 2024] [proxy_fcgi:debug] [pid 3158653:tid 3158694] mod_proxy_fcgi.c(1069): [client IPADDR:57147] AH01076: url: http://camera.mgt/video1s1.mjpg proxyname: (null) proxyport: 0
[Thu Apr 11 10:15:35.279552 2024] [proxy_fcgi:debug] [pid 3158653:tid 3158694] mod_proxy_fcgi.c(1074): [client IPADDR:57147] AH01077: declining URL http://camera.mgt/video1s1.mjpg
[Thu Apr 11 10:15:35.279559 2024] [proxy:debug] [pid 3158653:tid 3158694] proxy_util.c(2568): AH00942: http: has acquired connection for (camera.mgt:80)
[Thu Apr 11 10:15:35.279565 2024] [proxy:debug] [pid 3158653:tid 3158694] proxy_util.c(2626): [client IPADDR:57147] AH00944: connecting http://camera.mgt/video1s1.mjpg to camera.mgt:80
[Thu Apr 11 10:15:35.279686 2024] [proxy:debug] [pid 3158653:tid 3158694] proxy_util.c(2852): [client IPADDR:57147] AH00947: connected /video1s1.mjpg to camera.mgt:80
[Thu Apr 11 10:15:35.280125 2024] [proxy:debug] [pid 3158653:tid 3158694] proxy_util.c(3325): AH02824: http: connection established with 172.28.202.16:80 (camera.mgt:80)
[Thu Apr 11 10:15:35.280136 2024] [proxy:debug] [pid 3158653:tid 3158694] proxy_util.c(3514): AH00962: http: connection complete to 172.28.202.16:80 (camera.mgt)
[Thu Apr 11 10:15:45.295838 2024] [proxy:debug] [pid 3158653:tid 3158694] proxy_util.c(2584): AH00943: http: has released connection for (camera.mgt:80)

If I remove the GSSAPI directives for my Apache config so its just

<Location /camera>
        SSLRequireSSL
        SSLOptions +StdEnvVars
        ProxyPreserveHost On
        ProxyPass http://camera.mgt/video1s1.mjpg
        ProxyPassReverse http://camera.mgt/video1s1.mjpg

        <RequireAny>
                Require all granted
        </RequireAny>
</Location>

The Apache error log is identical, except for the last line about http: has released connection for (camera.mgt:80)

[Thu Apr 11 10:21:41.763191 2024] [proxy:debug] [pid 3165996:tid 3166118] mod_proxy.c(1520): [client IPADDR:57214] AH01143: Running scheme http handler (attempt 0)
[Thu Apr 11 10:21:41.763197 2024] [proxy_ajp:debug] [pid 3165996:tid 3166118] mod_proxy_ajp.c(795): [client IPADDR:57214] AH00894: declining URL http://camera.mgt/video1s1.mjpg
[Thu Apr 11 10:21:41.763202 2024] [proxy_fcgi:debug] [pid 3165996:tid 3166118] mod_proxy_fcgi.c(1069): [client IPADDR:57214] AH01076: url: http://camera.mgt/video1s1.mjpg proxyname: (null) proxyport: 0
[Thu Apr 11 10:21:41.763208 2024] [proxy_fcgi:debug] [pid 3165996:tid 3166118] mod_proxy_fcgi.c(1074): [client IPADDR:57214] AH01077: declining URL http://camera.mgt/video1s1.mjpg
[Thu Apr 11 10:21:41.763217 2024] [proxy:debug] [pid 3165996:tid 3166118] proxy_util.c(2568): AH00942: http: has acquired connection for (camera.mgt:80)
[Thu Apr 11 10:21:41.763223 2024] [proxy:debug] [pid 3165996:tid 3166118] proxy_util.c(2626): [client IPADDR:57214] AH00944: connecting http://camera.mgt/video1s1.mjpg to camera.mgt:80
[Thu Apr 11 10:21:41.763353 2024] [proxy:debug] [pid 3165996:tid 3166118] proxy_util.c(2852): [client IPADDR:57214] AH00947: connected /video1s1.mjpg to camera.mgt:80
[Thu Apr 11 10:21:41.763760 2024] [proxy:debug] [pid 3165996:tid 3166118] proxy_util.c(3325): AH02824: http: connection established with 172.28.202.16:80 (camera.mgt:80)
[Thu Apr 11 10:21:41.763775 2024] [proxy:debug] [pid 3165996:tid 3166118] proxy_util.c(3514): AH00962: http: connection complete to 172.28.202.16:80 (camera.mgt)

What's weird is, if I go through a process of destroying and re-creating the ticket on the client, everything starts working with GSSAPI.

For example, on the Windows 10/11 system, if I have a browser (Any of Chrome, Edge or Firefox) reporting the 400 error, if I CTRL-ALT-DEL, lock the screen, then unlock it again, 100% of the time reloading the page with the 400 error makes it start working, IN ALL 3 Browsers, until I reboot the system again. So it's not a browser specific issue. I am wondering if there is something going on between mod_auth_gssapi and the way the ticket is being created on the client system at login vs how it's re-created when I unlock the screen.

I have had the same thing happen with the RHEL9 workstations, but I haven't quite figured out the pattern of kdestory and kinit there to make it work reliability like I can under Windows.

So, any idea's on what I should be looking at or any other debugging suggestions?

Thanks!

0 replies

simo5 · 2024-04-11T16:27:06Z

simo5
Apr 11, 2024
Maintainer

I use ProxyPass and ProxyPassReverse in various places to proxy to a "local" service in order exactly to add GSSAPI auth on top, and everything "seem" to be working fine.

In those logs I see nothing from mod_auth_gssapi, seem like errors are raised from the Proxy itself?

I see you do not use sessions in the latest examples, so that will cause each request to go thorough multiple roundtrips receiving a 401 to start negotiation and then resolving it, but this should enver result in a 400 error.

I wonder if there is an issue setting up HTTP causing a downgrade to older HTTP level that does not allow to maintain a connection open.

Perhaps you should try to capture the traffic between the browser and the server (you can do that with Firefox developer tools for example), and see what differences (besides the SPNEGO negotiation part) you see.

It is also not clear to me what is returning the 400 error, can you see that in the access_log ?

1 reply

simo5 Apr 11, 2024
Maintainer

I do have these additional keywords on the ProxyPass directive: retry=0 timeout=30
Unclear if this may make a difference to your case.

bviviano · 2024-04-11T17:21:06Z

bviviano
Apr 11, 2024
Author

I use ProxyPass and ProxyPassReverse in various places to proxy to a "local" service

We also use Proxy for other GSSAPI services like Grafana for SSO and those are working fine. The only substantive difference between the working services and my WebCam setup is, everything else I have setup and working with mod_proxy and mod_auth_gssapi is running on the same host as the httpd process, just on a different port. So my Grafana is setup with

ProxyPreserveHost on
ProxyPass http://localhost:3000
ProxyPassReverse http://localhost:3000

But otherwise the <Location> directive as far as GSSAPI is identical to what I am trying to use with my Web Camera. The thing is it's just not working with the web cam. I've also tested Proxy and GSSAPI with other external web interfaces (Dell iDRAC, APC PDU Interface, etc) and I get the similar results as with the same as the web cam. It seems if I try and Proxy to something not directly on localhost with a GSSAPI authentication frontend, it fails and I can't figure out why.

I see you do not use sessions in the latest examples

Yes, I was trying to remove as many confounding factors as possible and make the configuration as simple as I can, until I can get something that works. Once I have something that works, I can start reapplying settings and test each step.

It is also not clear to me what is returning the 400 error, can you see that in the access_log

This is what is in the Apache access_log:

IPADDR - - [11/Apr/2024:13:05:02 -0400] "GET /camera/ HTTP/1.1" 401 381 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"
IPADDR - bviviano [11/Apr/2024:13:05:02 -0400] "GET /camera/ HTTP/1.1" 400 151 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36"

Perhaps you should try to capture the traffic between the browser and the server

Yeah, that was going to be my next step, to see if I can see anything about the difference between the two.

2 replies

simo5 Apr 11, 2024
Maintainer

What's not clear is what is returning that 400 error, I would have expected raise the apache log to debug level would have provided that information ...

bviviano Apr 15, 2024
Author

Agreed and since the problem is happening on all web sites I try and use mod_proxy on with mod_auth_gssapi, its not clear which module is to blame or if Apache itself is the problem.

I did do a manual install of the 1.6.5 version of mod_auth_gssapi from the tar.gz release bundle with the same results, so the fact that I was using the Red Hat provided 1.6.3 isn't the problem.

Since I am using RHEL, and we're paying for Enterprise level support, I am going to go ahead and open a case with Red Hat today and throw the whole mess over to them, see if they can reproduce the problem in their lab and maybe one of their developers will find a root cause that can be fixed.

If RH opens a bug associated with this issue, I'll update this discussion with the Bugzilla details, in case others have a similar issue and want to follow along.

In the meantime, if anyone has any suggestions on ways to make this work with a remote proxied site, I am happy to give it a go on my testing infrastructure.

Thanks!

bviviano · 2024-04-29T12:40:54Z

bviviano
Apr 29, 2024
Author

With Red Hat's assistance, I found a resolution to this issue, so I am updating for anyone else who comes across a similar problem. This issue can otherwise be closed.

Using network tracking tools and increased log levels in Apache, Red Hat and I determined the issue was mod_auth_gssapi passing the Authorization header through mod_proxy. The large size of the Authoriztion header generated with mod_auth_gssapi relative to what was passed with BasicAuth to the camera steam was causing the 400 / 401 error messages I was seeing on the client side.

Since there is no reason to pass the Authorization header through mod_proxy in this instance, the resolution was to unset the Authorization header in the Apache <Location> directive. So, the following works correctly in Apache when connecting to the live stream camera with Chrome, Edge and Firefox, tested on multiple operating systems (RHEL, OS X and Windows 11).

<Location /camera>
        SSLRequireSSL
        SSLOptions +StdEnvVars
        ProxyPreserveHost On
        ProxyPass http://camera.mgt/video1s1.mjpg
        ProxyPassReverse http://camera.mgt/video1s1.mjpg

        RequestHeader unset Authorization

        <RequireAny>
                Require all granted
        </RequireAny>
</Location>

Thanks for all those who responded to my request and hopefully the above will save someone else the headache I went through.

1 reply

simo5 Apr 29, 2024
Maintainer

Great analysis, and thanks a lot for reporting back here, this will definitely help other people!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Configuration of mod_auth_gssapi w/Proxy. #302

{{title}}

Replies: 6 comments 4 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Configuration of mod_auth_gssapi w/Proxy. #302

bviviano Apr 10, 2024

Replies: 6 comments · 4 replies

simo5 Apr 10, 2024 Maintainer

bviviano Apr 10, 2024 Author

bviviano Apr 11, 2024 Author

simo5 Apr 11, 2024 Maintainer

simo5 Apr 11, 2024 Maintainer

bviviano Apr 11, 2024 Author

simo5 Apr 11, 2024 Maintainer

bviviano Apr 15, 2024 Author

bviviano Apr 29, 2024 Author

simo5 Apr 29, 2024 Maintainer

bviviano
Apr 10, 2024

Replies: 6 comments 4 replies

simo5
Apr 10, 2024
Maintainer

bviviano
Apr 10, 2024
Author

bviviano
Apr 11, 2024
Author

simo5
Apr 11, 2024
Maintainer

simo5 Apr 11, 2024
Maintainer

bviviano
Apr 11, 2024
Author

simo5 Apr 11, 2024
Maintainer

bviviano Apr 15, 2024
Author

bviviano
Apr 29, 2024
Author

simo5 Apr 29, 2024
Maintainer