Forward Headers not working in Spring Boot + Kubernetes

Mazen Ammar :

Background

I recently migrated my Spring app to Spring Boot (w/ embedded Tomcat), and I am currently moving it into Kubernetes. As a part of the move into Kubernetes, I am separating out the Apache config into its own service and deployment in Kubernetes to act as a proxy for my Spring Boot app.

My current setup is Apache with a LoadBalancer service that accepts requests from the world. This takes those requests and forwards them to my Spring Boot app, which has a ClusterIP service.

Also important to note: my Apache redirects all http to https.

Issue

Whenever my Spring Boot app returns a redirect to the client, the location header in the response is http instead of https (only requests made over https will get through the Apache proxy to my app).

Example:

Non-logged in user goes to:

https://example.com/admin

If not authenticated, the admin page will redirect the user to the login page. This SHOULD be one redirect to:

https://example.com/login

However, as is, my app redirects the user to:

http://example.com/login

And then the user is redirected again by Apache to:

https://example.com/login

What I've Tried

I've checked my logs to make sure that the requests my app receives include the X-Forwarded-Proto: https header, which from what I understand should make the location header in the redirect response https.

As mentioned in several Stack Overflow posts, I tried adding server.use-forward-headers=true to my application.properties file, but this did nothing. I also tried adding server.tomcat.protocol-header=X-Forwarded-Proto with it, but this did nothing as well (and from what I read, is the default anyway).

Other Notes

My cluster's networkCIDR is included in Tomcat's list of internal proxies for RemoteIPValve
X-Forwarded-For doesn't seem to have an impact either, so I think the issue is with all forward headers

Mazen Ammar :

I figured it out. My mistake was assuming that networkCIDR was the IP address returned by request.getRemoteAddr(), when it was actually my k8s internal cluster IP. This makes sense given that the request is coming from Apache, which is also inside the cluster. The internal cluster IP was NOT in the list of internal proxies, and thus RemoteIPValve was not utilizing the forward headers.

By adding the server.tomcat.internal-proxies property to my application.properties with the default values specified in RemoteIpValve AND my internal cluster IP, everything worked as it should.

See https://docs.spring.io/spring-boot/docs/current/reference/html/howto-embedded-servlet-containers.html#howto-customize-tomcat-behind-a-proxy-server