Scaling of pods is not happening as per the http requests

[Manikiran88888]

Report

Scaling of pods is not happening according to the requirements

Expected Behavior

deployment yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80 
        resources:
            limits:
              memory: "50Mi" 
              cpu: "50m"

http scaled object yaml

kind: HTTPScaledObject
apiVersion: http.keda.sh/v1alpha1
metadata:
    name: nginx-http-scaledobject
spec:
    hosts: myhost.com
    targetPendingRequests: 1
    scaleTargetRef:
        deployment: nginx
        service: nginx
        port: 80
    replicas:
        min: 1
        max: 3

the scaled object is applied on the deployment

Actual Behavior

The pods should scale up as the load on the http requests is generated but the pods are not scaling

Steps to Reproduce the Problem

Logs from KEDA HTTP operator

example

HTTP Add-on Version

0.7.0

Kubernetes Version

1.29

Platform

None

Anything else?

No response

[JorTurFer]

Hello,
Could you share KEDA operator logs, HTTP add-on interceptor logs? What error do you see in the request?

[Manikiran88888]

test.log

These are the logs of KEDA operator please check it

[Manikiran88888]

Hello, Could you share KEDA operator logs, HTTP add-on interceptor logs? What error do you see in the request?

Uploading test.log…

Please find the logs attached, Thanks

[Manikiran88888]

http.log

Find the http-interceptor logs

[Manikiran88888]

I have followed 3 methods for scaling the http pods and the links for them are as follows

https://medium.com/@carlocolumna/how-to-scale-your-pods-based-on-http-traffic-d58221d5e7f1

In the above link I have faced issue while handling the host during the curl command with the host

https://medium.com/cuddle-ai/auto-scaling-microservices-with-kubernetes-event-driven-autoscaler-keda-8db6c301b18

In the above link I have faced the issue after this step

https://knowledgebase.eo-lab.org/en/latest/kubernetes/HTTP-Request-based-Autoscaling-on-K8S-using-Prometheus-and-Keda-on-EO-Lab.html
In the above link I have faced the issue at the Prometheus server address part and the forwarding of the port

[JorTurFer]

The interceptor hasn't received any request. Could you double check the connections? If everything looks well, could you share all the related manifests?

• ingress
• external service
• your workload service service
• HTTPScaledObject

[JorTurFer]

The interceptor hasn't received any request. Could you double check the connections? If everything looks well, could you share all the related manifests?

• ingress
• external service
• your workload service service
• HTTPScaledObject

@Manikiran88888 , could you share the info please?

[Manikiran88888]

Hey,
First of all I am sorry that I closed the issue by mistake and Thank you for re-opening it instead of the new one

I am sharing the zip folder which consists

• my yaml file for the deployment of a service, called myntra.yaml where you have to change the image name because the image which I have used is my docker hub image
• the httpscaledobject yaml file which is deploy-keda-httpscaledobject.yaml.
• I have shared the recent logs of the above mentioned yaml files

sample-http.zip

please check the files and let me know if any thing required

If you are able to scale the pods using the files which I have sent please share me the exact steps so that I will follow the same

[JorTurFer]

I only see 2 request in the interceptor:

{"level":"info","ts":1714551507.0846822,"logger":"LoggingMiddleware","caller":"middleware/logging.go:73","msg":"127.0.0.1:51490 - - [01/May/2024:08:18:27 +0000] \"GET / HTTP/1.1\" 404 9 \"\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36\""}
{"level":"info","ts":1714551507.1441922,"logger":"LoggingMiddleware","caller":"middleware/logging.go:73","msg":"127.0.0.1:51490 - - [01/May/2024:08:18:27 +0000] \"GET /favicon.ico HTTP/1.1\" 404 9 \"http://127.0.0.1/\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36\""}

Both returning 404.

I see that you are calling using 127.0.0.1. Are you using a local cluster or so for testing this? If yes, you have to set the Host header, like a proxy does. The interceptor relies on it to route the request. The host header has to match with any host set in the HTTPScaledObject

[JorTurFer]

Assuming that you HTTPScaledObject sets myhost.com as a host, you can try this:

curl -H `Host: myhost.com` http://127.0.0.1/favicon.ico

and that should trigger the scaling out and give you the favicon

[Manikiran88888]

I have tested it using the locust where I have given the load
and I have given in this format

because my curl -H command is giving me error in the terminal

[JorTurFer]

Powershell doesn't have a real curl command, that's why you see that error.
I'm not a PW expert, but I think that this should be equivalent:

$headers = @{
    "Host" = "myhost.com"
}
$response = Invoke-WebRequest -Uri "http://127.0.0.1/favicon.ico" -Headers $headers

You need to pass the Host header if you call directly using the IP address

[JorTurFer]

The other option can be editing the host file to include a mapping between your host and 127.0.0.1.
Calling directly to the IP address without any host won't work as the host is required

[Manikiran88888]

I want to know one more thing why should I use (http://127.0.0.1/favicon.ico)
I mean why favicon.ico
because I have port forwarded my application to the port 8090 then I have to use http://127.0.0.1:8090 if I am not wrong

I am asking out of curiosity I want to know why favicon.ico came here

Is it okay if I use curl -H Host: myhost.com http://127.0.0.1/8090/

[JorTurFer]

oh, yeah, you should use the port too.
About the path, my example was with favicon because I saw that path in logs, but you can use the path that you want, the important point is that your request do have to contain the Host header as you are calling directly via IP address.

[Manikiran88888]

I have tried the complete setup in the wsl and the curl command worked

But the pods did not increase.

is there any thing that I need to provide to generate the load

[JorTurFer]

Do you want to generate the load from your laptop directly? I suggest using ab within cluster

[JorTurFer]

apiVersion: batch/v1
kind: Job
metadata:
  name: load-generator
  namespace: WHERE_YOU_WANT
spec:
  template:
    spec:
      containers:
      - name: apache-ab
        image: ghcr.io/kedacore/tests-apache-ab
        imagePullPolicy: Always
        args:
          - "-n"
          - "20000" #Total requests
          - "-c"
          - "1"  #Concurrent clients
          - "-H"
          - "Host: YOUR_HOST" 
          - "http://YOUR_INGRESS_SERVICE.YOUR_INGRESS_NAMESPACE:YOUR_INGRESS_PORT/"
      restartPolicy: Never
      terminationGracePeriodSeconds: 5
  activeDeadlineSeconds: 600
  backoffLimit: 5

[JorTurFer]

This will generate the load directly inside the cluster, hitting your ingress and passing over all the components, but you can hit directly the add-on interceptor service if you prefer, as you wish

[Manikiran88888]

How can we hit the add-on interceptor service directly any reference ?

I would like to check it please

[JorTurFer]

About how to configure the scaling and metrics/thresholds, you could be interested on this feature that will be released soon as part of v0.8.0: #961

[JorTurFer]

How can we hit the add-on interceptor service directly any reference ?

apiVersion: batch/v1
kind: Job
metadata:
  name: load-generator
  namespace: WHERE_YOU_WANT
spec:
  template:
    spec:
      containers:
      - name: apache-ab
        image: ghcr.io/kedacore/tests-apache-ab
        imagePullPolicy: Always
        args:
          - "-n"
          - "20000" #Total requests
          - "-c"
          - "1"  #Concurrent clients
          - "-H"
          - "Host: YOUR_HOST" 
          - "http://INTERCEPTOR_SERVICE.KEDA_NAMESPACE:INTERCEPTOR_SERVICE_PORT/"
      restartPolicy: Never
      terminationGracePeriodSeconds: 5
  activeDeadlineSeconds: 600
  backoffLimit: 5

Basically you can use this job pointing to any level on top of interceptor just changing the last line to call to the layer that you want

- "http://INTERCEPTOR_SERVICE.KEDA_NAMESPACE:INTERCEPTOR_SERVICE_PORT/"

[Manikiran88888]

Hey,
I could see that the v0.8.0 is released is there any documentation to use the metrics in the yaml file or is there any example to have a look on it

Also one more thing is that How much ever I do I could not scale up the pods could you please have a look and try to do it on your PC because it is just a sample nginx image which I want to test
and I want a screen show to check how you are scaling up the pods
Test-http.zip

[JorTurFer]

You can see the reference page for the HTTPScaledObject here: https://kedacore.github.io/http-add-on/ref/v0.8.0/http_scaled_object.html

[JorTurFer]

I've just deployed your resources on my own cluster, and deployed this job into the cluster:

apiVersion: batch/v1
kind: Job
metadata:
  name: load-generator
spec:
  template:
    spec:
      containers:
      - name: apache-ab
        image: ghcr.io/kedacore/tests-apache-ab
        imagePullPolicy: Always
        args:
          - "-n"
          - "2000000" #Total requests
          - "-c"
          - "400"  #Concurrent clients
          - "-H"
          - "Host: myhost.com" 
          - "http://keda-add-ons-http-interceptor-proxy.keda:8080/test"
      restartPolicy: Never
      terminationGracePeriodSeconds: 5
  activeDeadlineSeconds: 600
  backoffLimit: 5

It has scaled as expected:

You have to ensure that your requests reach the nginx, as your HTTPScaledObject defines the path /test, the requests have to be to that path, if you call to '/' but your definition is for /test, the interceptor won't route the request neither scale.

Assuming that all the traffic reach the monitored workload, you can try modifying the HTTPScaledObject to use the new aggregated metrics, just adding this section:

spec: 
.....
  requestRate:
    granularity: 1s
    targetValue: 100
    window: 1m

This will be more accurate for the scaling, but this feature has been released today as part of v0.8.0

[Manikiran88888]

Okay
I will also try once again and check for the same

[JorTurFer]

Yeah, that's not correct, you can see the correct spec here: https://github.com/kedacore/http-add-on/blob/main/docs/ref/v0.8.0/http_scaled_object.md
the correct place is (I missed scalingMetric):

spec:
  scalingMetric:
    requestRate:
      granularity: 1s
      targetValue: 100
      window: 1m

[Manikiran88888]

My Httpobjectscode is

kind: HTTPScaledObject
apiVersion: http.keda.sh/v1alpha1
metadata:
    name: test-httpscale
spec:
    hosts:
    - myhost.com
    pathPrefixes:
    - /test
    scaleTargetRef:
        name: nginx-test-app
        kind: Deployment
        apiVersion: apps/v1
        service: nginx-test-app
        port: 80
    replicas:
        min: 0
        max: 10
    scaledownPeriod: 100
    requestRate:
        granularity: 1s
        targetValue: 100
        window: 1m`

Should I need to change anything here

[Manikiran88888]

Thanks for your support on the http topic

[Manikiran88888]

Hey,
I have a Query which I want some clarity and It is related to the above code

After applying the scaling if we want to access the webpage in the browser which has to trigger my Http request and should scale up the pods
what will be the url which I can use so that I can access my app in the browser
If you consider the same code as above I want to access the nginx app then which url should I need to use to open it in the browser

[Manikiran88888]

Instead of generating the load using the code, I want to open the nginx in the browser and I want to see if it is scaling my pods or not

[Manikiran88888]

Hey,
In my case the pods are scaling When I hit the load generator

I have forwarded my interceptor proxy

but when I try to use the curl command and check the response from the nginx server

these is a not found response I am getting but it is scaling the pods

Is there anything I am missing or Is there any issue could you please support on it

[JorTurFer]

Which is the issue? You are forwarding the request correctly. I mean, you open a port to the interceptor service, and you are calling to it and the interceptor is proxying the request to nginx because that message is from nginx, so it's working as expected.

I guess that you want to open your browser and call to localhost port-forwarded port. Am I right? If yes, it won't work unless you set the headers in your requests using a plugin or so. e.g: https://chromewebstore.google.com/detail/modheader-modify-http-hea/idgpnmonknjnojddfkpgkljpfnnfcklj?pli=1

The problem is that the interceptor routes the traffic based on the Host header. This header is set automatically when you call using a domain, but is empty using the IP directly, so reverse proxies don't know how to route it

[Manikiran88888]

Yes I want to open it in the browser to check if it is working
but I also tried to check with curl command still it is giving me the 404 NOT FOUND

I have also installed ingress so that it can accept the localhost

but still the response from curl

Is there anything I need to install or is there any step I am missing

[JorTurFer]

The problem is that your HTTPScaledObject doesn't have set localhost as valid domain. You can try modifying the HTTPScaledObject to include localhost within hosts section too. If you check the curl command, you are setting the Host to myhost.com but your browser doesn't do it. You can check this just opening the devtools and checking the request headers. You'll see that the browser sets the Host header to the domain (localhost in your case).

In any case, the about the picture using curl, that message is given by the backend, not by the interceptor. If you want to call to any route, you have to replace /test in the HTTPScaledObject's pathPrefix with just /. It will say to the interceptor that has to route any request for any path and not for paths starting from /test

[Manikiran88888]

Can I know what does this error indicating to

I am actually trying to run my project and the interceptor is giving bad gateway

[Manikiran88888]

It worked me for nginx server and it ran properly and when I try to deploy my application

apiVersion: apps/v1
kind: Deployment
metadata:
name: orders-poc
spec:
replicas: 1
selector:
matchLabels: &labels
app: orders-poc
template:
metadata:
labels: *labels
spec:
imagePullSecrets:
- name: my-testfapi-test
containers:
- name: orders-event
# Change the image name according to the image created
image: manikiranreddy58/ordspoc-db:1
resources:
limits:
memory: "200Mi"
cpu: "300m"

---

kind: Service
apiVersion: v1
metadata:
name: orders-poc
spec:
selector:
app: orders-poc
ports:
- protocol: TCP
port: 9000
targetPort: 9000

it is not running and the steps I followed the same how I did for nginx

[JorTurFer]

The message is quite clear, the connection has been refused 😄. Most probably because your pod doesn't listen the port 9090 despite it's the configured in k8s side. What do you see if you try to port-forward that port or make a request from any other pod within the cluster?

[Manikiran88888]

If I try to port forward it, the app is working fine and it is being accessible
That is why I want to make clear why is it showing connection refused ?

[JorTurFer]

Do you have network policies or any kind of network management (service mesh, firewall, etc)?

[Manikiran88888]

no actually I am running a fastapi service application and it is giving the error

[Manikiran88888]

sample-http.zip

This is the sample code of Fastapi which I am trying to run but it is not executing

I have included the readme inside the zip file

I have deployed it and scaled it also forwarded the port of the interceptor but when I try the curl command it is not responding it is giving bad gateway

[JorTurFer]

it is not running and the steps I followed the same how I did for nginx

This picture isn't related with this comment #1005 (comment)

The picture is pointing to port 9000 and your API doesn't expose that port. Please, don't change all the things all the time without clarifying the change because I'm lost and IDK what you are trying now.

[JorTurFer]

Could you please clarify which is the current scenario?:

• What are you deploying?
• What are you executing to validate it?
• Which is the exact error that you are seing?
• Which is the exact behaviour that you are facing?

[Manikiran88888]

I have just created the application on port 8008
Just gave a sample fastapi application

[Manikiran88888]

What are you deploying?

• I am deploying a fastapi application for which the code is inside the zip file with a folder name test-fastapi
• inside the test-fastapi there is a readme file which is uesful to create the docker image of a fastapi application which will run on port 8008
• once you create the fastapi application you can use that image in the http-deploy file so that it will deploy the fastapi application on your machine
• After that you can run the http-scale file to apply the httpscaledobject on the fastapi app

What are you executing to validate it?

• I want to scale up and down this application and also get the response of this application using the curl command
• the scaling is happening but there is no response from the curl command

Which is the exact error that you are seeing?

• I have forwarded the interceptor proxy
  

• The error is that the connection is refused
  

Which is the exact behavior that you are facing?

• when you run the curl command curl -H "Host: myhost.com" localhost:8080/

• it has to give me a response of hello world
  

• but it is prompting bad gateway

[JorTurFer]

There are 2 failures:

• your Dockerfile is binding the host to 127.0.0.1, which is not the one when you receive traffic from outside the pod (without a port-forward). I've tested your image and I can't reach the server either form a simple pod next to your deployment, only using port forward. You can try out my image instead yours: jorturfer/fts
• your workload doesn't define any readinessProbe, so the container is "ready" before the server is up un running. As the interceptor is monitoring the k8s endpoints, and they are ready when the pod is ready (despite your container isn't ready), the interceptor is proxying the request to a not ready container (because it needs up to 30 seconds to be ready, probably because you have limited the CPU to 50m). You must add readinessProbe to your workload.

Basically, if you change your container to this:

        - name: nginx-test-app
          image: jorturfer/fts # use your image name here
          ports:
            - containerPort: 8008
          readinessProbe:
            httpGet:
              path: /
              port: 8008
              scheme: HTTP

You will see how it works perfectly.

[Manikiran88888]

Okay
I will try it
Thanks

[Manikiran88888]

Good Morning,
I have added the redinessprobe to my code
If I use your Image I am getting the response properly

but If I use my image I am still getting this

Can you exactly spot if you have done any change in my code before creating your image
I understand that you are asking me not to bind my image to 127.0.0.1
But I want to know what did you use instead of that
can I see the code of your image jorturfer/fts
I want to check where it is differing from your code to my code and how your code is running with redinessprobe and why my code is not running with the same
what did you change in the Dockerfile while creating your image jorturfer/fts
It would be great if you can share the code of your image

[Manikiran88888]

Hey,
Good Afternoon
Could you share the sample code which you have used to create your image jorturfer/fts
It is a very important reference for the code which we are using

[Manikiran88888]

Good Morning,
Please could you send us the details of the code which you have used to create the image of fastapi
We want to see how you have created the image of it because to proceed further with KEDA in our project we want the sample code you have used to create the app of fastapi

[JorTurFer]

Hello,
Sorry, I was on a business trip with limited access to GH in general.
I just changed the Dockerfile

FROM python:3.10.12-slim

WORKDIR /app

COPY . /app/

RUN pip3 install fastapi uvicorn 

CMD [ "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8008" ]

The important change is this: "--host", "0.0.0.0"

[Manikiran88888]

Thanks A lot