Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Returning 304 #19

Open
ReeSilva opened this issue May 2, 2019 · 4 comments
Open

Returning 304 #19

ReeSilva opened this issue May 2, 2019 · 4 comments

Comments

@ReeSilva
Copy link

ReeSilva commented May 2, 2019

Hey guys, I'm looking for u to see if anyone can help, because I tried a lot and nothing seems to work. I have the follow stack:
An AWS ECS Fargate Service exposing the Rendora (with cache disabled for test), in front of this a Load Balancer and in front of this Load Balancer I have Cloudfront also with cache disabled for test. This is my actual Rendora config:

target:
  url: http://localhost:8080 #This is the address for the React SPA Application, Fargate doesn't use container linkings
backend:
  url: http://localhost:8080 #This is the address for the React SPA Application, Fargate doesn't use container linkings
headless:
  waitAfterDOMLoad: 3000
  internal:
    url: http://localhost:9222
cache:
  type: none
debug: true
server:
  enable: true
filters:
  userAgent:
    defaultPolicy: blacklist
    exceptions:
      keywords:
        - googlebot
        - Yahoo! Slurp
        - bingbot
        - yandex
        - baiduspider
        - facebookexternalhit
        - twitterbot
        - rogerbot
        - linkedinbot
        - embedly
        - quora link preview
        - showyoubot
        - outbrain
        - pinterest/0.
        - developers.google.com/+/web/snippet
        - slackbot
        - vkShare
        - W3C_Validator
        - redditbot
        - Applebot
        - WhatsApp
        - flipboard
        - tumblr
        - bitlybot
        - SkypeUriPreview
        - nuzzel
        - Discordbot
        - Google Page Speed
        - Qwantify
        - pinterestbot
        - Bitrix link preview
        - XING-contenttabreceiver
        - Chrome-Lighthouse'
        - laboritbot

When I call the Rendora API /render it works like a charm, when I reach my page through a browser, it works also. But, when I try to curl using one of my defined bots, it returns 304 everytime. I noticed that in browser the initial call also returns 304, but render oks, and on the API of Rendora it also renders even returning 304.

Is there anything that am I doing wrong?

Thank u in advance.

@u8983478934
Copy link

probably this has nothing to do with rendora, it just saves the status code it gets from the backend and inject it back to the whitelisted clients (e.g. crawlers). what status code number do you see returned from the API? if it's 304 this proves my point. Your backend is the one that returns 304 and rendora just returns it back as it is back to the client

@ReeSilva
Copy link
Author

ReeSilva commented May 4, 2019

@gophobic Thank u for ur answer. I may have expressed myself wrong. The problem is not the status itself, but the fact that the response is empty, but when I call the Rendora API the response is complete.

EDIT: I don't know if that can help with something, but Chrome Headless returns this error: Failed to connect to socket /var/run/dbus/system_bus_socket: no such file or directory

EDIT2: Everytime it works on a first try then starts to fail

@stas
Copy link

stas commented Jul 21, 2019

@ReeSilva, until #21 is merged, consider patching the chrome Docker image to not cache (last 3 options):

FROM rendora/chrome-headless:latest

CMD [ \
  "--headless", \
  "--no-sandbox", \
  "--disable-setuid-sandbox", \
  "--disable-dev-shm-usage", \
  "--disable-gpu", \
  "--remote-debugging-address=0.0.0.0", \
  "--remote-debugging-port=9222", \
  "--disable-remote-fonts", \
  "--user-data-dir=/tmp", \
  "--disable-application-cache", \
  "--media-cache-size=1", \
  "--disk-cache-size=1" \
  ]

@elion
Copy link

elion commented Oct 26, 2021

But, when I try to curl using one of my defined bots, it returns 304 everytime.

Some of your bots are not lower case:

  • SkypeUriPreview
  • WhatsApp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants