Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unable to run external commands in background or from crontab #212

Open
ng5 opened this issue Dec 14, 2023 · 9 comments
Open

unable to run external commands in background or from crontab #212

ng5 opened this issue Dec 14, 2023 · 9 comments

Comments

@ng5
Copy link

ng5 commented Dec 14, 2023

i have setup broadcast mirror server, there is no problem with that. there is no problem with sender too.

  1. sender.sh - OK
    echo X | websocat -k -E -t - wss://127.0.0.1:9000

  2. receiver.sh
    websocat -E -k -t wss://127.0.0.1:9000 | xargs -r -n1 print_args.sh

receiver.sh works only from interactive shell.
i) ./receiver.sh - OK
ii) nohup ./receiver.sh & - NOT working
iii) adding receiver.sh to crontab doesn't work either.

i tried to redirect to named pipe in receiver.sh but that didn't work either. similar tools like netcat works without any problem in sender/receiver.sh.

@vi
Copy link
Owner

vi commented Dec 14, 2023

What do you mean by "not working"?

Variants:

  1. websocat does not start
  2. websocat starts, but exits early
  3. websocat starts and connects to the socket, but xargs finishes early
  4. both websocat and xargs persist, but xarg does not start any new processes
  5. xargs starts child processes, but it does not work properly

?

Typically I use strace tool to debug what is happening, including for backgrounded processes.

@ng5
Copy link
Author

ng5 commented Dec 14, 2023

From Crontab

  1. both websocat and xargs immediately exits

From nohup

  1. both websocat and xargs shows up (under ps -eaf command), but they don't work as expected. I would like to invoke
    print_args.sh on each message received.

I suspect it is not able to pipe correctly in the background.

Instead if use a similar tool like netcat (for send/receive)
works consistently across interactive shell, nohup or crontab.

  1. sender_nc.sh:
    nc 127.0.0.1 9000
    x
    y
    z

  2. receiver_nc.sh
    nc -l 9000 | xargs -r -n1 print_args.sh

@vi
Copy link
Owner

vi commented Dec 14, 2023

Note that you may want try foreachmsg: overlay if you want something to happen on each message. Something like

websocat -E -k -t wss://127.0.0.1:9000 foreachmsg:cmd:./print_stdin.sh

@ng5
Copy link
Author

ng5 commented Dec 15, 2023

foreachmsg:cmd immediately exits as reported earlier by other people.
All 3 scenarios are shown in the screenshot attached.

  1. without xargs or foreach
  2. with xargs - only works in interactive shell
  3. foreachmsg - exits immediately.

For now I ended up writing another websocket client to execute the command after receiving the message (in background). this is not ideal but not a deal breaker either. Atleast I was able to run web socket server and sender without writing code. It is only the receiver part which doesn't seem to work in background mode.

image

@vi
Copy link
Owner

vi commented Dec 15, 2023

Note that instead of broadcast:mirror: it may be better to use wsbroad.

@vi
Copy link
Owner

vi commented Dec 15, 2023

Tried some command lines: foreachmsg: exists as it tries both directions by default (data transfer to WebSocket and from WebSocket) and -E commands it to exit if any of that direction finishes. But reverse direction (to WebSocket) finishes immediately in some cases, as stdin is not available in those modes.

Maybe you want -un instead of -E, i.e. use only one direction (from left to right) and prevent sending quit message.

Also you need to modify print_args.sh for foreachmsg: - the data will be on stdin, not in argv.

@ng5
Copy link
Author

ng5 commented Jan 17, 2024

Note that instead of broadcast:mirror: it may be better to use wsbroad.

I'm running this in production from last 3 weeks weeks. On an average it crashes once every 24 hours without providing anything in stdout or stderr. I have written a crontab script that runs every minute and if websocat server is down it simply restarts it. Therefore it is not causing a serious problem. However do you think wsbroad is more stable than running websocat with broadcast:mirror. Is there a known problem?

@vi
Copy link
Owner

vi commented Jan 17, 2024

Websocat is not expected to require periodic restarts. Maybe something wrong is happening? What do you mean by "crashes"? Process exits? (with what exit code?). Is it a high-load scenario or Weboscat is mostly waiting?


However do you think wsbroad is more stable than running websocat with broadcast:mirror

It is much simpler than Websocat and it uses modern dependencies (tokio-tungstenite).

Note that it differs from broadcast:mirror: - each seen URL gets a separate broadcast domain instead of being ignored like in Websocat. You can also tune some options of how messages should be forwarded.

@ng5
Copy link
Author

ng5 commented Jan 17, 2024

load is very minimal hardly 100 concurrent subscribers. yes process exits without writing anything to stdout or stderr. i'm redirecting both 1 and 2 to a file. unfortunately core dump is not enabled on production server which could have revealed the actual cause of the problem. it is very hard to reproduce the error. i will replace websocat with wsbroad and see how it goes.

if you want to close the issue please feel free to close it as i'm not able to provide any stack trace because core is not enabled on production server. i use the websocat behind haproxy tunnel not sure if this matters. reason haproxy is used to provide consistent ssl connection on 443 port.

Once again thank you for all the help and great work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants