segfault caused by os.execvpe in windows

[matthewdeanmartin]

I'm using windows 11 and os.execvpe will cause a segfault. I happen to be using gitbash and pipx, but I get the same behavior if I don't use pipx.

After a bit of hacking, I got this to work, but with 2 second additional time cost over using aws directly. I see that 9 month ago execvpe was introduced probably to make things better for linux or mac users

I'm guessing you don't have access to a windows machine, you might want to consider a github action on a windows image to check to see if awslocal --help can run without segfault.

Another possibility is to use something like shellingham to detect the shell before doing things that the OS isn't expected to be able to do.

def run(cmd, env=None):
    """
    Replaces this process with the AWS CLI process, with the given command and environment
    """
    # if not env:
    #     env = {}
    # print("os.execvpe")
    # os.execvpe(cmd[0], cmd, env)
    if env is None:
        env = os.environ.copy()
    else:
        # Make sure we include the current environment to avoid missing essential variables
        full_env = os.environ.copy()
        full_env.update(env)

    print("subprocess.run")
    result = subprocess.run(cmd, env=full_env, text=True, capture_output=True)
    print(result.stdout)  # Print standard output
    print(result.stderr, file=sys.stderr)  # Print standard error to the error stream
    return result

When I use powershell or cmd, I get "ModuleNotFoundError: No module named 'boto3'", probably because the .bat file gets launched and doesn't actual activate the right environment, but I also got that even when installing to the system python (which is a no-no). I didn't pursue why the entry point scripts fail to get an activated environment, since I never use powershell or cmd.exe.

[MarcelStranak]

Hello @matthewdeanmartin,

I, too, utilize LocalStack on the Windows OS. I do not use gitbash and pipx, but in a normal powershell terminal or cmd terminal all works as expected.

Could you clarify why installing Python is discouraged? The AWS CLI tool is also offered as a Python pip package, and awslocal serves as a thin wrapper around it.

I recommend upgrading your WSL version to the latest version and using the endpoint URL http://localhost.localstack.cloud:4566 to potentially enhance connection speed.

Alternatively, you could use AWS CLI version 2 and configure a profile in your ~/.aws/credentials and ~/.aws/config files.
You can find the steps here https://docs.localstack.cloud/user-guide/integrations/aws-cli/#configuring-a-custom-profile.

I hope this helps with your setup.

[matthewdeanmartin]

Well I'm glad to hear it works on your machine.

I see you all have a merge request to add github Actions to test on other operating systems & shells, you might want to merge that and include windows in the build matrix, who knows, you might see a break since os.execvpe isn't expected to behave the same on windows as linux:

ref: https://stackoverflow.com/questions/67371606/how-to-replace-os-execvpe-on-windows-efficiently-if-the-child-process-is
https://stackoverflow.com/questions/7004687/os-exec-on-windows
python/cpython#101191

Could you clarify why installing Python is discouraged?

I mean people shouldn't be installing all sorts of things into your system python, the python that exists before the first venv is created. I have dozens of cli commands installed on my machine, if they all install to the same site-packages, then each succeeding app will uninstall the preferred versions of the previous. This is why virtual environments exist. Pipx exists to keep cli tools out of the current venv and in their own isolated venv & also out of the system python.

[alexrashed]

@matthewdeanmartin We actually do have a few windows machine, and haven't seen this report before, which makes it hard for us to act on it.
The pipeline is a very good point! The PR you referenced was just merged by @dominikschubert (#81), so it would be there as a basis to be extended with a Windows runner.
Do you think you might contribute to this open source project with a PR adding a windows runner reproducing your issue? :)