Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSSQL pod fail on crashloopBackOff #70

Open
1 task done
sebglon opened this issue Dec 4, 2023 · 6 comments
Open
1 task done

MSSQL pod fail on crashloopBackOff #70

sebglon opened this issue Dec 4, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@sebglon
Copy link
Contributor

sebglon commented Dec 4, 2023

Steps To Reproduce

deploy minimal helm values:

database:
  enabled: true
  volume:
    backups:
      size: 20Gi
    data:
      size: 20Gi
    log:
      size: 10Gi
general:
  admins: xxxx
  domain: bitwarden.env.fr
  email:
    replyToEmail: [email protected]
    smtpHost: toto.env.fr
    smtpPort: "587"
  volumeAccessMode: ReadWriteOnce
secrets:
  secretName: custom

Expected Result

mssql pod is up and running

Actual Result

mssql log:

SQL Server 2019 will run as non-root by default.
This container is running as user mssql.
To learn more visit https://go.microsoft.com/fwlink/?linkid=2099216.
2023-12-04 16:54:47.94 Server      The licensing PID was successfully processed. The new edition is [Express Edition].
2023-12-04 16:54:48.81 Server      Setup step is copying system data file 'C:\templatedata\master.mdf' to '/var/opt/mssql/data/master.mdf'.
2023-12-04 16:54:48.82 Server      ERROR: Setup FAILED copying system data file 'C:\templatedata\master.mdf' to '/var/opt/mssql/data/master.mdf':  5(Access is denied.)
ERROR: BootstrapSystemDataDirectories() failure (HRESULT 0x80070005)
Stream closed EOF for bitwarden/bitwarden-self-host-mssql-0 (bitwarden-self-host-mssql)

kube event:

    state:
      waiting:
        message: back-off 5m0s restarting failed container=bitwarden-self-host-mssql
          pod=bitwarden-self-host-mssql-0_bitwarden(61aea36e-717f-4732-81ff-ab64ca6bbc90)
        reason: CrashLoopBackOff

Screenshots or Videos

No response

Additional Context

No response

Chart Version

self-host-0.1.7-Beta

Environment Details

kubernetes self-hosted: 1.22.2

Issue Tracking Info

  • I understand that work is tracked outside of Github. A PR will be linked to this issue should one be opened to address it, but Bitwarden doesn't use fields like "assigned", "milestone", or "project" to track progress.
@sebglon sebglon added the bug Something isn't working label Dec 4, 2023
@sebglon
Copy link
Contributor Author

sebglon commented Dec 4, 2023

If i replace the default image mcr.microsoft.com/mssql/server:2019-CU17-ubuntu-20.04 by this one bitwarden/mssql:2023.8.3 all work well

I don't know why

@jhbeskow
Copy link
Collaborator

jhbeskow commented Dec 6, 2023

I have seen this caused by having DB files out there created from a previous install that is running under a different security context (i.e. a different service account) or issues with the storage class permissions. Which flavor of Kubernetes are you running?

@ebCrypto
Copy link

ebCrypto commented Jan 4, 2024

I am experiencing the same problem here. Running on Talos 1.6.1.

apiVersion: v1
kind: Namespace
metadata:
  name: bitwarden
  labels:
    toolkit.fluxcd.io/tenant: dev-team
    pod-security.kubernetes.io/enforce: privileged
    pod-security.kubernetes.io/audit: privileged
    pod-security.kubernetes.io/warn: privileged
 #
    # Configure database
    #
    database:
      # deploy the database pod?  if false, a connection string to a SQL Server will need to be provided through one of the configuration providers.
      enabled: true
      # Labels to add to the MSSQL deployment
      labels: {}
      # Image name, tag, and pull policy
      image:
        name: mcr.microsoft.com/mssql/server
        # Tag of the image to use. (Defaults to general.coreVersion)
        tag: 2019-CU17-ubuntu-20.04
      # The container is limited to the resources below.  Adjust for your environment.
      resources:
        requests:
          memory: "2G"
          cpu: "100m"
        limits:
          memory: "2G"
          cpu: "500m"

      # The MSSQL volumes for the PVCs
      volume:
        backups:
          # Storage size
          size: 1Gi
        data:
          # Storage size
          size: 10Gi
        log:
          # Storage size
          size: 10Gi
      # Specifies the strategy used to replace old Pods by new ones. The value can be "OnDelete" or "RollingUpdate". "OnDelete" is the default value.
      # When set to OnDelete, the SQL pod will not be recreated automatically on update.  It must be manually deleted to update.
      updateStrategy: OnDelete
      # This will set the Kubernetes container security context
      securityContext:
      # Run the pod under a service account you create.  This is especially useful for OpenShift deployments
      podServiceAccount:

@Kaylleur
Copy link

I think this charts has some missing init containers to set up correct rights on volume before starting main containers. I have the same error on all the pods about permission denied on each volumes

@momelod
Copy link

momelod commented Feb 25, 2024

In my case i was attaching a PVC for the data volume and was getting the same permission error. I can get the pod to start, but I know there must be a better way.

helm template bitwarden/self-host --values /tmp/bitwarden-values.yaml > /tmp/bitwarden-rendered.yaml

Then i edit /tmp/bitwarden-rendered.yaml and add spec.template.spec.securityContext.fsGroup: 10001:

      securityContext:
        fsGroup: 10001

Finally kubectl apply -f /tmp/bitwarden-rendered.yaml

And here is just a sample from the file where the change needs to be made:

---
# Source: self-host/templates/mssql.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  ...
spec:
  serviceName: release-name-self-host-mssql
  template:
     spec:
      securityContext:
        fsGroup: 10001
      containers:
        - name: release-name-self-host-mssql
          image: "mcr.microsoft.com/mssql/server:2022-CU11-ubuntu-22.04"

@Niich
Copy link

Niich commented Feb 26, 2024

I was also having this issue.

I am using the k8s sigs nfs provisioner and I was able to get past this error by setting runAsUser: 10001 under the database.securityContext available in the values file.

I also removed the old files so the pod could recreate them with the correct UID.

#
# Configure database
#
database:
  # deploy the database pod?  if false, a connection string to a SQL Server will need to be provided through one of the configuration providers.
  enabled: true
  # Labels to add to the MSSQL deployment
  labels: {}
  # Image name, tag, and pull policy
  image:
    name: mcr.microsoft.com/mssql/server
    # Tag of the image to use. (Defaults to general.coreVersion)
    tag: 2022-CU11-ubuntu-22.04


  # This will set the Kubernetes container security context
  securityContext:
    runAsUser: 10001
  # Run the pod under a service account you create.  This is especially useful for OpenShift deployments
  podServiceAccount:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

6 participants