Docs: Docker build with yarn 3 pnp support/example

[zigang93]

What is the improvement or update you wish to see?

I follow the latest docker buildnext.config.js with this outputStandalone enable

module.exports = {
  experimental: {
    outputStandalone: true,
  },
}

I had no clue what's file need to include when migrate to yarn 3 PnP

current Dockerfile:

# Install dependencies only when needed
FROM node:16-alpine AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile

# Rebuild the source code only when needed
FROM node:16-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN yarn build

# Production image, copy all the files and run next
FROM node:16-alpine AS runner
WORKDIR /app

ENV NODE_ENV production

RUN addgroup -g 1001 -S nodejs
RUN adduser -S nextjs -u 1001

# You only need to copy next.config.js if you are NOT using the default configuration
# COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/public ./public
COPY --from=builder /app/package.json ./package.json

# Automatically leverage output traces to reduce image size 
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs

EXPOSE 3000

ENV PORT 3000

# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry.
# ENV NEXT_TELEMETRY_DISABLED 1

CMD ["node", "server.js"]

Is there any context that might help us understand?

yarn 3 PnP with docker docs support
I am using next.js 12.0.8

Does the docs page already exist? Please link to it.

https://github.com/vercel/next.js/tree/canary/examples/with-docker

[zigang93]

I try to change the Dockerfile to this:

# Install dependencies only when needed
# FROM node:16.13-alpine AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
# RUN apk add --no-cache libc6-compat
# WORKDIR /app
# COPY package.json yarn.lock .yarnrc.yml .pnp.cjs .pnp.loader.mjs ./
# RUN echo 'YARN VERSION IN DEPS: ' && yarn --version

# Rebuild the source code only when needed
FROM node:16.13-alpine AS builder
WORKDIR /app
# COPY --from=deps /app/.yarn  ./.yarn
COPY . .
RUN yarn install --immutable
RUN yarn build

# Production image, copy all the files and run next
FROM node:16.13-alpine AS runner
WORKDIR /app

ENV NODE_ENV production

RUN addgroup -g 1001 -S nodejs
RUN adduser -S nextjs -u 1001

# You only need to copy next.config.js if you are NOT using the default configuration
COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/public ./public
COPY --from=builder /app/package.json ./package.json


# Automatically leverage output traces to reduce image size 
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

COPY --from=builder /app/.env.production.local ./

USER nextjs

EXPOSE 3000

ENV PORT 3000

# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry.
ENV NEXT_TELEMETRY_DISABLED 1

CMD ["node", "server.js"]

But it shows error from logs when I try to run it:

Error: Cannot find module 'next/dist/server/next-server'
Require stack:
- /app/server.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/app/server.js:4:20)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/app/server.js' ]

[perevezencev]

If you are using PnP you should run node -r ./.pnp.cjs server.js

[vHades0199]

I try node -r ./.pnp.cjs server.js but it did not work for me. Any other solution for PnP?

[matthewdfleming]

I just put together the first steps on getting zero install/PnP working with the docker builds. The main trick was to handle the dependencies as your stack trace indicates. I did this by disabling the global cache which moves the dependency cache to the local area, and then copy that local cache with the .pnp.cjs file to the same relative location in the image.

Also when I use the latest version of Typescript (4.8.4 at the time of this post), I needed to use the canary version of yarn via:
yarn set version canary prior to doing the docker build. Maybe I could have downgraded Typescript to work with 3.x..

Also as an aside, my stuff uses yarn workspaces (monorepo) with the packages holding the each workspace. You should be able to figure things out for not a monorepo.

When this runs I get a ~ 180 MB runnable image.

.yarnrc.docker.yml

enableStrictSsl: false
nodeLinker: pnp
httpTimeout: 300000
enableInlineBuilds: true
enableGlobalCache: false
packageExtensions:
  eslint-config-next@*:
    dependencies:
      next: "*"
      react: "*"
      react-dom: "*"
yarnPath: .yarn/releases/yarn-4.0.0-rc.22.cjs

Dockerfile

FROM node:16-alpine  AS builder

WORKDIR /app

ENV NEXT_TELEMETRY_DISABLED 1

# copy dependencies from the build image
COPY package.json .

COPY yarn.lock .
COPY .yarnrc.docker.yml .yarnrc.yml
COPY .yarn/releases ./.yarn/releases

# copy the app code over
COPY tsconfig.base.json .
COPY tsconfig.json .
COPY .eslint* .
COPY packages ./packages

ENV NODE_TLS_REJECT_UNAUTHORIZED=0

# install dependencies
RUN yarn install --immutable

# build the web gateway application app
RUN yarn workspace @web-gateway-monorepo/web-gateway-application build


FROM node:16-alpine  AS runner

WORKDIR /app

ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED 1
ENV NODE_TLS_REJECT_UNAUTHORIZED=0

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder --chown=nextjs:nodejs /app/.pnp.cjs ./
COPY --from=builder --chown=nextjs:nodejs /app/.yarn/cache ./.yarn/cache
COPY --from=builder --chown=nextjs:nodejs /app/packages/web-gateway-application/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/packages/web-gateway-application/.next/static ./.next/static
COPY --from=builder --chown=nextjs:nodejs /app/packages/web-gateway-application/public ./public

USER nextjs

EXPOSE 3000

ENV PORT 3000

CMD ["node", "-r", "./.pnp.cjs", "server.js"]

next.config.js - the output standalone is important

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  swcMinify: true,
  output: 'standalone',
  poweredByHeader: false
}

module.exports = nextConfig

Here's some miscellaneous files so you can make sense of the workspaces and general layout.

root package.json

{
  "name": "web-gateway-monorepo",
  "version": "0.0.1",
  "license": "MIT",
  "private": true,
  "workspaces": {
    "packages": [
      "packages/web-gateway-application"
    ]
  },
  "packageManager": "[email protected]",
  "repository": {
    "url": "https://whatever.git"
  },
  "scripts": {
    "lint": "yarn run eslint --ext .js,.jsx,.ts,.tsx .",
    "web-dev": "yarn workspace @web-gateway-monorepo/web-gateway-application dev",
    "prepare": "husky install"
  },
  "lint-staged": {
    "*.{ts,js,jsx,tsx}": [
      "yarn prettier --write --ignore-unknown",
      "eslint"
    ]
  },
  "devDependencies": {
    "eslint": "^8.24.0",
    "eslint-config-next": "^12.3.1",
    "eslint-config-prettier": "^8.5.0",
    "eslint-plugin-unused-imports": "^2.0.0",
    "husky": "^8.0.1",
    "next": "^12.3.1",
    "prettier": "^2.7.1",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "typescript": "^4.8.4"
  }
}

packages/web-gateway-application/package.json (where the next.js application lives)

{
  "name": "@web-gateway-monorepo/web-gateway-application",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "next": "^12.3.1",
    "react": "^18.2.0",
    "react-dom": "^18.2.0"
  },
  "devDependencies": {
    "@types/node": "18.7.23",
    "@types/react": "18.0.21",
    "eslint": "^8.24.0",
    "eslint-config-next": "^12.3.1",
    "typescript": "4.8.4"
  }
}

[balazsorban44]

As far as I know Yarn 3 is still in development. The docker example focuses on Yarn 1 which based on npm version stats is wildly more popular than v2 as well (1,5 million vs ~20000 weekly downloads, only looking their latest versions).

https://www.npmjs.com/package/yarn

[zigang93]

@balazsorban44 I see. too bad.. my team keep complain that docker build is too slow for them.. I try to optimise the build time from docker by using yarn 3 pnp..

[teddybee]

You could try out pnpm, its very fast on SSD.

[vanxh]

I created a gist to deploy NextJS apps using yarn v3 and PnP with Docker.
Gist - https://gist.github.com/vanxh/0c3a62cc6bd6b8aa143c2e278d9e9dfa

[ash67]

@vanxh is there a way not to have devDependencies packages in the .yarn folder when you are running the prod build.

My .yarn folder dev-dependencies packages which increases the overall size of docker image.

[marcinw99]

Any update on this? trying with Next.js 13.1.5 and yarn 3.4.1 the issue still persists.

[sszczep]

Still not fixed. Any updates on that matter?

[lewandom]

Running with node -r ./.pnp.cjs server.js + copying .yarn/cache works for me. It's not an acceptable final solution though, since yarn cache includes dev dependencies. It's a major hurdle both size-wise and security-wise.

It looks like the ultimate solution may need to address #34600. Today, output: standalone only supports npm/node_modules.

[sszczep]

I was ultimately forced to create a seperate next and yarn config files (standalone + node-modules for Docker; pnp for development). I don't like it, doesn't come with PnP advantages and makes building images longer, but I couldn't see a more suitable solution.

Dockerfile:

FROM node:lts-alpine@sha256:47d97b93629d9461d64197773966cc49081cf4463b1b07de5a38b6bd5acfbe9d as builder

ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED 1

USER node
WORKDIR /home/node

COPY --chown=node:node . .
RUN mv .docker.yarnrc.yml .yarnrc.yml
RUN mv .docker.next.config.js next.config.js

RUN yarn && yarn build

FROM node:lts-alpine@sha256:47d97b93629d9461d64197773966cc49081cf4463b1b07de5a38b6bd5acfbe9d as runner

EXPOSE 3000

ENV NODE_ENV production
ENV NEXT_TELEMETRY_DISABLED 1

USER node
WORKDIR /home/node

COPY --from=builder --chown=node:node /home/node/public ./public
COPY --from=builder --chown=node:node /home/node/.next/standalone ./
COPY --from=builder --chown=node:node /home/node/.next/static ./.next/static

CMD ["node", "server.js"]

.dockerignore

.next
next.config.js
.yarnrc.yml

Should take less space than the cache method though.

[AkiraVoid]

Here may be a better workaround cause I found that .pnp.cjs only support CommonJS modules:

next.config.js:

/** @type {import('next').NextConfig} */
const nextConfig = {
    reactStrictMode: true,
    output: "standalone",
    // Other configuration
};

module.exports = nextConfig;

.yarnrc.yml:

# These configurations are generated by yarn with commands:
#
# yarn plugin import workspace-tools
# yarn set version latest
#
# You should replace 3.5.1 with the real version of your yarn,
# if you want to manually set these configurations. 
plugins:
    - path: .yarn/plugins/@yarnpkg/plugin-workspace-tools.cjs
      spec: "@yarnpkg/plugin-workspace-tools"

yarnPath: .yarn/releases/yarn-3.5.1.cjs

Dockerfile:

FROM node:18-alpine AS base

ENV NODE_ENV=production

RUN corepack enable
RUN corepack prepare yarn@stable --activate

FROM base AS deps

WORKDIR /app
COPY ./package.json .
COPY ./.yarn/plugins ./.yarn/plugins
COPY ./.yarn/releases ./.yarn/releases/
COPY ./.yarnrc.yml .
RUN yarn install

FROM base AS builder

WORKDIR /app

COPY . .
COPY --from=deps /app/.yarn ./.yarn/
COPY --from=deps /app/yarn.lock .

RUN yarn build

FROM base AS runner

WORKDIR /app

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static/
COPY --from=builder /app/public ./public/

FROM runner AS production

WORKDIR /app

COPY --from=builder /app/.yarn/plugins .yarn/plugins/
COPY --from=builder /app/.yarnrc.yml .
COPY --from=builder /app/.yarn/releases .yarn/releases/

# Delete this folder cause it contains the `next` dependency which was output by Next.js builder.
# If we install dependencies later, yarn will try to rename cache file `***.tmp` to `***.zip`.
# But the name has been claimed already when it comes to `next` to be installed,
# which causes a failure of building image.
RUN rm -rf ./.yarn/cache

RUN yarn workspaces focus --all --production

USER nextjs

EXPOSE 3000

# Replace 3.5.1 with the real version of your yarn.
CMD ["node", "/app/.yarn/releases/yarn-3.5.1.cjs", "--", "node", "/app/server.js"]

With this you will only install production dependencies while building docker image.

[KrisLau]

Thanks for this it was huge help for my project since I kept running into failed to solve: process "/bin/sh -c yarn errors. To fix I basically just added these lines from your example:

RUN corepack enable
RUN corepack prepare yarn@stable --activate