Sitemap index

[omerp-explorium]

Goals

When using the multiple sitmap approach, Nextjs should produce a root sitemap that will point to all sitemap urlsets e.g /sitemap.xml

Non-Goals

Background

As you can see in the image, we should also have this sitemap pointing to multiple sitemaps

Taken from Google documentation on large sitemaps

Proposal

I propose Nextjs will keep track of all the generated sitemaps, and generate the proposed sitemap of sitemaps xml.

What do you think?

[Fedeorlandau]

I made a PR that adds support for sitemap-index.ts which you can use to dynamically add your sitemaps to the sitemap index file.

#61391

[ricardo-fnd]

@Fedeorlandau But this means that we do not need /app/sitemap.xml working? Or sitemap-index.ts is the sitemap.xml which points for others sitemaps?

[Fedeorlandau]

Hi! This is just for having a sitemap-index file where you can manually generate the index and put your other sitemaps files. You still need to generate sitemap.xml files for your site.

[tomcru]

Thanks for the PR @Fedeorlandau. Hope to see this merged soon!

Could you possible give any pointers on how you have handled sitemap index creation until now?

[Fedeorlandau]

@tomcru absolutely. The way I found it to work is by creating a /app/sitemap-index.xml/route.ts with a route GET handler and returning a xml like response.

[tomcru]

For anyone else coming across this, here's a quick example I used to test a sitemap index via /app/sitemap-index.xml/route.ts as Fedeorlandau suggested.

const PAGE_SIZE = 1000;

const generateSitemapLink = (url: string) =>
  `<sitemap><loc>${url}</loc></sitemap>`;
  
export async function GET() {
  const gamesResponse = await fetch(
    `${API.base}/products?${new URLSearchParams({
      pageSize: String(PAGE_SIZE),
      pageNumber: String(1),
      currency: Currency.usd,
      order: 'nameAsc',
    })}`,
  );

  const games = (await gamesResponse.json()) as GetProductsOkResponse;

  const sitemapIndexXML = `<?xml version="1.0" encoding="UTF-8"?>
    <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
        ${generateSitemapLink(`${URLS.base}/sitemap.xml`)}

        ${Array.from({ length: games.data.pageCount }, (_, i) => i + 1)
          .map((id) =>
            generateSitemapLink(`${URLS.base}/games/sitemap/${id}.xml`),
          )
          .join('')} 
    </sitemapindex>`;

  return new Response(sitemapIndexXML, {
    headers: { 'Content-Type': 'text/xml' },
  });
}

Result:

[antoinerousseau]

I don't understand the utility of generateSitemaps without a sitemaps index, am I missing something?

[hermbo]

Agreed, seems like this should just be automatic. Then you only need to provide google with the index file.

[antoinerousseau]

Meanwhile, I went with my own custom index at src/app/sitemap.xml/route.ts:

import { env } from "~/env";
import { getLastMod } from "~/lib/sitemap";
import { generateSitemaps as getArtistSitemaps } from "../api/sitemaps/artists/sitemap";
import { generateSitemaps as getEventSitemaps } from "../api/sitemaps/events/sitemap";
import { generateSitemaps as getVenueSitemaps } from "../api/sitemaps/venues/sitemap";

export const revalidate = 3600; // cache for 1 hour

function getFileName(id?: number) {
  if (id == null) {
    return "sitemap.xml";
  }
  /**
   * https://nextjs.org/docs/app/api-reference/file-conventions/metadata/sitemap#generating-multiple-sitemaps
   * In production, your generated sitemaps will be available at /.../sitemap/[id].xml. For example, /product/sitemap/1.xml.
   * In development, you can view the generated sitemap on /.../sitemap.xml/[id]. For example, /product/sitemap.xml/1. This difference is temporary and will follow the production format.
   * See the generateSitemaps API reference for more information.
   */
  return env.VERCEL_ENV === "development"
    ? `sitemap.xml/${id}`
    : `sitemap/${id}.xml`;
}
function getLoc(path: string, id?: number) {
  return `${env.URL}/api/sitemaps/${path}/${getFileName(id)}`;
}
function getSitemap(path: string, id?: number) {
  return /* XML */ `<sitemap><loc>${getLoc(path, id)}</loc><lastmod>${getLastMod()}</lastmod></sitemap>`;
}
function getSitemaps(ids: { id: number }[], path: string) {
  return ids.map(({ id }) => getSitemap(path, id)).join("");
}

export async function GET() {
  const xml = /* XML */ `<?xml version="1.0" encoding="UTF-8"?>
    <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
      ${getSitemap("cities")}
      ${getSitemaps(await getEventSitemaps(), "events")}
      ${getSitemaps(await getArtistSitemaps(), "artists")}
      ${getSitemaps(await getVenueSitemaps(), "venues")}
    </sitemapindex>
  `;

  return new Response(xml, {
    headers: {
      "Content-Type": "application/xml",
    },
  });
}