[POC] feat(website): configure llms plugin

[weronikaolejniczak]

Summary

Relates to #8828

This is a PoC of https://github.com/rachfop/docusaurus-plugin-llms to see how well it works with our documentation site.

See llms.txt specification here: https://llmstxt.org/

Tip

Think of it as a sitemap.xml or robots.txt file but for LLMs. We can generate LLM context based on the file.

Why are we making this change?

Internal request:

Is there any effort to create an llm.txt of EUI docs ala this experiment over here with our main docs? Would be very helpful in dev workflows when generating EUI interfaces. Right now I just provide relevant EUI examples similar to what I'm trying to build, but having llm-ready variants of the docs would really improve this

QA

• verify that the llms.txt is there and is correct
• verify that all pages expose .md version (example: button.md)

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 883b7c2
• Documentation website

History

• 💚 Build #636 succeeded f9d23ea

cc @weronikaolejniczak

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 883b7c2

cc @weronikaolejniczak

[weronikaolejniczak]

Most likely, we need to add description in the frontmatter for the majority of pages. Otherwise, it takes the beginning of the file and for some components, that is a good summary, for some it's ::tip or functional details and not a component overview.

We would also need the files prepended with _ to be excluded.

[weronikaolejniczak]

For the "exposing Markdown files" what we can do is programmatically copy the .mdx files from the /docs folder to the /static folder, and even transform it if we need to (e.g. add details that might be important to LLM, like EUI version). We might use something as simple as copy-webpack-plugin which should look more or less like:

const CopyWebpackPlugin = require('copy-webpack-plugin');
const path = require('path');

module.exports = {
  // ...
  plugins: [
    async function myPlugin(context, options) {
      return {
        name: 'copy-md-files',
        configureWebpack() {
          return {
            plugins: [
              new CopyWebpackPlugin({
                patterns: [
                  {
                    from: path.resolve(__dirname, 'docs'),
                    to: path.resolve(__dirname, 'static/docs'),
                    globOptions: {
                      ignore: ['**/*.ts', '**/*.gif', '**/*.png', '**/*.yml'],
                    },
                  },
                ],
              }),
            ],
          };
        },
      };
    },
  ],
};

[github-actions]

👋 Hey there. This PR hasn't had any activity for 90 days. We'll automatically close it if that trend continues for another week. If you feel this issue is still valid and needs attention please let us know with a comment.