[misc]: Processing files in middleware

[didinewlander]

I understand that this issue may be closed if it should be filed in another category

• I understand

My issue

I have this documentUploader route, that gets word,pdf and image files:

documentUploader: f({
    pdf: { maxFileSize: "8MB", maxFileCount: 4 },
    image: { maxFileSize: "8MB", maxFileCount: 4 },
    "application/msword": { maxFileSize: "8MB", maxFileCount: 4 },
  })
    .middleware(async ({ files }) => {
      await auth();

      const processedFiles = [...files];

      const images = files.filter((file) =>
        file.name.match(/\.(png|jpg|jpeg)$/i)
      );
      for (const file of images) {
        const resizedBuffer = await sharp(file) // << Problematic Line
          .resize(100, 100)
          .toBuffer();

        const resizedFile = {
          ...file,
          buffer: resizedBuffer,
          name: `small-${file.name}`,
        };
        processedFiles.push(resizedFile);
      }
      return { files: processedFiles };
    })
    .onUploadComplete(() => {}),
} satisfies FileRouter;

Disclaimer: I got to this version using ChatGPT, but the overall idea is from 'Josh Tried Coding' channel https://www.youtube.com/watch?v=7hS9b6n7HrM for image compression optimized for nextjs applications.

The Issue

The problem is that in the middleware I haven't figured out how to work with the buffer itself, only with the FileUploadData class, which provides only the name, size, and type of the file, meaning I can't actually make any modifications or create more files in the middleware.

The Solution

Enable in the core.ts file, where all the routes are handled, an option to work with the files themselves BY REQUEST and not by default, so only if there is a BE file optimizer or handler the developer could process the files by selection.

Use cases

• Image compression
• File parsers and validators
• Automation processes and file forwarding

[markflorkowski]

The file buffers aren't available in the middleware to avoid server ingress/egress costs. To achieve what you need, you'd either need to process files client-side or have your own server endpoint ingest, modify, and then upload them using UTApi functions.

We're considering an alternative that would allow optional server-side file handling, despite the added costs, but it's not yet in development. Your feedback is quite helpful here, as it highlights demand for this feature.

cc @t3dotgg