@circulo-ai/upload

A type-safe, multi-provider file upload framework for Node.js and Next.js, with first-class support for presigned URLs and multipart uploads.

Features

🧠 Type-safe storage contexts: compile-time safety when routing files across multiple buckets, containers, or backends.
🌐 Multi-provider support: AWS S3, Azure Blob, Local storage, Vercel Blob
🪣 Multi-bucket/container: Organize files across different storage contexts
📦 Multipart uploads: Large file support with resumable uploads
🔐 Presigned URLs: Direct client-to-storage uploads
📝 TypeScript: Full type safety with generics
🎯 Zero dependencies: Only peer dependencies for storage providers you use
🔒 Secure: Built-in path traversal protection and filename sanitization
🧩 Extensible: Lifecycle hooks and structured errors for predictable DX

Installation

npm install @circulo-ai/upload

# Install the storage provider(s) you need:
npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner    # For S3
npm install @azure/storage-blob                                 # For Azure Blob
# Local storage has no dependencies

Core Concepts

StorageProvider: low-level adapter for one backend (S3, Azure Blob, Vercel Blob, Local). Use directly for single-bucket/simple cases.
StorageManager (recommended): orchestrates multiple named providers/contexts (e.g., uploads, public, temp) with shared helpers (presign, multipart) and type-safe context selection.
Route adapters: @circulo-ai/upload/next and @circulo-ai/upload/hono expose HTTP handlers for uploads, presigned URLs, multipart, and serving files.

Environment Compatibility

Surface	Supported	Notes
Node.js (server)	✅	Primary target
Edge runtimes (Vercel/Cloudflare)	⚠️	Depends on provider; Local/S3 presign need Node APIs, Vercel Blob works
Browser	⚠️	Use presigned URLs or route adapters; providers are server-side only

Quick Start

Single Storage Provider

import { S3StorageProvider } from "@circulo-ai/upload";

const storage = new S3StorageProvider({
  bucket: "my-bucket",
  region: "us-east-1",
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  },
});

// Upload a file
const result = await storage.upload({
  file: buffer,
  fileName: "document.pdf",
  contentType: "application/pdf",
});

console.log(result.key); // Storage key
console.log(result.path); // Serve path

Multiple Storage Contexts (Recommended)

import {
  StorageManager,
  S3StorageProvider,
  LocalStorageProvider,
} from "@circulo-ai/upload";

// Define your storage contexts
type MyContexts = "user-uploads" | "public-assets" | "temp-files";

const manager = new StorageManager<MyContexts>({
  providers: {
    "user-uploads": new S3StorageProvider({
      bucket: "user-files",
      region: "us-east-1",
    }),
    "public-assets": new S3StorageProvider({
      bucket: "public-cdn",
      region: "us-east-1",
    }),
    "temp-files": new LocalStorageProvider({
      basePath: "./temp",
    }),
  },
  defaultContext: "user-uploads",
});

// Upload to specific context
await manager.upload({
  file: buffer,
  fileName: "avatar.png",
  contentType: "image/png",
  context: "user-uploads", // Type-safe context
});

// Download from context
const file = await manager.download({
  key: "avatar.png",
  context: "user-uploads",
});

API Reference

Storage Providers

S3StorageProvider

import { S3StorageProvider } from "@circulo-ai/upload";

const s3 = new S3StorageProvider({
  bucket: "my-bucket",
  region: "us-east-1",

  // Optional: For S3-compatible services (MinIO, R2, etc.)
  endpoint: "https://s3.example.com",
  forcePathStyle: true,

  // Optional: Explicit credentials (uses AWS credential chain if omitted)
  credentials: {
    accessKeyId: "KEY",
    secretAccessKey: "SECRET",
  },

  // Optional: Path prefix within bucket
  pathPrefix: "uploads",
});

AzureBlobStorageProvider

import { AzureBlobStorageProvider } from "@circulo-ai/upload";

const blob = new AzureBlobStorageProvider({
  containerName: "my-container",
  accountName: "myaccount",
  accountKey: "KEY",

  // Or use connection string
  connectionString: "DefaultEndpointsProtocol=https;...",

  // Optional: Path prefix within container
  pathPrefix: "uploads",
});

LocalStorageProvider

import { LocalStorageProvider } from "@circulo-ai/upload";

const local = new LocalStorageProvider({
  basePath: "./uploads",

  // Optional: Path prefix within base path
  pathPrefix: "files",

  // Optional: Custom serve base URL
  serveBaseUrl: "/api/files",
});

Vercel Blob

import { VercelBlobStorageProvider } from "@circulo-ai/upload";
import type { NextRequest } from "next/server";

const storage = new VercelBlobStorageProvider({
  // Often unnecessary on Vercel; SDK uses BLOB_READ_WRITE_TOKEN by default
  // token: process.env.BLOB_READ_WRITE_TOKEN,
  pathPrefix: "uploads",
  multipart: true, // let Vercel handle big uploads
});

export async function POST(req: NextRequest) {
  const form = await req.formData();
  const file = form.get("file");
  if (!(file instanceof File)) {
    return new Response("file is required", { status: 400 });
  }

  const buffer = Buffer.from(await file.arrayBuffer());

  const info = await storage.upload({
    file: buffer,
    fileName: file.name,
    contentType: file.type,
  });

  return Response.json(info);
}

StorageManager

import { StorageManager } from "@circulo-ai/upload";

type Contexts = "primary" | "backup" | "cache";

const manager = new StorageManager<Contexts>({
  providers: {
    primary: s3Provider,
    backup: blobProvider,
    cache: localProvider,
  },
  defaultContext: "primary",
});

// Upload
await manager.upload({
  file: buffer,
  fileName: "file.pdf",
  contentType: "application/pdf",
  context: "primary", // Optional, uses default if omitted
  customKey: "custom-key", // Optional
  preserveKey: false, // Optional, skip timestamp prefix
  metadata: { userId: "123" }, // Optional
});

// Download
const buffer = await manager.download({
  key: "file-key",
  context: "primary",
});

// Delete
await manager.delete({
  key: "file-key",
  context: "primary",
});

// Presigned URLs (for cloud providers)
const { url, key } = await manager.generatePresignedUploadUrl({
  fileName: "upload.pdf",
  contentType: "application/pdf",
  fileSize: 1024000,
  context: "primary",
  expirationSeconds: 3600, // Optional
});

const downloadUrl = await manager.generatePresignedDownloadUrl({
  key: "file-key",
  context: "primary",
  expirationSeconds: 3600,
});

Multipart Uploads

For large files (>5MB recommended):

// 1. Initiate
const { uploadId, key } = await manager.initiateMultipartUpload({
  fileName: "large-file.zip",
  contentType: "application/zip",
  fileSize: 100 * 1024 * 1024, // 100MB
  context: "primary",
});

// 2. Get URLs for parts (e.g., 5MB chunks)
const partUrls = await manager.getMultipartPartUrls({
  uploadId,
  key,
  partNumbers: [1, 2, 3, 4], // Upload 4 parts
  context: "primary",
});

// 3. Upload parts (client-side)
const parts = await Promise.all(
  partUrls.map(async ({ partNumber, url }) => {
    const response = await fetch(url, {
      method: "PUT",
      body: partData[partNumber],
    });
    return {
      PartNumber: partNumber,
      ETag: response.headers.get("ETag"),
    };
  }),
);

// 4. Complete
const result = await manager.completeMultipartUpload({
  uploadId,
  key,
  parts,
  context: "primary",
});

// Or abort if needed
await manager.abortMultipartUpload({
  uploadId,
  key,
  context: "primary",
});

Route handlers, hooks, and errors

FileRouteHandler lets you plug in logging/analytics and get consistent error codes from the generated Next.js/Hono routes.

import { FileRouteHandler } from "@circulo-ai/upload";

const handler = new FileRouteHandler({
  storageManager: manager,
  hooks: {
    beforeUpload: (file, context) =>
      console.log("uploading", file.name, context),
    afterUpload: (upload, context) =>
      console.log("uploaded", upload.key, context),
    onError: (error, context) =>
      console.error("upload error", { error, context }),
  },
});

Errors are instances of UploadError (also exported) and the HTTP responses from createNextFileHandler / createHonoFileRoutes are shaped like:

{
  "error": "File size ...",
  "code": "FILE_TOO_LARGE",
  "details": { "maxSize": 104857600 }
}

Browser → API → Storage flow (presigned example)

// client (browser)
async function uploadFile(file: File) {
  const presign = await fetch("/api/files/presigned", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      fileName: file.name,
      contentType: file.type,
      fileSize: file.size,
    }),
  }).then((r) => r.json());

  await fetch(presign.presignedUrl, {
    method: "PUT",
    headers: presign.uploadHeaders,
    body: file,
  });

  return presign.fileInfo; // contains stable serve path + key
}

Custom file validation

Use the validateFile hook on FileRouteHandler to enforce your own rules. Return null to allow, or a FileValidationError/UploadError to block.

import { FileRouteHandler, UploadError } from "@circulo-ai/upload";

const handler = new FileRouteHandler({
  storageManager: manager,
  validateFile: ({ fileName, contentType, fileSize, context, phase }) => {
    if (fileSize > 50 * 1024 * 1024) {
      return new UploadError("FILE_TOO_LARGE", "Max 50MB");
    }
    if (context === "avatars" && !contentType.startsWith("image/")) {
      return {
        code: "UNSUPPORTED_FILE_TYPE",
        message: "Only images are allowed for avatars",
        supportedTypes: ["image/jpeg", "image/png", "image/webp"],
      };
    }
    return null;
  },
});

Security & Safety

Max size: default 100 MB (MAX_FILE_SIZE), enforced for uploads/presigns/multipart.
Type checks: optional MIME/extension validation (not content sniffing)
Path traversal: Local provider sanitizes keys and enforces base directory boundaries.
Header hygiene: Filenames sanitized before Content-Disposition; metadata sanitized per provider.
Public vs private: Presigned URLs give time-limited access; Local fallback uses server-side serve route.
Hooks: Use onError/beforeUpload to log/audit/deny suspicious uploads.

Validation Utilities

import {
  validateFileType,
  validateFileSize,
  formatFileSize,
  getFileExtension,
} from "@circulo-ai/upload";

// Validate file type
const error = validateFileType("document.pdf", "application/pdf");
if (error) {
  console.error(error.message);
}

// Validate size
const sizeError = validateFileSize(fileSize, 100 * 1024 * 1024);

// Format size
console.log(formatFileSize(1536000)); // "1.5 MB"

// Get extension
const ext = getFileExtension("photo.jpg"); // "jpg"

Next.js route handler

Add a catch-all route (e.g., app/api/files/[...path]/route.ts) and point both GET and POST to the generated handler.

import { LocalStorageProvider, StorageManager } from "@circulo-ai/upload";
import { createNextFileHandler } from "@circulo-ai/upload/next";

const handler = createNextFileHandler({
  storageManager: new StorageManager({
    providers: {
      uploads: new LocalStorageProvider({ basePath: "./uploads" }),
    },
    defaultContext: "uploads",
  }),
});

export const GET = handler;
export const POST = handler;

Available paths under that route:

POST /delete – delete by key (JSON: { key, context? })
POST /download – get a download URL (JSON: { key, name?, context? })
POST /presigned – single upload URL (JSON: { fileName, contentType, fileSize }, query type/context)
POST /presigned/batch – multiple upload URLs (JSON: { files: [...] }, query type)
POST /multipart?action=initiate|get-part-urls|complete|abort – multipart helpers
POST /upload – multipart form with file field
GET /serve/:key – stream file contents (optional context query)

If you keep a dedicated file for one route (for example, /presigned), pass defaultRoute: "presigned" when creating the handler.

Configuration Examples

AWS S3 with Environment Variables

const s3 = new S3StorageProvider({
  bucket: process.env.S3_BUCKET!,
  region: process.env.AWS_REGION!,
  // Credentials automatically loaded from environment or IAM role
});

Cloudflare R2

const r2 = new S3StorageProvider({
  bucket: "my-r2-bucket",
  region: "auto",
  endpoint: `https://${accountId}.r2.cloudflarestorage.com`,
  credentials: {
    accessKeyId: process.env.R2_ACCESS_KEY_ID!,
    secretAccessKey: process.env.R2_SECRET_ACCESS_KEY!,
  },
});

MinIO

const minio = new S3StorageProvider({
  bucket: "my-bucket",
  region: "us-east-1",
  endpoint: "http://localhost:9000",
  forcePathStyle: true,
  credentials: {
    accessKeyId: "minioadmin",
    secretAccessKey: "minioadmin",
  },
});

When This May Not Be a Fit

You only need a simple <input type="file"> + form POST
You want a hosted upload widget / UI
You need built-in virus scanning or media processing

License

MIT

Contributing

Contributions welcome! Please open an issue or PR.

Name		Name	Last commit message	Last commit date
Latest commit History 32 Commits
src		src
CHANGELOG.md		CHANGELOG.md
README.md		README.md
package.json		package.json
tsconfig.json		tsconfig.json
tsdown.config.ts		tsdown.config.ts
vitest.config.ts		vitest.config.ts

Folders and files

Latest commit

History

Repository files navigation

@circulo-ai/upload

Features

Installation

Core Concepts

Environment Compatibility

Quick Start

Single Storage Provider

Multiple Storage Contexts (Recommended)

API Reference

Storage Providers

S3StorageProvider

AzureBlobStorageProvider

LocalStorageProvider

Vercel Blob

StorageManager

Multipart Uploads

Route handlers, hooks, and errors

Browser → API → Storage flow (presigned example)

Custom file validation

Security & Safety

Validation Utilities

Next.js route handler

Configuration Examples

AWS S3 with Environment Variables

Cloudflare R2

MinIO

When This May Not Be a Fit

License

Contributing

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages