@silgi/blob

The first-party blob module. Add it to modules and set its options under the blob config key. Files of any size, with their type and metadata, on the disk, on S3 (or any S3 API: R2, MinIO) or on R2 through its binding. A body is streamed, never held whole in memory, except where get says so.

// silgi.config.ts
export default defineConfig({
  modules: ["@silgi/blob"],
  runtimeConfig: { s3Secret: "" },
  blob: {
    driver: "s3",
    bucket: "uploads",
    endpoint: "https://<account>.r2.cloudflarestorage.com",
    region: "auto",
    accessKeyId: "AKIA…",
    secretAccessKey: { runtimeConfig: "s3Secret" },
    dev: { driver: "fs" },
  },
});

default

The module, as defineModule({ meta: { name: "blob", configKey: "blob" } }). Its setup checks the options (a wrong value is a build error naming the key), picks the driver, refuses @silgi/blob/server in client code, and fails the build when the driver cannot run on the preset.

Without a blob key the store is fs at .data/blob on bun, node, static and under silgi dev, and the cloudflare-r2 binding BLOB on cloudflare. On vercel and netlify the build fails: @silgi/blob needs blob.driver on the vercel preset: its file system does not keep files; use "s3". A cloudflare build whose wrangler config (wrangler.json or wrangler.jsonc, with the cloudflare.env block) declares no r2_buckets entry for the binding fails: blob uses the cloudflare-r2 binding "BLOB", which the app's wrangler config does not declare; add an r2_buckets entry with binding "BLOB".

BlobConfig

type BlobConfig = (
  | { driver: "fs"; base?: string }
  | { driver: "s3"; bucket?: Secret; region?: Secret; endpoint?: Secret; accessKeyId?: Secret; secretAccessKey?: Secret;
      sessionToken?: Secret; virtualHostedStyle?: boolean; base?: string }
  | { driver: "cloudflare-r2"; binding?: string; base?: string }
) & { dev?: BlobConfig };
type Secret = string | { runtimeConfig: string };

s3 options not given come from S3_<NAME> or AWS_<NAME> in the environment (S3_BUCKET, AWS_SECRET_ACCESS_KEY, …); { runtimeConfig: "s3Secret" } reads runtimeConfig.s3Secret at the first request, and an unset one throws blob.secretAccessKey reads runtimeConfig.s3Secret, which is not set. A stream body that does not end within 10 MiB is written on s3 and cloudflare-r2 as a multipart upload in 10 MiB parts. dev replaces the config under silgi dev and in the prerender; cloudflare-r2 cannot run under silgi dev, so it needs a dev.

BlobObject, BlobListItem, BlobList, BlobPutOptions, BlobServeOptions

interface BlobObject { pathname: string; contentType: string; size: number; etag: string; uploadedAt: Date; customMetadata: Record<string, string> }
interface BlobListItem { pathname: string; size: number; etag: string; uploadedAt: Date }
interface BlobList { blobs: BlobListItem[]; cursor?: string; hasMore: boolean; folders?: string[] }
interface BlobPutOptions { contentType?: string; addRandomSuffix?: boolean; customMetadata?: Record<string, string> }
interface BlobServeOptions { cacheControl?: string }

etag is quoted, the HTTP form. A contentType is one type/subtype with optional ; name=value parameters, stored in lowercase type, subtype and parameter names; a list (image/png, text/html) or anything else throws TypeError: Invalid blob content type "image/png, text/html": use one type/subtype with optional ; parameters, from the option and from a Blob's own type alike. A customMetadata key holds a-z, 0-9 and -, a value printable ASCII with no leading or trailing space, and keys and values together hold at most 2048 bytes (S3's limit), so each one is the same on every driver; otherwise put throws Invalid customMetadata key "Author": use a-z, 0-9 and "-", Invalid customMetadata value for "k": use printable ASCII with no leading or trailing space or customMetadata is 2049 bytes; keys and values together may hold at most 2048 (S3's limit).

BlobStore

interface BlobStore {
  put(pathname: string, body: string | ArrayBuffer | Uint8Array | Blob | ReadableStream<Uint8Array>, options?: BlobPutOptions): Promise<BlobObject>;
  get(pathname: string): Promise<Blob | undefined>;
  head(pathname: string): Promise<BlobObject | undefined>;
  list(options?: { prefix?: string; limit?: number; cursor?: string; folded?: boolean }): Promise<BlobList>;
  delete(pathnames: string | string[]): Promise<void>;
  serve(event: SilgiEvent, pathname: string, options?: BlobServeOptions): Promise<Response>;
  presign(pathname: string, options: { method: "GET" | "PUT"; expiresIn?: number; contentType?: string }): Promise<string>;
}

A pathname is /-separated, a leading / dropped, at most 1024 bytes of UTF-8; an empty segment, ., .., a \, NUL or a lone surrogate throws TypeError: Invalid blob pathname "a/../b".

• put stores the body and returns the object; a put to an existing pathname replaces it. contentType defaults to the body's type (a Blob or File), else the type of the extension, else application/octet-stream. addRandomSuffix: true puts -<8 base36> before the extension (a/photo-k3j9x0q2.jpg).
• get returns a Blob, lazy on fs and read whole on s3 and cloudflare-r2; to answer a request with a large object, use serve. get and head return undefined for a missing pathname.
• list pages in pathname order: limit 1 – 1000 (default 1000), cursor the opaque value of the page before, folded: true groups what is below the next / into folders (each ending in /). Each entry is a BlobListItem, what every backend's listing gives in one request; head gives a pathname's type and metadata.
• delete of a missing pathname does nothing.
• serve answers GET and HEAD with the bytes streamed, content-type, content-length, etag, last-modified and accept-ranges: bytes. A matching If-None-Match, or without one an If-Modified-Since not before the upload, answers 304; one Range answers 206 with content-range, an unsatisfiable one 416 with content-range: bytes */<size>, several 200 with the whole body; an If-Range sends the whole body. Every answer carries content-security-policy: default-src 'none'; style-src 'unsafe-inline'; sandbox and x-content-type-options: nosniff, whatever the type and the app's security headers, so an uploaded page cannot run script or read the app's origin. cache-control is private, no-cache (no shared cache keeps the blob, a browser revalidates with the ETag); cacheControl replaces it, for example public, max-age=31536000, immutable for a pathname that is never written twice. The headers and the bytes are one version: the bytes are read only if the pathname still holds the ETag the headers carry (If-Match on s3, onlyIf on cloudflare-r2, the object file the metadata names on fs); a put in between makes serve read again, and after three tries it answers 503 Blob changed while it was read. A missing pathname throws a 404 Blob not found.
• presign returns a URL for one GET or PUT of the pathname straight on storage, so the bytes never pass the server; it is signed by presignS3 of silgi/server on every preset and valid expiresIn seconds, 1 to 604800 (default 3600; otherwise RangeError: presign expiresIn must be 1 to 604800 seconds (got 0)). It works on the s3 driver only, R2 through its S3 endpoint; fs and cloudflare-r2 throw presign is not available on the fs driver; use serve(event, pathname), or the s3 driver. A PUT always signs a type, so the client must send exactly that Content-Type and storage refuses another: contentType, or without it the type put would store (the extension's, else application/octet-stream). contentType is checked as put's and must be given in canonical form, the value that is signed: text/plain;charset=utf-8 throws TypeError: Invalid presign content type "text/plain;charset=utf-8": pass "text/plain; charset=utf-8", the form that is signed. A GET takes no contentType (TypeError: presign contentType is signed on a PUT only: a GET sends no body). The URL does not limit the size: head the object when the client says it is done, pass the result to ensureBlob, and delete it when the rule fails. A browser on another origin needs the bucket's CORS rules to allow the method.

@silgi/blob/server

useBlob

function useBlob(): BlobStore;

The app's store on the driver the build chose, opened at its first use. A catch-all route param is a string[] of the URL's segments, so a route serving a pathname joins it.

import { defineHandler } from "silgi/server";
import { useBlob } from "@silgi/blob/server";

export const file = defineHandler({
  method: "GET",
  route: "/files/**:pathname",
  raw: true,
  async handler({ pathname }, { event }) {
    await requireUser(event);
    return useBlob().serve(event, pathname.join("/"));
  },
});

ensureBlob

function ensureBlob(blob: Blob | BlobObject, options: { maxSize?: number | `${number}${"B" | "KB" | "MB" | "GB"}`; types?: string[] }): void;

Throws a 400 File size 9.2MB is over 8MB (the size in maxSize's unit, KB = 1024 B) or File type text/plain is not allowed (allowed: image). A types entry is a MIME type (image/png) or a top-level type (image). Without both it throws TypeError: ensureBlob needs maxSize or types. It takes a Blob or File, or the BlobObject that head returns (its contentType is the type checked), so an object uploaded on a presigned URL is checked after the upload. The type checked is the one the client sent, so it is not a safety check; serve sandboxes active types whatever it said.

handleMultipartUpload, MultipartUploadOptions

function handleMultipartUpload(event: SilgiEvent, options: MultipartUploadOptions): Promise<Response>;

interface MultipartUploadOptions {
  action: string;                                   // the route's :action param
  pathname?: (name: string) => string;              // the client's name → the stored pathname
  ensure?: { maxSize?: number | `${number}${"B" | "KB" | "MB" | "GB"}`; types?: string[] }; // ensureBlob's options
  put?: { addRandomSuffix?: boolean; customMetadata?: Record<string, string> };
  direct?: boolean;                                 // s3 only: parts go from the browser to storage
  expiresIn?: number;                               // seconds a direct part URL is valid, default 900
  secret: { runtimeConfig: string };                // the runtime config key of the secret that signs upload tokens
}

A file over one request's body limit, or one that needs progress, is sent in parts of 5 MiB or more, each its own request. A single request needs no helper: a handler with a File in its input is the upload, bound by its form.maxBodySize and the preset's request body limit (4.5 MB on vercel, 6 MB on netlify). One raw handler serves every step:

import { defineHandler } from "silgi/server";
import { handleMultipartUpload } from "@silgi/blob/server";

export const upload = defineHandler({
  method: "*",
  route: "/upload/:action",
  raw: true,
  async handler({ action }, { event }) {
    const user = await requireUser(event);
    return handleMultipartUpload(event, {
      action,
      pathname: (name) => `uploads/${user.id}/${name}`,
      ensure: { maxSize: "2GB" },
      direct: true,
      secret: { runtimeConfig: "uploadSecret" },
    });
  },
});

• pathname is the client's name; options.pathname maps it to the stored pathname on every step, so the handler decides where a user may write on each request. With put.addRandomSuffix the suffix is added to the name at create, and the pathname answered carries it.

• create needs size, the file's bytes (create needs size (the file's bytes)); ensure is checked there, before any part, with ensureBlob's 400 messages. The type stored is contentType, else the extension's, checked as put's. partSize is 5 MiB to 5 GiB (default 10 MiB) and the file at most 10000 parts (A file of 52434042880 bytes needs more than 10000 parts of 5242880 bytes). Through the function, partSize must also fit the request body limit (server.maxRequestBodySize, or 100 MB on cloudflare): create partSize 209715200 is over the request body limit of 100000000 bytes on the cloudflare preset; send smaller parts, or use direct: true.

• The uploadId is a token the server signs (HMAC-SHA-256) over the stored pathname, size, partSize, storage's id and an expiry 24 hours after create, with a key derived from runtimeConfig[secret.runtimeConfig]. Set it from the environment (runtimeConfig: { uploadSecret: "" } and SILGI_UPLOAD_SECRET), at least 32 characters; unset or shorter, every step answers 503 Upload endpoint is not configured and the server logs handleMultipartUpload needs runtimeConfig.uploadSecret (at least 32 characters). A literal throws TypeError: handleMultipartUpload secret must be { runtimeConfig: "<key>" }; a secret in the code ends up in the repository. Changing the secret ends the uploads in flight. Each part must hold exactly its share: partSize bytes, the rest of size for the last; a part through the function of other bytes is 400 (413 above its share), a direct part's size is signed into its URL as content-length, so storage refuses another. complete takes exactly the parts 1 to n. So the object that takes the pathname is always the size and type create checked: a refused upload never replaces what is at the pathname, and nothing refused is ever stored there.

• Another step answers 404 Unknown upload step "list", a step with the wrong method 405 The upload step "create" takes POST with allow, a malformed body, a part of the wrong size or past the last, or a part list storage refuses 400 Invalid upload request, an uploadId that does not verify, has expired or belongs to another pathname 404 Upload not found. abort of an upload that is gone answers 204. An error thrown by the app's pathname function is a 500 whose message is not sent, unless it is an HTTP error (createError), which answers as it is.

• Through the function s3 and cloudflare-r2 use their native multipart upload and fs keeps parts in <base>/multipart/<uploadId>/<n> until complete joins them or abort removes them; a folder older than 24 hours is dropped at the next create. On vercel and netlify a part leaves no room under the request body limit, so a call without direct throws handleMultipartUpload on the vercel preset needs direct: true: a part of at least 5 MiB is over the function's 4.5 MB request body (netlify: … is within 1 MB of the function's 6 MB request body).

• Direct (direct: true) needs the s3 driver (handleMultipartUpload direct: true needs the s3 driver (blob.driver is "fs")): create, complete and abort go through the handler, each part from the browser to storage on a URL signed by presignS3, so no part passes the function. The bucket's CORS rules must allow PUT from the app's origin and expose ETag; on S3, and in an R2 bucket's CORS policy:

  [{ "AllowedOrigins": ["https://app.example.com"], "AllowedMethods": ["PUT"], "AllowedHeaders": ["*"], "ExposeHeaders": ["ETag"], "MaxAgeSeconds": 3600 }]
  

• Uploads are not resumable: an upload cut by a reload or a closed tab is left for storage to expire (an S3 or R2 lifecycle rule for incomplete multipart uploads).

@silgi/blob/client

useMultipartUpload, MultipartUploadClientOptions, MultipartFileOptions, MultipartUpload

function useMultipartUpload(url: string, options?: MultipartUploadClientOptions): (file: File, options?: MultipartFileOptions) => MultipartUpload;

interface MultipartUploadClientOptions { partSize?: number; concurrency?: number }
interface MultipartFileOptions { pathname?: string; onProgress?: (sent: number, total: number) => void }
interface MultipartUpload { completed: Promise<BlobObject>; abort(): Promise<void> }

The client of handleMultipartUpload at url (/upload for the handler above): plain fetch with no UI library, the same in every adapter.

import { useMultipartUpload } from "@silgi/blob/client";

const upload = useMultipartUpload("/upload", { concurrency: 3 });
const { completed, abort } = upload(file, { onProgress: (sent, total) => console.log(sent / total) });
const object = await completed;

• It sends size: file.size, partSize and the file's type at create, then each part through the handler, or, when the upload answer carries a url, to that URL with PUT, reading its ETag header. A storage answer without a readable ETag rejects completed with Storage answered part 3 without a readable ETag; expose ETag in the bucket's CORS rules.
• partSize is 5 MiB to 5 GiB, S3's limits (default 10 MiB; the last part may be smaller); concurrency is the parts in flight (default 1). Otherwise it throws RangeError: useMultipartUpload partSize must be a whole number of bytes from 5 MiB to 5 GiB (got 1048576) or RangeError: useMultipartUpload concurrency must be a whole number from 1 (got 0). A file over 10000 parts rejects completed before any request: RangeError: A file of 52434042880 bytes needs more than 10000 parts of 5242880 bytes; raise partSize.
• pathname is the name sent at create, default file.name. onProgress(sent, total) is called after each part is stored, sent the bytes of the stored parts: fetch reports no progress inside a request.
• A part that failed on the network or with a 5xx is sent once more (a 4xx is not); then the upload is aborted and completed rejects with the handler's message (File size 2.0MB is over 1MB) and its status. abort() before complete is sent stops the parts in flight, calls the abort step and resolves when it answered; completed then rejects with an AbortError. Once complete is sent, abort() no longer stops the upload: it resolves when completed settles, and completed resolves with the object. abort() never rejects.