Skip to content

node-fs.mdx

Latest commit

 

History

History
479 lines (373 loc) · 14 KB

node-fs.mdx

File metadata and controls

479 lines (373 loc) · 14 KB
title Node:FS
sidebarTitle node:fs
description Read and write files in Edge Scripts using the Node.js-compatible file system API
No breaking changes are expected, but additional features will be implemented soon.

Overview

Edge Scripting provides support for the Node.js file system API through the node:fs and node:fs/promises modules. This allows you to read, write, and manipulate files and directories within your Edge Scripts.

The file system API is based on Deno's sandboxed environment with Node.js compatibility, supporting both the modern Promise-based API (node:fs/promises) and the traditional callback-based API (node:fs).

We recommend using the Promise-based API for cleaner async/await syntax.

Virtual File System

Each script has now access to a Virtual File System that allow to handle file based operations.

This file system lives on the Virtual Memory available of the worker.

Currently the file directory is composed of two folder:

/ # Root directory
├── home/
│   └── user/ # Your current working directory, R/W access
└── tmp/ # Temporary directory, R/W access

You can create directories where you want as permissions are not enforced yet. All files are opened in w+.

We suggest you to avoid creating directory in root directory `/` as we are going to create new folders with differents functions, like `/dev/*` and other read only directories.

In the Virtual File System, paths are limited to 4096 chars, for instance foo/bar is 7 characters. It's also limited to 48 segments (a/b/c is 3 segments).

Capacity checks for creating, copying and moving are done before the operation. This can result in trying to copy a large file resulting in an out of memory error, even though there is space left to create new files or directories.

Importing the Module

EdgeScripting supports both the modern Promise-based API and the traditional callback-based API.

Promise-based API (Recommended)

import * as fs from "node:fs/promises";

// Use with async/await
const data = await fs.readFile("/tmp/file.txt", "utf-8");

Callback-based API

import * as fs from "node:fs";

// Use with callbacks
fs.readFile("/tmp/file.txt", "utf-8", (err, data) => {
  if (err) {
    console.error("Error:", err);
    return;
  }
  console.log("Data:", data);
});

Limitations

EdgeScripting's file system implementation operates in a sandboxed environment with limitations. Understanding these constraints is essential for building reliable applications.

Always handle file system errors gracefully. The sandboxed environment may behave differently from traditional Node.js environments.

Symbolic Links and Hard Links

Symbolic links and hard links are not supported in the EdgeScripting runtime yet.

The following operations will throw errors or behave unexpectedly: - `symlink()`, `symlinkSync()` - Creating symbolic links - `readlink()`, `readlinkSync()` - Reading symbolic links - `link()`, `linkSync()` - Creating hard links - `lstat()` may not distinguish between files and symlinks correctly

Workaround: Use copyFile() instead of creating links, or restructure your application to avoid link dependencies.

File Statistics - Timestamps

File modification and access times are not reliably available yet.

The following `Stats` object properties may return incorrect or placeholder values: - `atime` - Last access time - `mtime` - Last modification time - `ctime` - Last status change time - `atimeMs`, `mtimeMs`, `ctimeMs` - Millisecond timestamps - `birthtime`, `birthtimeMs` - File creation time

Reliable Stats properties:

  • size - File size in bytes
  • isFile() - Check if entry is a file
  • isDirectory() - Check if entry is a directory

Example of unreliable usage:

// DON'T rely on timestamps
const stats = await fs.stat("file.txt");
console.log(stats.mtime); // May be incorrect!

// DO rely on size and type checks
console.log(stats.size); // Reliable
console.log(stats.isFile()); // Reliable

File and Directory Permissions

All files and directories effectively have read/write permissions in the sandboxed environment (644) and are owned by the current (non root) user (uid=1000, gid=1000).

Permission-related operations have limited effect: - `chmod()`, `chmodSync()` - Not supported - `chown()`, `chownSync()` - Not supported - `Stats.mode` - Won't reflect actual permissions - Permission checks via `access()` - Will mostly tell you if a file exists or not.

File Watching

File system watching is not supported.

The following operations are unavailable: - `watch()` - Watch for file changes - `watchFile()` - Poll for file changes - `unwatchFile()` - Stop watching - `FSWatcher` class

Workaround: Use polling with stat() if you need to detect changes, but be mindful of performance implications and CPU time limits.

Performance Considerations

**Memory limits:** The whole Virtual FileSystem lives inside your script memory. The current file system limits of your script is set up to 64MB. Reading large files may cause memory exhaustion if you store it in memory. Leverage streaming to avoid allocating too much memory at a time.

CPU time: File I/O counts toward the 30-second CPU time limit per request.

Best practices:

  • Stream large files instead of reading entirely into memory
  • Clean up temporary files to avoid storage bloat
  • See Limits for more details on resource constraints

Quickstart

Here are practical examples showing common file system patterns in EdgeScripting.

Example 1: Reading a File

import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

BunnySDK.net.http.serve(async (request: Request) => {
  try {
    // Read file as UTF-8 string
    const content = await fs.readFile("/tmp/data.txt", "utf-8");

    return new Response(content, {
      headers: { "content-type": "text/plain" }
    });
  } catch (error) {
    return new Response(`Error reading file: ${error.message}`, {
      status: 500
    });
  }
});

Example 2: Writing a File

import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

BunnySDK.net.http.serve(async (request: Request) => {
  const data = await request.json();

  try {
    // Write JSON data to file
    await fs.writeFile(
      "/tmp/output.json",
      JSON.stringify(data, null, 2),
      "utf-8"
    );

    return new Response("File written successfully", { status: 201 });
  } catch (error) {
    return new Response(`Error writing file: ${error.message}`, {
      status: 500
    });
  }
});

Example 3: Working with Directories

import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

BunnySDK.net.http.serve(async (request: Request) => {
  try {
    // Create directory (recursive option creates parent directories)
    await fs.mkdir("/tmp/myapp/data", { recursive: true });

    // Write a file in the new directory
    await fs.writeFile("/tmp/myapp/data/config.json", "{}", "utf-8");

    // List directory contents
    const files = await fs.readdir("/tmp/myapp/data");

    return new Response(JSON.stringify({ files }), {
      headers: { "content-type": "application/json" }
    });
  } catch (error) {
    return new Response(`Error: ${error.message}`, { status: 500 });
  }
});

Example 4: Error Handling Patterns

import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

BunnySDK.net.http.serve(async (request: Request) => {
  const filepath = "/tmp/cache/data.txt";

  try {
    // Check if file exists by trying to access it
    await fs.access(filepath);

    // File exists, read it
    const content = await fs.readFile(filepath, "utf-8");
    return new Response(content, {
      headers: { "x-cache": "hit" }
    });
  } catch (error) {
    if (error.code === "ENOENT") {
      // File doesn't exist, create it
      const defaultContent = "Default cached data";
      await fs.writeFile(filepath, defaultContent, "utf-8");

      return new Response(defaultContent, {
        status: 201,
        headers: { "x-cache": "miss" }
      });
    }

    // Re-throw other errors
    throw error;
  }
});

Example 5: Using FileHandle for Chunked Reading

This is not an optimized script at all. This is more to show an example of chunked reading, please avoid using something similar in production. 
import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

// Global promise to track the download operation
let downloadPromise: Promise<void> | null = null;
const filePath = "/tmp/large-file.txt";

async function downloadLoremIpsum() {
  try {
    const response = await fetch('https://lorem-api.com/api/lorem', {
      method: 'GET',
      headers: {
        'Content-Type': 'application/json'
      }
    });

    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }

    const content = await response.text();
    
    // Calculate how many times we need to repeat to reach ~10MB
    const targetSize = 10 * 1024 * 1024; // 10MB in bytes
    const contentSize = Buffer.byteLength(content, 'utf-8');
    const repeatCount = Math.ceil(targetSize / contentSize);
    
    // Create the repeated content
    let finalContent = '';
    for (let i = 0; i < repeatCount; i++) {
      finalContent += content;
    }
    
    // Write to file
    await fs.writeFile(filePath, finalContent, 'utf-8');
  } catch (error) {
    // Clear the promise on error so it can be retried
    downloadPromise = null;
    throw error;
  }
}

async function ensureFileExists() {
  try {
    // Check if file exists
    await fs.access(filePath);
  } catch (error) {
    // File doesn't exist, initiate download
    if (!downloadPromise) {
      // Start download and store the promise
      downloadPromise = downloadLoremIpsum();
    }
    // Wait for the download to complete (whether we started it or another request did)
    await downloadPromise;
  }
}

BunnySDK.net.http.serve(async (request: Request) => {
  try {
    // Ensure file exists (download if necessary, blocking until complete)
    await ensureFileExists();

    // Create SSE stream
    const stream = new ReadableStream({
      async start(controller) {
        const encoder = new TextEncoder();
        let fileHandle;
        
        try {
          // Open file for reading
          fileHandle = await fs.open(filePath, "r");
          
          // Get file stats to know the total size
          const stats = await fileHandle.stat();
          const totalSize = stats.size;
          
          // Send initial event with file info
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({
            type: "start",
            totalSize,
            chunkSize: 1024,
            timestamp: new Date().toISOString()
          })}\n\n`));
          
          const buffer = new Uint8Array(1024);
          let position = 0;
          let chunkNumber = 0;
          
          // Read file 1KB at a time
          while (position < totalSize) {
            const { bytesRead } = await fileHandle.read(buffer, 0, 1024, position);
            
            if (bytesRead === 0) break; // End of file
            
            // Convert to string
            const content = new TextDecoder().decode(buffer.slice(0, bytesRead));
            
            // Send chunk as SSE event
            controller.enqueue(encoder.encode(`data: ${JSON.stringify({
              type: "chunk",
              chunkNumber,
              bytesRead,
              position,
              content,
              progress: ((position + bytesRead) / totalSize * 100).toFixed(2) + "%",
              timestamp: new Date().toISOString()
            })}\n\n`));
            
            position += bytesRead;
            chunkNumber++;
            
            // Small delay to prevent overwhelming the client
            await new Promise(resolve => setTimeout(resolve, 10));
          }
          
          // Send completion event
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({
            type: "complete",
            totalChunks: chunkNumber,
            totalBytesRead: position,
            timestamp: new Date().toISOString()
          })}\n\n`));
          
        } catch (error) {
          // Send error event
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({
            type: "error",
            message: error.message,
            timestamp: new Date().toISOString()
          })}\n\n`));
        } finally {
          // Close file handle
          await fileHandle?.close();
          controller.close();
        }
      }
    });
    
    return new Response(stream, {
      headers: {
        "Content-Type": "text/event-stream",
        "Cache-Control": "no-cache",
        "Connection": "keep-alive",
        "X-Accel-Buffering": "no"
      }
    });
    
  } catch (error) {
    return new Response(`Error: ${error.message}`, { status: 500 });
  }
});

References