Plugin Development Guide

Introduction

This comprehensive guide walks you through extending Spry, a Markdown-based notebook system built on the unified ecosystem. Spry allows you to create powerful, executable documents by combining Markdown with custom processing logic.

What you'll learn:

How to create remark plugins that transform Markdown AST (Abstract Syntax Tree)
How to build task directive inspectors for custom executable cell types
How to integrate with Spry's event system for reactive programming
Best practices for type-safe, performant plugin development

Core Concepts

What is Spry?

Spry is a programmable notebook system that treats Markdown files as executable documents. It leverages:

unified/remark — Industrial-strength Markdown processing
Task Directives — Turn code blocks into executable tasks
Type Safety — Full TypeScript support with runtime validation
Event-Driven — React to execution events in real-time

Extension Points

Spry provides three main extension mechanisms:

FileCode

Remark Plugins

Process and enrich Markdown during parsing

Task Directive Inspectors

Define how code cells become executable

Radio

Event Bus Listeners

React to lifecycle events (start, complete, error)

Part 1: Remark Plugin Development

Remark plugins transform the Markdown Abstract Syntax Tree (AST) as documents are processed. They can add metadata, validate structure, or prepare content for execution.

Basic Plugin Structure

Every remark plugin follows this pattern:

import { Plugin } from "unified";
import { Root } from "types/mdast";

interface MyPluginOptions {
  option1?: string;
  option2?: boolean;
}

export const myPlugin: Plugin<[MyPluginOptions?], Root> = (options = {}) => {
  return function transformer(tree: Root) {
    // Transform the tree
  };
};

export default myPlugin;

Key Points:

Plugins are factory functions that return a transformer
The transformer receives the AST tree and modifies it in place
Options provide configuration flexibility
The Root type represents the top-level Markdown document

Traversing the AST with `visit`

The unist-util-visit utility makes tree traversal simple:

import { visit } from "unist-util-visit";
import { Code, Root } from "types/mdast";

export const myPlugin: Plugin<[], Root> = () => {
  return (tree) => {
    visit(tree, "code", (node: Code) => {
      // Process each code block
      console.log(`Found code block: ${node.lang}`);
    });
  };
};

Common node types:

"code" — Code blocks (fenced with backticks)
"paragraph" — Text paragraphs
"heading" — Headings (h1-h6)
"list" — Ordered and unordered lists
"link" — Hyperlinks

The visitor pattern lets you process specific node types without manually recursing through the tree.

Attaching Metadata to Nodes

Plugins often need to attach computed data to nodes for later use:

import { visit } from "unist-util-visit";

export const myPlugin: Plugin = () => {
  return (tree) => {
    visit(tree, "code", (node) => {
      // Initialize data object if needed
      node.data ??= {};

      // Attach custom metadata
      node.data.myCustomData = {
        processedAt: new Date().toISOString(),
        someComputation: computeSomething(node),
      };
    });
  };
};

Why attach data?

Downstream plugins can read this metadata
Task inspectors use it to determine executability
Renderers can use it for custom formatting

Type-Safe Node Data

Type safety prevents runtime errors and improves developer experience. Spry provides utilities for attaching validated data to AST nodes.

Using `nodeDataFactory`

This creates a typed accessor for node data:

import { nodeDataFactory } from "./lib/remark/mdast/safe-data.ts";

// Define your data type
interface MyMetadata {
  processed: boolean;
  value: number;
}

// Create a typed data accessor
const MY_KEY = "myMetadata" as const;
export const myMetadataNDF = nodeDataFactory<typeof MY_KEY, MyMetadata>(MY_KEY);

// Use in plugin
export const myPlugin: Plugin = () => {
  return (tree) => {
    visit(tree, "code", (node) => {
      // Attach typed data
      myMetadataNDF.set(node, {
        processed: true,
        value: 42,
      });
    });
  };
};

// Check and read data elsewhere
if (myMetadataNDF.is(node)) {
  const data = node.data.myMetadata; // TypeScript knows the type
  console.log(data.value); // 42
}

Benefits:

TypeScript autocompletion for data fields
Type checking at compile time
No need for type assertions

Using `safeNodeDataFactory` with Zod

For runtime validation, combine with Zod schemas:

import { z } from "@zod/zod";
import { safeNodeDataFactory } from "./lib/remark/mdast/safe-data.ts";

// Define schema
const myMetadataSchema = z.object({
  processed: z.boolean(),
  value: z.number().min(0),
});

type MyMetadata = z.infer<typeof myMetadataSchema>;

// Create validated data accessor
export const myMetadataSNDF = safeNodeDataFactory<"myMetadata", MyMetadata>(
  "myMetadata",
  myMetadataSchema,
  {
    onAttachSafeParseError: ({ node, error }) => {
      console.error("Validation failed:", error);
      return null; // Don't attach invalid data
    },
  }
);

When to use: Parsing metadata from node attributes, validating user-provided configuration, or ensuring data integrity across plugin boundaries.

Complete Plugin Example: Code Statistics

Let's build a practical plugin that calculates statistics for code blocks:

// lib/remark/plugin/node/code-stats.ts

import type { Code, Root } from "types/mdast";
import { Plugin } from "unified";
import { visit } from "unist-util-visit";
import { nodeDataFactory } from "../../mdast/safe-data.ts";

interface CodeStats {
  lineCount: number;
  charCount: number;
  hasShebang: boolean;
  language: string;
}

export const CODE_STATS_KEY = "codeStats" as const;
export const codeStatsNDF = nodeDataFactory<typeof CODE_STATS_KEY, CodeStats>(
  CODE_STATS_KEY
);

export interface CodeStatsOptions {
  onStats?: (node: Code, stats: CodeStats) => void;
}

export const codeStats: Plugin<[CodeStatsOptions?], Root> = (options = {}) => {
  const { onStats } = options;

  return function transformer(tree: Root) {
    visit(tree, "code", (node: Code) => {
      const value = node.value || "";
      const lines = value.split("\n");

      const stats: CodeStats = {
        lineCount: lines.length,
        charCount: value.length,
        hasShebang: value.startsWith("#!"),
        language: node.lang || "unknown",
      };

      // Attach to node
      codeStatsNDF.set(node, stats);

      // Optional callback
      onStats?.(node, stats);
    });
  };
};

export default codeStats;

Using the Code Statistics Plugin

import { remark } from "remark";
import codeStats, { codeStatsNDF } from "./code-stats.ts";

const processor = remark().use(codeStats, {
  onStats: (node, stats) => {
    console.log(`${stats.language}: ${stats.lineCount} lines`);
  },
});

const tree = processor.parse(`
\`\`\`bash
#!/usr/bin/env bash
echo "hello"
\`\`\`
`);

processor.runSync(tree);

// Access stats on nodes
visit(tree, "code", (node) => {
  if (codeStatsNDF.is(node)) {
    console.log(node.data.codeStats.hasShebang); // true
  }
});

Plugin Features:

Counts lines and characters
Detects shebang lines for executability
Tracks language for syntax highlighting
Provides callback for real-time processing

Part 2: Task Directive Inspectors

Task Directive Inspectors determine which code blocks become executable tasks and how they should run.

Understanding the Inspector Interface

type TaskDirectiveInspector<Provenance, Frontmatter, CellAttrs, I> = (
  ctx: {
    cell: PlaybookCodeCell<Provenance, CellAttrs>;
    pb: Playbook<Provenance, Frontmatter, CellAttrs, I>;
    registerIssue: (message: string, error?: unknown) => void;
  }
) => TaskDirective | false;

Context provided:

cell — The code cell being inspected
pb — The entire playbook (notebook) for cross-referencing
registerIssue — Report problems without throwing errors

Return values:

TaskDirective — Instructions for executing this cell
false — This inspector doesn't handle this cell (try next inspector)

Creating a Custom Inspector

Here's an inspector for a custom deno-task language:

// lib/task/deno-task-inspector.ts

import { TaskDirectiveInspector } from "./cell.ts";
import { languageRegistry } from "../universal/code.ts";

// Define the language spec
const denoTaskLangSpec = {
  id: "deno-task",
  extensions: [".ts"],
  comment: { line: ["//"], block: [] },
};

export function denoTaskInspector<Provenance>(): TaskDirectiveInspector<
  Provenance
> {
  return ({ cell, registerIssue }) => {
    // Only handle deno-task language
    if (cell.language !== "deno-task") {
      return false;
    }

    // Require a task identity
    const pi = cell.parsedPI;
    if (!pi?.firstToken) {
      registerIssue("deno-task cells require a task name");
      return false;
    }

    // Return the task directive
    return {
      nature: "TASK",
      identity: pi.firstToken,
      language: denoTaskLangSpec,
      deps: pi.flags.dep
        ? Array.isArray(pi.flags.dep)
          ? pi.flags.dep
          : [pi.flags.dep]
        : undefined,
    };
  };
}

Key Concepts:

nature: "TASK" — This cell should be executed
identity — Unique task name (from first token after language)
deps — Task dependencies (will wait for these to complete)
language — Defines syntax and execution environment

Registering Inspectors

Inspectors are checked in order until one returns a directive:

import { TaskDirectives, partialsInspector, spawnableTDI } from "./lib/task/mod.ts";
import { denoTaskInspector } from "./deno-task-inspector.ts";
import { fbPartialsCollection } from "./lib/markdown/notebook/mod.ts";

const partials = fbPartialsCollection();
const td = new TaskDirectives(partials, [
  partialsInspector(),      // Handle PARTIAL blocks first
  denoTaskInspector(),      // Our custom inspector
  spawnableTDI(),           // Default shell tasks
]);

Order matters: Earlier inspectors have priority. Return false to pass to the next inspector. The first inspector returning a directive wins.

Part 3: Event Bus Integration

Spry uses an event bus for reactive programming. Plugins can listen to task execution events.

Task Execution Events

import { eventBus } from "./lib/universal/event-bus.ts";
import { TaskExecEventMap } from "./lib/universal/task.ts";

// Create or get the task bus
const tasksBus = eventBus<TaskExecEventMap>();

tasksBus.on("task:start", ({ task, ctx }) => {
  console.log(`[${ctx.runId}] Starting: ${task.taskId()}`);
});

tasksBus.on("task:complete", ({ task, result }) => {
  console.log(`Completed: ${task.taskId()}`);
  console.log(`Exit code: ${result.exitCode}`);
});

tasksBus.on("task:error", ({ task, error }) => {
  console.error(`Failed: ${task.taskId()}`, error);
});

Shell Execution Events

For more granular control, listen to shell events:

import { ShellBusEvents } from "./lib/universal/shell.ts";

const shellBus = eventBus<ShellBusEvents>();

shellBus.on("spawn", ({ command }) => {
  console.log(`Spawning: ${command}`);
});

shellBus.on("stdout", ({ data }) => {
  process.stdout.write(data);
});

shellBus.on("stderr", ({ data }) => {
  process.stderr.write(data);
});

Use cases:

Real-time output streaming
Progress tracking
Performance monitoring
Debugging and logging

Testing Plugins

Thorough testing ensures plugins work correctly and don't break under edge cases.

Unit Test Structure

// lib/remark/plugin/node/code-stats_test.ts

import { assertEquals, assert } from "@std/assert";
import { remark } from "remark";
import codeStats, { codeStatsNDF } from "./code-stats.ts";
import { visit } from "unist-util-visit";

Deno.test("codeStats plugin", async (t) => {
  const processor = remark().use(codeStats);

  await t.step("calculates line count", () => {
    const md = "```bash\nline1\nline2\nline3\n```";
    const tree = processor.parse(md);
    processor.runSync(tree);

    visit(tree, "code", (node) => {
      assert(codeStatsNDF.is(node));
      assertEquals(node.data.codeStats.lineCount, 4);
    });
  });

  await t.step("detects shebang", () => {
    const md = "```bash\n#!/usr/bin/env bash\necho hi\n```";
    const tree = processor.parse(md);
    processor.runSync(tree);

    visit(tree, "code", (node) => {
      assert(codeStatsNDF.is(node));
      assertEquals(node.data.codeStats.hasShebang, true);
    });
  });

  await t.step("tracks language", () => {
    const md = "```python\nprint('hi')\n```";
    const tree = processor.parse(md);
    processor.runSync(tree);

    visit(tree, "code", (node) => {
      assert(codeStatsNDF.is(node));
      assertEquals(node.data.codeStats.language, "python");
    });
  });
});

Running Tests

# Run all tests
deno test --parallel --allow-all

# Run specific test file
deno test lib/remark/plugin/node/code-stats_test.ts --allow-all

# Watch mode for development
deno test --watch --allow-all

Testing best practices: Test edge cases (empty code blocks, missing languages), use descriptive test names, group related tests with t.step, and test both success and failure paths.

Best Practices

Plugin Design Principles

Single Responsibility

Each plugin should do one thing well

✅ Good: codeStats only calculates statistics

❌ Bad: Plugin that calculates stats AND executes code

Idempotent

Running a plugin multiple times should be safe

Check if data already exists before recomputing
Don't accumulate state across runs

Type-Safe Data

Always use nodeDataFactory or safeNodeDataFactory

Prevents typos in property names
Enables autocompletion
Catches errors at compile time

Options Pattern

Accept configuration via options object

Provides sensible defaults
Makes plugins flexible
Documents available configuration

No Side Effects

Plugins should be pure transformers

Don't write files directly
Don't make network requests
Emit data for later processing

Error Handling

Use registerIssue for recoverable errors:

if (!isValid(cell)) {
  registerIssue("Cell is missing required field");
  return false;
}

Use safeNodeDataFactory with error handlers:

onAttachSafeParseError: ({ error }) => {
  console.warn("Invalid data:", error);
  return null; // Don't attach
}

Only throw for truly unrecoverable situations:

if (systemResourcesExhausted()) {
  throw new Error("Cannot continue");
}

Performance Optimization

Early Return — Check node type early in visitors

visit(tree, "code", (node) => {
  if (node.lang !== "javascript") return; // Fast rejection
  // Expensive processing...
});

Avoid Reprocessing — Check if data already exists

if (myDataNDF.is(node)) {
  return; // Already processed
}

Batch Operations — Collect during visit, process after

const nodes = [];
visit(tree, "code", (node) => nodes.push(node));
// Now process all at once with optimized algorithm
processBatch(nodes);

GitHub Issues — Report bugs or request features
Discussions — Ask questions and share ideas
Discord — Real-time chat with the community
Documentation — Keep this guide handy as a reference

Happy plugin development! 🚀

How is this guide?