Skip to main content

Repository Structure and Adding Features

This document explains the project structure and how to add new features to Docusaurus without polluting the documentation or breaking maintainability.

Project Structure Overview​

TechStackMain/
├── tools/                          # Source of truth for tool docs
│   ├── *.md                        # One file per tool with frontmatter metadata
│   └── README.md                   # Tools directory guide
├── docusaurus/                     # Docusaurus site
│   ├── config/                     # Configuration files
│   │   └── sections.ts             # Sections configuration
│   ├── docs/                       # Documentation pages
│   │   ├── tools/                  # Auto-generated by category (DO NOT EDIT)
│   │   │   ├── website/            # Website category
│   │   │   │   ├── index.md        # Category landing page
│   │   │   │   └── *.md            # Tool pages
│   │   │   ├── self-hosting/       # Self-hosting category
│   │   │   ├── operational-tools/  # Operational tools category
│   │   │   └── ...                 # Other categories
│   │   ├── docusaurus/             # Docusaurus-specific docs
│   │   │   ├── features/           # Feature documentation
│   │   │   ├── index.md            # Docusaurus overview
│   │   │   ├── repo-structure.md   # This document
│   │   │   └── todo.md             # To-do list
│   │   ├── Guides/                 # Operational guides
│   │   └── *.md                    # Top-level pages
│   ├── src/                        # Source code
│   │   ├── theme/                  # Custom components
│   │   │   └── MDXComponents/      # MDX components
│   │   │       ├── FAQStructuredData/
│   │   │       ├── ToolMetadata/   # Tool metadata component
│   │   │       ├── CategorySummary/# Category summary component
│   │   │       ├── ToolsOverview/  # Tools overview component
│   │   │       └── SearchBar/      # Algolia search component
│   │   └── css/                    # Custom styles
│   ├── scripts/                    # Build scripts
│   │   ├── sync-tools.cjs          # Auto-sync with categorization
│   │   ├── sync-todo.cjs           # Sync TODO.md to Docusaurus
│   │   ├── validate-mdx.cjs         # MDX validator
│   │   └── index-algolia.cjs        # Algolia search indexing
│   ├── sidebars.ts                 # Sidebar configuration
│   └── docusaurus.config.ts        # Docusaurus configuration
├── scripts/                         # Automation scripts
│   ├── match-csv-to-tools.js       # Match CSV to workspace tools
│   ├── add-frontmatter-to-tools.js # Add metadata to tool files
│   └── resources/                  # Tool-specific resources
│       ├── tool-csv-mapping.json   # CSV mapping data
│       └── unmatched-tools.json    # Tools needing creation
├── architecture.md                  # Tech stack architecture
└── TOOL_CATEGORIZATION.md          # Tool categorization system docs

Key Principles​

1. Separation of Concerns​

Source Files (tools/*.md):

  • Edit these directly
  • Include frontmatter with metadata (category, costs, renewals, account types)
  • Auto-synced to categorised Docusaurus structure on build
  • One file per tool
  • Metadata synced from TwentyCRM CSV export

Generated Files (docusaurus/docs/tools/*/index.md and *.md):

  • ⚠️ NEVER EDIT - Auto-generated, overwritten on every build
  • Created by sync-tools.cjs script
  • Organised into category subdirectories (website/, self-hosting/, etc.)
  • Include Docusaurus frontmatter and ToolMetadata component
  • Category index pages auto-generated with CategorySummary component

Custom Pages (docusaurus/docs/*.md):

  • Standalone documentation pages
  • Not auto-synced
  • Manual frontmatter required
  • Examples: intro.md, architecture.md, tools.md

Feature Documentation (docusaurus/docs/docusaurus/features/):

  • Reusable components and features
  • Setup guides for new repos
  • Not tool-specific

2. Adding New Features​

When adding a new feature to Docusaurus (like the FAQ component):

  1. Create the component in docusaurus/src/theme/MDXComponents/
  2. Document the feature in docusaurus/docs/docusaurus/features/
  3. Update setup guide if it's a reusable feature
  4. Add to to-do list if it needs future work

Example: Adding FAQ Component

docusaurus/
├── src/theme/MDXComponents/
│   └── FAQStructuredData/          # Component code
│       ├── index.js
│       └── types.d.ts
└── docs/docusaurus/features/
    └── faq-structured-data.md      # Feature documentation

3. Adding Tool Documentation​

When documenting a new tool:

  1. Create tools/<tool-name>.md with frontmatter: 
    ---
    title: Tool Name
    category: WEBSITE
    categoryDisplay: Website
    monthlyCost: 25
    monthlyCurrency: GBP
    annualCost: 0
    annualCurrency: GBP
    renewalDate: null
    accountType:
      - INTERNAL
      - CLIENT_ACCESS
    accessLink: https://tool.com/dashboard
    moreInfo: Brief description
    ---
  2. Commit - Auto-sync runs on build, organises by category
  3. Verify - Check docusaurus/docs/tools/<category>/<tool-name>.md was created
  4. Never edit the generated file directly

See TOOL_CATEGORIZATION.md for full details on metadata fields and CSV integration.

4. Adding Custom Pages​

For standalone documentation pages:

  1. Create docusaurus/docs/<page-name>.md
  2. Add frontmatter: 
    ---
    id: page-name
    title: Page Title
    sidebar_position: 5
    ---
  3. Update sections config in docusaurus/config/sections.ts for top-level sections
  4. Update sidebar in docusaurus/sidebars.ts if needed (auto-generated from sections.ts)

5. Adding New Documentation Sections​

To add top-level sections like "Prompts", "Processes", or "Projects":

  1. Edit docusaurus/config/sections.ts
    • Add a new section entry or change status from 'planned' to 'active'
    • Set appropriate position (lower numbers appear first in sidebar)
    • Set landingPage to point to your index page (e.g., 'Projects/projects')
    • Set autogenerate: true to auto-generate sidebar items
  2. Create directory: docusaurus/docs/<section-name>/
  3. Create index page: docusaurus/docs/<section-name>/index.md
    • Add frontmatter with id matching the landingPage (without path)
    • Set sidebar_position: 0 to appear first in the section
  4. Create category config: docusaurus/docs/<section-name>/_category_.json
    • Use "type": "doc", "id": "<section-id>" to link to manual index
    • Or use "type": "generated-index" for auto-generated index
  5. Add content files to the directory
  6. Build - Sidebar automatically updates from sections config

Example from sections.ts:

{
  id: 'projects',
  label: 'Projects',
  dirName: 'Projects',
  status: 'active',
  position: 5,
  description: 'Internal projects and automation workflows',
  landingPage: 'Projects/projects',  // Points to docs/Projects/index.md with id: projects
  autogenerate: true,
}

Adding Sub-Projects to a Section:

For sections like "Projects" that contain multiple sub-projects:

  1. Create sub-project folder: docusaurus/docs/<section-name>/<project-name>/
  2. Create sub-project index: docusaurus/docs/<section-name>/<project-name>/index.md
    • Add frontmatter with unique id and sidebar_position (higher than section index)
  3. Create sub-category config: docusaurus/docs/<section-name>/<project-name>/_category_.json
    • Set "position" to match the sidebar_position in the index.md

Example structure for Projects section:

docusaurus/docs/Projects/
├── _category_.json                    # Section category config
├── index.md                            # Section landing page (id: projects, sidebar_position: 0)
└── cold-outreach-automation-flow/      # Sub-project
    ├── _category_.json                 # Sub-project category config (position: 2)
    └── index.md                        # Sub-project page (id: cold-outreach-automation-flow, sidebar_position: 2)

Feature Development Workflow​

Step 1: Plan the Feature​

  • Document the use case
  • Identify if it's reusable or one-off
  • Determine where it belongs (component, page, guide)

Step 2: Implement​

  • Create component/page in appropriate location
  • Follow existing patterns and conventions
  • Test locally with npm start

Step 3: Document​

  • Create feature documentation in docusaurus/docs/docusaurus/features/
  • Update setup guide if reusable
  • Add examples and usage instructions

Step 4: Integrate​

  • Update sidebar if needed
  • Add to relevant documentation
  • Test in production build

Step 5: Maintain​

  • Add to to-do list if future work needed
  • Document any gotchas or limitations
  • Keep examples up to date

Avoiding Documentation Pollution​

✅ Good Practices​

  • Feature docs go in docusaurus/docs/docusaurus/features/
  • Tool docs go in tools/ (auto-synced)
  • Guides go in docusaurus/docs/Guides/
  • Architecture goes in top-level architecture.md or docusaurus/docs/architecture.md

❌ Avoid​

  • Don't mix feature docs with tool docs
  • Don't create one-off pages for reusable features
  • Don't edit generated files
  • Don't duplicate documentation across locations

Maintaining Project Structure​

Regular Tasks​

  1. Review generated files - Ensure auto-sync is working
  2. Check for duplicates - Search before creating new docs
  3. Update to-do list - Track planned features and tasks
  4. Clean up - Remove deprecated features and docs

Automation Opportunities​

  • Auto-sync - Already automated ✅
  • MDX validation - Already automated ✅
  • Title extraction - Already automated ✅
  • Build and deploy - Already automated ✅

Future Automation Ideas​

  • Auto-generate tool index from tools/ directory
  • Auto-update last verified dates
  • Auto-check for broken links
  • Auto-format Markdown files

File Naming Conventions​

  • Tool docs: tools/<tool-name>.md (lowercase, hyphens)
  • Feature docs: docusaurus/docs/docusaurus/features/<feature-name>.md
  • Guides: docusaurus/docs/Guides/<guide-name>/
  • Components: docusaurus/src/theme/MDXComponents/<ComponentName>/

Version Control Strategy​

  • Main branch - Production-ready documentation
  • Feature branches - For new features or major updates
  • Pull requests - Review before merging
  • Auto-deploy - Cloudflare Pages builds on push to main