documentation/ARCHITECTURE.md · main · 2026-10-13

GitSocial Architecture

GitSocial is one Go library that implements GITMSG.md, with thin clients on top. The CLI and the TUI call it directly, JSON-RPC serves it over stdio, and the static site reads the bucket that it publishes to over the S3 remote.

Development · Code Rules · Directory Structure · Package Reference · Cache · TUI

Development

Branching and builds

Development is trunk-based. main is the integration branch and stays linear; each feature is on a feature/<name> branch, rebased on main and merged fast-forward.

git switch main && git pull --ff-only                    # refresh before branching and after any merge
git switch -c feature/<name>
git fetch origin && git rebase main                      # keep current with main
git switch main && git merge --ff-only feature/<name>    # integrate
git branch -d feature/<name>
git worktree add .local/worktrees/<name> -b feature/<name> main   # a parallel branch in its own directory
git worktree remove .local/worktrees/<name>                       # after the merge
go build -o bin/gitsocial ./cli/gitsocial     # the CLI
go build -o bin/ ./...                        # compile-check everything; mains land in bin/, never the repo root
bin/gitsocial tui

Test and lint

scripts/check.sh is the gate, in two tiers: the quick tier (--quick) and the full tier. The pre-push hook runs the quick tier on each push that carries code; run the full tier before you merge to main and at release. TESTING.md tells what each tier runs.

scripts/check.sh --quick                    # the push tier
scripts/check.sh                            # the full tier
git config core.hooksPath scripts/hooks     # install the pre-push hook, once per clone

TESTING.md has the six stages and the failures of each, the guarded tests, the coverage ratchet with child processes, the environment variables and the artifact paths. Each baseline is a ratchet: a count can go down but not up, and --update accepts a lower count.

Design notes

A branch that changes core/objstore, core/site, core/gitmsg or core/cache, or that changes what readers see during concurrent writes, where data is stored, or what the protocol writes, adds three steps to the branch flow.

A change to the appearance of the site needs no note; it follows the rules in STATIC-SITE.md.

Go back to the note, not to another fix, when one of these is true:

Definition of done

A change is done when each of these lines is true; a review checks them in order.

Working in a session

One set of rules for every session.

Code Rules

Layers

cli/gitsocial
  library/tui, library/rpc
    library/client, library/import, library/proposals
      library/extensions/*
        library/core/*
          stdlib and the modules in go.mod

Each layer imports only the layers below it. core imports nothing above itself. Two standing exceptions:

Inside core the packages form a stack, and each imports only what is below it: log; protocol, text, result; cache, git; storage, gitmsg, identity; settings, notifications; fetch; objstore, search; site. scripts/import-graph.sh --check reads this sentence and fails on an edge against it. A sub-package sits in its parent's tier, and an edge between a sub-package and its parent is exempt. With no argument the script prints the current edges.

Do

Never

PackageMutable package-level state
core/gitthe executor seam, the command timeout, the s3 helper alias memo
core/gitmsgthe per-workdir caches
core/cachethe database singleton, the extension schema registry
core/logthe process logger
core/notificationsthe provider registry
core/identitythe in-flight map, the DNS policy flag
core/identity/forgethe forge registry
core/objstorethe two credential warn-once flags
extensions/reviewthe fetched-refs memo
tui/tuicorethe theme struct, the registries (contexts, views, cards, nav targets, message handlers), the width-margin flag
tui/tuicore/diffthe background-suppression flag

Patterns

Core packages (cache, git, protocol)  return error
Extension public API (social.*)       return Result[T], built with result.Ok and result.Err
Internal helpers                      return error

Directory Structure

gitsocial/                     # module github.com/gitsocial-org/gitsocial
├── cli/gitsocial/             # the CLI; builds the binary
├── library/
│   ├── core/
│   │   ├── git/               # git operations
│   │   ├── protocol/          # GitMsg parsing and refs
│   │   ├── gitmsg/            # protocol-level storage (config refs, lists, forks, push)
│   │   ├── cache/             # SQLite
│   │   ├── storage/           # bare repo management
│   │   ├── objstore/          # S3 client and the s3:// remote helper
│   │   ├── site/              # the static site: its assets, artifacts and pages
│   │   ├── fetch/             # fetch orchestration and processing
│   │   ├── identity/          # identity verification; forge/ holds the forge adapters
│   │   ├── notifications/     # notification aggregation
│   │   ├── search/            # cross-extension search
│   │   ├── settings/          # user settings and config paths
│   │   ├── log/, text/, result/
│   ├── extensions/            # social, pm, release, review, memo
│   ├── proposals/             # cross-repo proposals: accept and decline
│   ├── import/                # forge import; github/ and gitlab/ adapters
│   ├── client/                # the fetch and push sequences the thin clients run
│   ├── rpc/                   # JSON-RPC server
│   ├── tui/                   # TUI
│   └── internal/testutil/     # shared test fixtures
├── documentation/
├── scripts/                   # check.sh, prose-check.sh, import-graph.sh, test.sh, coverage.sh, release.sh, install.sh, site-test.sh, site-coverage.sh
└── specs/

Outside the tree:

~/.config/gitsocial/           # honors XDG_CONFIG_HOME
├── credentials.json           # S3 credentials per endpoint host
└── personal/                  # the personal bare repo: settings and personal memos (GITSOCIAL_PERSONAL_REPO overrides)

~/.cache/gitsocial/            # --cache-dir overrides
├── cache.db                   # SQLite
├── repositories/              # bare clones of followed repositories
├── forks/                     # bare clones of registered forks
├── imports/                   # import mapping files, one per repository URL
└── memo/session/              # session memo repos

Package Reference

PackageKey TypesKey Exports
core/git
Git operations
Commit, FileDiff, Hunk, DiffLine, DiffStatsGetCommits, CreateCommit, ReadRef, WriteRef, GetDiff, WorkingStatus, GetWorkingFileDiff, StageFiles, UnstageFiles, CommitIndex, GetFileDiff, GetFileContent, ListTree, ObjectType, ListTags, TagOrder, CompareTagsDesc, CountCommits, GetDiffStats, MergeBranches, SquashMerge, RebaseMerge, ForceMerge, RebaseBranch, RangeDiff, PatchesEqual, GetBehindCount, GetMergeBase, GetUserName, GetGitConfig, GetCommitSignerKey
core/protocol
Message parsing
Header, Message, Origin, Trailer, RefSpan, MentionSpanParseMessage, ParseHeader, CreateHeader, FormatMessage, ParseRef, CreateRef, FormatShortRef, QuoteContent, ApplyOrigin, ExtractTrailers, Trailer, FindRefs, FindMentions
core/cache
SQLite operations
Repository, Commit, TrailerRefOpen, DB, ExecLocked, QueryLocked, InsertCommits, FilterUnfetchedCommitsByRepo, MarkCommitsStaleByRepo, MarkCommitsStaleByHome, GetCommitOnAnyBranch, ResetRepositoryData, ToNullString, ToNullInt64, GetTrailerRefsTo, TrailerRef, GetRepositoryBranches
core/gitmsg
Protocol-level storage
ResolveRepoURL, Push, ReadExtConfig, WriteList, GetHistory, GetExtBranch, IsBranchConfigurable, IsExtInitialized, GetForks, AddFork, AddForks, RemoveFork
core/storage
Bare repo management
EnsureRepository, GetStorageDir, FetchRepository
core/objstore
S3 remote
Client, Config, HelperEnv, Progress, LocalCommitSource, PushOutcome, PostPushHook, SiteOverrideNewClient, ClientForRemote, ParseS3URL, RunHelper, HelperEnvFromOS, ListRemoteRefs, ReadRemoteRefs, RebuildRefManifest, LogDumbTransportInfo, RefsHeadDigest, ReadPackedObject, ThinUpstreamURL, CompressJSON, ReadCompressedJSON, PutCompressed, UploadConcurrency, RunParallel, PushArtifactObjects, PutObjectToRemote
core/site
Static site
SiteCustomization, ForkPull, ForkPullSource, ForkPullAreaRebuild, PostPushMaintenance, SetRemoteHead, WriteSiteStats, ReadWorkspaceSiteCustomization, WriteWorkspaceSiteCustomization, NormalizeSiteURL, NormalizeSiteImage, NormalizeSiteGlobs, ValidSiteAccent
core/fetch
Fetch orchestration
FetchAll, FetchRepository, FetchForks, FetchForkBranches, CommitProcessor, PostFetchHook, DefaultBranch
core/settings
User settings
Get, Set, ListAll
core/search
Cross-extension search
Search, Params, Result, Item, Group, GroupedItem, FormatResult, IsValidGroupBy
core/result
Result type
Result[T], ErrorOk, Err, ErrWithDetails
core/notifications
Notification aggregation
Notification, Provider, FilterRegisterProvider, GetAll, GetUnreadCount, MarkAsRead, MarkAsUnread, MarkAllAsRead, MarkAllAsUnread, MentionProcessor, ExtractMentions, TrailerProcessor
core/identity
Identity verification
Identity, ResolvedIdentity, DNSIdentity, Binding, Source, VerifyCandidateVerifyBinding, IsVerified, IsVerifiedCommit, LookupBinding, VerifyCandidates, NormalizeSignerKey, NormalizeEmail, ResolveIdentity
core/identity/forge
Forge adapters
Forge, GPGKey, CommitVerificationForge, Register, Lookup, LookupForRepo, ParseRepoURL, NewGitHub, GPGKey, CommitVerification
extensions/social
Posts, lists, timeline
Post, SocialItemGetPosts, CreatePost, CreateComment, GetComments, Fetch
extensions/pm
Issues, milestones, sprints
Issue, Milestone, Sprint, PMNotificationGetIssues, CreateIssue, GetMilestones, GetSprints, MessageToPMItem, FetchRepository, Processors
extensions/release
Releases
Release, ReleaseItem, ReleaseNotificationCreateRelease, EditRelease, GetReleases, GetSingleRelease, MessageToReleaseItem, FetchRepository, Processors
extensions/review
Pull requests, feedback
PullRequest, Feedback, ReviewSummary, StackEntry, ReviewNotificationCreatePR, GetPR, UpdatePR, MergePR, ClosePR, RetractPR, MarkReady, ConvertToDraft, UpdatePRTips, SyncPRBranch, GetPRVersions, ComparePRVersions, GetVersionAwareReviews, CreateFeedback, GetReviewSummary, MessageToReviewItem, GetPullRequests, GetPullRequestsWithForks, OpenForkPulls, OpenForkHeads, FetchMissingForkHeads, ResolvePRDiff, GetStack, RebaseStack, Processors
extensions/memo
Memos across tiers
Memo, MemoItem, Tier, SessionInfoCreateMemo, EditMemo, RetractMemo, PromoteMemo, ListMemos, GetSingleMemo, InitProject, InitPersonal, InitSession, ListSessions, GCSession, PushPersonal, FetchPersonal, PushSession, FetchSession, SyncAllTierReposToCache, AddInherit, RemoveInherit, ListInherits, IsInherited
client
Fetch and push sequences
FetchOptions, Options, Result, SiteOutcomeFetch, SyncWorkspace, ResolveRemotes, Preview, Push, PushAll, PublishSite, DeleteTrackingRefs
proposals
Cross-repo proposals
OutcomeAccept, Decline
import
Forge import
SourceAdapter, Stats, MappingFileRun, SourceAdapter, ReadMapping, WriteMapping, MappingKey, ResolveHost, MapLabels
import/github
GitHub adapter
New, CheckGH, Adapter.FetchPM, Adapter.FetchReleases, Adapter.FetchReview, Adapter.FetchSocial
TermContextMeaning
originalGITSOCIAL fieldthe post being commented on, reposted or quoted
canonicalversioningthe first version of a message
rawversioningthe commit's own message, before any edit applies
editsGITMSG fieldthe reference to the canonical version an edit replaces
publishthe sitethe site step of a push, guarded by the site.publish config key
rebuildthe sitethe verb of the site step: it uploads the shell and the artifacts, and sends no refs

A push is the sequence: the data push, then the site step after it.

Cache

You can delete the storage under repositories/ at any time; GitSocial uses cache.db, not the storage, to decide what to fetch. The cache is append-only. A commit that leaves its source branch (rebase, force-push) is stale: cache.MarkCommitsStale or MarkCommitsStaleByRepo sets its stale_since. In the workspace, a row is stale when its branch is not the home of its commit, and MarkCommitsStaleByHome sets the value. Timeline and list queries exclude stale commits, and thread and detail views show them dimmed.

SQLite: WAL, 64 MB page cache, temp store in memory, 16 connections, 256 MB mmap.

Schema

Every extension table is keyed by (repo_url, hash, branch) into core_commits and carries the extension's prefix. Column lists are in core/cache/db.go and each extension's schema.go.

core_commits.edits stores the raw header value; core_commits_version is authoritative by repository and hash. Use cache.ResolveToCanonical and cache.GetLatestVersion.

Edit resolution applies only to same-repository edits (GITMSG.md §1.5), so a cross-repository edit is a proposal that has no effect until the owner accepts it. proposals.Accept writes an accepting edit: a same-repository edit of the owner that carries accepts=<proposal>. The accepting edit has priority in resolution, and its processing writes the core_edit_acceptances row. proposals.Decline publishes a marker at refs/gitmsg/core/declines/*. Both clear the marker of the proposer, and accept has priority over decline.

A change to an item of a registered fork adopts the item and does not make a proposal (GITMSG.md §1.5). The first change writes a copy on the branch of this repository, and the copy carries adopts=<original> and a GitMsg-Ref: reference section with the author of the original. Each later change edits the copy.

Resolved views

core_commits has the generated columns effective_message, effective_author_name, effective_author_email and effective_timestamp, which use the content of the latest edit and the origin fields in place of the raw values. Each extension has a <ext>_items_resolved view that joins its table onto core_commits and projects those columns under the display names:

CREATE VIEW {ext}_items_resolved AS
SELECT c.effective_message AS resolved_message, c.effective_author_name AS author_name, c.effective_timestamp AS timestamp, ...,
       COALESCE(e.type, 'default') AS type, e.field1, ...
FROM core_commits c
LEFT JOIN {ext}_items e ON c.repo_url = e.repo_url AND c.hash = e.hash AND c.branch = e.branch;

The denormalized columns resolved_message, has_edits and is_retracted are written only by applyEditToCanonical in core/cache/versions.go.

core_commits.action is the timeline action of a commit, NULL for none. The commit insert writes it for a first version, and applyEditToCanonical writes it for each edit of a canonical on each pass, so the value does not depend on the order of arrival. Only the timeline admits an edit row, through cache.TimelineItemFilter.

RuleTest
The action of an edit is the same for each order of arrivalTestAction_orderOfArrival
A text, label or assignee edit has no actionTestAction_textEditHasNone
A proposal and a retraction have no actionTestAction_proposalAndRetractionHaveNone
A merge, a ready mark and a review have their actionTestAction_pullRequestAndReview
The timeline has the item at its creation time and each action at its own timeTestTimeline_actionEntries
A stale row of an action is in no timelineTestTimeline_excludesStaleAction
A merge that closes an issue gives one entryTestTimeline_mergeHidesIssueClose
The actions of a retracted item are in no timelineTestTimeline_excludesActionOfRetractedItem
No list other than the timeline shows an edit rowTestGetPMItems_excludesActionRows
The site app derives the same action as the cacheTestParityActions, unit_timeline_actions.js

Use the view when the WHERE clause is on core_commits columns. Join core_commits to the extension table directly when the WHERE clause is selective on extension columns (pm_items.state = 'open') or the query is a recursive CTE over extension relationships; otherwise, the planner scans core_commits. social.GetComments, and the thread and notification readers inside social, are the examples.

Refs and keys

References are [repo_url]#<type>:<value>: https://github.com/user/repo#commit:abc123def456 or, workspace-relative, #commit:abc123def456. Types: commit, branch, tag, file, list.

A repository has two names:

A repository has an identity, and a person has a verified binding; do not use the word "identity" for a third concept.

A virtual commit is a commit that a GitMsg-Ref trailer references and that is not fetched yet. It is stored with is_virtual = 1 and full metadata, and a fetch of the commit changes the value to 0.

State refs under refs/gitmsg/:

Each element has its own ref, so concurrent adds from several clones do not collide. Metadata is at _meta. Git refuses a child ref under a parent ref that has the same name.

Fork refs sit outside refs/gitmsg/, keyed by fetch.URLHash of the fork identity:

Fetch rules

RepositoryCacheStorage
Workspacefull history, local branches and originthe workspace directory
Followed with #branch:*full history, all branchespersistent
Followed on one branchfull history, incrementalpersistent
Not followeda 30-day windowmay be deleted at any time

All-branch following stores each commit under its real refname. Deduplication and stale marking of a followed repository work per repository through FilterUnfetchedCommitsByRepo and MarkCommitsStaleByRepo. Switching a repository between one branch and * runs cache.ResetRepositoryData, and the next fetch rebuilds the data of the repository.

Workspace home branch

The workspace sync stores each commit under one branch, its home, and the home is a function of the current refs. A cache built from empty has the same live rows as a cache that followed each change. core/fetch/home.go computes the home, and SyncWorkspaceOrigin and each SyncWorkspace* function ingest through it, with the workspace sync function of each extension and the mention and trailer processors.

RuleValueTest
Walked refsrefs/heads/* and refs/remotes/origin/*; no other remote, no tag, no state refTestWorkspaceSync_otherRemoteAddsNothing, TestWorkspaceSync_skipsStateRefs
Logical branchthe ref name without refs/heads/ or refs/remotes/origin/TestWorkspaceSync_contentBranchHome
Default branchthe HEAD of origin, then main, then master, then the first code branch by nameTestDefaultBranch_ignoresCheckout
Homethe first branch that reaches the commit: the default branch, then each gitmsg/* branch by name, then each other branchTestHomeBranch_precedence
Order of the other branchesa branch whose tips another branch reaches comes first; unrelated branches sort by nameTestHomeBranch_stackAncestorFirst
Live rowthe row whose branch is the home; each other row of the hash is staleTestWorkspaceSync_mergeMovesHome, TestWorkspaceSync_selfHeals
Read markercopied to the new home when a row goes staleTestMarkCommitsStaleByHome_readMarkerFollowsTheCommit
Gatethe names, tips and symrefs of the walked refs, in core_sync_tipsTestWorkspaceSync_gateEqualsWalkSet
Stable branchesthe default branch and the gitmsg/* branches; the sync reads only the commits after their last tipsTestWorkspaceSync_rewindTakesTheFullPath
Full comparisonon a first build, a tip from older code, a stable branch that lost a commit, or a new set of stable branches; it reads each rowTestWorkspaceSync_rewindTakesTheFullPath, TestSyncTip_oldFormatTriggersSync
Tipadvances only when each commit has its home rowTestWorkspaceSync_tipNeedsFullState
Finalizethe stale marks and the tip are written only if no walked ref moved during the syncTestWorkspaceSync_refChangeDuringSyncSkipsFinalize
Origin syncingests with the sync functions in Options.WorkspaceSyncs; with none, the gate stays open for the callerTestSyncWorkspaceOrigin_runsTheWorkspaceSyncs
Rebuildthe live rows equal those of a cache built from emptyTestWorkspaceSync_rebuildEqualsIncremental
Lookup of a commitby repository and hash, the live row firstTestGetCommitOnAnyBranch_prefersLive
Reference targetmatched by repository and hash, the live row first; the stored reference keeps its branchTestGetComments_movedCommit, TestRecount_movedTarget, TestGetTrailerRefsTo_branchlessTrailer, TestResolveRefLocation
Edit resolutionthe canonical is matched by repository and hash, and its resolved state is on each row of the hashTestApplyEdit_everyRowOfTheHash, TestInsertCommits_editAppliesAcrossBranch, TestSyncWorkspace_editSurvivesMerge
Stale sourceno mention notification, no trailer notification and no trailer referenceTestMentionProvider_excludesStaleCommit, TestGetTrailerRefsTo_excludesStaleSource
Stale rowin no list, count or notification; a lookup by hash takes the live row firstTestGetPMItems_excludesStaleCommit, TestGetPMItemByHashPrefix_liveFirst, TestNotifications_excludesStaleComment

A different repository, a mirror or the upstream of a fork, is not workspace content when it is only a git remote. It gets into the cache through a list or a fork registration, under its own URL.

Extension rules

TUI

Two panels on Bubbletea: navigation on the left, content on the right. Keys are in TUI-KEYS.md, layouts in TUI-DIAGRAMS.md, the headless test suite in TUI-TESTS.md.

library/tui/
├── app.go, host.go      # the tea.Model, view dispatch, shared state
├── tuicore/             # infrastructure: utils, components, registries, the message bus
├── tuiviews/            # the core routable views
├── tuinav/              # navigation items and the registry extensions register into
├── tuisocial/, tuipm/, tuirelease/, tuireview/, tuimemo/, tuiproposal/
├── tuikeydoc/           # keybinding documentation generator
└── test/                # headless integration tests
PrefixPurposeExample
view_a routable view, in tuiviews/ or an extension's tui<ext>/view_timeline.go
component_a reusable stateful componentcomponent_nav_panel.go
registry_a global registryregistry_nav_target.go
form_a modal formform_issue.go
version_item_a history-picker version itemversion_item_issue.go
util_helpers, and the shared app state they take: State, Router, themeutil_render.go

The table governs library/tui/*/, not test/ and not the tuicore/diff/ sub-package. A component_ renders and takes keys; shared app state stays in a util_ file.

A new extension gets a tui/tui<ext>/ directory with its view_*.go files and a util_register.go exposing Register(host), called from app.go.