Incremental file indexes
IncrementalFileIndexComponent<T> from @mnaoumov/obsidian-lib/components/incremental-file-index-component keeps a consumer-defined projection of each accepted file. It subscribes before loading a saved cache, reuses records whose mtime and size still match, and reads the other files sequentially. It handles vault create, modify, delete and rename events, including folder subtrees, and uses text supplied by metadata-cache changed events directly.
Pass app, an accepts(file) predicate, a synchronous indexFile({ file, content }) projection, onSet(path, record) and onRemove(path) notifications for your secondary index, and asynchronous load() and save(records) callbacks. The saved map holds IndexedFileRecord<T> values with mtime, size and record. Treat records as immutable. The consumer owns the schema validation, storage location, corruption recovery and any domain-wide state, such as the names an unlinked-mentions index has already searched for. Device-local derived indexes belong outside the synced vault; see Plugin data stores.
Attach the component to the lifetime that owns the index and await loadWithPromises() before answering queries. If the projection needs metadata, first wait for ensureMetadataCacheReady(app) from @mnaoumov/obsidian-lib/metadata-cache, in the owner’s layout-ready callback. The component does not impose metadata readiness on projections that need only text. isBuilt reports completion of startup reconciliation. A load failure leaves it false and never overwrites a saved cache with a partial build.
Pending reads invalidate their previous record immediately, notifying onRemove. Queries must include accepted files absent from the index as conservative candidates until onSet makes their new projection usable. Newer events, renames, deletions, stat changes and unloads invalidate older read completions. Stat matching cannot detect an external rewrite that preserves both modification time and size. Use rescan() to rebuild projections for every file when the projection’s inputs change, such as when an unlinked-mentions index learns new names; the consumer owns scan markers and coordinates overlapping domain scans.
Two optional parameters remove reads a consumer does not need. shouldAwaitMetadataChange(file) returns true for files whose create and modify events Obsidian follows with a metadata-cache changed event carrying their text, as for Markdown notes. Such an event then drops the stale projection and reads nothing, and the changed event restores it. A modify that leaves the content unchanged raises no changed event, so that file stays without a record until the next startup or rescan(); the candidate rule above keeps queries correct meanwhile. Set shouldMoveRecordsOnRename when a projection depends on text alone. A rename then moves the record with its stats, for a file or a whole folder, in whichever order Obsidian raises the folder’s and its descendants’ events. A file renamed to a path accepts rejects loses its record, and a renamed file with no known record is read. A read that fails after its file was renamed or deleted is discarded; one that fails for a file still in place rejects as before.
loadWithPromises() loads a component that was unloaded again, as Obsidian’s load() does. When the owner adds the index with addChild while loaded, the child starts loading there, so check the owner is still loaded before awaiting the child’s loadWithPromises(); otherwise an unload during the child’s load() callback restarts it.
Changes request a save on a five-second debounce that is not postponed by continuous edits. flush() serializes writes and follows a successful write with another snapshot if records changed while it was pending. A failed write rejects without marking its revision saved; a later flush retries it. getRecords() returns a shallow map snapshot. Unload cancels the timer, unregisters events, stops adopting pending reads and requests a final flush only for a completed index. Await flush() explicitly before teardown when the caller needs a persistence guarantee; Obsidian’s synchronous unload cannot await it.
Obsidian’s word characters and case folding
Section titled “Obsidian’s word characters and case folding”@mnaoumov/obsidian-lib/obsidian-word-characters exports isObsidianWordCharacter(codeUnit), tokenizeObsidianWords(text), getObsidianNameTokens(name) and createObsidianMentionRegExp(name). These preserve the fixed word-character class and boundary behavior of Obsidian’s unlinked mentions. A name with no word tokens, or whose case folding crosses the word boundary class, returns null from getObsidianNameTokens; keep those queries on the full-scan path. The mention regex captures the literal name in group 1.
canonicalizeObsidianText(text) canonicalizes literal text for the non-Unicode ignore-case regexes used by both core Search and unlinked mentions. It preserves UTF-16 length and accents. Greek final sigma and sigma canonicalize together; Kelvin sign does not become ASCII K, long s does not become ASCII S, and sharp s does not expand to SS. It is neither locale folding nor Unicode normalization. Do not apply it to regex source: regex syntax and character classes need their own analysis. Core Search still decides whether a candidate matches, including case-sensitive subqueries, filename matches and canvas content.