SearchHub

A local search engine for your browser bookmarks and history. Import bookmarks/history from Firefox, Zen, Chrome, or Chromium, search them with full-text queries, and optionally forward searches to external engines like Wikipedia or SearXNG (aggregates results from dozens of backends). Content can be automatically tagged via a two-stage ONNX pipeline — a fast bi-encoder scores all tags, then a cross-encoder reranker refines the top candidates (opt-in; see auto-tagging section).

The ONNX models are embedded via the optional tagging feature (included by default). If you don't need auto-tagging, you can build a ~30% smaller binary by excluding it.

Quick start

Prerequisites: Rust (or grab a pre-built binary).

git clone https://vit.am/~ololduck/search_hub/repository.git search_hub
cd search_hub
cargo install --path .

Then import your browser bookmarks and start the web UI:

search_hub import firefox     # or "chrome", "zen"
search_hub serve              # open http://127.0.0.1:8080

Smaller build (no ONNX tagging):

cargo install --path . --no-default-features

CLI reference

All commands use ~/.local/share/search_hub/bookmarks.db by default. Override with --db-path or set db_path in the config file.

Web UI

Search queries are forwarded to external engines: Wikipedia, crates.io, and optionally SearXNG if configured. Works as a custom search provider in Firefox/Zen via OpenSearch (auto-discovered at /opensearch.xml).

When JavaScript is available, results load progressively: local bookmark results appear instantly, then external engine results stream in as each engine responds. The server supports a ?js=true parameter that skips external engines during the initial page render (~50ms instead of ~1700ms), letting the client fetch them via a streaming connection instead. The inline JavaScript automatically adds ?js=true to the URL after the first load.

For the fastest load from your browser's search bar (before JS runs), configure the search engine URL with ?js=true:

• Firefox: Settings → Search → Manage Search Engines → Add, use http://localhost:8080/search?q=%s&js=true
• OpenSearch template: http://localhost:8080/search?q={searchTerms}&js=true

Without ?js=true, the server waits for all external engines before sending the page — a fallback for browsers without JavaScript (still works, just slower on first load).

Configuration

Run search_hub init-config to create ~/.config/search_hub/config.toml with all available options commented out. The config supports:

• [tagging] — auto-tagging settings (enable, threshold, model selection, reranker)
• [[engines]] — external search engines (SearXNG, Wikipedia, MDN, crates.io, generic)
• [[bangs]] — custom search shortcuts
• [[tagging.tags]] — custom tag definitions with optional negatives
• [web] — server bind address, page size, worker threads
• exclude_urls — hosts to skip when fetching content
• db_path — bookmark database location

See search_hub config-schema for the complete JSON Schema of the config file format.

Search shortcuts

Shortcodes are compact aliases auto-generated from configured [[engines]].

Auto-generated shortcodes

Override shortcode, bang_url, or bang_enabled per engine:

[[engines]]
type = "wikipedia"
lang = "fr"
shortcode = "wikifr"       # overrides "wfr"
bang_enabled = false       # disable ! redirect (still searchable via @)

Custom bangs (standalone ! redirects without @ support):

[[bangs]]
trigger = "gh"
url = "https://github.com/search?q={}"
name = "GitHub"

If two engines share a shortcode, SearchHub panics at startup naming both. Set shortcode on one to resolve.

Auto-tagging

When [tagging] enabled = true, SearchHub automatically tags bookmarks during insert and import using a two-stage ONNX pipeline:

1. Bi-encoder — BAAI/bge-small-en-v1.5 (~33 MB) embeds both the bookmark content and each tag's example set. Scores use mean-pooling (average similarity across all tag examples), requiring consensus rather than a single match.

2. Cross-encoder reranker — BAAI/bge-reranker-v2-m3 (~2 GB, optional) re-ranks the top 10 candidates for higher precision. Falls back to bi-encoder scores on failure. Disable with reranker_enabled = false to save disk space.

Tags can include negative examples. The score is pos_mean - neg_mean * 0.3, so negative matches penalize a tag without dominating. Tags without negatives are unaffected. A confidence threshold ([tagging] threshold, default: 0.6) controls which tags are applied.

Tag taxonomy

SearchHub ships with 21 built-in tags. Add or override them with [[tagging.tags]] in config.

Custom tags with negatives

[[tagging.tags]]
name = "my-special-tag"
examples = ["example text one", "example text two"]
negatives = ["something similar but different", "another close miss"]

Disk space

The bi-encoder model is ~33 MB. Enabling the reranker adds ~2 GB (~2.1 GB total in cache). Set reranker_enabled = false to minimize disk usage. The first tagging command downloads models to ~/.cache/search_hub/.

Deployment

Systemd user services

Web server (auto-start on login):

VERSION=$(search_hub --version | cut -d\  -f2)
mkdir -p ~/.config/systemd/user
wget -O ~/.config/systemd/user/search-hub-web.service \
  https://vit.am/~ololduck/search_hub/repository/browse/v$VERSION/contrib/search-hub-web.service
systemctl --user daemon-reload
systemctl --user enable --now search-hub-web.service

Auto-import (daily Firefox/Zen bookmark import):

wget -O ~/.config/systemd/user/search-hub-import.service \
  https://vit.am/~ololduck/search_hub/repository/browse/v$VERSION/contrib/search-hub-import.service
wget -O ~/.config/systemd/user/search-hub-import.timer \
  https://vit.am/~ololduck/search_hub/repository/browse/v$VERSION/contrib/search-hub-import.timer
systemctl --user daemon-reload
systemctl --user enable --now search-hub-import.timer

Auto-update (weekly release check):

wget -O ~/.config/systemd/user/search-hub-update.service \
  https://vit.am/~ololduck/search_hub/repository/browse/v$VERSION/contrib/search-hub-update.service
wget -O ~/.config/systemd/user/search-hub-update.timer \
  https://vit.am/~ololduck/search_hub/repository/browse/v$VERSION/contrib/search-hub-update.timer
systemctl --user daemon-reload
systemctl --user enable --now search-hub-self-update.timer

Check status with systemctl --user status search-hub-web. View logs with journalctl --user -u search-hub-web -f.

Podman / Docker

A container image is available at oci.vit.am/search-hub:latest (port 8080, runs as search_hub user).

podman run -d --name search-hub \
  -p 8080:8080 \
  -v ~/.config/search_hub:/home/search_hub/.config/search_hub:ro \
  -v ~/.local/share/search_hub:/home/search_hub/.local/share/search_hub \
  oci.vit.am/search-hub:latest serve

SIGHUP reloads the config without restarting: podman kill -s HUP search-hub

Or use docker compose up -d (see docker-compose.yaml at the project root, includes a commented SearXNG service). For systemd-native container management, see the Quadlet file at contrib/search-hub.container.

Contributing

Contributions are welcome! This is a personal project, but bug reports and feature requests are genuinely appreciated -- they help make search_hub better for everyone.

The best way to contribute is to report a bug or describe an unsupported use case. Pull requests are welcome too, though for larger changes it's a good idea to reach out first to discuss the approach. The project uses a develop branch for active work, with main reserved for releases.

I'm reachable via mail, IRC (ololduck on Libera.Chat), or the Fediverse (@ololduck@vit.am).

Resources

• Downloads: vit.am/~ololduck/search_hub/latest
• Repository browser: vit.am/~ololduck/search_hub/repository
• Git clone: git clone https://vit.am/~ololduck/search_hub/repository.git search_hub

Changes in 0.7.1

Changed

• Restructured flat config fields into hierarchical [tagging] and [web] sections.
• Lowered default tagging threshold from 0.6 to 0.35 for better tag coverage.
• Cross-encoder reranker disabled by default.

Fixed

• Excluded URLs (e.g. localhost) are now skipped during the first import pass instead of only during content fetching, preventing unwanted entries from entering the database.