search_hub

at 6806854 Raw

# Changelog

All notable changes to this project are documented here.

The format is based on [Keep a Changelog](https://keepachangelog.com/),
and this project adheres to [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Added

- Cross-encoder reranker pipeline (`BAAI/bge-reranker-v2-m3`, ~2 GB) for higher tagging precision. Falls back to bi-encoder scores on failure. Configurable via `[tagging] reranker_enabled` and `reranker_model`.
- Negative examples support in tag definitions (`negatives` field in `[[tagging.tags]]`). Score formula uses mean-pooling: `pos_mean - neg_mean * 0.3`.
- `search_hub config-schema` command that prints the full JSON Schema for the config file format.
- `schemars` dependency for deriving `JsonSchema` on all config structs.
- abbaye script builder to auto-generate `search_hub.schema.json` during release.
- `$schema` property in `init-config` output pointing to the published schema URL.
- `override_tags` option for `[tagging]` to use only user-provided tags.
- `tag_weight` config option for controlling FTS5 tags column ranking weight.

### Changed

- Config restructured: `tagging_enabled` / `tagging_threshold` flattened keys moved into `[tagging]` section as `enabled` / `threshold`. Web server options moved to `[web]` section. Custom `[[tags]]` moved under `[tagging]` as `[[tagging.tags]]`.
- `TaggingEngine::new()` signature extended with `reranker_enabled` and `reranker_model` parameters.
- Tag scoring switched from max-pooling to mean-pooling (average consensus across examples, not best match).
- README rewritten with improved flow: quick start up top, CLI reference before auto-tagging, bloated inline config replaced with concise overview.
- `contrib/release.sh` now generates `search_hub.schema.json` during release.

## [0.7.0] - 2026-06-26

### Added
- `tagging` cargo feature (included by default) gating `fastembed`/ONNX Runtime. Build with `--no-default-features` to produce a ~30% smaller binary without semantic auto-tagging.
- Second abbaye `cargo` builder with `no_default_features = true` so release page offers both binaries.

### Changed
- Self-updater appends `-no-default` to the download URL when the running binary was compiled without the `tagging` feature.
- Pruned `tokio` from `features = ["full"]` to only needed features (`rt-multi-thread`, `macros`, `signal`, `sync`, `time`, `fs`).
- Pruned `actix-web` from default features to only `macros` + `compat-routing-macros-force-pub`.


## [0.6.0] - 2026-06-26

### Added
- Progressive search via SSE (`/api/search/stream`): local bookmark results appear instantly, external engine results stream in as they arrive, using `EventSource` with zero external dependencies.
- `?js=true` query parameter for the `/search` handler: skips external engines during page render (~50ms instead of ~1700ms). JS automatically inserts this into the URL via `history.replaceState`/`pushState`, enabling fast loading on all subsequent interactions.
- Integration tests for the SSE endpoint covering content-type, bookmark events, done events, empty queries, no-match queries, and bang redirects.


## [0.5.2] - 2026-06-24

### Fixed
- Removed nested Tokio runtime in `run_external_engines` to fix "Cannot start a runtime from within a runtime" panic on search.

### Added
- Full-stack integration tests for HTML and JSON search endpoints using `actix_web::test`.

### Changed
- Made `SearchApiResponse` fields public and added `Deserialize` for test use.
- Added `SearchApiResult::title()` convenience method.


## [0.5.1] - 2026-06-24

### Fixed
- Server now binds to both IPv4 (127.0.0.1) and IPv6 (::1) localhost addresses, fixing connection issues in browsers that prefer IPv6 (e.g. Zen Browser).


## [0.5.0] - 2026-06-23

### Added
- Bang (`!`) and at-sign (`@`) shortcuts: `!w query` redirects to Wikipedia, `@w query` searches only Wikipedia. Shortcodes auto-generated from configured engines, overridable per-engine or via `[[bangs]]` config.
- `shortcode`, `bang_url`, `bang_enabled` per-engine override fields.
- Configurable standalone bangs via `[[bangs]]` in `config.toml`.
- Dynamic shortcuts help in the web UI (press `?`).
- JSON API returns `{ type: "bang", redirect_url }` for bang queries.


## [0.4.2] - 2026-06-23

### Added
- *(none)*

### Fixed
- *(none)*

### Changed
- *(none)*


## [0.4.2] - 2026-06-23

### Fixed
- Robots.txt URL now includes port number (non-standard ports were silently hitting default port)
- MutexGuard no longer held across await points in Fetcher (potential deadlock under concurrent use)
- Clippy warnings resolved across the entire codebase (~60 fixes)
- Custom abbaye theme templates removed in favor of built-in defaults (added missing `.browse-link` CSS)

### Changed
- `run_import` now takes `&Config` instead of 12 individual parameters
- `Config` derives `Default` instead of manual implementation
- Fetcher runtime changed from `enable_time()` to `enable_all()` (required for reqwest IO)

### Added
- 3 Fetcher tests with local HTTP server: basic fetch, robots.txt blocking, cache behavior

## [0.4.1] - 2026-06-22

### Fixed
- Progress bar no longer overshoots total during import (separate bar for fetch+tag pass)
- Stack overflow in inner tokio runtime replaced with current-thread runtime + enabled timers
- Mutex poison resilience in Fetcher's `save_cache` and `fetch_async`


## [0.4.0] - 2026-06-21

### Added
- `self-update` CLI command: checks abbaye Atom feed for new releases, downloads and replaces the binary
- `contrib/search-hub-self-update.{service,timer}` for weekly automated updates
- Wikipedia search engine (configurable `lang`, defaults to English)
- MDN Web Docs search engine (configurable `locale`, defaults to en-US)
- Generic HTML-scraped search engine (user-provided URL template + CSS selector)
- Container build (`Containerfile`, `.dockerignore`) and push to `oci.vit.am`
- `docker-compose.yaml` for one-command deployment
- `contrib/search-hub.container` (Podman Quadlet) for systemd-native container management
- SIGHUP reload: re-read config and swap engines without restart

### Changed
- Engine config switched from `HashMap` to `Vec<EngineConfig>` tagged enum
- Config option `enabled_engines` removed (the engine list itself defines what's active)
- `reqwest` switched from `native-tls` to `rustls-tls` (no system OpenSSL required)

### Changed (CLI)
- `release.sh` builds container image with `buildah` as part of release process


## [0.3.0] - 2026-06-19

### Added
- Unit and integration tests for search engines (24 unit + 4 integration)
- `tagging_enabled` config option (default: `false`) to opt into auto-tagging

### Changed
- Search results from multiple engines are now interleaved round-robin
- abbaye.toml: fix `$shema` -> `$schema` typo, add `clone_url` for git UI

### Documentation
- Update README to reflect opt-in auto-tagging


## [0.2.3] - 2026-06-19

### Added
- SearXNG integration (optional, configurable in `[engines.searxng]`)


## [0.2.2] - 2026-06-19

### Fixed
- Align abbaye version_index template with abbaye v0.8 API (dist_files -> dist_categories)
- Add abbaye build and rsync to release script

### Documentation
- Update README with SearXNG mention, binary downloads, and repository links

## [0.2.1] - 2026-06-19

### Added
- Abbaye release site theme matching the search_hub web UI aesthetic
  - Monospace, earthy palette, dark/light mode
  - Covers all 7 Jinja2 templates + markdown

## [0.2.0] - 2026-06-19

### Added
- Release automation script (`contrib/release.sh`) with version bump, test, build, tag
- Engine filter badges with client-side hide/show by engine
- Keyboard shortcuts: `/` focus search, `j`/`` next, `k`/`` prev, `o`/`Enter` open, `?` toggle help, `Esc` close
- Help overlay showing all shortcuts
- `accesskey="s"` on search input
- `init-config` CLI command to generate config file
- `-c`/`--config` flag for CLI
- Async main with tokio
- Engine config refactored to `[engines.*]` map
- Single-pass import with channel-based error reporting

### Changed
- Results split into side-by-side local/external columns in web UI
- Slogan: "search_hub // personal web index"

## [0.1.0] - 2026-06-17

### Added
- CLI importer for Firefox, Zen, Chrome, Chromium bookmarks
- Web UI with full-text search (FTS5 via SQLite)
- crates.io integration via public JSON API
- Auto-tagging via local ONNX embeddings (fastembed)
- Single-pass import: insert, fetch content, HTML->MD conversion, tagging
- Bookmark database in `~/.local/share/search_hub/bookmarks.db`
- OpenSearch protocol support for Firefox/Zen
- Systemd service and timer for automated Zen imports