Tools

Tools: Report: Samba, Spotlight & Manticore

2026-04-13 0 views admin
Tools: Report: Samba, Spotlight & Manticore

Part 1 — The Journey

The Goal

The Context: Proxmox, LXC, and Alpine Linux

Manticore Search: A Lightweight Elasticsearch

The First Problem: spotlight.so Does Not Exist on Alpine

Migrating to Debian 13

The Technical Problems, One by One

1. The Missing Mappings File

2. Manticore Does Not Support Field Names With Dots

3. The prefix Query Is Not Supported on Full-Text Fields

4. Elasticsearch's query_string Syntax Is Not Compatible

5. json_loadb failed: the Content-Length

6. Accented Characters and ensure_ascii

7. The Nested Structure of path.real

The Python Proxy: The Solution

The Result

Part 2 — Practical Guide

Prerequisites

Architecture

CT Manticore — Installation

CT Samba/Debian — Installation

Final Verification I have a NAS built on Proxmox with LXC containers. The idea was simple: share files via Samba and be able to search them from macOS Finder as if they were on a local disk, using Spotlight. Search for "Battiato" and find all the MP3s, PDFs, text files — everything indexed, everything reachable in a second. Samba has supported Apple's Spotlight protocol since version 4.x, with an option called spotlight backend that allows delegating indexing to an external engine. Three backends are supported: noindex (no search), tracker (GNOME Tracker), and elasticsearch. None of these was exactly what I was looking for. My setup uses Proxmox as a hypervisor with LXC containers for services. The NAS initially ran on an Alpine Linux container — lightweight, fast, perfect for a fileserver. The first obstacle came right away: I want Spotlight search, so I need to choose a backend. Tracker is out of the question on Alpine — it depends on D-Bus, the GNOME stack, systemd, none of which is practically available on Alpine with musl libc. Elasticsearch is heavy and requires a JVM. But there is an alternative. Manticore Search is a full-text search engine compatible with the Elasticsearch REST API, written in C++. It is much lighter, requires no JVM, and has excellent support for text indexing. The idea was to use it as the Elasticsearch backend for Samba, in a separate LXC container. The two-container architecture makes sense: the search service is separate from the fileserver, they can be updated independently, and Manticore can be reached by multiple shares if needed. After configuring Samba with spotlight backend = elasticsearch and adding vfs objects = spotlight to the share, the result was: The Samba package on Alpine does not include the VFS module for Spotlight. It is not in the repositories, there is no separate package, and compiling it from source on Alpine with musl libc is an inadvisable adventure. I recreated the NAS container on Debian 13 (Bookworm). Samba 4.22 on Debian is compiled with Spotlight support — you can see this from the smbd -b output which shows WITH_SPOTLIGHT and HAVE_SPOTLIGHT_BACKEND_ES. The module is compiled statically into smbd, not as a separate .so, so it does not need to be added to vfs objects. The first error after the migration: Samba looks for a JSON file that maps Spotlight attributes (such as kMDItemFSName, kMDItemTitle) to Elasticsearch fields. The file was not included in the Debian package. Solution: download it from the official Samba repository and place it at the correct path. Samba sends queries with fields like path.real, file.filename, meta.title. Manticore does not accept field names containing dots — not as SQL, not with backticks, in no way. Solution: rename the fields in the index (path_real, file_filename, etc.) and create an HTTP proxy that translates the names in both requests and responses. Samba filters results by share using a prefix query on the path.real field. Manticore does not support this type of query on full-text fields. The proxy must intercept and remove this filter. Samba sends queries like "query": "Fran* OR content:Fran*" with "fields": ["file.filename", "content"]. Manticore does not understand the content:Fran* syntax inside a query_string. The proxy must rewrite the query into Manticore's format: @file_filename Fran* | @content Fran*. Even with a correct JSON response, Samba kept throwing json_loadb failed. The problem was that the proxy was not including the Content-Length header in the response. Samba reads exactly N declared bytes — without Content-Length, parsing fails. Python by default serializes non-ASCII characters as Unicode sequences (È → E\u0300). Samba does not handle NFD (decomposed) form correctly. Solution: json.dumps(data, ensure_ascii=False). The final error: Missing path.real in JSON result. Looking at the Samba source code reveals that the parser expects: Not the flat key "path.real". The proxy must therefore transform path_real into the nested object {"path": {"real": "..."}} in the response. A small HTTP proxy written in Python, listening on port 9200, which intercepts Samba's requests and translates them for Manticore, then translates the responses back into the format Samba expects. About 80 lines of standard library code, no external dependencies. Searching for "Battiato" from Finder shows Franco Battiato's MP3 files. Searching for "Robot" finds Ufo Robot songs and Asimov's Robot novels. Search works on filenames, ID3 titles, PDF and text file contents, and PHP and JavaScript source code. Recommended distro: Debian 12 or Ubuntu 22.04 Download the Manticore configuration file: Spotlight mappings file: Download the required files: Templates let you quickly answer FAQs or store snippets for re-use. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse

Code Block

Copy

Error loading module '/usr/lib/samba/vfs/spotlight.so': No such file or directory Error loading module '/usr/lib/samba/vfs/spotlight.so': No such file or directory Error loading module '/usr/lib/samba/vfs/spotlight.so': No such file or directory mdssvc_es_init: Opening mapping file [/usr/share/samba/mdssvc/elasticsearch_mappings.json] failed mdssvc_es_init: Opening mapping file [/usr/share/samba/mdssvc/elasticsearch_mappings.json] failed mdssvc_es_init: Opening mapping file [/usr/share/samba/mdssvc/elasticsearch_mappings.json] failed "_source": { "path": { "real": "/share/music/..." } } "_source": { "path": { "real": "/share/music/..." } } "_source": { "path": { "real": "/share/music/..." } } macOS Finder (Spotlight) │ SMB + Spotlight RPC ▼ CT Debian — smbd + rpcd_mdssvc │ HTTP :9200 ▼ CT Debian — ms-proxy.py │ HTTP :9308 ▼ CT Manticore — Manticore Search ▲ CT Debian — spotlight-indexer.sh │ inotify + ffprobe/pdftotext/catdoc ▼ /share (filesystem) macOS Finder (Spotlight) │ SMB + Spotlight RPC ▼ CT Debian — smbd + rpcd_mdssvc │ HTTP :9200 ▼ CT Debian — ms-proxy.py │ HTTP :9308 ▼ CT Manticore — Manticore Search ▲ CT Debian — spotlight-indexer.sh │ inotify + ffprobe/pdftotext/catdoc ▼ /share (filesystem) macOS Finder (Spotlight) │ SMB + Spotlight RPC ▼ CT Debian — smbd + rpcd_mdssvc │ HTTP :9200 ▼ CT Debian — ms-proxy.py │ HTTP :9308 ▼ CT Manticore — Manticore Search ▲ CT Debian — spotlight-indexer.sh │ inotify + ffprobe/pdftotext/catdoc ▼ /share (filesystem) wget https://repo.manticoresearch.com/manticore-repo.noarch.deb dpkg -i manticore-repo.noarch.deb apt update apt install manticore manticore-extra wget https://repo.manticoresearch.com/manticore-repo.noarch.deb dpkg -i manticore-repo.noarch.deb apt update apt install manticore manticore-extra wget https://repo.manticoresearch.com/manticore-repo.noarch.deb dpkg -i manticore-repo.noarch.deb apt update apt install manticore manticore-extra wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/manticore.conf mv manticore.conf /etc/manticoresearch/manticore.conf wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/manticore.conf mv manticore.conf /etc/manticoresearch/manticore.conf wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/manticore.conf mv manticore.conf /etc/manticoresearch/manticore.conf systemctl enable manticore systemctl start manticore systemctl enable manticore systemctl start manticore systemctl enable manticore systemctl start manticore apt install samba samba-vfs-modules \ inotify-tools curl ffmpeg \ poppler-utils catdoc \ python3 jq apt install samba samba-vfs-modules \ inotify-tools curl ffmpeg \ poppler-utils catdoc \ python3 jq apt install samba samba-vfs-modules \ inotify-tools curl ffmpeg \ poppler-utils catdoc \ python3 jq mkdir -p /usr/share/samba/mdssvc curl -s https://raw.githubusercontent.com/samba-team/samba/master/source3/rpc_server/mdssvc/elasticsearch_mappings.json \ -o /usr/share/samba/mdssvc/elasticsearch_mappings.json mkdir -p /usr/share/samba/mdssvc curl -s https://raw.githubusercontent.com/samba-team/samba/master/source3/rpc_server/mdssvc/elasticsearch_mappings.json \ -o /usr/share/samba/mdssvc/elasticsearch_mappings.json mkdir -p /usr/share/samba/mdssvc curl -s https://raw.githubusercontent.com/samba-team/samba/master/source3/rpc_server/mdssvc/elasticsearch_mappings.json \ -o /usr/share/samba/mdssvc/elasticsearch_mappings.json wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/msctl wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/ms-proxy.py wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/ms-proxy.service wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/smb.conf wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/spotlight-indexer.service wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/spotlight-indexer.sh mv msctl /usr/bin/ mv ms-proxy.py /usr/local/bin/ mv ms-proxy.service /etc/systemd/system/ mv smb.conf /etc/samba/ mv spotlight-indexer.service /etc/systemd/system/ mv spotlight-indexer.sh /usr/local/bin/ wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/msctl wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/ms-proxy.py wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/ms-proxy.service wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/smb.conf wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/spotlight-indexer.service wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/spotlight-indexer.sh mv msctl /usr/bin/ mv ms-proxy.py /usr/local/bin/ mv ms-proxy.service /etc/systemd/system/ mv smb.conf /etc/samba/ mv spotlight-indexer.service /etc/systemd/system/ mv spotlight-indexer.sh /usr/local/bin/ wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/msctl wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/ms-proxy.py wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/ms-proxy.service wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/smb.conf wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/spotlight-indexer.service wget https://raw.githubusercontent.com/senzidee/samba-spotlight-manticore/refs/heads/main/spotlight-indexer.sh mv msctl /usr/bin/ mv ms-proxy.py /usr/local/bin/ mv ms-proxy.service /etc/systemd/system/ mv smb.conf /etc/samba/ mv spotlight-indexer.service /etc/systemd/system/ mv spotlight-indexer.sh /usr/local/bin/ systemctl daemon-reload systemctl enable ms-proxy spotlight-indexer systemctl start ms-proxy spotlight-indexer systemctl restart smbd systemctl daemon-reload systemctl enable ms-proxy spotlight-indexer systemctl start ms-proxy spotlight-indexer systemctl restart smbd systemctl daemon-reload systemctl enable ms-proxy spotlight-indexer systemctl start ms-proxy spotlight-indexer systemctl restart smbd # Check that services are running systemctl status ms-proxy spotlight-indexer smbd # Count documents in the index msctl count # Search for a file msctl search README.md # Real-time logs during a search from the Mac journalctl -u smbd -f journalctl -u ms-proxy -f # Check that services are running systemctl status ms-proxy spotlight-indexer smbd # Count documents in the index msctl count # Search for a file msctl search README.md # Real-time logs during a search from the Mac journalctl -u smbd -f journalctl -u ms-proxy -f # Check that services are running systemctl status ms-proxy spotlight-indexer smbd # Count documents in the index msctl count # Search for a file msctl search README.md # Real-time logs during a search from the Mac journalctl -u smbd -f journalctl -u ms-proxy -f - Proxmox with two LXC containers - Samba CT: Debian 13, at least 512MB RAM - Manticore CT: Debian/Ubuntu, at least 1GB RAM

🏷️ Tags

toolsutilitiessecurity toolsreportsambaspotlightmanticorejourney

More from Tools

Tools: Embedded Linux Development - Part 2 - Complete Guide

Tools: Embedded Linux Development - Part 2 - Complete Guide

2026-04-13 0
Tools: The Developer's Guide to Running LLMs Locally: Ollama, Gemma 4, and Why Your Side Projects Don't Need an API Key

Tools: The Developer's Guide to Running LLMs Locally: Ollama, Gemma 4, and Why Your Side Projects Don't Need an API Key

2026-04-13 0
Tools: Ultimate Guide: Cómo hacer tu primer juego en Godot con IA en un fin de semana

Tools: Ultimate Guide: Cómo hacer tu primer juego en Godot con IA en un fin de semana

2026-04-13 0
Tools: I Got Tired of Flashing My Pico Manually, So I Built a GUI (2026)

Tools: I Got Tired of Flashing My Pico Manually, So I Built a GUI (2026)

2026-04-13 0

Trending

1

CVE-2025-61481: Critical Remote Code Execution Vulnerability in MikroTik RouterOS & SwitchOS

2025-10-27 • 189 views
2

CVE-2025-43939: Dell Unity OS Command Injection (High)

2025-10-30 • 148 views
3

Google disputes false claims of massive Gmail data breach

2025-10-30 • 130 views
4

Microsoft: DNS outage impacts Azure and Microsoft 365 services

2025-10-30 • 88 views
5

3.5B Accounts, 1 Critical Flaw: Meta Closes WhatsApp Data-Harvesting

2025-11-25 • 81 views