SCM Brings Local AI Search to macOS Photos and Video

Overview

A new open-source application called SCM (Screen Memories) is bringing powerful, local artificial intelligence search capabilities to macOS users. The tool allows individuals to deep-search their entire photo and video libraries using natural language queries, all while maintaining strict privacy standards by keeping data on-device.

Built with Electron and Bun, SCM differentiates itself from cloud-dependent services like Apple Photos or Google Photos by offering a fully offline experience. There are no accounts required, no subscriptions, and no data uploads. Once the initial model weights are downloaded, the application operates entirely offline, ensuring that personal memories never leave the user’s machine.

The project has garnered early attention in the developer community for its approach to handling large media libraries. With a lightweight footprint and a modular architecture, it aims to solve the frustration of sifting through thousands of files to find that one specific moment captured on camera.

How It Works

SCM employs a multi-modal search strategy that goes beyond simple file naming. It utilizes local vision models to embed images and video frames, allowing users to search by describing a memory rather than recalling a filename. When a user types a query, the system performs an instant pre-pass on filenames before handing off to the vision model for deeper semantic analysis.

The application segments videos into distinct scenes, embedding each segment individually. This means a search for "dog playing in the park" will not just return a video file but will jump directly to the specific timecode where that action occurs. A noise gate ensures that irrelevant segments are filtered out, preventing the results grid from being flooded with low-quality matches.

For text-heavy searches, SCM offers two specialized modes. OCR (Optical Character Recognition) uses Tesseract to identify visible text within images and video frames. This mode matches query tokens literally and works independently of the AI engine, making it functional even in offline or limited-connectivity scenarios.

Search Modes and Features

The interface organizes search capabilities into five distinct modes, catering to different retrieval needs:

• Files: The default mode searches whole photos and videos by meaning. Results are ranked via cosine similarity against image embeddings, with additional scoring boosts for filename keywords and exact phrase matches. Every result tile includes a "why it matched" badge, providing transparency into the ranking logic.
• Scenes: This mode focuses on temporal precision within video files. Users can search for specific shots, and the app limits contributions from any single video to three scenes to maintain result diversity.
• OCR: Ideal for finding documents, signs, or text messages within images. It supports 35+ languages and highlights matched words in amber on the thumbnail tiles.
• Dialogue: Leveraging the Whisper model, this feature transcribes spoken audio in videos. It offers tiered exactness levels: finding exact contiguous lines, exact words within an eight-second window, or any words spoken in the same video.
• LLMs (Ask): An opt-in feature that allows users to chat with their library. Using a local llama.cpp sidecar, the app answers questions by citing specific dialogue lines, OCR text, and filenames. Responses are streamed token-by-token, complete with a live tokens-per-second readout.

The application also features a self-maintaining library system. Watched folders automatically import new content, and the system uses content hashing to deduplicate files even if they are renamed. Switching between different AI models triggers background re-embedding without blocking the search interface.

Installation and Requirements

SCM is currently packaged for macOS only, with certain menu-bar and tray features exclusive to the platform. The development stack relies on Bun as both the package manager and runtime, alongside Vite for the renderer bundle and Electron for the desktop wrapper.

Installation is streamlined for Apple Silicon users via Homebrew. The project’s cask is configured to automatically clear the macOS quarantine flag, which eliminates the need for manual Gatekeeper interventions during installation or upgrades.

brew tap allenv0/scm
brew trust allenv0/scm
brew install --cask allenv0/scm/scm

For users concerned with least privilege, the cask can be trusted individually without trusting the entire tap repository. First-time users should note that the default CLIP vision model requires approximately 435MB of storage for weights download. However, this is a one-time cost, after which all inference runs locally on the Mac’s hardware.

Technical Architecture

Under the hood, SCM is designed for modularity and performance. The project structure separates the main process, preload scripts, and the React-based renderer. The indexer component handles the heavy lifting of scanning directories, extracting metadata, and generating embeddings.

The application supports CJK (Chinese, Japanese, Korean) queries by treating them as overlapping bigrams, ensuring that searches for locations like "Taipei Main Station" correctly match partial components. Additionally, the search results include a near-duplicate diversity filter, ensuring that users see a variety of results rather than multiple near-identical frames from the same video source.

The project is public on GitHub, inviting community contribution and inspection. This transparency aligns with the broader "local-first" software movement, which prioritizes user sovereignty over their data.

"Your media never leaves the machine. Weights download once; everything after that is offline." — SCM README

As AI tools become increasingly prevalent in consumer software, applications like SCM highlight a growing demand for privacy-preserving alternatives. By leveraging the increasing computational power of modern Macs, SCM demonstrates that sophisticated AI search does not require a cloud subscription or a constant internet connection.