FrameSense is a command-line tool for processing large moving image collections into machine-readable datasets. It is designed to run on consumer hardware or to scale up to high-performance computing (HPC) environments, so that institutions with very different levels of infrastructure can use it. The idea came out of a collaboration with King’s Digital Lab to develop a sustainable pipeline for analysing video at scale. FrameSense takes a collection of video files (large or small) as input and produces structured and crucially reproducible datasets — sequences of frames, metadata, and derived features — that can be fed into downstream machine learning and analysis pipelines. It is modular and configurable, supporting different sampling rates, output formats, and compute environments. The development and design of FrameSense is largely the work of Geoffroy Noël, senior research engineer at KDL.
Why FrameSense?
Many audiovisual archives hold collections that are too large to process manually and too heterogeneous for tools designed for other domains. FrameSense addresses this gap: a tool built specifically for the needs of moving image research and screen heritage, designed to be accessible to a small regional archive and a national archive alike. We are currently using it in Intelligent Systems for Screen Archives.
The tool’s development arc — from proof of concept to prototype to usable software — was included as a case study in the Alan Turing Institute’s roadmap Towards a National Research Software Engineering Capability in Arts and Humanities Research, a report that outlines a strategic plan to establish a national, Arts & Humanities RSE capability, ensuring that digital expertise is accessible and sustainable within A&H research.
Roadmap and public engagement
The tool is very new and just at the begining of its development. There is still a lot testing ahead and development for it to mature. We have approached different communities steer this development, for example at the Audiovisual Digital Humanities SIG at DH2025 (Lisbon), the largest international Digital Humanities conference. It will feature again in the next edition of ADHO in Daejeon, South Korea.