Scalable Image AI via Self-designing Storage

Abstract: Image AI has the potential to improve every aspect of human life, providing more productive and safer services and tools. Image AI is, however, very expensive. It costs millions of dollars to train and deploy a single model with hundreds of thousands of pounds of carbon emission. We identify that the root cause of the problem is a long-overlooked and largely unexplored dimension: storage. The majority of the cost of reading and processing an image depends on how it is stored on disk and in main memory, as storage determines how much data is moved and processed. Most images today are stored as JPEG files. But, JPEG is designed for the human eye; it aims to maximally compress images with minimal loss in visual quality. In contrast, we observe that, during image AI, it is AI algorithms that “see” the images, rather than humans. Furthermore, JPEG is a single design. AI problems, however, are diverse; every problem is unique in terms of how data should be stored and processed. Using a fixed design, such as JPEG, for all problems results in excessive data movement and wasteful image AI pipelines.

This talk presents Image Calculator, a self-designing storage system that creates and manages storage for image AI tasks. Unlike state-of-the-art that uses a fixed storage format, Image Calculator builds a design space of thousands of storage formats, each capable of storing and representing data differently, at different training and inference speed, accuracy, and space trade-offs. Given an AI task, Image Calculator searches and finds the optimal storage format that minimizes training and inference times and maximizes accuracy. Image Calculator consists of two main components: (i) search & training, and (ii) model-serving. Search & training efficiently searches within the design space by using locality among storage formats. Formats that have similar features also perform similarly. It clusters information-dense formats and quickly identifies high-quality candidates with scalable search time. Model-serving deploys storage formats at inference servers. It exploits the inherent frequency structure in image data. It breaks images into pieces, i.e., frequency components, and processes images frequency by frequency instead of image by image as conventionally done. This allows dramatically reducing data communication between clients & servers with fast and efficient inference.

We evaluate Image Calculator across a diverse set of datasets, tasks, models, and hardware. We show that Image Calculator can generate storage formats that reduce end-to-end inference and training time by up to 14.2x and consume space by up to 8.2x with little or no loss in accuracy for image classification, object detection, and instance segmentation, compared to state-of-the-art image storage formats, such as JPEG and its recent variants. Image Calculator’s storage formats reduce individual time components, such as PCIe time, by up to 271x. Image Calculator is even more successful on small hardware devices providing sub-millisecond inference on CPUs, making inference scalable and cheap on commodity hardware. Tailoring storage to AI tasks results in heavily compressed and specialized data. Despite that, we show that Image Calculator is able to reconstruct images with high visual quality. Image Calculator’s incremental computation scheme allows moving just enough data for every image, further reducing data movement cost, and providing fast and scalable inference serving.

This talk will mainly be based on the following two papers:

[1] The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format. Utku Sirin, Stratos Idreos. SIGMOD, 2024 [Link]
[2] Frequency-Store: Scaling Image AI by A Column-Store for Images. Utku Sirin, Victoria Kauffman, Aadit Saluja, Florian Klein, Jeremy Hsu, Stratos Idreos. CIDR, 2025 [Link]

Bio: Utku Sirin is a postdoctoral researcher at the Data Systems lab at Harvard University, advised by Stratos Idreos. Utku’s work on the Image Calculator reimagines Image AI through self-designing AI storage, which always takes the best shape given the AI context and goals, bringing 10x speedup end-to-end. Utku was awarded the Microsoft Research PhD Fellowship in 2017 and the Swiss National Science Foundation Postdoctoral Fellowship in 2021 and 2023. Utku has also been a winner of the ACM SIGMOD Students Research Competition and a recipient of an IEEE ICDE best reviewer award. Before joining Harvard, Utku obtained his PhD from the Data-Intensive Applications and Systems lab at EPFL, advised by Anastasia Ailamaki on hardware-conscious data systems. 

-- For the zoom passcode, contact the organizer at markakis@mit.edu