Aws S3 Vectors At Scale: Real Performance Numbers At 10 Million...

AWS S3 Vectors promises "billions of vectors with sub-second queries" and up to 90% cost savings over traditional vector databases. These claims sound good on paper, but implementation details matter. How does performance actually scale? What's the accuracy trade-off? Are there operational gotchas?

This post presents empirical benchmarks testing S3 Vectors from 10,000 to 10 million vectors, comparing performance and accuracy against FAISS and NMSLib. All code used boto3 on us-east-1, measuring real-world query latency including network overhead.

S3 Vectors is AWS's managed vector search service that stores and queries vector embeddings directly in S3. Key characteristics:

The value proposition is operational simplicity and cost reduction. You don't manage infrastructure, handle index building, or worry about scaling - you just store vectors in S3 and query them.

DINOv3 (self-supervised vision transformer) for image embeddings:

Chose DINOv3 for strong performance on image retrieval tasks without fine-tuning.

Important caveat: Local execution eliminates network latency, giving FAISS/NMSLib inherent speed advantages unrelated to algorithm quality.

Testing from 10K to 10M vectors with 384-dimensional embeddings, topK=5:

S3 Vectors query time grows from 112ms at 10K vectors to 382ms at 10M vectors - a 3.4x increase for a 1000x data increase.

Query latency scales sublinearly: Moving from 10K to 10M vectors (1000x increase) results in only 3.4x latency increase. This suggests efficient indexing that doesn't degrade linearly with dataset size.

Source: Dev.to