About

Snowmelt Labs studies how training data shapes what models learn. We're a small team with backgrounds from Stanford, MIT and Google DeepMind.

Measuring data

A dataset is worth what it does to the model that trains on it, and the cheap proxies people use instead tend to be wrong exactly where it matters. We measure it with short training runs at fixed compute and hold whole sources out of training. In our video ranking work, for instance, two rankers that agree at 0.944 across a pool share only 64% of the top 2,000 clips, and those top clips are what anyone would ship.

Datasets and RL environments

We build data and environments where the answer can be checked. Our materials science tasks give agents real problems and grade what they hand back by recomputing it, and every reference method had to reproduce published numbers first. WebPII is about 45,000 rendered checkout and account pages with every personal field boxed, including fields caught half typed.

Audits

We audit the datasets and environments labs already have, starting with what they leak and what their graders let through. Scrubbing agent traces for release, we found that 96% of the secrets in them had no prefix a list could match, so a scrubber that reported clean was blind to most of them. In Terminal-Bench 2.0, 19 of 65 tasks gave full reward to a wrong solution that a model reviewer judged a clear break of a stated requirement.

Working with us

Write to us if you want a corpus measured or audited before you buy or train on it, or if a model you're training needs data or environments. The reasoning behind all of it is in the manifesto.