Evals and benchmarks vendors
Evaluation tasks and benchmarks measure model or agent behavior. Public samples help show what is tested; a benchmark listing is not evidence of vendor quality.
Evals and benchmarks vendors
125 vendors. 29 state they have a SOC 2 Type I or Type II report and 33 state they can run in a customer's cloud or on-prem. These are sourced statements, not independent verification. Pending and unspecified SOC 2 claims are excluded.
— means not publicly listed or not confirmed; it never means zero or “No”. Checked is the date a company’s facts were last checked, and each fact on its profile links to its source.
| Compare | Vendor | Offerings | Domains | Data sources | Funding | Checked |
|---|---|---|---|---|---|---|
| AbundantAbundant builds reinforcement-learning environments, evaluation benchmarks, and training datasets for AI labs, AI startups, and large enterprises developing and improving AI agents. | RL envs, Training data, Evals | Coding, Computer use, Enterprise workflows, Security | Open data | $500K | 2026-09-29 | |
| AfterQueryAfterQuery sells expert-created and synthetic training datasets, reinforcement-learning environments, and custom evaluations to AI labs and enterprises developing foundation models and agents. | RL envs, Training data, Evals, Human data | Multi-domain, Coding, Computer use, Browser, Finance, Legal, Medical | Expert-made, Synthetic | $30.5M | 2026-10-13 | |
| AIChampAIChamp provides robotics and AI teams with proprietary simulated environments and physics-verified evaluations for robot policies, running large-scale trials and returning scores, videos, trajectories, and failure analysis before real-world deployment. | RL envs, Evals, Simulation | Robotics and physical | Synthetic | — | 2026-09-29 | |
| Align AIAlign AI sells expert-grounded training data, simulated RL environments, continuous evaluations, and in-runtime verification tools to AI labs and enterprises building agents for regulated, high-stakes work. | RL envs, Evals, Human data | Finance, Legal, Medical, Enterprise workflows | Expert-made | — | 2026-09-29 | |
| Andon LabsAndon Labs sells custom evaluations and benchmarks to AI labs and developers, testing AI agents in simulations and real-world deployments across business operations, spatial reasoning, computer use, and robotics. | Evals, Simulation | Multi-domain, Computer use, Robotics and physical | Synthetic | $500K | 2026-09-29 | |
| AndromedeAndromede provides AI labs and agent developers with reinforcement-learning environments, tasks, and verifiers built from real-world data for training and evaluating long-horizon agents. | RL envs, Evals | Other | Synthetic | — | 2026-09-29 | |
| AnthromindAnthromind supplies AI developers, research teams, and businesses with expert-curated training and preference data, custom LLM evaluations, and fine-tuning support through tailored projects and managed data-curation services. | Human data, Training data, Evals | Multi-domain, Coding | Customer data, Expert-made | $200K | 2026-09-29 | |
| AppenAppen supplies AI labs and product teams with human-produced training data, annotation, expert feedback, RL environments, and model evaluations for machine-learning and agentic AI systems. | RL envs, Human data, Evals, Training data, Annotation | Multi-domain, Medical, Legal, Science, Finance, Coding, Browser, Security, Enterprise workflows | Expert-made, Licensed data | — | 2026-10-08 | |
| ApturaAptura provides expert-built training and evaluation data, benchmarks, and reinforcement-learning environments to AI labs and developers building systems for finance, healthcare, and legal work. | Training data, Evals, RL envs, Human data | Finance, Medical, Legal | Expert-made | $6.7M | 2026-09-29 | |
| ArchalArchal sells resettable API sandboxes that mimic third-party software services, giving AI developers and software teams repeatable environments to test, evaluate, and train agents without using live accounts. | RL envs, Evals, RL infra | Multi-domain | App replicas, Synthetic | $500K | 2026-09-29 | |
| ArenaArena sells AI-model evaluations to labs, enterprises, and developers, using real-world human preferences to assess model performance and provide analytics, feedback samples, and SLA-backed results. | Human data, Evals | Coding, Legal, Medical, Science, Multi-domain | — | $450M | 2026-10-13 | |
| Arga LabsArga provides AI developers and engineering teams with stateful replicas of business software, letting them train and test agents on realistic workflows and validate changes without modifying live production systems. | RL envs, Evals, Simulation | Multi-domain, Enterprise workflows | Synthetic | $10.5M | 2026-09-29 | |
| ARIMLABSBuilds production-style simulated workspaces and verifiable benchmarks for AI agents, helping AI developers train and test long-horizon tasks in cybersecurity, incident response, software engineering, and tool use. | RL envs, Evals, Simulation | Multi-domain, Security, Coding | Expert-made | — | 2026-10-13 | |
| AviroAviro sells reinforcement-learning environments, post-training datasets, and evaluations for AI labs and teams building long-running agents, with tasks spanning coding, computer use, and enterprise knowledge work. | RL envs, Training data, Evals | Coding, Computer use, Enterprise workflows, Science | Expert-made | $500K | 2026-09-29 | |
| BenchFlowBuilds simulated, workflow-based environments and benchmarks, plus a runtime and managed post-training service, for AI labs and developers training and evaluating agents. | RL envs, Evals, RL infra | Multi-domain, Enterprise workflows | App replicas | $1M | 2026-09-29 | |
| Besimple AIBesimple AI supplies AI labs and voice-AI developers with licensed conversational audio, expert annotation, and benchmarks for training and evaluating speech models. | Human data, Training data, Evals, Annotation | Multi-domain | Expert-made | $3.5M | 2026-09-29 | |
| Bespoke LabsBespoke Labs sells realistic, company-scale reinforcement-learning environments and agent-training, optimization, and evaluation infrastructure to frontier AI labs and enterprises. | RL envs, RL infra, Evals | Multi-domain | — | $40M | 2026-09-29 | |
| BioStack PlatformsBioStack sells clinical and preclinical data, healthcare AI training environments, and evaluation tools to AI labs and life-sciences teams developing medical and drug-discovery models. | RL envs, Training data, Evals | Medical, Science | — | $500K | 2026-09-29 | |
| BugcrowdBugcrowd provides cybersecurity training environments and an exploitation benchmark to AI developers, alongside crowdsourced vulnerability testing services for organizations. | RL envs, Evals | Security | — | $230M | 2026-09-29 | |
| CentificCentific sells multimodal AI training data, human annotation, model evaluations, and simulated enterprise RL environments to frontier AI labs and enterprise AI teams. | RL envs, Training data, Evals, Human data, Annotation | Multi-domain, Finance, Medical, Enterprise workflows, Legal, Robotics and physical | Licensed data | $60M | 2026-09-29 | |
| Chakra LabsChakra sells computer-use reinforcement-learning environments, human interaction trajectories, and evaluation resources that AI researchers use to train and test agents. | RL envs, Training data, Evals | Computer use, Enterprise workflows, Games, Creative | App replicas, Expert-made | $10.1M | 2026-10-13 | |
| ClaruClaru provides custom and licensed video, robotics demonstrations, and human annotations to AI labs building robotics, embodied AI, and world models. | Training data, Human data, Annotation, Evals | Robotics and physical, Games, Creative, Multi-domain, Coding, Legal, Medical, Math, Science, Security, Other | Licensed data, Expert-made, Synthetic, Open data | $170M | 2026-10-13 | |
| CollinearCollinear sells stateful simulated environments, agent evaluations, and synthetic training data to frontier AI labs and enterprise AI teams building and improving AI agents. | RL envs, Training data, Evals | Multi-domain, Security, Finance, Enterprise workflows | Synthetic | $10M | 2026-09-29 | |
| Contra LabsContra Labs provides AI teams with professional creatives’ judgments, model evaluations, benchmark datasets, human-feedback training data, and recorded design workflows to improve and measure generative AI. | Human data, Evals, Training data | Creative | Customer data, Synthetic | $44.5M | 2026-09-29 | |
| Cortex AICortex AI provides AI labs with real-world workplace video and robot trajectories for training embodied AI, plus human-supervised robot rollouts and evaluations that capture recovery from failures. | Human data, Training data, Evals | Robotics and physical | Expert-made | $6.5M | 2026-10-13 | |
| CuaCua sells cloud and self-hosted desktop sandboxes, computer-use runtimes, RL environments and benchmarks, and trajectory data to AI developers building, training, and evaluating agents that operate computers. | RL envs, Evals, RL infra | Computer use, Multi-domain | Expert-made | $500K | 2026-10-06 | |
| CueBenchCueBench builds reinforcement-learning environments and task benchmarks for AI labs and developers to train and assess frontier models on difficult scientific-reasoning and performance-engineering problems. | RL envs, Evals | Science, Coding | — | $500K | 2026-09-29 | |
| DARE LabsDARE Labs offers Cerebro, a data refinery that turns operational data into supervised-training datasets, RL data and environments graded on real outcomes. | Training data, RL envs, Evals | Multi-domain | Customer data, Synthetic | — | 2026-10-06 | |
| DatacurveDatacurve sells expert-built coding data, reinforcement-learning environments, benchmarks, and evaluations to foundation-model labs and AI developer-tool companies. | RL envs, Training data, Evals, Human data | Coding, Science, Security | Expert-made | $17.7M | 2026-09-29 | |
| DataOcean AIDataOcean AI sells off-the-shelf datasets and custom data collection, labeling, and model evaluation services to AI developers and enterprises, supporting LLM, speech, vision, multimodal, and autonomous-driving systems. | Training data, Annotation, Evals, Human data | Multi-domain, Coding, Finance, Medical, Legal | Customer data, Expert-made | — | 2026-09-29 | |
| David AIDavid AI designs, collects, annotates, and licenses human-recorded audio datasets, and evaluates speech models for AI labs and developers building speech recognition, translation, synthesis, and conversational systems. | Human data, Training data, Evals | Other | Expert-made | $80.5M | 2026-09-29 | |
| Deccan AIDeccan AI sells AI labs and developers expert-created post-training datasets, human feedback, reinforcement-learning environments and evaluations; it also offers enterprises monitoring and AI-enabled operations tools. | RL envs, Training data, Evals, Human data | Multi-domain, Coding, Enterprise workflows, Computer use, Browser | Expert-made | $25M | 2026-09-29 | |
| Defined.aiDefined.ai sells licensed, multilingual AI training datasets and provides custom data collection, annotation, evaluation and model fine-tuning services to organizations developing AI. | Training data, Human data, Annotation, Evals | Multi-domain | Licensed data, Expert-made | $85M | 2026-09-29 | |
| DisseiDissei sells finance-focused RL environments, evaluation benchmarks, and real-transaction training data and analyst trajectories to AI labs and model developers, as well as financial institutions building or assessing models. | RL envs, Evals, Training data, Human data | Finance | Licensed data | — | 2026-09-29 | |
| e2fe2f, which renamed itself TrustScale in June 2026, sells custom AI training datasets, expert annotation, model evaluation and benchmarking, and reinforcement learning feedback through its ArgusRL platform, covering 200 languages and text, image, audio and video data. | Training data, Evals, RL infra, Human data, Annotation | Multi-domain | Expert-made | $50M | 2026-10-13 | |
| EdotEnvEdotEnv provides financial-market-based reinforcement-learning environments and research benchmarks that AI labs and research teams use to train and assess agents on quantitative-research tasks. | RL envs, Evals | Finance | — | $500K | 2026-09-29 | |
| Eigent AIEigent sells an open-source multi-agent desktop workforce for automating enterprise workflows, and builds custom RL environments, benchmarks, and training/evaluation loops for enterprise teams and AI developers. | RL envs, Evals, Training data | Multi-domain, Browser, Enterprise workflows | Open data, Synthetic | — | 2026-10-09 | |
| EmulatedEmulated offers AI teams post-training datasets, production-like software-engineering environments, and benchmarks to train and evaluate agents on coding, infrastructure operations, machine-learning engineering, and research tasks. | RL envs, Training data, Evals | Coding, Enterprise workflows, Science | Synthetic | $850K | 2026-09-29 | |
| EnactEnact works with robotics model developers to identify failures on physical robots, create targeted recovery demonstrations and training data, then fine-tune and evaluate models for real-world tasks. | RL envs, Training data, Evals | Robotics and physical | — | $500K | 2026-09-29 | |
| EncordEncord sells a multimodal data platform and managed collection, annotation, and evaluation services to AI teams preparing and improving training data for physical AI, computer vision, and generative models. | Training data, Annotation, Evals | Multi-domain, Robotics and physical | — | $110M | 2026-09-29 | |
| exponentialExponential sells AI developers multi-agent security environments for RL post-training, security-focused training data, and adaptive red-team evaluations that test AI models and agent systems against realistic attacks. | RL envs, Training data, Evals | Security | — | — | 2026-10-13 | |
| FleetFleet sells simulated training environments and evaluation tools where AI agents practice realistic tasks to frontier AI labs, hyperscalers, and enterprises. | RL envs, Evals, RL infra | Multi-domain | Synthetic | $60M | 2026-09-29 | |
| Foundry AIFoundryAI creates expert-reviewed coding and other technical-workflow data, RL-ready task suites, private model evaluations, and regression tests for AI development teams. | Training data, RL envs, Evals, Human data | Coding | Expert-made | — | 2026-09-29 | |
| General ReasoningGeneral Reasoning offers AI labs and developers API-hosted reinforcement-learning environments, plus infrastructure to deploy environments and train or evaluate agents at scale. | RL envs, Evals, RL infra | Multi-domain | Synthetic | $10.9M | 2026-09-29 | |
| Good Start LabsGood Start Labs builds game-based reinforcement-learning environments, turns gameplay into human and synthetic training data, and publishes evaluation benchmarks for AI labs. It also helps game publishers capture gameplay data and put AI agents into their games. | RL envs, Human data, Evals, Training data | Games | Customer data, Licensed data | $3.6M | 2026-09-29 | |
| Gradient HealthGradient Health sells de-identified medical imaging and multimodal clinical datasets through Atlas to medical AI developers, helping them train, test, validate, and prepare regulatory evidence for healthcare models. | Training data, Evals | Medical | — | $8.4M | 2026-09-29 | |
| Gray Swan AIGray Swan sells AI-model and agent security to frontier labs and enterprises: adversarial testing before release, plus runtime filtering and monitoring to block attacks, unsafe outputs, and policy violations. | Evals | Security | Expert-made | $50M | 2026-09-29 | |
| HalluminateHalluminate sells finance-focused computer-use and tool-use RL environments, simulated business applications, training data, and agent benchmarks to frontier AI labs and enterprise AI teams. | RL envs, Training data, Evals | Computer use, Browser, Enterprise workflows, Finance | Expert-made | $38.5M | 2026-10-08 | |
| HandshakeHandshake supplies frontier AI developers with expert-created training and evaluation data, reinforcement-learning environments, and licensed, de-identified operational records from real companies. | Human data, Evals, RL envs, Training data | Multi-domain, Legal, Finance, Science | Licensed data, Expert-made | $435.5M | 2026-10-13 | |
| HUDHUD sells an SDK and cloud platform for building, running, and grading reinforcement-learning environments. It turns agent runs into evaluation results and training data, and connects environment and data suppliers with AI labs. | RL envs, Training data, Evals, RL infra | Multi-domain, Computer use, Finance | — | $16M | 2026-10-13 | |
| Humyn LabsHumyn Labs collects and annotates real-world multimodal human data—especially first-person video, sensor and motion signals, and speech—and supplies training datasets and evaluation benchmarks to AI labs, robotics developers, and other AI teams. | Training data, Annotation, Evals | Multi-domain, Robotics and physical | Expert-made | — | 2026-09-29 | |
| Huzzle LabsHuzzle Labs sells reinforcement-learning environments, expert-produced training data, evaluations, and custom-trained models to AI labs and enterprises seeking agents that can handle real software and business workflows. | RL envs, Human data, Evals, Training data, Annotation | Coding, Computer use, Enterprise workflows | Customer data | $6M | 2026-10-08 | |
| IdlerIdler sells reinforcement-learning environments, benchmarks and evaluation datasets based on production work to AI labs and other teams training and evaluating models. | RL envs, Evals | Multi-domain, Enterprise workflows, Legal, Coding, Security | — | $9.5M | 2026-09-29 | |
| iMeritAcquired by EXLiMerit sells expert-led data annotation, training-data creation, human feedback, model fine-tuning and evaluation services, alongside Ango Hub workflow software, to frontier AI labs, AI developers and enterprises building multimodal and domain-specific models. | Human data, Training data, Evals, Annotation | Multi-domain, Medical | Synthetic | $24.5M | 2026-09-29 | |
| IncalmoIncalmo builds reinforcement-learning cyber environments and security evaluations for AI labs and security teams to train and assess cybersecurity-capable models. | RL envs, Evals | Security | Synthetic | $8.7M | 2026-09-29 | |
| InnodataInnodata provides AI developers and enterprises with training data, human feedback, annotation, reinforcement-learning environments, model evaluations, and red-teaming services for building and improving AI models and agents. | RL envs, Human data, Training data, Evals, Annotation | Coding, Computer use, Browser, Enterprise workflows, Finance, Legal, Medical, Science, Math, Security, Robotics and physical, Multi-domain | Customer data, Expert-made, Synthetic | — | 2026-10-13 | |
| Invisible TechnologiesInvisible sells expert-created training data, model fine-tuning, evaluations, and custom reinforcement-learning environments to AI labs and developers; it also provides AI software and implementation support for enterprises. | RL envs, Training data, Evals, Human data, Annotation | Multi-domain, Coding, Finance, Legal, Enterprise workflows | Expert-made | $144M | 2026-09-29 | |
| LabelboxLabelbox sells simulated reinforcement-learning environments, expert-generated preference and training data, and evaluation services to frontier AI labs; it also offers enterprise teams software for running agents on real company workflows. | RL envs, Human data, Evals, Training data, Annotation, RL infra | Multi-domain, Coding, Computer use, Security, Enterprise workflows, Science, Robotics and physical, Finance, Legal | Licensed data | $189M | 2026-09-29 | |
| LatchLatchBio sells cloud data and compute software, bioinformatics workflows, and AI-based analysis to biotech R&D teams, laboratories, and scientific kit or instrument providers. It also publishes evaluations and offers assessment services for biology-focused AI agents. | Evals | Science, Security, Medical | Expert-made | $163M | 2026-09-29 | |
| LightwheelLightwheel sells physics-accurate simulation assets and environments, synthetic and human demonstration data, and evaluation tools to robotics and Physical AI teams training and validating robot models. | Human data, Evals, Simulation, Training data, RL infra | Robotics and physical | Expert-made | $293M | 2026-09-29 | |
| LightWrkLightWrk sells expert-produced reasoning annotations, preference and training signals, and evaluation workflows to robotics, physical-AI, and world-model teams. | Human data, Training data, Evals, Annotation | Robotics and physical | Synthetic | — | 2026-09-29 | |
| LuelLuel licenses ready-made multimodal datasets and organizes custom collection campaigns for AI labs and developers, using paid contributors to supply rights-cleared audio, video, speech, sensor, and computer-use data. | Human data, Training data, Evals | Multi-domain, Computer use, Medical, Robotics and physical | Expert-made | $31.7M | 2026-09-29 | |
| LXTLXT supplies AI developers and enterprises with custom and off-the-shelf training data, human annotation, evaluation, and RLHF services across text, speech, images, and video. | Training data, Annotation, Evals, Human data | Multi-domain, Legal, Finance, Medical, Robotics and physical | Expert-made | — | 2026-09-29 | |
| MaingenMaingen builds industrial-operation simulations, RL environments and benchmarks for frontier AI labs, helping them train and assess agents on tasks such as running solar portfolios and factories. | RL envs, Evals, Simulation, Training data | Enterprise workflows, Other | — | $500K | 2026-09-29 | |
| MarkovMarkov sells computer-use datasets and task environments—including expert CAD, productivity, coding, and gaming workflows—to AI labs and developers training or evaluating desktop agents. | RL envs, Training data, Evals, Human data | Multi-domain, Computer use, Coding, Browser, Games | Expert-made | $500K | 2026-09-29 | |
| MaxinsightsMaxinsights collects and annotates real-world egocentric video and multimodal human-motion data for robotics and physical-AI developers, and markets simulation and model-evaluation capabilities for robot and world models. | Human data, Training data, Evals | Robotics and physical | — | — | 2026-10-13 | |
| MechanizeMechanize builds simulated software-engineering tasks and benchmarks for frontier AI labs, helping them train and evaluate coding agents using graded performance. | RL envs, Evals | Coding | Expert-made | $9.1M | 2026-09-29 | |
| MeckaMecka collects and structures real-world human activity into robotics training datasets, and provides data tools and evaluation and deployment support for robotics labs, frontier AI teams, and enterprises building embodied systems. | Human data, Training data, Evals | Robotics and physical | Expert-made | $60M | 2026-09-29 | |
| MercorMercor sells expert-created training data, reinforcement-learning environments, and evaluations and benchmarks to frontier AI labs and other AI developers. It also offers enterprise data licensing and evaluation services. | RL envs, Human data, Training data, Evals | Multi-domain, Finance, Legal, Medical, Coding, Browser, Enterprise workflows | Licensed data, Expert-made, Synthetic | $483.6M | 2026-10-13 | |
| MetaphiMetaphi sells custom and reusable simulated work environments, training datasets, and agent evaluations to frontier AI labs and enterprise AI teams, helping them train and assess agents on realistic software, finance, and content-generation tasks. | RL envs, Training data, Evals | Coding, Enterprise workflows, Finance, Creative | Licensed data, Expert-made | — | 2026-09-29 | |
| micro1micro1 provides expert human data and post-training services, real-world RL environments, robotics data, and AI-agent evaluations for frontier AI labs, enterprises, and robotics teams. | RL envs, Human data, Evals, Training data | Multi-domain, Finance, Legal, Medical, Coding, Robotics and physical | Expert-made | $135M | 2026-10-13 | |
| Mind LabMind Lab provides pharmaceutical, financial, public-sector and gaming companies with custom benchmarks, task environments, data development and model post-training, alongside its own training platform. | RL envs, Evals, RL infra | Multi-domain | Infra, n/a | $60M | 2026-10-13 | |
| MonteMonte sells post-training and continual-learning infrastructure to AI labs and enterprise AI teams, helping them evaluate models against work tasks, train specialized agents, and improve them using feedback from real workflows. | RL infra, Evals | Enterprise workflows | Customer data | $500K | 2026-10-06 | |
| Moonlake AIMoonlake provides world-model and simulation tools that turn real-world inputs or text into simulation-ready 3D assets and environments for robotics, autonomy, and AI research teams. | RL envs, Training data, Evals | Robotics and physical, Computer use, Creative | Synthetic | $28M | 2026-09-29 | |
| NormalNormal publishes CAD benchmarks, datasets, and technical reports for AI developers working on hardware design; CAD Arena evaluates how agents reconstruct real mechanical parts from engineering drawings as editable CAD models. | Evals | Robotics and physical | Licensed data | — | 2026-09-29 | |
| Olam LabsOlam Labs builds simulated multi-agent environments, behavioral evaluations and game-derived datasets for AI labs and researchers training or evaluating models’ social, agentic and safety capabilities. | RL envs, Training data, Evals, Simulation | Coding, Computer use, Enterprise workflows, Finance, Games, Security, Multi-domain | — | — | 2026-10-13 | |
| Ooak DataOoak Data licenses and anonymizes real company workflows, then turns them into RL training and evaluation environments for frontier AI labs and teams building enterprise agents. | RL envs, Training data, Evals | Enterprise workflows | Licensed data | $500K | 2026-09-29 | |
| OpenTrain AIOpenTrain connects AI labs and developers with vetted human experts for RLHF, model evaluation, red teaming, and data annotation. Customers hire directly or use a managed team working in their existing tools. | Human data, Training data, Evals, Annotation | Multi-domain | Customer data, Expert-made | — | 2026-10-06 | |
| Paper InstrumentsPaper Instruments post-trains language models and builds evaluation benchmarks, post-training infrastructure, and agent software for AI developers working on knowledge-intensive tasks. | Evals, Training data, RL infra | Finance, Legal, Medical, Enterprise workflows | Synthetic, Expert-made | $3.6M | 2026-09-29 | |
| ParetoPareto supplies AI labs with specialist-created training data, evaluations, and RL environments that turn expert judgment into signals for assessing and improving frontier models. | RL envs, Human data, Evals | Multi-domain, Medical, Legal, Finance, Coding, Science | Expert-made | $17.8M | 2026-09-29 | |
| ParsewaveParsewave creates custom, expert-written datasets, solution traces, and evaluations for frontier AI labs and developers to train and test coding and other professional-domain models. | Training data, Human data, Evals | Multi-domain | Expert-made | — | 2026-10-13 | |
| Patronus AIPatronus sells AI evaluation and benchmarking tools, plus simulated digital worlds and RL environments that AI labs and developer teams use to test, train, and improve models and agents. | RL envs, Evals, Simulation, RL infra | Multi-domain, Finance, Coding, Enterprise workflows | Expert-made, Open data | $70M | 2026-09-29 | |
| Perle AIPerle supplies AI labs and developers with specialist-generated multimodal training data, annotation, RLHF and model evaluations through expert-led programs and managed workflows. | Human data, Training data, Evals, Annotation | Multi-domain, Medical, Legal, Robotics and physical | Expert-made | $17.5M | 2026-09-29 | |
| PlatoPlato builds simulated software and computer environments from real-world data so AI agent developers can train, test, and score browser and computer-use agents. | RL envs, Evals | Multi-domain, Browser, Computer use, Enterprise workflows | Synthetic | $2.5M | 2026-09-29 | |
| Poindexter LabsPoindexter Labs supplies expert data annotation and evaluation services to frontier AI teams, alongside syncronus, a collaborative platform for expert-led model training. | Human data, Evals, Annotation | Finance, Legal, Multi-domain | Expert-made | — | 2026-10-06 | |
| PolymathBuilds simulated environments, tasks, and verifiers for AI labs to train and evaluate long-horizon agents; its Horizon-SWE benchmark tests end-to-end software engineering workflows. | RL envs, Evals, Simulation | Multi-domain, Coding | Expert-made | $8M | 2026-09-29 | |
| pre.devpre.dev sells runnable, graded coding environments and training and evaluation data built from licensed private software repositories to AI labs and developers training coding agents. | RL envs, Training data, Evals | Coding | Customer data | — | 2026-09-29 | |
| Prime IntellectPrime Intellect sells AI labs and developers an open stack for reinforcement-learning environments, model evaluations, hosted training, inference, sandboxes, and GPU compute to build and improve agentic models. | RL envs, Evals, RL infra | Multi-domain, Math, Coding, Games | — | $150M | 2026-09-29 | |
| ProlificProlific gives AI labs and developers access to verified participants and subject-matter experts to collect human feedback, training data, and model evaluations through a platform, API, or managed research service. | Human data, Training data, Evals, Annotation | Multi-domain, Legal, Medical, Finance, Robotics and physical | Expert-made | $33.4M | 2026-09-29 | |
| ProvisionProvision supplies real-world multimodal training data, physical-world evaluations, and deployment support for teams developing physical AI and robotics. | Training data, Evals | Robotics and physical | — | — | 2026-10-09 | |
| ProximalProximal builds coding tasks, training data and RL environments that AI labs and enterprises use to train software-engineering agents. | RL envs, Training data, Evals | Coding, Science | Synthetic | — | 2026-10-08 | |
| QuesmaQuesma publishes coding-agent benchmarks and builds RL training environments for frontier labs; it also markets tools that help development teams understand agent costs, workflows and outcomes. | RL envs, Evals | Coding, Security, Games | — | $2.3M | 2026-10-06 | |
| ReasonCoreReasonCore provides expert-curated AI training datasets, reinforcement-learning environments, and adaptive evaluations for frontier AI labs, emerging AI labs, and enterprises developing or deploying advanced models. | RL envs, Training data, Evals, Human data, Annotation | Robotics and physical, Science, Coding, Finance, Math, Enterprise workflows | Customer data, Expert-made | — | 2026-09-29 | |
| RefreshRefresh sells simulated software environments and verifiable evaluations for training and testing AI agents, alongside real-workflow coding datasets, to frontier AI labs and enterprises. | RL envs, Training data, Evals | Multi-domain, Coding, Computer use, Medical, Enterprise workflows | Customer data | $500K | 2026-09-29 | |
| ReveloRevelo sells AI labs code-focused training data, evaluations, benchmarks, and RL environments, with expert-led data creation and managed workflows to help improve coding models and agents. | RL envs, Training data, Evals, Human data | Multi-domain, Coding | Synthetic, Expert-made | $46M | 2026-09-29 | |
| Rise Data LabsRise Data Labs sells human- and synthetic-data production, annotation, and model-evaluation services to AI labs and enterprise model teams building and improving generative AI. | Human data, Training data, Evals, Annotation, RL envs | Multi-domain | Customer data | — | 2026-09-29 | |
| RunloopRunloop provides isolated development sandboxes, orchestration, and benchmarking tools so AI developers and labs can build, run, score, and refine coding agents, including reinforcement-learning and fine-tuning workflows. | RL envs, Evals, RL infra | Coding, Browser, Computer use | Synthetic | $7M | 2026-10-13 | |
| SamaSama sells human-in-the-loop data annotation, validation, evaluation, and training-data services to enterprise AI teams building computer-vision, multimodal, and generative AI models. | Training data, Evals, Annotation, Human data | Multi-domain, Robotics and physical, Creative | Customer data, Expert-made | $84.8M | 2026-09-29 | |
| SapienSapien supplies human-created and expert-reviewed training data to AI developers, including data annotation, collection, curated datasets, and quality checks for AI outputs. | Human data, Training data, Evals, Annotation | Multi-domain, Robotics and physical | Expert-made | $15.5M | 2026-09-29 | |
| ScaleScale sells AI labs and businesses custom training data, human feedback, annotation, RL environments, and model evaluations, alongside software to build, test, and deploy AI agents and applications. | RL envs, Training data, Evals, Human data, Annotation | Multi-domain, Coding, Computer use, Browser, Enterprise workflows, Finance | Synthetic, App replicas | $15.9B | 2026-09-29 | |
| Sepal AIAcquired by MercorSepal supplies frontier AI labs and enterprise model teams with expert-created training and evaluation data, benchmarks, and reinforcement-learning environments for developing and testing models on specialized tasks. | RL envs, Human data, Evals | Multi-domain | Expert-made, Synthetic | — | 2026-09-29 | |
| SieveSieve supplies AI labs and model developers with curated or custom-collected video, audio, image, and interaction data, plus annotations, evaluation sets, and interactive environments. | Training data, Evals, RL envs, Human data | Creative, Computer use, Robotics and physical, Games, Multi-domain | Licensed data, Expert-made | — | 2026-10-13 | |
| SnorkelSnorkel sells expert-built training datasets, agent environments, benchmarks and evaluations—through ready-to-use products and custom data-development services—to frontier AI labs and enterprise AI teams. | Training data, Evals, RL envs, Human data | Multi-domain, Coding, Science, Finance, Enterprise workflows, Medical, Legal | Expert-made, Synthetic | $587M | 2026-09-29 | |
| SuperAnnotateSuperAnnotate sells AI data software and managed expert services to AI labs and developers, covering multimodal annotation, training data, model and agent evaluations, and reinforcement-learning environments. | RL envs, Human data, Evals, Annotation, Training data | Multi-domain, Medical, Legal, Science, Coding, Browser, Computer use, Robotics and physical, Math, Enterprise workflows, Finance | Expert-made | $67.5M | 2026-09-29 | |
| SurgeSurge sells AI labs and developers expert-built training data, human feedback, evaluation benchmarks, and interactive reinforcement-learning environments to train and assess AI models and agents. | RL envs, Training data, Evals, Human data, Annotation | Multi-domain, Coding, Enterprise workflows, Finance, Medical, Science, Math, Legal | Expert-made | — | 2026-09-29 | |
| SymbalSymbal supplies AI labs with multimodal post-training datasets, private evaluations and benchmarks, and human-in-the-loop data; it also builds enterprise agents using customers’ first-party workflows. | Training data, Evals, Simulation | Multi-domain, Science, Math, Finance, Medical, Legal, Robotics and physical, Coding | Customer data | — | 2026-09-29 | |
| Synthetic SciencesAn AI co-scientist workbench for research teams, automating literature review, hypothesis generation, experiments, analysis, and paper drafting; the company also develops scientific evaluations and long-horizon RL environments for AI labs. | RL envs, Evals | Science | — | $1.9M | 2026-09-29 | |
| Tacit LabsTacit builds expert-authored life-science evaluations, scientific agent environments, and structured datasets to help frontier AI labs train and assess models on long-horizon biology, chemistry, and drug-development tasks. | RL envs, Evals, Training data, Human data | Science | Expert-made | — | 2026-09-29 | |
| TaskUsTaskUs provides outsourced AI data operations for AI developers and enterprises, including data collection and labeling, expert feedback and RLHF, model evaluations and red-teaming, and robotics data services. | Training data, Human data, Evals, Annotation | Robotics and physical, Medical, Enterprise workflows, Multi-domain | Expert-made | $279M | 2026-10-13 | |
| Taste LabsTaste Labs sells preference and reasoning data, rubrics, and evaluation environments to frontier AI labs for subjective design quality, and provides agent and app developers a Brand API and MCP tools to extract, search, and verify brand systems. | RL envs, Training data, Evals, Human data | Creative | Expert-made | $18.5M | 2026-09-29 | |
| TELUS DigitalAcquired by TELUSTELUS Digital supplies AI labs and developers with expert-built training data, annotations, agentic RL environments, red-team testing and evaluations, plus custom multimodal and physical-AI datasets and managed data-collection services. | RL envs, Human data, Training data, Evals, Annotation | Coding, Enterprise workflows, Finance, Legal, Medical, Math, Robotics and physical, Science, Security, Other, Multi-domain | Expert-made, Customer data | — | 2026-10-13 | |
| The General Data CompanyThe General Data Company licenses anonymized records and workflow-based datasets from European businesses to AI labs for model training and evaluation, including held-out benchmarks and reinforcement-learning environments. | RL envs, Training data, Evals | Finance, Legal, Medical, Enterprise workflows | Licensed data, Expert-made | — | 2026-09-29 | |
| ThetaTheta sells custom simulated work environments and hands-on implementation to AI labs and enterprises, helping them train and evaluate specialized agents on realistic, multi-step business workflows. | RL envs, Evals, RL infra | Multi-domain, Computer use, Browser, Enterprise workflows, Finance, Medical, Coding | Customer data, Expert-made | $500K | 2026-09-29 | |
| TolokaToloka supplies AI labs and developers with human-expert and synthetic training data, agent environments, evaluations, and red-teaming to build and assess AI models and agents. | RL envs, Human data, Evals, Training data, Annotation | Multi-domain, Coding, Computer use, Enterprise workflows, Math, Science, Robotics and physical | Synthetic | $72M | 2026-09-29 | |
| Trajectory LabsTrajectory Labs runs safety evaluations, expert red-teaming and RL environments that help AI labs find model vulnerabilities and measure safer behavior. | RL envs, Evals, Human data | Multi-domain, Security | Expert-made | — | 2026-09-29 | |
| TuringTuring supplies AI labs with training data, evaluations, and RL environments for coding, enterprise knowledge work, and STEM. It also helps large enterprises build and deploy AI agents in important workflows. | RL envs, Training data, Evals, Human data | Multi-domain, Coding, Enterprise workflows, Science, Math, Finance, Legal, Medical, Computer use, Security | Customer data | $225M | 2026-10-13 | |
| Uber AI SolutionsUber AI Solutions provides custom AI data collection and annotation, specialist human feedback and RLHF, red-teaming, and model evaluation for frontier labs, enterprises, robotics firms, and autonomous-vehicle developers. | Annotation, Training data, Evals, Human data, RL envs | Coding, Enterprise workflows, Finance, Legal, Medical, Science, Robotics and physical, Multi-domain, Other | Expert-made, Customer data | — | 2026-10-09 | |
| UlamUlam sells reasoning datasets, expert-reviewed traces, benchmarks, and verifier-backed RL environments—primarily for mathematics and STEM—to AI labs and developers. | RL envs, Training data, Evals, Human data | Coding, Science, Math, Multi-domain | Synthetic | — | 2026-10-13 | |
| Vals AIVals AI sells evaluation software, expert-built benchmarks, and review tools to AI labs and enterprise engineering teams testing models and AI applications on practical, domain-specific tasks. | Evals, RL infra | Multi-domain, Finance, Legal, Medical, Coding, Math | Expert-made | $45M | 2026-09-29 | |
| Veris AIVeris sells AI developers a simulation and reinforcement-learning platform that recreates enterprise tools, systems, and users, helping teams train, test, and improve agents on realistic workflows before production. | RL envs, Evals, RL infra | Multi-domain, Enterprise workflows, Finance, Coding | Customer data, Synthetic | $8.5M | 2026-09-29 | |
| Verita AIVerita supplies vetted domain experts and manages project operations for AI labs, producing annotation, human-feedback and evaluation work, training data, and support for reinforcement-learning environments. | RL envs, Training data, Annotation, Human data, Evals | Multi-domain | Licensed data | — | 2026-09-29 | |
| Vetto AIVetto sells AI labs and developers expert post-training data, human evaluation, custom RL environments, and verifier-graded benchmarks to help train and assess models on practical tasks. | RL envs, Training data, Evals, Human data, Annotation | Multi-domain, Science, Medical, Legal, Finance, Coding, Computer use | Expert-made | $3.6M | 2026-09-29 | |
| Vibrant LabsVibrant Labs builds RL-ready simulated environments, training tasks, and benchmarks for AI labs and developers working on long-horizon agents, including agents that use browsers, computers, or enterprise tools. | RL envs, Training data, Evals | Multi-domain, Computer use, Browser, Enterprise workflows | Synthetic, App replicas | $2.5M | 2026-09-29 | |
| VmaxVmax markets Campaign, a self-play task-generation and RL platform that uses developers’ proprietary data and evaluations to create training environments and refine agents on new task examples. | RL envs, RL infra, Evals | Coding, Math | Customer data | $250K | 2026-10-13 | |
| XDOFXDOF collects robot teleoperation data and runs real-world policy evaluations for robotics labs and AI companies, and also supplies robotic systems and data tooling. | Training data, Human data, Evals, Annotation | Robotics and physical | Expert-made | $70M | 2026-10-13 |
No vendor matches every filter. Filters in different groups must all match. Remove one to widen the list:
— not publicly listed or not confirmed. Open a profile for each fact’s source. The register is unranked; sorting only reorders.
How to read domain and SOC 2 labels
Multi-domain means work across several task areas, not every area. Vendors appear in a specific domain filter only when that area has supporting evidence. SOC 2 entries are vendor statements, not independently verified audits: Unspecified means claimed without a stated type; Pending means in progress; Not publicly listed means we found no public statement. None of those means a completed Type I/II report. SOC 2 and deployment details remain on profiles and in comparison.