Home / Blog / AWS announces AI data preparation capabilities in Amazon…
Tech News

AWS announces AI data preparation capabilities in Amazon SageMaker

Amazon Web Services expanded Amazon SageMaker AI with automated data preparation tools and synthetic generation APIs across two US AWS cloud regions.

By Dillip Chowdary • Nov 18, 2026 • Source: AWS Machine Learning Blog

AWS announces AI data preparation capabilities in Amazon SageMaker
Image: AWS Machine Learning Blog

At a glance

Product
Amazon SageMaker AI data preparation
Interfaces
SageMaker Python SDK, AWS AI-ML agent skill
Review software
Label Studio Community Edition
Availability
US East (N. Virginia), US West (Oregon)
Enterprise user
Splunk Agent Observability

Amazon Web Services expanded Amazon SageMaker AI with native data preparation capabilities designed to build datasets for generative AI model development, according to AWS Machine Learning Blog's report. The addition gives engineers modular tools to generate synthetic records, automate data labeling at scale, diagnose quality defects, and incorporate human reviewer corrections directly within local notebooks or development environments.

This article details the core data preparation tools, coding agent integrations, end-to-end pipeline architectures, and underlying security controls. It is written for machine learning practitioners and technical leaders creating task-specific datasets to fine-tune open-weight models, run preference alignment routines, or construct evaluation benchmarks without hand-crafting disparate pipelines.

01Five standalone data preparation building blocks

Auto-labeling processes datasets from unannotated collections, small labeled seed sets, or unclassified inputs across classification, named entity recognition, extractive question answering, ordinal rating, and summarization tasks. For tasks with fixed potential outputs, an ensemble jury of multiple models labels each record, using their collective agreement level to calculate an automatic confidence score.

Records falling below user-defined confidence thresholds route automatically to a human review queue hosted in SageMaker Studio using the open-source Label Studio Community Edition. External subject matter experts access the review environment via partner application links, submitting corrections that feed directly back into labeling algorithms while producing consensus metrics and labeling reports.

Synthetic data generation creates training, evaluation, and test sets grounded in seed records and enterprise documents, outputting inspectable code pipelines. Complementary data quality evaluation flags duplicates, label noise, and answer leakage via business rules and LLM-as-a-judge metrics, while transformation tools execute natural-language instructions, standard deduplication, splitting, or custom Python scripts.

02Notebook environments and coding agent integration

Data scientists can execute preparation tasks directly through the SageMaker Python SDK or delegate orchestration to coding assistants via the AWS AI-ML agent skill. The agent interprets natural-language specifications to select and execute capabilities across SageMaker Studio JupyterLab, Kiro, or local development environments, aligning execution steps with established data science practices.

Teams inspecting agent-generated workflows retain full visibility into underlying code, specifications, and pipeline configurations, allowing manual edits before execution. This programmatic architecture enables developers to integrate automated quality assessments into existing pipelines or trigger standalone synthetic generation routines without managing separate data engineering infrastructure.

Enterprise adoption includes Splunk, a Cisco company, which implemented the synthetic data generation capability to create statistically representative datasets for its Agent Observability evaluation models. Splunk engineering leadership noted that generating diverse, task-specific synthetic data preserved essential statistical patterns while reducing time spent on manual data curation.

03Chained pipelines for fine-tuning and evaluation

Advertisement

Tech Pulse Daily

Get tomorrow's pulse first

Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.

Developers can combine modular capabilities into multistep pipelines, such as building golden evaluation benchmarks from raw records. In this pattern, auto-labeling processes raw data from small seed inputs, accepting high-confidence rows automatically while directing uncertain entries to human reviewers whose corrections iteratively refine subsequent passes.

A full pipeline transforms unorganized source documents into trained models by linking generation, transformation, validation, and training stages. Synthetic tools produce grounded domain examples, transformation steps reformat outputs for supervised fine-tuning, direct preference optimization, or reinforcement learning, and quality evaluation strips duplicate entries and leakage before compute runs.

After passing automated validation checks, prepared datasets route directly to SageMaker AI model customization for fine-tuning. This end-to-end integration ensures specialized open-weight models receive clean, domain-adapted training data, allowing smaller architectures to achieve target accuracy without paying higher inference costs associated with prompting massive general-purpose foundation models.

04Four-step evaluation dataset preparation run

To illustrate the preparation workflow, consider a team evaluating an agentic customer support assistant tasked with answering technical setup queries. The team must verify whether the assistant preserves product manual warnings stating that a factory reset erases saved user settings, requiring grounded evaluation pairs derived from official product guides.

  1. Define the requirements Supply the product guide as context, define required questions and reference answers, and specify that reset answers preserve data-loss warnings.
  2. Generate and inspect a sample Produce a preliminary batch to inspect generated code, verify topic coverage, and ensure reference answers follow source documentation.
  3. Assess and review Run automated quality evaluations across generated pairs, inspect record findings, and route disputed assessments to subject matter experts.
  4. Apply changes and repeat Refine the underlying specification to eliminate duplicates, generate supplementary examples, and reassess the final dataset before conducting evaluations.

This iterative cycle demonstrates how specification adjustments and automated quality assessments refine test datasets before benchmarking starts. By updating rules and regenerating examples against verified criteria, developers ensure evaluation suites accurately measure model compliance on critical instructions without requiring manual authoring of hundreds of test prompts.

05Customer storage boundaries and regional availability

AI data preparation executes under customer-managed AWS Identity and Access Management execution roles, maintaining organizational security boundaries across automated operations. Source documents, synthetic outputs, and generated annotations remain strictly within customer-owned Amazon Simple Storage Service buckets located in the user's chosen AWS Region.

Developers initiate workflows by configuring data preparation skills inside supported development environments or invoking SageMaker Python SDK endpoints directly from custom application code. SDK calls accept structured specifications as parameters, allowing teams to add targeted quality evaluations or synthetic generation routines to existing continuous delivery pipelines.

Note: The AI data preparation capabilities are generally available in the US East (N. Virginia) and US West (Oregon) Regions. Workflows run under customer-controlled AWS Identity and Access Management execution roles, keeping source datasets and generated outputs inside private Amazon Simple Storage Service buckets.

Frequently asked questions

What built-in task types does SageMaker AI auto-labeling support?

Auto-labeling includes built-in support for classification, named entity recognition, extractive question answering, ordinal rating, and summarization, alongside custom task definitions. For tasks with fixed candidate answers, a multi-model jury determines record confidence scores based on model agreement levels.

How does human review integrate with automated labeling workflows?

Records falling below designated confidence thresholds are routed to a Label Studio Community Edition review queue inside SageMaker Studio. Feedback and corrections submitted by subject matter experts feed directly back into the auto-labeling pipeline to improve future iterations.

Where are source records and generated synthetic datasets stored?

All source documents, synthetic datasets, and label annotations remain inside customer-owned Amazon Simple Storage Service buckets. Operations execute under customer-managed AWS Identity and Access Management execution roles within the selected AWS Region.

Which AWS Regions currently support SageMaker AI data preparation?

The AI data preparation capabilities are generally available in the US East (N. Virginia) and US West (Oregon) Regions.

Sources

Dillip Chowdary

Author

Dillip Chowdary

Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.

Related on Tech Bytes

Advertisement

5-min tech signal

Weekday briefing for engineers who skip the noise.

No spam · Unsubscribe anytime

Advertisement

✈️ CareerPilot

Your AI job-search copilot

Match your resume against live Ashby, Greenhouse & Lever openings — fit scores, job-specific resume optimization and email alerts.

Find matching jobs →

Free Tools

Browse all tools →