New advertisers: 50% off your first month with code FIRSTMONTHSee slots
aws-samples avatar

nova-migration

End-to-end migration assistant (/nova-migrate) for porting prompts from other LLMs to Amazon Nova, with baseline capture, rubric-based evaluation, and a refine loop. Requires the nova-prompting plugin.

GitHub stars
460
View on GitHub

Add marketplace

/plugin marketplace add aws-samples/amazon-nova-samples

Run it inside Claude Code once, then install the plugin below.

Install

/plugin install nova-migration@aws-samples-amazon-nova-samples

Run it inside Claude Code after adding the marketplace.

Files

Discovered from source

Nova Migration Assistant

You are a migration orchestrator. Your job is to take a prompt (or small app) written for another LLM and port it to Amazon Nova 1 or Nova 2, while proving via evaluation that quality holds up — and iterating on the migration if it doesn't.

You do not do the prompt rewriting yourself. You delegate to /nova1-prompt or /nova2-prompt. You do not invent new test data from the prompt itself — that is circular. You either use what the user brings, or you generate from the task description.


INPUTS YOU NEED (ask in order, skip if already provided)

  1. Target Nova generation — Nova 1 or Nova 2. If the user is unsure, briefly surface the tradeoff (Nova 1 = Micro/Lite/Pro/Premier choice; Nova 2 = Lite only, has reasoning + 1M context). Do not over-explain.
  2. Source prompt — file path, repo dir to scan, or pasted inline. If a repo, find prompt-like strings (look for system prompts, template strings, .txt/.md under prompts/, etc.) and confirm the list with the user before proceeding.
  3. Task description — one or two sentences: what does this prompt do? This drives synthetic test generation and rubric derivation. Never feed the prompt itself to the test generator — that is circular and inflates scores.
  4. Test data — one of:
    • a path to tests.jsonl / tests.yaml / tests.yml (auto-detect by extension; schema in test_schema.md)
    • generate N — synthesize N tests from the task description (default 10). You MUST show the generated set to the user for curation (delete/edit/add) before running anything.
    • a path to a dir of pre-recorded input/output pairs (the no-API-key path — skip baseline capture)
  5. Source model — one of:
    • a Bedrock model ID (e.g. anthropic.claude-3-5-sonnet-20240620-v1:0, meta.llama3-1-70b-instruct-v1:0) — uses existing AWS creds
    • openai:<model-id> or anthropic:<model-id> — requires OPENAI_API_KEY or ANTHROPIC_API_KEY env var; fail early with a clear message if missing
    • --baseline-from-tests — no source-model calls; expects expected_output in each test (the no-API-key fallback)

If any input is missing and can't be reasonably inferred, ask. Do not guess defaults for source model or generation.


THE FLOW

All artifacts go under ./nova-migrate-runs/{YYYY-MM-DD-HHMMSS}/. Each step writes its output as a named file so the run is inspectable after the fact.

STEP 1 — Ingest

  • Parse source prompt(s) and test file into a normalized in-memory spec.
  • Create run dir: ./nova-migrate-runs/{ts}/.
  • Write spec.json: {target_generation, source_prompt, task_description, tests: [...], source_model}.

STEP 2 — Baseline

  • If source model is callable: run each test input through it via helper.py invoke (or batch). Save to baseline.jsonl — one record per test with {test_id, input, output, latency_ms, usage}.
  • If --baseline-from-tests: copy expected_output from each test into baseline.jsonl with a source: "user_provided" marker.
  • If a test has no expected output AND no source model is available, fail loudly and tell the user what's missing.

STEP 3 — Optimize (first pass)

  • Invoke /nova1-prompt or /nova2-prompt (whichever matches target generation). Pass the source prompt + task description.
  • Save the optimized prompt to prompt_v1.md.
  • Save any notes the optimizer returned to optimize_notes_v1.md.

STEP 4 — Derive rubric

  • From the task description (NOT the prompt), propose 4–6 rubric criteria. Good criteria are task-specific and checkable — e.g. "Output is valid JSON matching the schema in test.expected_schema", not "Output is good".
  • If any test file supplies its own rubric (YAML format allows this), merge: test-level overrides global.
  • Show the rubric to the user and accept edits before scoring. Save to rubric.md.

STEP 5 — Evaluate (iteration 1)

  • Run each test through Nova with prompt_v{N}.md via helper.py batch. Save to nova_results_v{N}.jsonl.
  • Score with helper.py judge: for each test, LLM-as-judge compares Nova output vs baseline against the rubric, returning per-criterion scores (1–5) + a short critique. Save to scores_v{N}.jsonl.
  • Compute aggregates: mean per criterion, overall mean, count of regressions (Nova ≥ 1 point below baseline on any criterion).

STEP 6 — Refine loop

Loop up to max_iterations (default 3, configurable):

  • Stop conditions (any one):
    • Zero regressions vs baseline.
    • Iteration count exhausted.
    • Score plateau: improvement < 0.2 points over previous iteration.
  • If stopping, proceed to STEP 7.
  • Otherwise, build a critique bundle: the failing tests, Nova's outputs, baseline outputs, judge critiques. Feed back to /nova1-prompt or /nova2-prompt with a prefix like: "Previous optimization regressed on these cases. Preserve what worked for the others. Here are the failures: ..."
  • Save the new prompt as prompt_v{N+1}.md, then re-run STEP 5.

Regression threshold default: Nova scores ≥ 1 point lower than baseline on any rubric criterion for that test. Configurable per run.

STEP 7 — Report

Write REPORT.md with:

  • Summary verdict: "ship" / "ship with caveats" / "do not ship" + one-line reason.
  • Final prompt (and path).
  • Per-test before/after scores, side-by-side outputs for regressions.
  • Iteration history: what changed between versions, what the optimizer was told each pass.
  • Known weaknesses and suggested follow-ups (e.g. "rubric criterion X consistently low — consider domain-specific post-processing").

Surface the verdict and report path in your final message to the user.


HOW YOU CALL THE HELPER

All model calls go through skills/nova-migrate/helper.py. Do not shell out to provider SDKs directly from the skill.

# Single invocation
uv run skills/nova-migrate/helper.py invoke \
  --model "<bedrock-id-or-provider:model>" \
  --system-file path/to/system.txt \
  --input-file path/to/input.txt \
  --out result.json

# Batch over a test set
uv run skills/nova-migrate/helper.py batch \
  --model "<...>" \
  --prompt-file prompt_v1.md \
  --tests tests.jsonl \
  --out nova_results_v1.jsonl

# LLM-as-judge scoring
uv run skills/nova-migrate/helper.py judge \
  --rubric rubric.md \
  --baseline baseline.jsonl \
  --candidate nova_results_v1.jsonl \
  --out scores_v1.jsonl

The helper handles provider dispatch, retries, parallel batching, and returns consistent JSON. If it errors, surface the error to the user; do not silently retry.


GUARDRAILS

  • Never generate test inputs by feeding the source prompt to a generator. Always from the task description.
  • Never skip the rubric curation step. Users should see and accept the rubric.
  • Never declare "ship" if any regression remains — use "ship with caveats" and list them.
  • Never fork /nova1-prompt or /nova2-prompt logic into this skill. Delegate.
  • Never overwrite a previous run's dir. Always timestamp.
  • If the user's app has multiple prompts, migrate them one at a time in separate runs. Don't batch a whole app into one eval — the signal gets muddied.

STARTING THE CONVERSATION

If invoked with $ARGUMENTS, treat it as the source prompt or a path to one, and begin at STEP 1. Otherwise, open by asking for target generation and source prompt, in that order. Keep the intro short — one or two sentences.

From the aws-samples/amazon-nova-samples README

Amazon AI Samples

Home of Amazon first-party foundation models and AI services. This repository contains sample code, notebooks, workshops, and reference applications for building with Amazon first-party foundation models and AI services on Amazon Bedrock, Amazon SageMaker AI, and third-party platforms.

This repository was renamed from aws-samples/amazon-nova-samples. Existing links and git clone URLs redirect automatically. To update a local clone, run: git remote set-url origin https://github.com/aws-samples/amazon-ai-samples.git

Repository map

FolderWhat's insidePrimary services
multimodal-understanding/Getting-started notebooks, a self-paced workshop, repeatable patterns, and sample apps for Nova understanding models: text, image, video, and document understanding, tool calling, RAG, agents, and prompt optimizationAmazon Bedrock (Nova Lite, Pro, Premier, Nova 2 Lite)
speech-to-speech/Real-time, bidirectional voice conversation samples in Python, Java, Node.js, .NET, and Ruby, plus integration patterns, workshops, and a Nova Sonic 2.5 prompting guideAmazon Bedrock (Nova Sonic, Nova 2 Sonic, Nova Sonic 2.5)
text-to-speech/A guided quickstart, focused feature samples (SSML, speech marks, lexicons, async and bidirectional streaming synthesis, voice cloning), and end-to-end speech applicationsAmazon Polly, with Transcribe, Translate, Bedrock, and AgentCore in the use cases
multimodal-embeddings/Embedding text, images, video, audio, and documents, and patterns for semantic, multilingual, and visual search, clustering, and classificationAmazon Bedrock (Nova Multimodal Embeddings), Amazon S3 Vectors, OpenSearch Serverless
customization/Fine-tuning (SFT, PEFT, DPO), reinforcement fine-tuning (RFT), distillation, evaluation, and deployment of custom Nova models, plus HyperPod setup and utilitiesAmazon Bedrock, Amazon SageMaker AI (Training Jobs, HyperPod)
nova-act/Tutorials and use cases for building agents that take actions in web browsers: QA testing, adaptive UI testing, and price comparisonAmazon Nova Act, Amazon Bedrock AgentCore Browser
skills/Agent Skills and Claude Code plugins for AI coding assistants: Nova prompt optimization, migrating from OpenAI, Gemini, or Titan to Nova, and turning a text agent into a voice agentWorks with Claude Code, Kiro, and other Agent Skills–compatible tools

Where to start

I want to…Start here
Make my first call to a Nova modelmultimodal-understanding/getting-started/
Learn Nova models end to end in a guided workshopmultimodal-understanding/workshop/
Find a ready-made pattern (RAG, agents, IDP, citations, batch inference, …)multimodal-understanding/repeatable-patterns/
Build a real-time voice agentspeech-to-speech/amazon-nova-2-sonic/
Write prompts for Nova Sonic 2.5speech-to-speech/amazon-nova-2.5-sonic/
Convert text to natural-sounding speechtext-to-speech/getting-started/
Build semantic or multimodal searchmultimodal-embeddings/getting-started/
Fine-tune or distill a Nova modelcustomization/
Automate browser workflows with an agentnova-act/tutorials/path-to-production/
Migrate existing OpenAI, Gemini, or Titan code to Novaskills/

Each folder has its own README with prerequisites, setup steps, and a full index of its samples.

Highlights by folder

Multimodal understanding

  • Getting started: setup, the Converse API, built-in tool calling, and integrations with Strands Agents and LangChain.
  • Workshop: a three-part self-paced workshop: getting started with Nova 2, multimodal intelligence, and agentic workflows deployed to Amazon Bedrock AgentCore.
  • Repeatable patterns: task-focused guides covering document understanding and IDP, image grounding, video temporal reasoning, citations, prompt caching, LLM-as-a-judge, batch inference, and agents built with Strands, LangGraph, CrewAI, smolagents, and the OpenAI Agents SDK.
  • Sample apps and the Nova meta-prompter, which transforms prompts written for other models into Nova-optimized prompts.

Speech-to-speech (Nova Sonic)

Nova Sonic uses the InvokeModelWithBidirectionalStream API for low-latency voice conversations.

  • amazon-nova-2-sonic/: the current samples. Includes sample code (console Python, WebSocket in Java, Node.js, and .NET, and AgentCore), repeatable patterns (knowledge bases, session continuation, conversation transfer, chat-history and token-usage logging, customer service), and workshops (Python server, React client, LiveKit, AgentCore).
  • amazon-nova-2.5-sonic/: a getting-started and migration guide and a prompting guide for Nova Sonic 2.5.
  • The top-level sample-codes/, repeatable-patterns/, and workshops/ folders hold the original Nova Sonic samples.

Text-to-speech (Amazon Polly)

  • Getting started: a guided first request that covers Polly engines, voice discovery, and synthesis.
  • Features: eight focused examples: SynthesizeSpeech, DescribeVoices, SSML, speech marks, lexicons, asynchronous synthesis to S3, bidirectional streaming, and voice cloning.
  • Use cases: a cascading STT → LLM → TTS voice agent, a podcast generator, a live speech translator, a lip-synced 3D avatar, and a voice assistant for the Reachy Mini robot.
  • Skills: agent skills that help AI coding assistants work with these samples.

Multimodal embeddings

Numbered getting-started notebooks for each modality, and repeatable patterns for video search with S3 Vectors, multilingual search, clustering, and intent classification, visual product search, multimodal document search with LangChain and LlamaIndex, and industry examples.

Customization

Bedrock fine-tuning and distillation (text-to-SQL, understanding, Nova Canvas, citations, and function-calling recipes); SageMaker AI customization for Nova 1.0 and Nova 2.0 (data prep → SFT → RFT → evaluation → deployment); HyperPod cluster setup; and utilities such as a dataset validator, an FT-to-eval converter, and job monitoring. See the customization README for a use-case-to-folder map.

Nova Act

A path-to-production tutorial (setup, workflow basics, tool use) and use cases for agentic QA testing, adaptive UI testing, and price comparison, using Amazon Bedrock AgentCore Browser.

Skills and Claude Code plugins

The skills/ folder holds Agent Skills for AI coding assistants. Several are also published as Claude Code plugins through this repository's marketplace:

/plugin marketplace add aws-samples/amazon-ai-samples
/plugin install nova-prompting@aws-samples-amazon-nova-samples
PluginPurpose
nova-promptingPrompt optimization for Nova 1 and Nova 2 Lite (/nova1-prompt, /nova2-prompt)
nova-sonic25-promptingPrompt best practices for Nova Sonic 2.5 voice agents (/nova-sonic25-prompt)
openai-to-nova-migrationMigrate OpenAI code and prompts to Nova 2 Lite
gemini-to-nova-migrationMigrate Google Gemini code and prompts to Nova 2 Lite
titan-nova-mme-migrationMigrate Titan embedding code to Nova Multimodal Embeddings

The marketplace keeps its original identifier, aws-samples-amazon-nova-samples, so existing plugin installs keep working.

Prerequisites

  1. An AWS account and credentials configured for the AWS CLI or SDK (for example, with aws configure or an IAM role).
  2. A Region where the model or service you're using is available. Check the individual sample README for supported Regions and model IDs.
  3. Python 3.10 or later for most samples. Some samples use Java, Node.js, or .NET, or need a newer Python version; their READMEs say so.
  4. IAM permissions for the services the sample uses (see below).

IAM permissions

The AWS identity you use, whether it's an IAM user, an assumed role, or a SageMaker AI execution role, needs permission to call the services in the sample. For experimentation only, a policy like the one below covers the core Bedrock and Polly samples. For anything beyond experimentation, scope Action and Resource to what the sample actually needs.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BedrockInvoke",
            "Effect": "Allow",
            "Action": ["bedrock:InvokeModel*"],
            "Resource": "*"
        },
        {
            "Sid": "PollySynthesis",
            "Effect": "Allow",
            "Action": ["polly:*"],
            "Resource": "*"
        }
    ]
}

Many samples need more than this, such as Amazon S3, Amazon Transcribe, Amazon Translate, Amazon SageMaker AI, OpenSearch Serverless, or Amazon Bedrock AgentCore. Each sample's README lists its own permissions.

⚠️ With Amazon SageMaker AI, the notebook execution role is usually separate from the role you use to sign in to the AWS Management Console. Grant permissions to both if you also want to use the console.

For fine-grained Bedrock permissions, see the Amazon Bedrock User Guide.

Costs

These samples are for demonstration and learning, and are not intended for production use as-is. Running them makes billed AWS API calls, and some samples deploy infrastructure (for example, Fargate, OpenSearch Serverless, HyperPod clusters, or training jobs) that keeps costing money until you delete it. Review each sample's README before running it, and follow its cleanup steps when you're done.

Contributing

We welcome community contributions. See CONTRIBUTING.md for guidelines.

Note: Please contact the repository maintainers before proposing new top-level folders or changes to them.

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.

👏 Contributors

Shout out to these awesome contributors:

image

Details

Marketplace
aws-samples/amazon-nova-samples
Source
skills/nova-prompter/plugins/nova-migration
Category
Community

More from aws-samples/amazon-nova-samples

All 6

Similar to nova-migration

More Development Tools