AI/ML skills including prompt engineering, RAG, and chunking strategies

version
4.0.1
GitHub stars
372

Install

/plugin install developer-kit-ai@developer-kit

Run it inside Claude Code after adding the marketplace with /plugin marketplace add giuseppe-trisciuoglio/developer-kit.

Contents

Discovered from source

Skills 3

Chunking Strategy for RAG Systems

Overview

Provides chunking strategies for RAG systems, vector databases, and document processing. Recommends chunk sizes, overlap percentages, and boundary detection methods; validates semantic coherence; evaluates retrieval metrics.

When to Use

Use when building or optimizing RAG systems, vector search pipelines, document chunking workflows, or performance-tuning existing systems with poor retrieval quality.

Instructions

Choose Chunking Strategy

Select based on document type and use case:

  1. Fixed-Size Chunking (Level 1)

    • Use for simple documents without clear structure
    • Start with 512 tokens and 10-20% overlap
    • Adjust: 256 for factoid queries, 1024 for analytical
  2. Recursive Character Chunking (Level 2)

    • Use for documents with structural boundaries
    • Hierarchical separators: paragraphs → sentences → words
    • Customize for document types (HTML, Markdown, JSON)
  3. Structure-Aware Chunking (Level 3)

    • Use for structured content (Markdown, code, tables, PDFs)
    • Preserve semantic units: functions, sections, table blocks
    • Validate structure preservation post-split
  4. Semantic Chunking (Level 4)

    • Use for complex documents with thematic shifts
    • Embedding-based boundary detection with 0.8 similarity threshold
    • Buffer size: 3-5 sentences
  5. Advanced Methods (Level 5)

    • Late Chunking for long-context models
    • Contextual Retrieval for high-precision requirements
    • Monitor computational cost vs. retrieval gain

Reference: references/strategies.md.

Implement Chunking Pipeline

  1. Pre-process documents

    • Analyze structure, content types, information density
    • Identify multi-modal content (tables, images, code)
  2. Select parameters

    • Chunk size: embedding model context window / 4
    • Overlap: 10-20% for most cases
    • Strategy-specific settings
  3. Process and validate

    • Apply chunking strategy
    • Validate coherence: run evaluate_chunks.py --coherence (see below)
    • Test with representative documents
  4. Evaluate and iterate

    • Measure precision and recall
    • If precision < 0.7: reduce chunk_size by 25% and re-evaluate
    • If recall < 0.6: increase overlap by 10% and re-evaluate
    • Monitor latency and memory usage

Reference: references/implementation.md.

Validate Chunk Quality

Run validation commands to assess chunk quality:

# Check semantic coherence (requires sentence-transformers)
python -c "
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')
chunks = [...]  # your chunks
embeddings = model.encode(chunks)
similarity = (embeddings @ embeddings.T).mean()
print(f'Cohesion: {similarity:.3f}')  # target: 0.3-0.7
"

# Measure retrieval precision
python -c "
relevant = sum(1 for c in retrieved if c in relevant_chunks)
precision = relevant / len(retrieved)
print(f'Precision: {precision:.2f}')  # target: >= 0.7
"

# Check chunk size distribution
python -c "
import numpy as np
sizes = [len(c.split()) for c in chunks]
print(f'Mean: {np.mean(sizes):.0f}, Std: {np.std(sizes):.0f}')
print(f'Min: {min(sizes)}, Max: {max(sizes)}')
"

Reference: references/evaluation.md.

Examples

Fixed-Size Chunking

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=256,
    chunk_overlap=25,
    length_function=len
)
chunks = splitter.split_documents(documents)

Structure-Aware Code Chunking

import ast

def chunk_python_code(code):
    tree = ast.parse(code)
    chunks = []
    for node in ast.walk(tree):
        if isinstance(node, (ast.FunctionDef, ast.ClassDef)):
            chunks.append(ast.get_source_segment(code, node))
    return chunks

Semantic Chunking

def semantic_chunk(text, similarity_threshold=0.8):
    sentences = split_into_sentences(text)
    embeddings = generate_embeddings(sentences)
    chunks, current = [], [sentences[0]]
    for i in range(1, len(sentences)):
        sim = cosine_similarity(embeddings[i-1], embeddings[i])
        if sim < similarity_threshold:
            chunks.append(" ".join(current))
            current = [sentences[i]]
        else:
            current.append(sentences[i])
    chunks.append(" ".join(current))
    return chunks

Best Practices

Core Principles

  • Balance context preservation with retrieval precision
  • Maintain semantic coherence within chunks
  • Optimize for embedding model context window constraints

Implementation

  • Start with fixed-size (512 tokens, 15% overlap)
  • Iterate based on document characteristics
  • Test with domain-specific documents before deployment

Pitfalls to Avoid

  • Over-chunking: context-poor small chunks
  • Under-chunking: missing information in oversized chunks
  • Ignoring semantic boundaries and document structure
  • One-size-fits-all for diverse content types

Constraints and Warnings

Resource Considerations

  • Semantic methods require significant compute resources
  • Late chunking needs long-context embedding models
  • Complex strategies increase processing latency
  • Monitor memory for large document batches

Quality Requirements

  • Validate semantic coherence post-processing
  • Test with representative documents before deployment
  • Ensure chunks maintain standalone meaning
  • Implement error handling for malformed content

References

Prompt Engineering

Overview

Use this skill to design prompt systems that are clear, testable, and reusable. It covers prompt drafting, optimization, evaluation, and production-oriented patterns for few-shot prompting, reasoning workflows, templates, and system prompts.

Keep the main workflow in this file and load the targeted reference files only for the pattern you are applying.

When to Use

Use this skill when:

  • A user asks to write, rewrite, or improve a prompt
  • A prompt needs better structure, reliability, or output formatting
  • Few-shot examples or reasoning scaffolds are needed
  • A system prompt or reusable prompt template must be created
  • An existing prompt needs measurable optimization and testing

Read the relevant files in references/ when you need deeper guidance on a specific pattern.

Core Patterns

1. Few-Shot Learning

Example Selection Strategy
  • Use references/few-shot-patterns.md for comprehensive selection frameworks
  • Balance example count (3-5 optimal) with context window limitations
  • Include edge cases and boundary conditions in example sets
  • Prioritize diverse examples that cover problem space variations
  • Order examples from simple to complex for progressive learning
Few-Shot Example (Sentiment Classification)
Classify the sentiment as Positive, Negative, or Neutral.

Text: "I love this product! It exceeded my expectations."
Sentiment: Positive
Reasoning: Enthusiastic language, positive adjectives, satisfaction

Text: "The app keeps crashing when I upload large files."
Sentiment: Negative
Reasoning: Complaint about functionality, frustration indicator

Text: "It arrived on time, as described."
Sentiment: Neutral
Reasoning: Factual statement, no strong emotion either way

Text: "{user_input}"
Sentiment:
Reasoning:

2. Chain-of-Thought Reasoning

Implementation Patterns
  • Reference references/cot-patterns.md for detailed reasoning frameworks
  • Use "Let's think step by step" for zero-shot CoT initiation
  • Provide complete reasoning traces for few-shot CoT demonstrations
  • Implement self-consistency by sampling multiple reasoning paths
  • Include verification and validation steps in reasoning chains
CoT Template Structure
Let's approach this step-by-step:

Step 1: {break_down_the_problem}
Analysis: {detailed_reasoning}

Step 2: {identify_key_components}
Analysis: {component_analysis}

Step 3: {synthesize_solution}
Analysis: {solution_justification}

Final Answer: {conclusion_with_confidence}

3. Prompt Optimization

Optimization Process
  • Use references/optimization-frameworks.md for comprehensive optimization strategies
  • Measure baseline performance before optimization attempts
  • Implement single-variable changes for accurate attribution
  • Track metrics: accuracy, consistency, latency, token efficiency
  • Use statistical significance testing for A/B validation
  • Document optimization iterations and their impacts

Track these metrics: accuracy, consistency, token efficiency, robustness, safety. See references/optimization-frameworks.md for measurement utilities.

4. Template Systems

Template Design Principles
  • Reference references/template-systems.md for modular template frameworks
  • Use clear variable naming conventions (e.g., {user_input}, {context})
  • Implement conditional sections for different scenario handling
  • Design role-based templates for specific use cases
  • Create hierarchical template composition patterns
Template Structure Example
# System Context
You are a {role} with {expertise_level} expertise in {domain}.

# Task Context
{if background_information}
Background: {background_information}
{endif}

# Instructions
{task_instructions}

# Examples
{example_count}

# Output Format
{output_specification}

# Input
{user_query}

5. System Prompt Design

System Prompt Components
  • Use references/system-prompt-design.md for detailed design guidelines
  • Define clear role specification and expertise boundaries
  • Establish output format requirements and structural constraints
  • Include safety guidelines and content policy adherence
  • Set context for background information and domain knowledge
System Prompt Framework
You are an expert {role} specializing in {domain} with {experience_level} of experience.

## Core Capabilities
- List specific capabilities and expertise areas
- Define scope of knowledge and limitations

## Behavioral Guidelines
- Specify interaction style and communication approach
- Define error handling and uncertainty protocols
- Establish quality standards and verification requirements

## Output Requirements
- Specify format expectations and structural requirements
- Define content inclusion and exclusion criteria
- Establish consistency and validation requirements

## Safety and Ethics
- Include content policy adherence
- Specify bias mitigation requirements
- Define harm prevention protocols

Implementation Workflows

Workflow 1: Create New Prompt from Requirements

  1. Analyze Requirements

    • Identify task complexity and reasoning requirements
    • Determine target model capabilities and limitations
    • Define success criteria and evaluation metrics
    • Assess need for few-shot learning or CoT reasoning
  2. Select Pattern Strategy

    • Use few-shot learning for classification or transformation tasks
    • Apply CoT for complex reasoning or multi-step problems
    • Implement template systems for reusable prompt architecture
    • Design system prompts for consistent behavior requirements
  3. Draft Initial Prompt

    • Structure prompt with clear sections and logical flow
    • Include relevant examples or reasoning demonstrations
    • Specify output format and quality requirements
    • Incorporate safety guidelines and constraints
  4. Validate and Test

    • Test with at least 3 inputs: one happy path, one edge case, one adversarial
    • Measure accuracy and token usage against defined success criteria
    • Change one variable at a time, re-test, keep only what improves metrics
    • Document optimization decisions and their rationale

Workflow 2: Optimize Existing Prompt

  1. Performance Analysis

    • Measure current prompt performance metrics
    • Identify failure modes and error patterns
    • Analyze token efficiency and response latency
    • Assess consistency across multiple runs
  2. Optimization Strategy

    • Apply systematic A/B testing with single-variable changes
    • Use few-shot learning to improve task adherence
    • Implement CoT reasoning for complex task components
    • Refine template structure for better clarity
  3. Implementation and Testing

    • Re-run the same test cases from step 1 against the optimized prompt
    • If accuracy < baseline, revert the change and try a different hypothesis
    • If accuracy >= baseline but < 90%, return to step 2 with a new strategy
    • Document the winning change and its measured impact

Workflow 3: Scale Prompt Systems

  1. Modular Architecture Design

    • Decompose complex prompts into reusable components
    • Create template inheritance hierarchies
    • Implement dynamic example selection systems
    • Build automated quality assurance frameworks
  2. Production Integration

    • Implement prompt versioning and rollback capabilities
    • Create performance monitoring and alerting systems
    • Build automated testing frameworks for prompt validation
    • Establish update and deployment workflows

Quality Gates

  • Accuracy >90% on 10+ diverse test cases before shipping
  • <5% variance across 3+ repeated runs
  • All edge cases and adversarial inputs handled gracefully
  • Output format matches spec on every test case

Best Practices

  • Optimize one variable at a time so results stay attributable
  • Keep prompts explicit about task, context, constraints, and output format
  • Prefer a small number of strong examples over many repetitive ones
  • Test prompts against happy-path, edge-case, and adversarial inputs
  • Move long pattern details to references/ instead of bloating SKILL.md

Constraints and Warnings

  • Do not assume longer prompts are better; extra detail often adds ambiguity
  • Avoid exposing hidden reasoning requirements when a concise rationale is enough
  • Validate prompts on representative inputs before claiming improvement
  • Keep model-specific assumptions explicit because behavior varies across models

Integration with Other Skills

This skill integrates seamlessly with:

  • langchain4j-ai-services-patterns: Interface-based prompt design
  • langchain4j-rag-implementation-patterns: Context-enhanced prompting
  • langchain4j-testing-strategies: Prompt validation frameworks
  • unit-test-parameterized: Systematic prompt testing approaches

Resources and References

  • references/few-shot-patterns.md: Comprehensive few-shot learning frameworks
  • references/cot-patterns.md: Chain-of-thought reasoning patterns and examples
  • references/optimization-frameworks.md: Systematic prompt optimization methodologies
  • references/template-systems.md: Modular template design and implementation
  • references/system-prompt-design.md: System prompt architecture and best practices

Common Pitfalls and Solutions

PitfallFix
Wrong output formatAdd a concrete output example at the end of the prompt
Inconsistent answersAdd 2-3 few-shot examples showing expected reasoning
HallucinationAdd "If unsure, say 'I don't know'" + constrain the answer domain
Too verboseAdd explicit word/sentence limit + "Be concise" instruction
Missed edge casesAdd an edge-case few-shot example

Constraints

  • Test across target models — capabilities and token limits vary
  • Keep few-shot examples to 3-5 to manage context usage
  • Validate with domain-specific test cases before production

RAG Implementation

Build Retrieval-Augmented Generation systems that extend AI capabilities with external knowledge sources.

Overview

This skill covers: document processing, embedding generation, vector storage, retrieval configuration, and RAG pipeline implementation.

When to Use

  • Building Q&A systems over proprietary documents
  • Creating chatbots with factual information from knowledge bases
  • Implementing semantic search with natural language queries
  • Reducing hallucinations with grounded, sourced responses
  • Building documentation assistants and research tools
  • Enabling AI systems to access domain-specific knowledge

Instructions

Step 1: Choose Vector Database

Select based on your requirements:

RequirementRecommended
Production scalabilityPinecone, Milvus
Open-sourceWeaviate, Qdrant
Local developmentChroma, FAISS
Hybrid searchWeaviate with BM25

Step 2: Select Embedding Model

Use CaseModel
General purposetext-embedding-ada-002
Fast and lightweightall-MiniLM-L6-v2
Multilinguale5-large-v2
Best performancebge-large-en-v1.5

Step 3: Implement Document Processing Pipeline

  1. Load documents from source (file system, database, API)
  2. Clean and preprocess (remove formatting, normalize text)
  3. Split documents into chunks with appropriate strategy
  4. Generate embeddings for each chunk
  5. Store embeddings in vector database with metadata

Validation: Verify embeddings were generated successfully:

List<Embedding> embeddings = embeddingModel.embedAll(segments);
if (embeddings.isEmpty() || embeddings.get(0).dimension() != expectedDim) {
    throw new IllegalStateException("Embedding generation failed");
}

Step 4: Configure Retrieval Strategy

Choose the appropriate strategy:

  • Dense Retrieval: Semantic similarity via embeddings (default for most cases)
  • Hybrid Search: Dense + sparse retrieval for better coverage
  • Metadata Filtering: Filter by document attributes
  • Reranking: Cross-encoder reranking for high-precision requirements

Step 5: Build RAG Pipeline

  1. Create content retriever with your embedding store
  2. Configure AI service with retriever and chat memory
  3. Implement prompt template with context injection
  4. Add response validation and grounding checks

Validation: Test with known queries to verify context injection works correctly.

Error Handling: For batch ingestion, wrap in retry logic:

for (Document doc : documents) {
    int attempts = 0;
    while (attempts < 3) {
        try {
            store.add(embeddingModel.embed(doc).content(), doc.toTextSegment());
            break;
        } catch (EmbeddingException e) {
            attempts++;
            if (attempts == 3) throw new RuntimeException("Failed after 3 retries", e);
        }
    }
}

Step 6: Evaluate and Optimize

  1. Measure retrieval metrics: precision@k, recall@k, MRR
  2. Evaluate answer quality: faithfulness, relevance
  3. Monitor performance and user feedback
  4. Iterate on chunking, retrieval, and prompt parameters

Examples

Example 1: Basic Document Q&A

List<Document> documents = FileSystemDocumentLoader.loadDocuments("/docs");

InMemoryEmbeddingStore<TextSegment> store = new InMemoryEmbeddingStore<>();
EmbeddingStoreIngestor.ingest(documents, store);

DocumentAssistant assistant = AiServices.builder(DocumentAssistant.class)
    .chatModel(chatModel)
    .contentRetriever(EmbeddingStoreContentRetriever.from(store))
    .build();

String answer = assistant.answer("What is the company policy on remote work?");

Example 2: Metadata-Filtered Retrieval

EmbeddingStoreContentRetriever retriever = EmbeddingStoreContentRetriever.builder()
    .embeddingStore(store)
    .embeddingModel(embeddingModel)
    .maxResults(5)
    .minScore(0.7)
    .filter(metadataKey("category").isEqualTo("technical"))
    .build();

Example 3: Multi-Source RAG Pipeline

ContentRetriever webRetriever = EmbeddingStoreContentRetriever.from(webStore);
ContentRetriever docRetriever = EmbeddingStoreContentRetriever.from(docStore);

List<Content> results = new ArrayList<>();
results.addAll(webRetriever.retrieve(query));
results.addAll(docRetriever.retrieve(query));

List<Content> topResults = reranker.reorder(query, results).subList(0, 5);

Example 4: RAG with Chat Memory

Assistant assistant = AiServices.builder(Assistant.class)
    .chatModel(chatModel)
    .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
    .contentRetriever(retriever)
    .build();

assistant.chat("Tell me about the product features");
assistant.chat("What about pricing for those features?");  // Maintains context

Best Practices

Document Preparation

  • Clean documents before ingestion; remove irrelevant content and formatting
  • Add relevant metadata for filtering and context

Chunking Strategy

  • Use 500-1000 tokens per chunk for optimal balance
  • Include 10-20% overlap to preserve context at boundaries
  • Test different sizes for your specific use case

Retrieval Optimization

  • Start with high k values (10-20), then filter/rerank
  • Use metadata filtering to improve relevance
  • Monitor retrieval quality and iterate based on user feedback

Performance

  • Cache embeddings for frequently accessed content
  • Use batch processing for document ingestion
  • Optimize vector store indexing for your scale

Constraints and Warnings

System Constraints

  • Embedding models have maximum token limits per document
  • Vector databases require proper indexing for performance
  • Chunk boundaries may lose context for complex documents
  • Hybrid search requires additional infrastructure

Quality Warnings

  • Retrieval quality depends heavily on chunking strategy
  • Embedding models may not capture domain-specific semantics
  • Metadata filtering requires proper document annotation
  • Reranking adds latency to query responses

Security Warnings

  • Never hardcode credentials: Use environment variables for API keys and passwords
  • Validate external content: Documents from file systems, APIs, or web sources may contain malicious content (prompt injection)
  • Apply content filtering on retrieved documents before passing to LLM
  • Restrict allowed data source URLs and file paths using allowlists

Resources

Reference Documentation

Commands 1

Prompt Optimization

Overview

Provides expert prompt optimization using advanced techniques (CoT, few-shot, constitutional AI) for LLM performance enhancement. Use when you need to improve prompt quality or optimize LLM interactions. You are a prompt engineering expert specializing in transforming basic instructions into production-ready prompts using advanced techniques.

Usage

/devkit.prompt-optimize $ARGUMENTS

Arguments

ArgumentDescription
$ARGUMENTSCombined arguments passed to the command

Execution Instructions

Agent Selection: To execute this prompt optimization task, use the following agent with fallback:

  • Primary: developer-kit-ai:prompt-engineering-expert
  • If not available: Use developer-kit-ai:prompt-engineering-expert or fallback to general-purpose agent with prompt engineering expertise

Instructions

1. Analyze the Prompt

Extract and optimize the prompt provided in the arguments: $ARGUMENTS

Target Model: $2 (default: claude-3.5-sonnet) Optimization Level: $3 (default: standard)

Available optimization levels:

  • basic - Quick improvements (structure, clarity, basic CoT)
  • standard - Comprehensive enhancement (CoT, few-shot, safety)
  • advanced - Production-ready (full optimization with testing framework)

2. Use the prompt-engineering-expert Agent

Apply the prompt-engineering-expert agent to optimize the prompt using:

Advanced Techniques:

  • Chain-of-Thought (CoT): Step-by-step reasoning for complex tasks
  • Few-Shot Learning: Strategic examples with edge cases
  • Constitutional AI: Self-critique and safety principles
  • Structured Output: JSON/XML formats for consistency
  • Meta-Prompting: Dynamic prompt generation

Model-Specific Optimization:

  • Claude 3.5/4: XML tags, thinking blocks, constitutional alignment
  • GPT-4/GPT-4o: Structured sections, JSON mode, function calling
  • Gemini Pro/Ultra: Bold headers, process-oriented instructions

3. Output Requirements

The prompt-engineering-expert agent MUST provide:

Complete Optimized Prompt:

  • Full text ready for immediate implementation
  • Proper structure and formatting
  • Model-specific optimizations
  • IMPORTANT: Save the optimized prompt to a file named optimized-prompt.md

Optimization Report:

  • Original prompt assessment (strengths/weaknesses)
  • Applied techniques with impact metrics
  • Performance projections (success rate, quality, cost)
  • Testing recommendations and deployment strategy

Implementation Guidelines:

  • Model parameters and settings
  • Safety and compliance considerations
  • Monitoring and iteration recommendations

4. Specialized Optimization Patterns

For Document Analysis Tasks:

  • RAG integration with source citation
  • Cross-reference analysis capabilities
  • Information extraction frameworks

For Code Comprehension Tasks:

  • Architecture analysis patterns
  • Security vulnerability detection
  • Refactoring recommendation systems

For Multi-Step Reasoning:

  • Tree-of-thoughts exploration
  • Self-consistency verification
  • Error handling and recovery

5. Quality Assurance

The optimized prompt must:

  • Include the complete prompt text in a marked section
  • Address the original requirements comprehensively
  • Incorporate safety and ethical considerations
  • Provide clear testing and evaluation frameworks
  • Be production-ready with deployment guidance

Examples

/devkit.prompt-optimize example-input

Agents 1

You are an expert prompt engineer specializing in crafting high-performance prompts for LLMs and optimizing AI system performance.

When invoked:

  1. Analyze the prompt requirements and target use case
  2. Select appropriate prompting techniques (CoT, few-shot, etc.)
  3. Design the complete prompt with clear structure
  4. Provide the full prompt text in a marked section
  5. Include implementation notes and optimization guidance

Prompt Engineering Checklist

  • Advanced Techniques: Chain-of-thought, constitutional AI, meta-prompting
  • Document Analysis: Information extraction, semantic search, summarization
  • Code Comprehension: Architecture analysis, security review, documentation generation
  • Multi-Agent Systems: Role definition, collaboration protocols, workflow orchestration
  • Production Optimization: Token efficiency, cost control, performance monitoring
  • Safety & Ethics: Content moderation, bias mitigation, constitutional principles

Core Expertise

1. Advanced Prompting Techniques

  • Chain-of-Thought (CoT): Step-by-step reasoning for complex problem-solving
  • Constitutional AI: Self-correction and alignment principles
  • Few-Shot Learning: Carefully crafted examples for pattern learning
  • Meta-Prompting: Dynamic prompt generation and optimization
  • Self-Consistency: Multiple reasoning chains for reliability
  • Program-Aided Language Models: Integration with computational tools

2. Document & Information Retrieval

  • Document Analysis: Extract key information from technical specifications, contracts, reports
  • Semantic Search: Intent-based information retrieval from large corpuses
  • Cross-Reference Analysis: Correlate information across multiple documents
  • Intelligent Summarization: Preserve critical details while filtering noise
  • Knowledge Extraction: Retrieve specific information from complex documentation
  • Legal & Technical Analysis: Specialized prompts for contracts and specifications

3. Code Comprehension & Analysis

  • Architecture Analysis: Identify patterns, dependencies, and relationships
  • Security Review: Detect vulnerabilities and suggest remediation steps
  • Documentation Generation: Create clear technical documentation from code
  • Test Case Generation: Generate comprehensive tests from code analysis
  • Refactoring Suggestions: Identify code smells and improvement opportunities
  • Performance Analysis: Evaluate efficiency and optimization potential

4. Multi-Agent Systems

  • Role Definition: Create specialized agent personas and capabilities
  • Collaboration Protocols: Design inter-agent communication patterns
  • Workflow Orchestration: Task decomposition and agent coordination
  • Memory Management: Shared context and knowledge persistence
  • Conflict Resolution: Handle disagreements between agents
  • Performance Monitoring: Track and optimize multi-agent efficiency

5. Production Optimization

  • Token Efficiency: Minimize costs while maintaining performance
  • Response Time Optimization: Reduce latency for time-sensitive applications
  • A/B Testing: Frameworks for systematic prompt improvement
  • Performance Monitoring: Track key metrics and success rates
  • Scalability Design: Build prompts that work at production scale
  • Error Handling: Robust failure recovery and graceful degradation

6. Model-Specific Optimization

  • Anthropic Claude: Constitutional AI, XML structuring, computer use prompts
  • OpenAI GPT: Function calling, JSON mode, system message design
  • Open Source Models: Special tokens, quantization considerations
  • Multimodal Models: Vision-language integration, cross-modal reasoning

Skills Integration

This agent leverages knowledge from and can autonomously invoke the following specialized skills:

LangChain4j AI Skills (7 skills)

  • langchain4j-ai-services-patterns - Interface-based AI service design
  • langchain4j-rag-implementation-patterns - Retrieval-augmented generation
  • langchain4j-testing-strategies - AI-powered application testing
  • langchain4j-tool-function-calling - Tool integration patterns
  • langchain4j-spring-boot-integration - Spring Boot integration patterns
  • langchain4j-mcp-server-patterns - Model Context Protocol servers
  • langchain4j-vector-stores-configuration - Vector store optimization

Usage Pattern: This agent will automatically invoke relevant skills when creating prompts for AI-powered applications. For example, when building RAG prompts, it may use langchain4j-rag-implementation-patterns; when designing AI services, it may use langchain4j-ai-services-patterns and langchain4j-spring-boot-integration.

Prompt Design Process

Phase 1: Analysis & Requirements

  1. Understand the use case and identify the target LLM model
  2. Analyze input/output requirements and performance constraints
  3. Identify success criteria and evaluation metrics
  4. Consider safety and ethical implications

Phase 2: Prompt Design

  1. Select appropriate techniques (CoT, few-shot, meta-prompting)
  2. Design prompt architecture with clear structure and flow
  3. Write the complete prompt text following established patterns
  4. Include testing guidelines and edge case considerations

Phase 3: Implementation & Testing

  1. Display the complete prompt in a clearly marked section
  2. Provide implementation notes and parameter recommendations
  3. Include evaluation criteria and testing approaches
  4. Document safety considerations and failure modes

Best Practices

  • Always show the complete prompt text in a marked section
  • Consider token efficiency and cost optimization in all designs
  • Implement safety measures and ethical guidelines
  • Test thoroughly with edge cases and failure scenarios
  • Monitor performance and iterate based on metrics
  • Document usage guidelines for production deployment

For each prompt design, provide:

  • The Complete Prompt: Full text ready for immediate use
  • Implementation Notes: Techniques used and design rationale
  • Testing & Evaluation: Test cases and success metrics
  • Usage Guidelines: When and how to use effectively
  • Performance Optimization: Cost and efficiency considerations

Common Patterns

Critical Requirements (Must Include)

  • Complete prompt text in clearly marked section
  • Clear instructions with step-by-step guidance
  • Output format specification and examples
  • Error handling and edge case coverage
  • Safety considerations and ethical guidelines

High Priority (Should Include)

  • Token optimization for cost efficiency
  • Model-specific tuning parameters
  • Testing framework with evaluation metrics
  • A/B testing recommendations
  • Integration guidelines for production

Medium Priority (Consider Adding)

  • Alternative prompt variations for different constraints
  • Performance benchmarking against baseline
  • Scalability considerations for high volume
  • Multi-language support if applicable
  • Advanced features (multi-modal, tool integration)

Role

Specialized Prompt Engineering expert focused on prompt engineering and AI optimization. This agent provides deep expertise in Prompt Engineering development practices, ensuring high-quality, maintainable, and production-ready solutions.

Process

  1. Requirements Analysis: Understand the task requirements and constraints
  2. Planning: Design the approach and identify necessary components
  3. Implementation: Build the solution following best practices and patterns
  4. Testing: Verify the implementation with appropriate tests
  5. Review: Validate quality, security, and performance considerations
  6. Documentation: Ensure proper documentation and code comments

Output Format

Structure all responses as follows:

  1. Analysis: Brief assessment of the current state or requirements
  2. Recommendations: Detailed suggestions with rationale
  3. Implementation: Code examples and step-by-step guidance
  4. Considerations: Trade-offs, caveats, and follow-up actions

Sponsors

Details

Marketplace
giuseppe-trisciuoglio/developer-kit
Source
plugins/developer-kit-ai
Category
Community

More from giuseppe-trisciuoglio/developer-kit

All 11