
developer-kit-ai
AI/ML skills including prompt engineering, RAG, and chunking strategies
- version
- 4.0.1
- GitHub stars
- 372
Install
/plugin install developer-kit-ai@developer-kitRun it inside Claude Code after adding the marketplace with /plugin marketplace add giuseppe-trisciuoglio/developer-kit.
Contents
Discovered from sourceSkills 3
Chunking Strategy for RAG Systems
Overview
Provides chunking strategies for RAG systems, vector databases, and document processing. Recommends chunk sizes, overlap percentages, and boundary detection methods; validates semantic coherence; evaluates retrieval metrics.
When to Use
Use when building or optimizing RAG systems, vector search pipelines, document chunking workflows, or performance-tuning existing systems with poor retrieval quality.
Instructions
Choose Chunking Strategy
Select based on document type and use case:
-
Fixed-Size Chunking (Level 1)
- Use for simple documents without clear structure
- Start with 512 tokens and 10-20% overlap
- Adjust: 256 for factoid queries, 1024 for analytical
-
Recursive Character Chunking (Level 2)
- Use for documents with structural boundaries
- Hierarchical separators: paragraphs → sentences → words
- Customize for document types (HTML, Markdown, JSON)
-
Structure-Aware Chunking (Level 3)
- Use for structured content (Markdown, code, tables, PDFs)
- Preserve semantic units: functions, sections, table blocks
- Validate structure preservation post-split
-
Semantic Chunking (Level 4)
- Use for complex documents with thematic shifts
- Embedding-based boundary detection with 0.8 similarity threshold
- Buffer size: 3-5 sentences
-
Advanced Methods (Level 5)
- Late Chunking for long-context models
- Contextual Retrieval for high-precision requirements
- Monitor computational cost vs. retrieval gain
Reference: references/strategies.md.
Implement Chunking Pipeline
-
Pre-process documents
- Analyze structure, content types, information density
- Identify multi-modal content (tables, images, code)
-
Select parameters
- Chunk size: embedding model context window / 4
- Overlap: 10-20% for most cases
- Strategy-specific settings
-
Process and validate
- Apply chunking strategy
- Validate coherence: run
evaluate_chunks.py --coherence(see below) - Test with representative documents
-
Evaluate and iterate
- Measure precision and recall
- If precision < 0.7: reduce chunk_size by 25% and re-evaluate
- If recall < 0.6: increase overlap by 10% and re-evaluate
- Monitor latency and memory usage
Reference: references/implementation.md.
Validate Chunk Quality
Run validation commands to assess chunk quality:
# Check semantic coherence (requires sentence-transformers)
python -c "
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')
chunks = [...] # your chunks
embeddings = model.encode(chunks)
similarity = (embeddings @ embeddings.T).mean()
print(f'Cohesion: {similarity:.3f}') # target: 0.3-0.7
"
# Measure retrieval precision
python -c "
relevant = sum(1 for c in retrieved if c in relevant_chunks)
precision = relevant / len(retrieved)
print(f'Precision: {precision:.2f}') # target: >= 0.7
"
# Check chunk size distribution
python -c "
import numpy as np
sizes = [len(c.split()) for c in chunks]
print(f'Mean: {np.mean(sizes):.0f}, Std: {np.std(sizes):.0f}')
print(f'Min: {min(sizes)}, Max: {max(sizes)}')
"
Reference: references/evaluation.md.
Examples
Fixed-Size Chunking
from langchain.text_splitter import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(
chunk_size=256,
chunk_overlap=25,
length_function=len
)
chunks = splitter.split_documents(documents)
Structure-Aware Code Chunking
import ast
def chunk_python_code(code):
tree = ast.parse(code)
chunks = []
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.ClassDef)):
chunks.append(ast.get_source_segment(code, node))
return chunks
Semantic Chunking
def semantic_chunk(text, similarity_threshold=0.8):
sentences = split_into_sentences(text)
embeddings = generate_embeddings(sentences)
chunks, current = [], [sentences[0]]
for i in range(1, len(sentences)):
sim = cosine_similarity(embeddings[i-1], embeddings[i])
if sim < similarity_threshold:
chunks.append(" ".join(current))
current = [sentences[i]]
else:
current.append(sentences[i])
chunks.append(" ".join(current))
return chunks
Best Practices
Core Principles
- Balance context preservation with retrieval precision
- Maintain semantic coherence within chunks
- Optimize for embedding model context window constraints
Implementation
- Start with fixed-size (512 tokens, 15% overlap)
- Iterate based on document characteristics
- Test with domain-specific documents before deployment
Pitfalls to Avoid
- Over-chunking: context-poor small chunks
- Under-chunking: missing information in oversized chunks
- Ignoring semantic boundaries and document structure
- One-size-fits-all for diverse content types
Constraints and Warnings
Resource Considerations
- Semantic methods require significant compute resources
- Late chunking needs long-context embedding models
- Complex strategies increase processing latency
- Monitor memory for large document batches
Quality Requirements
- Validate semantic coherence post-processing
- Test with representative documents before deployment
- Ensure chunks maintain standalone meaning
- Implement error handling for malformed content
References
- strategies.md - Detailed strategies
- implementation.md - Implementation guidelines
- evaluation.md - Performance metrics
- tools.md - Libraries and frameworks
- research.md - Research papers
- advanced-strategies.md - 11 advanced methods
- semantic-methods.md - Semantic approaches
- visualization-tools.md - Visualization tools
Prompt Engineering
Overview
Use this skill to design prompt systems that are clear, testable, and reusable. It covers prompt drafting, optimization, evaluation, and production-oriented patterns for few-shot prompting, reasoning workflows, templates, and system prompts.
Keep the main workflow in this file and load the targeted reference files only for the pattern you are applying.
When to Use
Use this skill when:
- A user asks to write, rewrite, or improve a prompt
- A prompt needs better structure, reliability, or output formatting
- Few-shot examples or reasoning scaffolds are needed
- A system prompt or reusable prompt template must be created
- An existing prompt needs measurable optimization and testing
Read the relevant files in references/ when you need deeper guidance on a
specific pattern.
Core Patterns
1. Few-Shot Learning
Example Selection Strategy
- Use
references/few-shot-patterns.mdfor comprehensive selection frameworks - Balance example count (3-5 optimal) with context window limitations
- Include edge cases and boundary conditions in example sets
- Prioritize diverse examples that cover problem space variations
- Order examples from simple to complex for progressive learning
Few-Shot Example (Sentiment Classification)
Classify the sentiment as Positive, Negative, or Neutral.
Text: "I love this product! It exceeded my expectations."
Sentiment: Positive
Reasoning: Enthusiastic language, positive adjectives, satisfaction
Text: "The app keeps crashing when I upload large files."
Sentiment: Negative
Reasoning: Complaint about functionality, frustration indicator
Text: "It arrived on time, as described."
Sentiment: Neutral
Reasoning: Factual statement, no strong emotion either way
Text: "{user_input}"
Sentiment:
Reasoning:
2. Chain-of-Thought Reasoning
Implementation Patterns
- Reference
references/cot-patterns.mdfor detailed reasoning frameworks - Use "Let's think step by step" for zero-shot CoT initiation
- Provide complete reasoning traces for few-shot CoT demonstrations
- Implement self-consistency by sampling multiple reasoning paths
- Include verification and validation steps in reasoning chains
CoT Template Structure
Let's approach this step-by-step:
Step 1: {break_down_the_problem}
Analysis: {detailed_reasoning}
Step 2: {identify_key_components}
Analysis: {component_analysis}
Step 3: {synthesize_solution}
Analysis: {solution_justification}
Final Answer: {conclusion_with_confidence}
3. Prompt Optimization
Optimization Process
- Use
references/optimization-frameworks.mdfor comprehensive optimization strategies - Measure baseline performance before optimization attempts
- Implement single-variable changes for accurate attribution
- Track metrics: accuracy, consistency, latency, token efficiency
- Use statistical significance testing for A/B validation
- Document optimization iterations and their impacts
Track these metrics: accuracy, consistency, token efficiency, robustness, safety. See references/optimization-frameworks.md for measurement utilities.
4. Template Systems
Template Design Principles
- Reference
references/template-systems.mdfor modular template frameworks - Use clear variable naming conventions (e.g.,
{user_input},{context}) - Implement conditional sections for different scenario handling
- Design role-based templates for specific use cases
- Create hierarchical template composition patterns
Template Structure Example
# System Context
You are a {role} with {expertise_level} expertise in {domain}.
# Task Context
{if background_information}
Background: {background_information}
{endif}
# Instructions
{task_instructions}
# Examples
{example_count}
# Output Format
{output_specification}
# Input
{user_query}
5. System Prompt Design
System Prompt Components
- Use
references/system-prompt-design.mdfor detailed design guidelines - Define clear role specification and expertise boundaries
- Establish output format requirements and structural constraints
- Include safety guidelines and content policy adherence
- Set context for background information and domain knowledge
System Prompt Framework
You are an expert {role} specializing in {domain} with {experience_level} of experience.
## Core Capabilities
- List specific capabilities and expertise areas
- Define scope of knowledge and limitations
## Behavioral Guidelines
- Specify interaction style and communication approach
- Define error handling and uncertainty protocols
- Establish quality standards and verification requirements
## Output Requirements
- Specify format expectations and structural requirements
- Define content inclusion and exclusion criteria
- Establish consistency and validation requirements
## Safety and Ethics
- Include content policy adherence
- Specify bias mitigation requirements
- Define harm prevention protocols
Implementation Workflows
Workflow 1: Create New Prompt from Requirements
-
Analyze Requirements
- Identify task complexity and reasoning requirements
- Determine target model capabilities and limitations
- Define success criteria and evaluation metrics
- Assess need for few-shot learning or CoT reasoning
-
Select Pattern Strategy
- Use few-shot learning for classification or transformation tasks
- Apply CoT for complex reasoning or multi-step problems
- Implement template systems for reusable prompt architecture
- Design system prompts for consistent behavior requirements
-
Draft Initial Prompt
- Structure prompt with clear sections and logical flow
- Include relevant examples or reasoning demonstrations
- Specify output format and quality requirements
- Incorporate safety guidelines and constraints
-
Validate and Test
- Test with at least 3 inputs: one happy path, one edge case, one adversarial
- Measure accuracy and token usage against defined success criteria
- Change one variable at a time, re-test, keep only what improves metrics
- Document optimization decisions and their rationale
Workflow 2: Optimize Existing Prompt
-
Performance Analysis
- Measure current prompt performance metrics
- Identify failure modes and error patterns
- Analyze token efficiency and response latency
- Assess consistency across multiple runs
-
Optimization Strategy
- Apply systematic A/B testing with single-variable changes
- Use few-shot learning to improve task adherence
- Implement CoT reasoning for complex task components
- Refine template structure for better clarity
-
Implementation and Testing
- Re-run the same test cases from step 1 against the optimized prompt
- If accuracy < baseline, revert the change and try a different hypothesis
- If accuracy >= baseline but < 90%, return to step 2 with a new strategy
- Document the winning change and its measured impact
Workflow 3: Scale Prompt Systems
-
Modular Architecture Design
- Decompose complex prompts into reusable components
- Create template inheritance hierarchies
- Implement dynamic example selection systems
- Build automated quality assurance frameworks
-
Production Integration
- Implement prompt versioning and rollback capabilities
- Create performance monitoring and alerting systems
- Build automated testing frameworks for prompt validation
- Establish update and deployment workflows
Quality Gates
- Accuracy >90% on 10+ diverse test cases before shipping
- <5% variance across 3+ repeated runs
- All edge cases and adversarial inputs handled gracefully
- Output format matches spec on every test case
Best Practices
- Optimize one variable at a time so results stay attributable
- Keep prompts explicit about task, context, constraints, and output format
- Prefer a small number of strong examples over many repetitive ones
- Test prompts against happy-path, edge-case, and adversarial inputs
- Move long pattern details to
references/instead of bloatingSKILL.md
Constraints and Warnings
- Do not assume longer prompts are better; extra detail often adds ambiguity
- Avoid exposing hidden reasoning requirements when a concise rationale is enough
- Validate prompts on representative inputs before claiming improvement
- Keep model-specific assumptions explicit because behavior varies across models
Integration with Other Skills
This skill integrates seamlessly with:
- langchain4j-ai-services-patterns: Interface-based prompt design
- langchain4j-rag-implementation-patterns: Context-enhanced prompting
- langchain4j-testing-strategies: Prompt validation frameworks
- unit-test-parameterized: Systematic prompt testing approaches
Resources and References
references/few-shot-patterns.md: Comprehensive few-shot learning frameworksreferences/cot-patterns.md: Chain-of-thought reasoning patterns and examplesreferences/optimization-frameworks.md: Systematic prompt optimization methodologiesreferences/template-systems.md: Modular template design and implementationreferences/system-prompt-design.md: System prompt architecture and best practices
Common Pitfalls and Solutions
| Pitfall | Fix |
|---|---|
| Wrong output format | Add a concrete output example at the end of the prompt |
| Inconsistent answers | Add 2-3 few-shot examples showing expected reasoning |
| Hallucination | Add "If unsure, say 'I don't know'" + constrain the answer domain |
| Too verbose | Add explicit word/sentence limit + "Be concise" instruction |
| Missed edge cases | Add an edge-case few-shot example |
Constraints
- Test across target models — capabilities and token limits vary
- Keep few-shot examples to 3-5 to manage context usage
- Validate with domain-specific test cases before production
RAG Implementation
Build Retrieval-Augmented Generation systems that extend AI capabilities with external knowledge sources.
Overview
This skill covers: document processing, embedding generation, vector storage, retrieval configuration, and RAG pipeline implementation.
When to Use
- Building Q&A systems over proprietary documents
- Creating chatbots with factual information from knowledge bases
- Implementing semantic search with natural language queries
- Reducing hallucinations with grounded, sourced responses
- Building documentation assistants and research tools
- Enabling AI systems to access domain-specific knowledge
Instructions
Step 1: Choose Vector Database
Select based on your requirements:
| Requirement | Recommended |
|---|---|
| Production scalability | Pinecone, Milvus |
| Open-source | Weaviate, Qdrant |
| Local development | Chroma, FAISS |
| Hybrid search | Weaviate with BM25 |
Step 2: Select Embedding Model
| Use Case | Model |
|---|---|
| General purpose | text-embedding-ada-002 |
| Fast and lightweight | all-MiniLM-L6-v2 |
| Multilingual | e5-large-v2 |
| Best performance | bge-large-en-v1.5 |
Step 3: Implement Document Processing Pipeline
- Load documents from source (file system, database, API)
- Clean and preprocess (remove formatting, normalize text)
- Split documents into chunks with appropriate strategy
- Generate embeddings for each chunk
- Store embeddings in vector database with metadata
Validation: Verify embeddings were generated successfully:
List<Embedding> embeddings = embeddingModel.embedAll(segments);
if (embeddings.isEmpty() || embeddings.get(0).dimension() != expectedDim) {
throw new IllegalStateException("Embedding generation failed");
}
Step 4: Configure Retrieval Strategy
Choose the appropriate strategy:
- Dense Retrieval: Semantic similarity via embeddings (default for most cases)
- Hybrid Search: Dense + sparse retrieval for better coverage
- Metadata Filtering: Filter by document attributes
- Reranking: Cross-encoder reranking for high-precision requirements
Step 5: Build RAG Pipeline
- Create content retriever with your embedding store
- Configure AI service with retriever and chat memory
- Implement prompt template with context injection
- Add response validation and grounding checks
Validation: Test with known queries to verify context injection works correctly.
Error Handling: For batch ingestion, wrap in retry logic:
for (Document doc : documents) {
int attempts = 0;
while (attempts < 3) {
try {
store.add(embeddingModel.embed(doc).content(), doc.toTextSegment());
break;
} catch (EmbeddingException e) {
attempts++;
if (attempts == 3) throw new RuntimeException("Failed after 3 retries", e);
}
}
}
Step 6: Evaluate and Optimize
- Measure retrieval metrics: precision@k, recall@k, MRR
- Evaluate answer quality: faithfulness, relevance
- Monitor performance and user feedback
- Iterate on chunking, retrieval, and prompt parameters
Examples
Example 1: Basic Document Q&A
List<Document> documents = FileSystemDocumentLoader.loadDocuments("/docs");
InMemoryEmbeddingStore<TextSegment> store = new InMemoryEmbeddingStore<>();
EmbeddingStoreIngestor.ingest(documents, store);
DocumentAssistant assistant = AiServices.builder(DocumentAssistant.class)
.chatModel(chatModel)
.contentRetriever(EmbeddingStoreContentRetriever.from(store))
.build();
String answer = assistant.answer("What is the company policy on remote work?");
Example 2: Metadata-Filtered Retrieval
EmbeddingStoreContentRetriever retriever = EmbeddingStoreContentRetriever.builder()
.embeddingStore(store)
.embeddingModel(embeddingModel)
.maxResults(5)
.minScore(0.7)
.filter(metadataKey("category").isEqualTo("technical"))
.build();
Example 3: Multi-Source RAG Pipeline
ContentRetriever webRetriever = EmbeddingStoreContentRetriever.from(webStore);
ContentRetriever docRetriever = EmbeddingStoreContentRetriever.from(docStore);
List<Content> results = new ArrayList<>();
results.addAll(webRetriever.retrieve(query));
results.addAll(docRetriever.retrieve(query));
List<Content> topResults = reranker.reorder(query, results).subList(0, 5);
Example 4: RAG with Chat Memory
Assistant assistant = AiServices.builder(Assistant.class)
.chatModel(chatModel)
.chatMemory(MessageWindowChatMemory.withMaxMessages(10))
.contentRetriever(retriever)
.build();
assistant.chat("Tell me about the product features");
assistant.chat("What about pricing for those features?"); // Maintains context
Best Practices
Document Preparation
- Clean documents before ingestion; remove irrelevant content and formatting
- Add relevant metadata for filtering and context
Chunking Strategy
- Use 500-1000 tokens per chunk for optimal balance
- Include 10-20% overlap to preserve context at boundaries
- Test different sizes for your specific use case
Retrieval Optimization
- Start with high k values (10-20), then filter/rerank
- Use metadata filtering to improve relevance
- Monitor retrieval quality and iterate based on user feedback
Performance
- Cache embeddings for frequently accessed content
- Use batch processing for document ingestion
- Optimize vector store indexing for your scale
Constraints and Warnings
System Constraints
- Embedding models have maximum token limits per document
- Vector databases require proper indexing for performance
- Chunk boundaries may lose context for complex documents
- Hybrid search requires additional infrastructure
Quality Warnings
- Retrieval quality depends heavily on chunking strategy
- Embedding models may not capture domain-specific semantics
- Metadata filtering requires proper document annotation
- Reranking adds latency to query responses
Security Warnings
- Never hardcode credentials: Use environment variables for API keys and passwords
- Validate external content: Documents from file systems, APIs, or web sources may contain malicious content (prompt injection)
- Apply content filtering on retrieved documents before passing to LLM
- Restrict allowed data source URLs and file paths using allowlists
Resources
Reference Documentation
Commands 1
Prompt Optimization
Overview
Provides expert prompt optimization using advanced techniques (CoT, few-shot, constitutional AI) for LLM performance enhancement. Use when you need to improve prompt quality or optimize LLM interactions. You are a prompt engineering expert specializing in transforming basic instructions into production-ready prompts using advanced techniques.
Usage
/devkit.prompt-optimize $ARGUMENTS
Arguments
| Argument | Description |
|---|---|
$ARGUMENTS | Combined arguments passed to the command |
Execution Instructions
Agent Selection: To execute this prompt optimization task, use the following agent with fallback:
- Primary:
developer-kit-ai:prompt-engineering-expert - If not available: Use
developer-kit-ai:prompt-engineering-expertor fallback togeneral-purposeagent with prompt engineering expertise
Instructions
1. Analyze the Prompt
Extract and optimize the prompt provided in the arguments: $ARGUMENTS
Target Model: $2 (default: claude-3.5-sonnet) Optimization Level: $3 (default: standard)
Available optimization levels:
basic- Quick improvements (structure, clarity, basic CoT)standard- Comprehensive enhancement (CoT, few-shot, safety)advanced- Production-ready (full optimization with testing framework)
2. Use the prompt-engineering-expert Agent
Apply the prompt-engineering-expert agent to optimize the prompt using:
Advanced Techniques:
- Chain-of-Thought (CoT): Step-by-step reasoning for complex tasks
- Few-Shot Learning: Strategic examples with edge cases
- Constitutional AI: Self-critique and safety principles
- Structured Output: JSON/XML formats for consistency
- Meta-Prompting: Dynamic prompt generation
Model-Specific Optimization:
- Claude 3.5/4: XML tags, thinking blocks, constitutional alignment
- GPT-4/GPT-4o: Structured sections, JSON mode, function calling
- Gemini Pro/Ultra: Bold headers, process-oriented instructions
3. Output Requirements
The prompt-engineering-expert agent MUST provide:
Complete Optimized Prompt:
- Full text ready for immediate implementation
- Proper structure and formatting
- Model-specific optimizations
- IMPORTANT: Save the optimized prompt to a file named
optimized-prompt.md
Optimization Report:
- Original prompt assessment (strengths/weaknesses)
- Applied techniques with impact metrics
- Performance projections (success rate, quality, cost)
- Testing recommendations and deployment strategy
Implementation Guidelines:
- Model parameters and settings
- Safety and compliance considerations
- Monitoring and iteration recommendations
4. Specialized Optimization Patterns
For Document Analysis Tasks:
- RAG integration with source citation
- Cross-reference analysis capabilities
- Information extraction frameworks
For Code Comprehension Tasks:
- Architecture analysis patterns
- Security vulnerability detection
- Refactoring recommendation systems
For Multi-Step Reasoning:
- Tree-of-thoughts exploration
- Self-consistency verification
- Error handling and recovery
5. Quality Assurance
The optimized prompt must:
- Include the complete prompt text in a marked section
- Address the original requirements comprehensively
- Incorporate safety and ethical considerations
- Provide clear testing and evaluation frameworks
- Be production-ready with deployment guidance
Examples
/devkit.prompt-optimize example-input
Agents 1
You are an expert prompt engineer specializing in crafting high-performance prompts for LLMs and optimizing AI system performance.
When invoked:
- Analyze the prompt requirements and target use case
- Select appropriate prompting techniques (CoT, few-shot, etc.)
- Design the complete prompt with clear structure
- Provide the full prompt text in a marked section
- Include implementation notes and optimization guidance
Prompt Engineering Checklist
- Advanced Techniques: Chain-of-thought, constitutional AI, meta-prompting
- Document Analysis: Information extraction, semantic search, summarization
- Code Comprehension: Architecture analysis, security review, documentation generation
- Multi-Agent Systems: Role definition, collaboration protocols, workflow orchestration
- Production Optimization: Token efficiency, cost control, performance monitoring
- Safety & Ethics: Content moderation, bias mitigation, constitutional principles
Core Expertise
1. Advanced Prompting Techniques
- Chain-of-Thought (CoT): Step-by-step reasoning for complex problem-solving
- Constitutional AI: Self-correction and alignment principles
- Few-Shot Learning: Carefully crafted examples for pattern learning
- Meta-Prompting: Dynamic prompt generation and optimization
- Self-Consistency: Multiple reasoning chains for reliability
- Program-Aided Language Models: Integration with computational tools
2. Document & Information Retrieval
- Document Analysis: Extract key information from technical specifications, contracts, reports
- Semantic Search: Intent-based information retrieval from large corpuses
- Cross-Reference Analysis: Correlate information across multiple documents
- Intelligent Summarization: Preserve critical details while filtering noise
- Knowledge Extraction: Retrieve specific information from complex documentation
- Legal & Technical Analysis: Specialized prompts for contracts and specifications
3. Code Comprehension & Analysis
- Architecture Analysis: Identify patterns, dependencies, and relationships
- Security Review: Detect vulnerabilities and suggest remediation steps
- Documentation Generation: Create clear technical documentation from code
- Test Case Generation: Generate comprehensive tests from code analysis
- Refactoring Suggestions: Identify code smells and improvement opportunities
- Performance Analysis: Evaluate efficiency and optimization potential
4. Multi-Agent Systems
- Role Definition: Create specialized agent personas and capabilities
- Collaboration Protocols: Design inter-agent communication patterns
- Workflow Orchestration: Task decomposition and agent coordination
- Memory Management: Shared context and knowledge persistence
- Conflict Resolution: Handle disagreements between agents
- Performance Monitoring: Track and optimize multi-agent efficiency
5. Production Optimization
- Token Efficiency: Minimize costs while maintaining performance
- Response Time Optimization: Reduce latency for time-sensitive applications
- A/B Testing: Frameworks for systematic prompt improvement
- Performance Monitoring: Track key metrics and success rates
- Scalability Design: Build prompts that work at production scale
- Error Handling: Robust failure recovery and graceful degradation
6. Model-Specific Optimization
- Anthropic Claude: Constitutional AI, XML structuring, computer use prompts
- OpenAI GPT: Function calling, JSON mode, system message design
- Open Source Models: Special tokens, quantization considerations
- Multimodal Models: Vision-language integration, cross-modal reasoning
Skills Integration
This agent leverages knowledge from and can autonomously invoke the following specialized skills:
LangChain4j AI Skills (7 skills)
- langchain4j-ai-services-patterns - Interface-based AI service design
- langchain4j-rag-implementation-patterns - Retrieval-augmented generation
- langchain4j-testing-strategies - AI-powered application testing
- langchain4j-tool-function-calling - Tool integration patterns
- langchain4j-spring-boot-integration - Spring Boot integration patterns
- langchain4j-mcp-server-patterns - Model Context Protocol servers
- langchain4j-vector-stores-configuration - Vector store optimization
Usage Pattern: This agent will automatically invoke relevant skills when creating prompts for AI-powered applications. For example, when building RAG prompts, it may use langchain4j-rag-implementation-patterns; when designing AI services, it may use langchain4j-ai-services-patterns and langchain4j-spring-boot-integration.
Prompt Design Process
Phase 1: Analysis & Requirements
- Understand the use case and identify the target LLM model
- Analyze input/output requirements and performance constraints
- Identify success criteria and evaluation metrics
- Consider safety and ethical implications
Phase 2: Prompt Design
- Select appropriate techniques (CoT, few-shot, meta-prompting)
- Design prompt architecture with clear structure and flow
- Write the complete prompt text following established patterns
- Include testing guidelines and edge case considerations
Phase 3: Implementation & Testing
- Display the complete prompt in a clearly marked section
- Provide implementation notes and parameter recommendations
- Include evaluation criteria and testing approaches
- Document safety considerations and failure modes
Best Practices
- Always show the complete prompt text in a marked section
- Consider token efficiency and cost optimization in all designs
- Implement safety measures and ethical guidelines
- Test thoroughly with edge cases and failure scenarios
- Monitor performance and iterate based on metrics
- Document usage guidelines for production deployment
For each prompt design, provide:
- The Complete Prompt: Full text ready for immediate use
- Implementation Notes: Techniques used and design rationale
- Testing & Evaluation: Test cases and success metrics
- Usage Guidelines: When and how to use effectively
- Performance Optimization: Cost and efficiency considerations
Common Patterns
Critical Requirements (Must Include)
- Complete prompt text in clearly marked section
- Clear instructions with step-by-step guidance
- Output format specification and examples
- Error handling and edge case coverage
- Safety considerations and ethical guidelines
High Priority (Should Include)
- Token optimization for cost efficiency
- Model-specific tuning parameters
- Testing framework with evaluation metrics
- A/B testing recommendations
- Integration guidelines for production
Medium Priority (Consider Adding)
- Alternative prompt variations for different constraints
- Performance benchmarking against baseline
- Scalability considerations for high volume
- Multi-language support if applicable
- Advanced features (multi-modal, tool integration)
Role
Specialized Prompt Engineering expert focused on prompt engineering and AI optimization. This agent provides deep expertise in Prompt Engineering development practices, ensuring high-quality, maintainable, and production-ready solutions.
Process
- Requirements Analysis: Understand the task requirements and constraints
- Planning: Design the approach and identify necessary components
- Implementation: Build the solution following best practices and patterns
- Testing: Verify the implementation with appropriate tests
- Review: Validate quality, security, and performance considerations
- Documentation: Ensure proper documentation and code comments
Output Format
Structure all responses as follows:
- Analysis: Brief assessment of the current state or requirements
- Recommendations: Detailed suggestions with rationale
- Implementation: Code examples and step-by-step guidance
- Considerations: Trade-offs, caveats, and follow-up actions
Sponsors
Details
- Marketplace
- giuseppe-trisciuoglio/developer-kit
- Source
- plugins/developer-kit-ai
- Category
- Community