Desktop Control
Gives Claude control over your mouse, keyboard, and screen through PyAutoGUI. You can automate clicks, type text, execute hotkeys, take screenshots, and even use OCR to locate text on screen. Works across Windows, macOS, and Linux. The app control commands let you open and focus applications by name. Most useful for repetitive GUI tasks like form filling, testing interfaces, or scraping data from applications without APIs. The image matching feature is handy but needs exact visual matches, so adjust confidence levels if things don't line up. Remember PyAutoGUI's fail-safe: mouse to corner kills everything.
- GitHub stars
- 2
- last updated
- Jun 3, 2026
Install
$ npx -y skills add patrickporto/desktop-agent --skill 'Desktop Control' --agent claude-codeInstalls into .claude/skills of the current project.
Files
Desktop Control Skill
This skill provides comprehensive desktop automation capabilities through PyAutoGUI, allowing AI agents to control the mouse, keyboard, take screenshots, and interact with the desktop environment.
How to Use This Skill
As an AI agent, you can invoke desktop automation commands using the uvx desktop-agent CLI.
Command Structure
All commands follow this pattern:
uvx desktop-agent <category> <command> [arguments] [options]
Categories:
mouse- Mouse controlkeyboard- Keyboard inputscreen- Screenshots and screen analysismessage- User dialogsapp- Application control (open, focus, list windows)
Available Commands
🖱️ Mouse Control (mouse)
Control cursor movement and clicks.
# Move cursor to coordinates
uvx desktop-agent mouse move <x> <y> [--duration SECONDS]
# Click at current position or specific coordinates
uvx desktop-agent mouse click [x] [y] [--button left|right|middle] [--clicks N]
# Specialized clicks
uvx desktop-agent mouse double-click [x] [y]
uvx desktop-agent mouse right-click [x] [y]
uvx desktop-agent mouse middle-click [x] [y]
Sponsors
Promote your product hereReach 125k+ visitors monthlyPromote your product hereReach 125k+ visitors monthly
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any language
Agent CaffeineSponsoredClose your lid, come back to finished workSnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthlyWaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthlyknobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/patrickporto/desktop-agent
- Category
- AI & Agent Building
- First seen
- Jun 3, 2026
