News Extractor
Pulls article content from five major Chinese news platforms: WeChat Official Accounts, Toutiao, NetEase, Sohu, and Tencent News. You throw a URL at it and get back structured JSON plus a clean Markdown file with the article text, images, and metadata. Built on curl_cffi for browser simulation and includes retry logic, which matters when scraping these sites. Uses uv for dependency management, so everything runs in isolation. The output format is consistent across platforms, making it easy to build workflows on top of it. Worth noting this is explicitly for education and research, not production scraping at scale. Works well when you need to extract a few articles for analysis or archival without manually copying content.
- GitHub stars
- 246
- last updated
- May 2, 2026
Install
$ npx -y skills add nanmicoder/skills-agent-proto --skill news-extractor --agent claude-codeInstalls into .claude/skills of the current project.
Files
News Extractor Skill
从主流新闻平台提取文章内容，输出 JSON 和 Markdown 格式。
支持平台
| 平台 | ID | URL 示例 |
|---|---|---|
| 微信公众号 | https://mp.weixin.qq.com/s/xxxxx | |
| 今日头条 | toutiao | https://www.toutiao.com/article/123456/ |
| 网易新闻 | netease | https://www.163.com/news/article/ABC123.html |
| 搜狐新闻 | sohu | https://www.sohu.com/a/123456_789 |
| 腾讯新闻 | tencent | https://news.qq.com/rain/a/20251016A07W8J00 |
依赖安装
本 skill 使用 uv 管理依赖。首次使用前需要安装：
# 项目级使用（推荐）
cd .claude/skills/news-extractor
uv sync
# 或用户级使用
cd ~/.claude/skills/news-extractor
uv sync
说明: 此 skill 可放置在项目级 (
.claude/skills/) 或用户级 (~/.claude/skills/) 目录。项目级便于团队共享，用户级便于跨项目复用。
重要: 所有脚本必须使用 uv run 执行，不要直接用 python 运行。uv run 会自动使用项目虚拟环境中的依赖。
依赖列表
| 包名 | 用途 |
|---|---|
| pydantic | 数据模型验证 |
| requests | HTTP 请求 |
| curl_cffi | 浏览器模拟抓取 |
| tenacity | 重试机制 |
| parsel | HTML/XPath 解析 |
| demjson3 | 非标准 JSON 解析 |
使用方式
基本用法
# 提取新闻，自动检测平台，输出 JSON + Markdown
uv run .claude/skills/news-extractor/scripts/extract_news.py "URL"
# 指定输出目录
uv run .claude/skills/news-extractor/scripts/extract_news.py "URL" --output ./output
Sponsors
Promote your product hereReach 125k+ visitors monthlyPromote your product hereReach 125k+ visitors monthly
Agent CaffeineSponsoredClose your lid, come back to finished work
SnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthly
WaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthly
knobsSponsoredSee your site on any phone, any day, in any language
Agent CaffeineSponsoredClose your lid, come back to finished workSnugSponsoredThe clipboard manager for the AI eraPromote your product hereReach 125k+ visitors monthlyWaxSponsoredBlock anything with supervision, without erasing your iPhonePromote your product hereReach 125k+ visitors monthlyknobsSponsoredSee your site on any phone, any day, in any languageDetails
- Repository
- github.com/nanmicoder/skills-agent-proto
- Category
- AI & Agent Building
- First seen
- Jun 11, 2026
