Key Takeaways
- Developers increasingly ask Claude, ChatGPT and coding agents which library to use. If your project's docs and repo can't be read, you won't be recommended.
- Anthropic's Claude-User crawler grew more than 100x between December 2025 and April 2026, but it is still about 1% of tracked AI crawler traffic. Unblock it, don't obsess over it.
- A CLAUDE.md or AGENTS.md file is the cheapest visibility and quality lever you control. Keep it short and specific.
- Docs and a public GitHub presence are what ChatGPT cites most for software prompts. Your own site is your main citation source.
- Check your server or CDN logs first. Most visibility problems in open source are access problems, not content problems.
Why open-source visibility now includes coding agents
A few years ago, "visibility" for an open-source project meant GitHub stars, a decent README and maybe a Hacker News post. That still matters. But a growing share of discovery now happens inside a chat window or a terminal. Someone types "what's a good Go library for X" into ChatGPT, or tells Claude Code to add a feature and it picks a dependency on its own.
That second case is the one maintainers underestimate. A coding agent doesn't browse like a person. It reads your README, your docs, your type definitions, your examples and, if you provide one, your agent context file. If those are thin, blocked or contradictory, the agent either skips you or uses your API wrong. Then the user blames your project.
This guide is a practical checklist. It is written for maintainers with limited time, so it is ordered by effort versus payoff.
Step 1: Make sure Claude's crawlers can reach you
Start with the boring thing. If Anthropic's crawlers can't fetch your pages, nothing else on this list matters.
Promptwatch's data on Claude's citation crawler (covering Dec 10, 2025 to Apr 18, 2026) shows how fast this changed. In mid-December the Claude-User crawler made roughly 30 visits a day, about 0.04% of tracked AI citation crawler traffic. By mid-April it was making several thousand visits a day, a more than 100x increase in four months. Its peak daily share was 1.73%, and the share across the whole period was 0.45%. The Claude crawler growth report notes the crawler may visit cited pages when someone asks Claude a question or uses Claude Code.
Keep the scale in perspective. Claude is still roughly 1% of tracked AI citation crawler traffic. The AI crawler traffic report (IP-verified requests, Jun 8 to Sep 6, 2026) shows OpenAI at 79.8% of verified requests in the week of Aug 31 to Sep 6, down from 94.8% in mid-June. So the sensible stance is: treat Claude as a fast-growing secondary channel. Unblock it, measure it, move on.
What to check:
- Your robots.txt doesn't disallow Claude-User, ClaudeBot or other Anthropic agents by accident (a blanket rule copied from a template is the usual culprit).
- Your CDN or WAF bot rules (Cloudflare bot fight mode, for instance) aren't silently returning 403s to AI user agents.
- Your docs site isn't gated behind a JavaScript-only renderer that returns an empty shell to non-browser clients.
The free robots.txt generator for AI and the AI bots directory are useful for working out which user agents to allow.
Read your logs once, then monthly
Search your server or CDN logs for Anthropic user agents and note the baseline. Then compare it against the provider mix in the crawler report. If a provider is a large share of the chart but near zero in your logs, you are probably blocked or unreachable for that provider.
If you'd rather not grep logs by hand, Promptwatch has IP-verified AI crawler logs that work with Cloudflare, Vercel, Fastly and raw server logs, and they filter out spoofed bots. It is a paid platform built for brands, so for a small hobby project the manual log check is probably enough. For a project with a company behind it, it's worth a look.

DarkVisitors is another option focused specifically on tracking AI agents and bots hitting a site.

Step 2: Write a CLAUDE.md and AGENTS.md that earn their place
Claude Code reads CLAUDE.md as project memory, and many other agents read AGENTS.md. Anthropic documents this in its Claude Code memory docs. For an open-source project, this file is where you tell every contributor's agent how your repo works.
A recent arXiv census, What Open-Source Projects Tell Their Coding Agents, looked at what projects actually put in these files. One slice of it is revealing. In the review-oriented instructions, the most common requirements were:
| Category | % of files requiring | % prohibiting |
|---|---|---|
| Review process, reviewers, tools and gating | 22.2 | 2.7 |
| Correctness, edge cases and error handling | 20.2 | 2.0 |
| Context gathering and review method | 17.7 | 1.5 |
| Project guidelines, checklists and repo-specific checks | 17.5 | 1.2 |
| Test coverage and test quality | 13.0 | 1.5 |
| Security and privacy | 12.2 | 2.0 |
| Backward compatibility and breaking changes | 9.5 | 0.7 |
| Style, naming, formatting and readability | 7.0 | 7.7 |
| Agent permissions, posting and AI disclosure | 4.5 | 3.2 |
Two things stand out to me. First, projects mostly care about process and correctness, not style. Style is the only category where prohibitions (7.7%) outnumber requirements (7.0%), which suggests maintainers are tired of agents nitpicking formatting. Second, AI disclosure and agent permissions are barely addressed (4.5%), even though that's where open-source maintainers feel the most pain right now.
What goes in the file
Keep it short. There is research on this, too: Chakrabarti's paper "Why Does CLAUDE.md Keep Growing?" describes how these files bloat over time, and bloated context is worse context. A good starting skeleton:
# CLAUDE.md
## Build and test
- Install: `make dev`
- Run tests: `make test` (must pass before any PR)
- Lint: `make lint`
## Conventions
- Public API lives in `src/api/`. Never change signatures without a deprecation note.
- Errors are returned, not panicked. See `src/errors.go`.
## Contributing with an agent
- Disclose AI assistance in the PR description.
- Keep PRs under one logical change. Do not reformat unrelated files.
- Never edit generated files in `gen/`.
## Where to look
- Architecture: docs/architecture.md
- Release process: docs/releasing.md
A few opinions, earned the annoying way:
- Write rules as commands the agent can verify ("run
make test"), not as vibes ("write clean code"). - Don't rely on a "do not" line as a security control. Yan's paper "When 'Do Not' Is Not Deny" argues that rules written in CLAUDE.md are not the same as built-in permission controls. Use real permissions for anything dangerous.
- Pick one canonical file and have the other point at it. The census treats the two file names as a pair, and duplicating content across them is how they drift apart.
For a deeper view on how ChatGPT treats docs when recommending tools, see the section on citations below.
Step 3: Make your docs the best thing AI can read
The software citations report from Promptwatch Data (Sep 28 to Oct 4, 2026, organic software prompts) is the closest thing we have to a map of where ChatGPT looks for software answers:
| Source | Share of software citations |
|---|---|
| Docs and help centers (as a source type) | 2.64% |
| Vendor sites | 2.01% |
| Review sites and analysts | 1.57% |
| GitHub | 1.2% |
| App stores | 0.47% |
By domain, github.com led at 1.2%, followed by learn.microsoft.com (0.98%) and g2.com (0.73%). The 23 domains in the chart together earn 7.89%, so the long tail is huge. Percentages are small because citations are spread across thousands of sites. The takeaway isn't the absolute numbers, it's the ordering: documentation beats review sites by a wide margin, and a public GitHub repo is itself a top source.
There's also a durability angle. In Promptwatch's content half-life study, landing pages and docs stayed cited longest, while news lost half its pages within 36 days. Docs are the asset that keeps paying off.
Practical docs checklist:
- Every public function and CLI flag has a page with a runnable example. Agents copy examples almost verbatim.
- A "quickstart" that works in under five minutes, on a fresh machine, with pinned versions.
- A plain-text or markdown-friendly rendering. Note that markdown files are only 0.05% of AI search citations (Promptwatch's markdown data), so this is about agents reading your docs in a repo, not about search citations. Don't expect a raw .md file to rank in AI answers.
- A comparison page that says honestly when someone should use an alternative. AI answers love comparisons, and "X vs Y" prompts are where projects get chosen.
- Versioned docs with a clear "latest" URL, so models don't cite year-old instructions.
- An llms.txt if you want one. It's cheap, though nobody has proven it moves citations. The llms.txt generator makes one in a minute.
Step 4: Keep your GitHub presence sharp
GitHub is a top cited software domain, so the repo page matters as a landing page, not just a code host.
- A README that opens with one sentence on what the project is, who it's for and what it replaces.
- Topics and a description that match how people phrase the problem.
- A recent release and recent commits. A repo that looks dead reads as dead to humans and probably to models.
- Issue templates and a CONTRIBUTING file that mention agents explicitly.
A word on stars. Don't buy them. A 2026 ICSE paper, "Six Million (Suspected) Fake Stars on GitHub," documents a spiral of fake popularity that leads into spam and malware, and platforms and researchers are getting good at spotting it. Real docs and real users are cheaper in the long run.
Step 5: Use the Claude for Open Source program, if you qualify
Anthropic announced Claude for Open Source, giving 6 months of free Claude Max 20x to open-source maintainers and core contributors. Boris Cherny, who works on Claude Code, posted the announcement on Threads.

I haven't verified the eligibility criteria, application steps or deadline, so check Anthropic's official page before you plan around it. One contributor wrote up their acceptance on Medium and said they applied using Claude Code, which is a decent hint about how Anthropic thinks about the program. A third-party guide at Verdent also covers using Claude to review PRs and ease maintainer load, though treat it as one perspective.
This matters for visibility indirectly. Maintainers who use Claude Code daily on their own repo learn quickly where their docs and context files fall short. You find out your quickstart is wrong when the agent follows it and fails.
Claude itself is in our catalog if you want to look at the assistant side.
Step 6: Prepare for the human side of agent traffic
More agent-assisted contributors means more PRs of uneven quality. Two habits help.
- Put your AI policy in writing. Say whether disclosure is required, whether agent-authored PRs need a human sign-off, and what gets closed on sight. Only 4.5% of the census files mention agent permissions and disclosure, so a clear policy is rare.
- Be ready for disagreement. In her All Things Open 2026 talk on the pre-launch checklist for open source projects, benny Vasquez argues that most projects stumble because maintainers aren't prepared for dissenting opinions. Her advice: set expectations and a feedback process before the first PR, not after the first flame war.

The Open Source Initiative's Maintainer Month 2026 post frames the larger point: the whole thing is about visibility, appreciation and sustainability for the people who keep projects running. Visibility to AI tools is useful. Not burning out is more useful.
Step 7: Measure whether any of this works
You can't improve what you never look at, and AI visibility is oddly easy to ignore. A simple monthly routine:
- Ask ChatGPT, Claude, Gemini and Perplexity ten questions your users would ask ("best library for X", "X vs Y", "how do I do Z in language L"). Write down whether you're mentioned and whether the answer is correct.
- Check your logs for Anthropic, OpenAI and Perplexity user agents. Compare against the provider mix.
- Check which of your pages get cited. If your docs aren't cited but a blog post from three years ago is, update the blog post or redirect it.
- Re-test after every major docs change.
Tools worth knowing
Doing step 1 by hand works for a while. When you want it automated, these are the options in the catalog that fit different needs.
| Tool | Best for | Notes |
|---|---|---|
| Promptwatch | Companies or foundations backing a project | Prompt tracking, crawler logs, citation data, content gap analysis and publishing to WordPress, Webflow or Framer. Paid, with a 7-day free trial |
| Elmo | Teams who want self-hosted | Open-source AI visibility platform |
| Voyage (OnVoyage) | Open-source minded teams | Open-source GEO infrastructure |
| DarkVisitors | Seeing which bots hit your site | Focused on AI agents and LLM referrals |
| llmtxt.info | Quick llms.txt | Free generator and validator |
If your project has a company or foundation behind it, a platform like Promptwatch covers the whole loop: see what AI says, trace which pages get cited, fix gaps and measure the result. There's also a free AI visibility report that scans how ChatGPT, Gemini and Perplexity describe a brand, with no login. Pricing starts at $95/mo for Essential, but confirm current plans at promptwatch.com/pricing.
For a purely volunteer project, the open-source options are a more natural fit.


If you want to browse more options, the GEO software directory lists tools by category.
The maintainer's checklist
Print this, or paste it in an issue.
Access
- robots.txt allows Claude-User, ClaudeBot and the other AI agents you want
- CDN/WAF rules don't block AI user agents
- Docs render without JavaScript, or have a server-rendered fallback
- Baseline Anthropic crawler visits recorded from logs
Agent context
- CLAUDE.md exists, under a page long, with build, test and convention commands
- AGENTS.md points to the same content, not a divergent copy
- AI disclosure and PR policy written down
- Dangerous actions enforced with real permissions, not "do not" lines
Content
- Quickstart tested on a clean machine
- Every public API has an example
- An honest comparison page for the top two alternatives
- Versioned docs with a stable latest URL
- Optional llms.txt
Presence
- GitHub description, topics and README lead match how users search
- Recent release and commits
- Review-site profiles (G2, Capterra) current, if your project has a commercial side
Measurement
- Ten recurring prompts tested monthly across at least three assistants
- Cited pages reviewed and stale ones refreshed
What I would do first
If you only have an afternoon, do three things: check robots.txt and your CDN for blocked AI agents, write a 30-line CLAUDE.md, and run your quickstart on a clean machine. The first removes the biggest silent failure, the second makes agent-assisted contributions better, and the third fixes the page that every human and every model reads first.
And keep expectations honest. Claude is about 1% of tracked AI crawler traffic today, and ChatGPT still dominates. The projects that do well in both will be the ones whose docs are clear enough for a model to quote correctly.

