Skip to content

Instantly share code, notes, and snippets.

@eonist
Created December 23, 2025 11:57
Show Gist options
  • Select an option

  • Save eonist/08eae072f0cee365e1eb32470bb335d8 to your computer and use it in GitHub Desktop.

Select an option

Save eonist/08eae072f0cee365e1eb32470bb335d8 to your computer and use it in GitHub Desktop.
Potential.Issues.with.High.Traffic.md

Potential Issues with High Traffic

Your JavaScript implementation makes client-side API calls to GitHub's API, which will likely cause problems with high traffic due to GitHub's strict rate limits.1

Critical Rate Limit Constraints

GitHub enforces a 60 requests per hour limit for unauthenticated requests, shared per originating IP address. With your current implementation, every visitor triggers three API calls (repo stats and releases), meaning you'll hit the limit after just 20 visitors per hour from the same IP address or network.21

High Traffic Scenarios

With lots of incoming traffic, you'll face several issues:

  • Shared IP addresses: Visitors behind corporate networks, VPNs, or NAT will share the same IP, exhausting the rate limit rapidly1
  • Badge failure: Once the limit is hit, all subsequent visitors see "0" for downloads, stars, and forks until the hourly reset3
  • Poor user experience: Users receive 403 errors instead of updated statistics4

Recommended Solutions

Server-side caching: Implement a backend service that fetches GitHub data periodically and caches results, eliminating direct client-to-GitHub API calls. This approach allows authenticated requests with a 5,000 requests/hour limit.541

Static badge services: Use services like shields.io that already implement caching and handle rate limits, or generate static badges during your site build process.4

Authentication with GitHub Actions: Create a GitHub Actions workflow that fetches stats daily using an authenticated token and commits the results as static JSON files your page loads directly.67

The current implementation works for low-traffic scenarios but will fail under load, displaying stale fallback values to most visitors once rate limits are exceeded.84 9101112131415161718192021


I think it's 60 request per user ip. not total. right?

You're absolutely correct! I apologize for the confusion in my previous response.

Yes, the 60 requests per hour limit is per IP address, not a total shared limit. Each unique visitor IP gets their own 60 requests/hour allocation.22

Why It's Still a Concern

Even though each IP gets 60 requests/hour, you may still encounter issues with high traffic:

Shared IP Scenarios

Corporate networks and institutions: Hundreds or thousands of employees behind a single public IP share the same 60-request quota. If 20+ people from the same office visit your page within an hour, later visitors will see rate limit errors.22

Mobile carriers and ISPs: Some carriers use carrier-grade NAT (CGNAT), putting many users behind shared IPs, which exhausts the quota quickly.

VPN services: Popular VPN endpoints can have thousands of users sharing the same exit IP address.

Your Specific Implementation

Your page makes 2 API calls per visitor:

  • 1 call to /repos/edgeleap/diffsense (for stars/forks)
  • 1 call to /repos/edgeleap/diffsense/releases (for downloads)

This means each IP can serve 30 unique visitors per hour before hitting the limit. For individual users on residential IPs, this is perfectly fine. But for shared networks, it becomes problematic.22

When You're Likely Safe

If your traffic consists mostly of individual developers on home/residential connections, the per-IP rate limit should work well. Most won't reload the page 30 times in an hour, so you're unlikely to hit issues.22


is there a global total for all users limit for this call?

No, there is no global total limit for all users combined on these specific API endpoints.23

GitHub's rate limiting is structured as:

Per-IP Limits (What Applies to You)

  • 60 requests/hour per IP address for unauthenticated requests23
  • Each IP address gets its own independent quota23

Secondary Limits (Platform-Wide Protection)

GitHub does have secondary rate limits that apply globally to prevent abuse, but these are much higher and apply to different scenarios:23

  • Maximum 100 concurrent requests (across all users)23
  • 900 points per minute per endpoint23
  • No excessive compute resource consumption23

These secondary limits are designed to protect GitHub's infrastructure from DoS attacks and abuse, but they're not a "total requests across all users" limit. They're triggered by patterns like making too many concurrent connections or hammering a single endpoint too rapidly.23

For Your Use Case

Your repository info endpoints (/repos/edgeleap/diffsense and /repos/edgeleap/diffsense/releases) don't have a global "total requests from all visitors" limit. Each visitor's IP gets its own 60/hour quota independently. Even with thousands of visitors, as long as they're on different IPs, they won't interfere with each other's rate limits.2423 252627282930313233343536373839404142


but if it goes viral will it still work?

Yes, it will likely work if it goes viral, but there are some caveats to consider.

Why It Should Work

Your implementation makes simple GET requests to read-only repository endpoints (/repos/{owner}/{repo} and /repos/{owner}/{repo}/releases), which are among the least resource-intensive API calls. Since each visitor's IP gets its own independent 60/hour quota, even with 100,000 unique visitors, each one can load your page without interfering with others.43

Secondary Rate Limit Risks

GitHub's secondary rate limits are designed to prevent abuse and could potentially be triggered if your traffic pattern looks suspicious:4443

What Could Trigger Issues

900 points per minute per endpoint: If thousands of people hit your page simultaneously (like a Reddit/HN spike), you might exceed 900 requests/minute to the releases endpoint. This limit is shared across all IPs accessing that specific endpoint.4543

Abuse detection patterns: GitHub has undisclosed secondary limits that can trigger on patterns they consider abusive. A sudden massive spike from a viral post could potentially look suspicious to their systems.4345

100 concurrent requests: If you get a massive simultaneous traffic surge, you could theoretically hit the 100 concurrent connections limit.43

Real-World Likelihood

For a typical viral scenario (Product Hunt, Hacker News front page, etc.), you're probably fine because:

  • Traffic spikes are usually spread over hours, not seconds
  • The repository info endpoints are lightweight
  • GitHub handles popular repositories with millions of visitors without issues
  • Your 2 API calls per visitor is modest

If Problems Occur

Users would see your fallback values (0 for downloads/stars) and the page would still function perfectly. The badges failing gracefully is actually a good design choice.46

Bottom line: It should work fine for viral traffic, but there's a small chance of hitting secondary limits during extreme simultaneous spikes (thousands of requests in the same minute).4443 4748495051525354555657585960616263


is there alternative way to solve this? issue do not a have server.

Yes! Here are several serverless alternatives that don't require maintaining a server:

1. Use Shields.io (Easiest Solution)

Replace your JavaScript API calls with Shields.io badge images that handle caching automatically:6465

<!-- Downloads badge -->
<img src="https://img.shields.io/github/downloads/edgeleap/diffsense/total" alt="Downloads">

<!-- Stars badge -->
<img src="https://img.shields.io/github/stars/edgeleap/diffsense" alt="Stars">

<!-- Forks badge -->
<img src="https://img.shields.io/github/forks/edgeleap/diffsense" alt="Forks">

Shields.io has its own authenticated GitHub tokens with higher rate limits and implements caching, so you completely avoid rate limit issues.6664

2. GitHub Actions + Static JSON (No Runtime API Calls)

Create a GitHub Action that fetches stats periodically and commits them as a static JSON file:

# .github/workflows/update-stats.yml
name: Update Stats
on:
  schedule:
    - cron: '0 */6 * * *'  # Every 6 hours
  workflow_dispatch:

jobs:
  update:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Fetch Stats
        run: |
          curl https://api.github.com/repos/edgeleap/diffsense > stats.json
          curl https://api.github.com/repos/edgeleap/diffsense/releases > releases.json
      - name: Commit
        run: |
          git config user.name github-actions
          git config user.email github-actions@github.com
          git add stats.json releases.json
          git commit -m "Update stats" || exit 0
          git push

Then fetch stats.json from your own domain (no rate limits).6768

3. Cloudflare Workers (Free Tier)

Create a tiny serverless proxy on Cloudflare's free tier that caches GitHub API responses:6970

// Cloudflare Worker
export default {
  async fetch(request) {
    const cache = caches.default;
    let response = await cache.match(request);
    
    if (!response) {
      response = await fetch('https://api.github.com/repos/edgeleap/diffsense');
      response = new Response(response.body, {
        ...response,
        headers: {
          ...response.headers,
          'Cache-Control': 'max-age=3600', // Cache 1 hour
          'Access-Control-Allow-Origin': '*'
        }
      });
      await cache.put(request, response.clone());
    }
    return response;
  }
}

Deploy to stats.your-domain.workers.dev and call that instead of GitHub API directly.7069

Recommendation

Use Shields.io if you're okay with badge-style images. It's the fastest solution with zero maintenance. If you need the exact current design, go with GitHub Actions + static JSON - it's simple, free, and eliminates all rate limit concerns.6467 71727374757677787980818283


can I extract just the value from shield and use my own html design for the badges etc?

No, Shields.io doesn't provide a JSON endpoint that returns just the raw value. It only returns rendered badge images (SVG/PNG).8485

However, here are better alternatives to get just the values:

1. GitHub Actions + Static JSON (Best for Your Case)

This is the simplest serverless solution that gives you full control over your HTML design:

# .github/workflows/update-stats.yml
name: Update Stats
on:
  schedule:
    - cron: '0 */1 * * *'  # Every hour
  workflow_dispatch:

jobs:
  update:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Fetch and Process Stats
        run: |
          # Fetch repo stats
          curl -s https://api.github.com/repos/edgeleap/diffsense > repo.json
          
          # Fetch releases
          curl -s https://api.github.com/repos/edgeleap/diffsense/releases > releases.json
          
          # Extract just the values we need
          node -e "
            const repo = require('./repo.json');
            const releases = require('./releases.json');
            let downloads = 0;
            releases.forEach(r => {
              (r.assets || []).forEach(a => downloads += a.download_count || 0);
            });
            require('fs').writeFileSync('stats.json', JSON.stringify({
              stars: repo.stargazers_count,
              forks: repo.forks_count,
              downloads: downloads
            }));
          "
      - name: Commit
        run: |
          git config user.name github-actions
          git config user.email github-actions@github.com
          git add stats.json
          git commit -m "Update stats [skip ci]" || exit 0
          git push

Then update your HTML to fetch from your own static file:

// In your index.html, replace the current fetch calls with:
(async function () {
  try {
    const res = await fetch('https://edgeleap.github.io/stats.json');
    if (!res.ok) return;
    const data = await res.json();
    document.getElementById('downloads-count').textContent = data.downloads.toLocaleString();
    document.getElementById('stars-count').textContent = data.stars.toLocaleString();
    document.getElementById('forks-count').textContent = data.forks.toLocaleString();
  } catch (e) {
    // keep fallback numbers
  }
})();

Benefits:

  • Zero rate limits (you're fetching from your own domain)
  • Keep your exact HTML design
  • Updates hourly automatically
  • Completely free
  • No external dependencies

2. Create Your Own JSON Endpoint with Cloudflare Workers

Deploy this tiny worker (free tier) that returns JSON:

export default {
  async fetch(request) {
    const cache = caches.default;
    const cacheKey = new Request('https://stats-cache.example.com/diffsense');
    
    let response = await cache.match(cacheKey);
    
    if (!response) {
      const [repoRes, releasesRes] = await Promise.all([
        fetch('https://api.github.com/repos/edgeleap/diffsense'),
        fetch('https://api.github.com/repos/edgeleap/diffsense/releases')
      ]);
      
      const repo = await repoRes.json();
      const releases = await releasesRes.json();
      
      let downloads = 0;
      releases.forEach(r => {
        (r.assets || []).forEach(a => downloads += a.download_count || 0);
      });
      
      const stats = {
        stars: repo.stargazers_count,
        forks: repo.forks_count,
        downloads: downloads
      };
      
      response = new Response(JSON.stringify(stats), {
        headers: {
          'Content-Type': 'application/json',
          'Cache-Control': 'max-age=3600',
          'Access-Control-Allow-Origin': '*'
        }
      });
      
      await cache.put(cacheKey, response.clone());
    }
    
    return response;
  }
}

I recommend Option 1 (GitHub Actions) - it's simpler, requires no additional services, and works perfectly for your use case.86 87888990919293949596979899100101102103104


cloudeflare is just more deps. and cron job makes lots of commits. can I parse the shield svg somehow? I mean the value must be in there somewhere?

Yes, you can parse the value from the SVG! Shields.io SVG badges contain the text values in <text> elements.105

Here's a simple solution that fetches the SVG and extracts the value:

async function getBadgeValue(badgeUrl) {
  try {
    const response = await fetch(badgeUrl);
    const svgText = await response.text();
    
    // Parse SVG as XML
    const parser = new DOMParser();
    const svgDoc = parser.parseFromString(svgText, 'image/svg+xml');
    
    // Get all text elements - the last one typically contains the value
    const textElements = svgDoc.querySelectorAll('text');
    const valueText = textElements[textElements.length - 1].textContent.trim();
    
    // Remove commas and convert to number if needed
    return valueText.replace(/,/g, '');
  } catch (e) {
    return '0';
  }
}

// Use it in your page
(async function() {
  const downloads = await getBadgeValue('https://img.shields.io/github/downloads/edgeleap/diffsense/total');
  const stars = await getBadgeValue('https://img.shields.io/github/stars/edgeleap/diffsense');
  const forks = await getBadgeValue('https://img.shields.io/github/forks/edgeleap/diffsense');
  
  document.getElementById('downloads-count').textContent = downloads;
  document.getElementById('stars-count').textContent = stars;
  document.getElementById('forks-count').textContent = forks;
})();

This works because:

  • Shields.io SVGs contain <text> elements with the actual values106105
  • Shields.io has its own caching and authenticated tokens, so no rate limits for you
  • You keep your exact HTML design
  • Zero additional dependencies
  • No commits to your repo

The only downside: You're making 3 requests to Shields.io instead of 2 to GitHub, but Shields.io is designed specifically for this use case and handles the caching/rate limiting on their end.107108

This is actually a clever solution that gives you the best of both worlds - Shields.io's infrastructure + your custom design! 109110111112113114115116117118119120121122123124

Footnotes

  1. https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api 2 3 4

  2. https://www.lunar.dev/post/a-developers-guide-managing-rate-limits-for-the-github-api

  3. https://github.com/orgs/community/discussions/67071

  4. https://stackoverflow.com/questions/41550758/github-api-rate-limit-on-public-github-pages-site 2 3 4

  5. https://github.com/orgs/community/discussions/144428

  6. https://www.kubeblogs.com/how-to-avoid-github-token-rate-limiting-issues-complete-guide-for-devops-teams/

  7. https://docs.github.com/rest/guides/best-practices-for-using-the-rest-api

  8. https://dev.to/matmooredev/can-you-host-a-read-only-api-on-github-pages-1jib

  9. https://github.blog/changelog/2025-05-08-updated-rate-limits-for-unauthenticated-requests/

  10. https://news.ycombinator.com/item?id=43936992

  11. https://drdroid.io/stack-diagnosis/github-actions-github-api-rate-limit-exceeded

  12. https://github.com/actions-runner-controller/actions-runner-controller/issues/206

  13. https://github.blog/engineering/infrastructure/how-we-scaled-github-api-sharded-replicated-rate-limiter-redis/

  14. https://github.com/orgs/community/discussions/31474

  15. https://docs.github.com/en/pages/getting-started-with-github-pages/github-pages-limits

  16. https://github.com/modelcontextprotocol/typescript-sdk/issues/690

  17. https://news.ycombinator.com/item?id=13227863

  18. https://www.reddit.com/r/PowerShell/comments/1kn58fi/fyi_changes_to_github_rate_limits_scripts/

  19. https://www.pullchecklist.com/posts/api-checklist-github

  20. https://docs.github.com/en/actions/reference/limits

  21. https://github.com/edgeleap/edgeleap.github.io/blob/main/index.html

  22. https://github.com/edgeleap/edgeleap.github.io/blob/main/index.html 2 3 4

  23. https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api 2 3 4 5 6 7 8 9

  24. https://www.lunar.dev/post/a-developers-guide-managing-rate-limits-for-the-github-api

  25. https://docs.github.com/en/enterprise-server@3.19/rest/using-the-rest-api/rate-limits-for-the-rest-api

  26. https://docs.github.com/en/enterprise-server@3.18/admin/configuring-settings/configuring-user-applications-for-your-enterprise/configuring-rate-limits

  27. https://docs.github.com/en/rest/rate-limit/rate-limit

  28. https://docs.github.com/en/graphql/overview/rate-limits-and-query-limits-for-the-graphql-api

  29. https://github.com/opentofu/opentofu/issues/2802

  30. https://docs.github.com/rest/guides/best-practices-for-using-the-rest-api

  31. https://adobe-apiplatform.github.io/umapi-documentation/en/api/getUsersREST.html

  32. https://docs.github.com/en/rest

  33. https://www.endorlabs.com/learn/how-to-get-the-most-out-of-github-api-rate-limits

  34. https://stackoverflow.com/questions/13394077/is-there-a-way-to-increase-the-api-rate-limit-or-to-bypass-it-altogether-for-git

  35. https://docs.github.com/en/rest/using-the-rest-api/troubleshooting-the-rest-api

  36. https://news.ycombinator.com/item?id=43936992

  37. https://stackoverflow.com/questions/77595105/github-rate-limit-api-shows-that-core-has-only-60-api-limit-for-authenticated-us

  38. https://github.blog/changelog/2025-05-08-updated-rate-limits-for-unauthenticated-requests/

  39. https://www.kubeblogs.com/how-to-avoid-github-token-rate-limiting-issues-complete-guide-for-devops-teams/

  40. https://docs.github.com/en/enterprise-server@3.15/rest/using-the-rest-api/rate-limits-for-the-rest-api

  41. https://docs.github.com/en/rest/interactions/user

  42. https://docs.gitlab.com/security/rate_limits/

  43. https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api 2 3 4 5 6

  44. https://www.lunar.dev/post/a-developers-guide-managing-rate-limits-for-the-github-api 2

  45. https://stackoverflow.com/questions/73644164/where-can-i-view-the-secondary-rate-limit-of-github-rest-api 2

  46. https://github.com/edgeleap/edgeleap.github.io/blob/main/index.html

  47. https://docs.github.com/en/rest/metrics/traffic

  48. https://docs.github.com/rest/repos/repos

  49. https://github.com/peter-evans/create-pull-request/issues/855

  50. https://stackoverflow.com/questions/64944523/get-github-api-results-more-than-100

  51. https://www.traceable.ai/blog-post/how-to-mitigate-ddos-attacks-on-your-apis

  52. https://github.com/octokit/plugin-throttling.js/issues/108

  53. https://github.com/ashernoel/Viral-Trends-Clustering

  54. https://github.com/camaraproject/IdentityAndConsentManagement/issues/128

  55. https://github.com/nf-core/tools/issues/911

  56. https://www.endorlabs.com/learn/how-to-get-the-most-out-of-github-api-rate-limits

  57. https://support.catonetworks.com/hc/en-us/articles/18279912261917-GitHub-Configuring-the-Data-Protection-API-Connector

  58. https://docs.github.com/en/rest

  59. https://docs.github.com/github-ae@latest/apps/creating-github-apps/registering-a-github-app/rate-limits-for-github-apps

  60. https://www.reddit.com/r/github/comments/sg4frc/github_api_for_code_search_ratelimiting_problem/

  61. https://docs.github.com/rest/guides/best-practices-for-using-the-rest-api

  62. https://docs.github.com/en/rest/deployments/protection-rules

  63. https://news.ycombinator.com/item?id=43936992

  64. https://shields.io/badges/git-hub-repo-stars 2 3

  65. https://shields.io/badges/git-hub-users-stars

  66. https://shields.io/badges/git-hub-commit-activity

  67. https://alexwilson.tech/content/717e6a35-1cb5-4a28-9321-592d05ddd9dc/ 2

  68. https://moss.sh/reviews/how-to-deploy-static-sites-with-github-actions/

  69. https://github.com/qyjoy/AI-worker 2

  70. https://github.com/conroyp/serverless-api-proxy-cloudflare-workers 2

  71. https://docs.github.com/en/rest/actions/cache

  72. https://shields.io/badges/git-hub-deployments

  73. https://www.youtube.com/watch?v=CC2krFoqL28

  74. https://github.com/badges/shields/issues/6310

  75. https://github.com/always-cache/always-cache

  76. https://github.com/blue-pen5805/llm-proxy-on-cloudflare-workers

  77. https://www.youtube.com/watch?v=4cgpu9L2AE8

  78. https://shields.io/badges/git-hub-license

  79. https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching

  80. https://github.com/lopins/serverless-api-proxy

  81. https://github.com/jaemk/badge-cache

  82. https://github.com/badges/shields/issues/534

  83. https://shields.io/badges

  84. https://shields.io/badges/endpoint-badge

  85. https://shields.io

  86. https://alexwilson.tech/content/717e6a35-1cb5-4a28-9321-592d05ddd9dc/

  87. https://shields.io/badges

  88. https://dev.to/pujux/i-built-a-badge-api-for-your-github-readme-14fn

  89. https://stackoverflow.com/questions/59881949/how-are-build-code-coverage-badges-generated-by-3rd-party-sites

  90. https://github.com/badges/shields/issues/9852

  91. https://css-tricks.com/adding-custom-github-badges-to-your-repo/

  92. https://shields.io/blog/new-frontend

  93. https://shields.io/badges/dynamic-regex-badge

  94. https://stackoverflow.com/questions/69988959/how-can-i-get-json-response-from-github-api

  95. https://www.opslevel.com/resources/service-maturity-shields-io-badges

  96. https://matt-dray.github.io/badgr/reference/get_badge.html

  97. https://github.com/swagger-api/validator-badge

  98. https://dev.to/sarafian/render-badges-with-powershell-markdownps-and-shields-io-2bm

  99. https://github.com/badges/shields/issues/4528

  100. https://contributing.shields.io/tutorial-TUTORIAL.html

  101. https://shields.fly.dev/endpoint/

  102. https://shields.io/badges/my-get-version

  103. https://swharden.com/blog/2022-04-12-github-badge/

  104. https://meta.discourse.org/t/shields-io-unable-to-retrieve-discourse-statistics-api/128574?tl=en

  105. https://stackoverflow.com/questions/11727787/grab-svg-text-value-with-javascript 2

  106. https://contributing.shields.io/module-core_base-service_base-svg-scraping-BaseSvgScrapingService.html

  107. https://shields.io

  108. https://github.com/badges/shields

  109. https://shields.io/badges/w-3-c-validation

  110. https://blog.logrocket.com/guide-svgs-react/

  111. https://learn.microsoft.com/en-us/connectors/shieldsioip/

  112. https://stackoverflow.com/questions/tagged/shields.io

  113. https://www.w3schools.com/html/html5_svg.asp

  114. https://shields.io/docs/static-badges

  115. https://developer.mozilla.org/en-US/docs/Web/SVG

  116. https://shields.io/badges

  117. https://shields.io/badges/endpoint-badge

  118. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat

  119. https://matt-dray.github.io/badgr/reference/get_badge.html

  120. https://gist.github.com/deeplook/20c7a64eda8fc77fe020f5c644466b27

  121. https://contributing.shields.io/tutorial-TUTORIAL.html

  122. https://stackoverflow.com/questions/75602122/how-to-create-key-value-svg-images-in-readme-md-file

  123. https://shields.io/badges/dynamic-regex-badge

  124. https://stackoverflow.com/questions/64670680/how-to-place-text-on-specific-position-in-svg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment