Save to database #13

Merged
haylan merged 42 commits from save-to-database into main 2026-07-12 21:51:57 +00:00
Showing only changes of commit 20e943a2b5 - Show all commits
+21 -9
View File
@@ -99,6 +99,9 @@ vendor/bin/phpunit tests/Unit/Service/SvgRendererTest.php
# Run tests matching a filter # Run tests matching a filter
vendor/bin/phpunit --filter it_renders vendor/bin/phpunit --filter it_renders
# Static analysis (PHPStan level 8)
composer phpstan
``` ```
On Windows, run via WSL (If Docker Desctop is not): On Windows, run via WSL (If Docker Desctop is not):
@@ -190,20 +193,24 @@ There is no `composer.lock` in the repo. If you add or change dependencies, run
## Architecture ## Architecture
Single-controller Symfony app with no database. The request flow: Single-controller Symfony app. Two-tier cache: a 1h filesystem SVG cache in front of a SQLite raw-data store, in front of the provider APIs. The request flow:
``` ```
GET /graph.svg?theme=dark|light GET /graph.svg?theme=dark|light
└─ GraphController └─ GraphController
├─ host check (ALLOWED_HOSTS env, optional) ├─ host check (ALLOWED_HOSTS env, optional)
├─ cache lookup (filesystem, 1h TTL, key = "graph_{theme}") ├─ cache lookup (filesystem, 1h TTL, key = "graph_{theme}")
│ └─ on miss: │ └─ on miss: ContributionAggregator::aggregate()
│ ├─ GitHubProvider → GitHub GraphQL API (contributionCalendar query) │ ├─ per configured provider: ContributionStore::latestDate($name)
├─ GitLabProvider → GitLab REST /users/:id/events (paginated, 100/page) │ → $since = latest - 3 days (trailing overlap), or null on first run
─ GiteaProvider → Gitea REST /api/v1/users/:user/heatmap ─ GitHubProvider → GitHub GraphQL API (contributionCalendar query, bounded by $since/$until)
each returns array<string, int> (Y-m-d => count) ├─ GitLabProvider → GitLab REST /users/:id/events (paginated, 100/page, `after`/`before` bounded)
failures are caught and logged; remaining providers still render ├─ GiteaProvider → Gitea REST /api/v1/users/:user/heatmap (filtered client-side by $since/$until)
└─ merge by date (sum counts across providers) │ each returns array<string, int> (Y-m-d => count)
│ │ failures are caught and logged; remaining providers still render
│ ├─ ContributionStore::merge() persists fresh data per provider (SQLite upsert)
│ ├─ ContributionStore::all(sinceDays: 371) reads back the render window (source of truth)
│ ├─ ContributionStore::prune() drops rows older than CONTRIBUTIONS_RETENTION_DAYS
│ └─ SvgRenderer::render() │ └─ SvgRenderer::render()
└─ Response: image/svg+xml, Cache-Control: public max-age=3600 └─ Response: image/svg+xml, Cache-Control: public max-age=3600
``` ```
@@ -212,7 +219,11 @@ GET /graph.svg?theme=dark|light
**SvgRenderer:** builds a 53-column × 7-row grid aligned so the last column always ends on the Saturday of the current week. Five intensity levels (0 contributions → level 0, 13 → 1, 46 → 2, 79 → 3, 10+ → 4) mapped to GitHub's exact colour tokens. No external assets — the SVG is fully self-contained. **SvgRenderer:** builds a 53-column × 7-row grid aligned so the last column always ends on the Saturday of the current week. Five intensity levels (0 contributions → level 0, 13 → 1, 46 → 2, 79 → 3, 10+ → 4) mapped to GitHub's exact colour tokens. No external assets — the SVG is fully self-contained.
**Cache:** filesystem adapter (`var/cache/`), mounted as a Docker volume to survive container restarts. Theme is part of the cache key so dark and light are cached independently. **SVG cache:** filesystem adapter (`var/cache/`), mounted as a Docker volume to survive container restarts. Theme is part of the cache key so dark and light are cached independently.
**ContributionStore:** PDO SQLite at `var/data/contributions.db` (also a Docker volume), table `contributions (provider, date unixtime, count)`. Keyed by `(provider, date)`, upserted via `INSERT ... ON CONFLICT`. Bounds every re-fetch to a 3-day trailing window off the last stored date per provider — the root fix for the full-365-day-refetch `MaxExecutionTimeError` that used to hit on every cache miss. `CONTRIBUTIONS_RETENTION_DAYS` controls how far back rows are kept (empty = forever).
**`graph:contributions:refetch` command:** manual escape hatch (`src/Command/RefetchContributionsCommand.php`) that bypasses the incremental trailing-window fetch and re-pulls a full or explicit date range. Options: `--provider=github,gitlab` (comma-separated, default all configured), `--from=YYYY-MM-DD` (default 365 days ago), `--to=YYYY-MM-DD` (default today), `--all` (shorthand for `--from=2005-01-01`). Splits the requested range into ≤365-day chunks (GitHub's GraphQL window limit; also caps GitLab pagination per call) and merges each chunk into the store as it completes, so a large `--all` run can't reintroduce the original runaway-pagination timeout.
## Environment variables ## Environment variables
@@ -224,5 +235,6 @@ GET /graph.svg?theme=dark|light
| `GITLAB_URL` | No | Defaults to `https://gitlab.com` | | `GITLAB_URL` | No | Defaults to `https://gitlab.com` |
| `GITEA_USER` / `GITEA_TOKEN` / `GITEA_URL` | For Gitea | Token scope: `read:user` | | `GITEA_USER` / `GITEA_TOKEN` / `GITEA_URL` | For Gitea | Token scope: `read:user` |
| `ALLOWED_HOSTS` | No | Comma-separated hostnames; empty = allow all | | `ALLOWED_HOSTS` | No | Comma-separated hostnames; empty = allow all |
| `CONTRIBUTIONS_RETENTION_DAYS` | No | Days of history to keep in the SQLite store; empty = keep forever |
Copy `.env` to `.env.local` for local development — `.env.local` is gitignored. Copy `.env` to `.env.local` for local development — `.env.local` is gitignored.