Data · Changelog

Data changelog

How the dataset itself changes over time — new history capture, classification rule versions, coverage milestones. Entries are dated by when the change took effect; stored history is never restated after the fact.

100-day freshness cap

2026-10-15

No posting stays on the board more than 100 days past our first sighting of it (ingested_at), however long the employer keeps it listed. Age is measured from first sighting, never from the employer's posted date, which employers restamp. Enforcement is two-layered: every read path (board, API live views, exports) excludes past-limit postings outright, and ingest closes the underlying rows with the new close_reason 'aged_out'. A posting past the limit is never reactivated by a later crawl.

Effect on the data: closures stamped 'aged_out' are freshness retirements, distinct from 'board_removed' (the listing disappeared) and 'feed_aged_out' (an aggregator stopped re-sending it). Point-in-time history is unaffected — snapshots and the event log keep recording past-limit postings exactly as before, so historical exports do not change.

Point-in-time history capture begins

2026-10-15

Append-only history starts today. Every crawl now writes field changes (title, location, country, salary, material description changes) to the job_events log with old and new values, and a daily snapshot of the full board lands in job_snapshots. Snapshots are insert-only per day — a day, once written, is never rewritten.

Close rule formalized at the same time: a listing closes only after two consecutive fully-successful crawls of its board miss it, and the closure is stamped with a close_reason (board_removed, board_retired, or feed_aged_out for aggregator sources). First-seen (ingested_at) and last-seen (last_seen_at) timestamps are recorded on every row; history before 2026-10-15 was not captured and cannot be reconstructed.

Classification v1 stored on every row

2026-10-15

Seniority, workplace type, AI-role and early-career flags are now computed at ingest time by a versioned rule set (tags_version 1) and stored on the row, replacing display-time derivations. Seniority values: intern, entry, mid, senior, staff_principal, manager, director, vp_executive. Workplace type: remote, hybrid, onsite, or null when the posting does not say.

Rules are title/location/description based and deterministic; re-running them is idempotent. When the rules change, tags_version increments and the change is announced here — stored history is not restated under new rules.

Company domain enrichment, pass 1

2026-10-15

Company records gained a domain field, filled from the hosts of each company's own job-posting URLs (the employer's ATS board host and the hosts appearing in its posting links). The rule that produced each domain is recorded alongside it (companies.domain_source). Domains are never guessed from the company name alone, and an existing domain is never overwritten by a later pass.

Twice-daily crawl cadence

2026-10-15

The board crawls every registered company board and public feed twice daily — 11:15 AM and 5:15 PM ET — ahead of the 12:00 PM and 6:00 PM ET drops. New listings stage to the next drop; listings already known release immediately. Aggregator-sourced rows age out after 24 hours without a sighting.