feat: enhance daily digest logic for send/log integrity and location …

[TheTrueAI]

…relevance.

Closes #44

[Copilot]

Pull request overview

Improves the daily digest pipeline to better satisfy Issue #44 by enforcing per-subscriber location relevance during evaluation and tightening send/log semantics to reduce “sent log ≠ delivered email” mismatches and improve retry idempotency.

Changes:

• Track job URLs per normalized location bucket and only evaluate unseen jobs from a subscriber’s own location bucket.
• Split evaluated jobs into “good matches” vs “low score” and adjust logging so good matches are logged only after a successful send (low-score jobs are logged even on send failure).
• Add tests covering send-failure logging, retry behavior, and location relevance; mark R7 roadmap steps as done.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
daily_task.py	Implements location-bucket URL tracking and adjusts send/log ordering & retry semantics.
tests/test_daily_task.py	Adds regression tests for send/log integrity, retry behavior, and location relevance filtering.
docs/strategy/ROADMAP.md	Marks Issue #44 (R7) steps as completed.

[Copilot]

mark_subscriber_last_sent() is now called after log_sent_jobs(). If log_sent_jobs() fails (DB/network issue), the subscriber won't get last_sent_at updated, which can cause duplicate sends for weekly cadence (and reruns in general) even though the email already went out. Consider marking last_sent_at immediately after a successful send_daily_digest() and treating log_sent_jobs() as best-effort (e.g., wrap/log exceptions) so a logging failure doesn’t lead to repeat emails.

Suggested change

	# Send succeeded — log ALL evaluated jobs and mark subscriber as sent
	all_eval_ids = low_score_ids + good_match_ids
	if all_eval_ids:
	log_sent_jobs(db, sub_id, all_eval_ids)
	mark_subscriber_last_sent(db, sub_id)
	# Send succeeded — first mark subscriber as sent, then best-effort log ALL evaluated jobs
	try:
	mark_subscriber_last_sent(db, sub_id)
	except Exception:
	log.exception(
	" sub=%s — failed to mark last_sent_at; subscriber may receive duplicate digests",
	sub_id,
	)
	all_eval_ids = low_score_ids + good_match_ids
	if all_eval_ids:
	try:
	log_sent_jobs(db, sub_id, all_eval_ids)
	except Exception:
	log.exception(
	" sub=%s — failed to log sent jobs; will retry evaluation next run",
	sub_id,
	)

[Copilot]

The ej_urls = {id(ej): _job_url(ej) for ej in evaluated} pattern is hard to follow and relies on object identity as a key. It would be clearer (and less error-prone) to compute the URL alongside each EvaluatedJob (e.g., build a list of (ej, url) pairs or a direct ej -> url mapping) and reuse that when building *_ids and email_jobs.

[Copilot]

The comment says jobs are keyed by title|company, but the actual dedupe key includes title|company_name|location. Updating the comment would prevent confusion when reasoning about deduping and cache/log keys.

Suggested change

	# Collect all jobs keyed by title|company to avoid duplication
	# Collect all jobs keyed by title|company_name|location to avoid duplication

[Copilot]

location_urls is a set, so sub_urls iteration order is non-deterministic. That makes the evaluation/email job ordering non-reproducible across runs, which can complicate debugging and lead to flaky behavior if downstream assumes stable ordering. Consider sorting unseen_urls (or storing URLs in a deterministic sequence) before building unseen_jobs/calling evaluate_all_jobs.

Suggested change

	unseen_urls = [url for url in sub_urls if url_to_db_id.get(url) and url_to_db_id[url] not in sent_ids]
	unseen_urls = sorted(
	url
	for url in sub_urls
	if url_to_db_id.get(url) and url_to_db_id[url] not in sent_ids
	)