[AnikDey-exe]

Overview

Adds sport and gender filtering to the videos and articles GraphQL queries.

Neither collection could support this before. Videos had no sport stored at all (it was derived at query time in types.py), and articles stored sports_type but no gender. This derives both fields, stores them on new rows, and backfills the existing ones so the queries can filter in Mongo rather than in Python.

Changes Made

Query layer

youtubeVideos(sport: String, gender: String): [YoutubeVideoType]
articles(sportsType: String, gender: String): [ArticleType]

Both arguments are optional on both queries. Filtering happens in the repository as a Mongo query, matching the existing find_by_sports_type. find_by_sports_type is kept and delegates to the new find_recent_filtered, so nothing that called it changes.

YoutubeVideoType gains sport and gender; ArticleType gains gender. The existing sportsType field on videos is kept as a deprecated alias for sport so current clients keep working. Both resolvers prefer the stored value and fall back to deriving from the title, so rows the backfill could not classify still resolve.

Extraction

Articles take their sport from the Daily Sun feed's tags and fall back to the headline; gender goes the other way, headline first then tags. That ordering is based on measurement rather than preference: across the 20 articles in the live feed, tags gave the sport for 20/20 and the headline for 19/20, while the headline gave the gender for 19/20 and tags for 18/20. They never disagreed where both resolved.

The scraper already fetched that endpoint and discarded tags, so this costs no extra request.

Videos use the title only. The YouTube search endpoint's snippet carries no tags, and fetching them would need a second API call per video for a field the title already supplies.

For sports Cornell fields only one team in, the gender is inferred from the sport. "Cornell Volleyball" and "Cornell Wrestling" are not written with a gender, and that mapping is derived from SPORT_URLS rather than hardcoded, so adding a team keeps it in step.

Three bugs in extract_sport_from_title

These had to be fixed before backfilling, or the wrong values would have been written to every row:

• Sprint Football was classified as Football. The \bfootball\b pattern sat above sprint\s+football, so it matched first.
• Lacrosse and Rowing returned None, having no patterns at all. Added those plus baseball, softball, tennis, squash, golf, polo, gymnastics, fencing, equestrian, sailing, cross country, track and field, and swimming and diving.
• A fallback did "ice" in title and "cornell" in title as a substring check, so any title containing "Service", "Practice" or "Choice" classified as Ice Hockey. Cornell Athletics Community Service Day returned Ice Hockey. Removed, since \bhockey\b already covers the real cases.

Backfill

scripts/backfill_sport_gender.py populates both collections from existing titles, re-fetching the Daily Sun feed to recover tags (stored articles match feed entries by slug). Dry run by default:

python3 scripts/backfill_sport_gender.py            # shows the plan, writes nothing
python3 scripts/backfill_sport_gender.py --apply

It is idempotent, so it can be re-run after any change to the extraction helpers.

Test Coverage

Manual, against live data. No automated tests, consistent with the rest of this area of the codebase.

Backfill applied to 78 documents, after which:

	sport	gender
videos	73/75	73/75
articles	5/5	5/5

Re-running then reported to update: 0, confirming idempotency.

Queries executed against that data:

youtubeVideos                                        75
youtubeVideos(sport: "Ice Hockey")                   30
youtubeVideos(gender: "Womens")                      26
youtubeVideos(sport: "Ice Hockey", gender: "Womens")  9
youtubeVideos(sport: "Lacrosse", gender: "Mens")      2
youtubeVideos(sport: "Curling")                       0

articles                                              4
articles(sportsType: "Ice Hockey")                    2
articles(gender: "Mens")                              2
articles(sportsType: "Ice Hockey", gender: "Womens")  1

Articles return 4 of 5 because find_recent applies a 3-day window, which is existing behaviour.

Both scrapers were run with the network mocked. The article scraper produced 20/20 documents with a real sport and 20/20 with a gender, and no "sports" placeholders. The video scraper resolved Lacrosse/Mens and Volleyball/Womens, and correctly returned nulls for a title that names no sport. A video was round-tripped through the service into Mongo and read back with both fields intact.

Also verified: the app imports, the GraphQL schema builds, and the existing test suite passes.

Reproduce

python3 scripts/backfill_sport_gender.py --apply
python3 app.py

Then at http://localhost:8000/graphql:

{
  youtubeVideos(sport: "Ice Hockey", gender: "Womens") { title sport gender url }
  articles(sportsType: "Ice Hockey", gender: "Womens") { title sportsType gender url }
}

Next Steps

Two videos stay unclassified because their titles name no sport, "Cornell Athletics Visits Meinig Fieldhouse" and "Beyond the Ice: Day in the Life of Jonathan Castagna". They return sport: null and appear only in unfiltered results. Worth knowing if a client expects every video to carry a sport.

No indexes were added on sport or gender. At 75 and 5 documents that makes no difference, but they are worth adding if these collections grow.

[coderabbitai]

Important

• Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 3f934e7c-fc54-4a0e-ad69-43314dd11c44

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn