Implement SEO, Sitemap, SSR/Prerendering & Structured Data for Course Search Indexing

[DreamBot706]

Description

Currently, when students search Google for queries like cs101 materials iitg or MA101 coursehub, CourseHub pages do not show up in search results. Shared links (WhatsApp, Discord, Telegram) also don't render rich previews. This issue tracks implementing end-to-end Search Engine Optimization (SEO) so all course pages are indexed, rank effectively on search engines, and preview correctly when shared.

Key Requirements

0. Server-Side Rendering / Prerendering for Bot Requests

• Social crawlers (WhatsApp, Discord, Telegram, Twitter/X) do not execute JavaScript — they only read the raw HTML of the initial response. react-helmet-async alone will not produce visible OG tags for them, since it injects tags client-side after JS runs.
• Googlebot does execute JS but relies on a slower, less reliable two-wave indexing process for CSR content, especially on lower-authority sites.
• Implement SSR (or a build-time/on-request prerender fallback, e.g. prerender.io or a static-generation step) for /browse/* routes so bots receive fully-rendered HTML with title, meta, and OG tags already present in the initial response.
• This is a prerequisite for Requirements Updated course model and routes #2 and UI components (Landing, Dashboard) #4 to function correctly — do this first.

1. Dynamic Course Sitemap (sitemap.xml) & robots.txt

• Generate/serve a sitemap.xml listing all public course routes (e.g. /browse/CS101, /browse/MA101), including lastmod timestamps.
• Sitemap should regenerate automatically when courses/materials are added or updated (not a static file that goes stale).
• Add robots.txt allowing search crawlers to index public /browse/* routes, and explicitly disallowing any private/authenticated routes.
• Add Google Search Console verification meta tag support.
• Add <link rel="canonical"> on each course page to prevent duplicate-content issues if a course is reachable via more than one URL pattern.
• Ensure invalid course codes (e.g. /browse/CS999) return a real HTTP 404, not a 200 response with "not found" text (avoids soft-404 penalties).

2. Search-Optimized Metadata (react-helmet-async + SSR from #0)

• Dynamically update page titles with search keywords:
  • Browse Course Page: {Course Code} - {Course Name} Study Materials | CourseHub IIT Guwahati
  • Example: CS101 - Introduction to Computing Study Materials | CourseHub IIT Guwahati
• Set <meta name="description"> targeting key terms: IIT Guwahati, course materials, notes, PYQs, syllabus.
• Clarify which content is publicly indexable vs. behind login — if materials require auth, only the course shell/summary should be indexed, not gated content.

3. Structured Data (Schema.org Course JSON-LD)

• Inject JSON-LD schema on course pages so search engines identify the content as an educational course:
  • provider: "IIT Guwahati"
  • courseCode: "CS101"
  • name: "Introduction to Computing"

4. Social Sharing Previews (Open Graph & Twitter Cards)

• Add og:title, og:description, og:image, and og:url tags (served via SSR, see #0) so sharing links on WhatsApp, Discord, or Telegram displays rich card previews with course details.

Acceptance Criteria

• /browse/* pages return fully-rendered HTML (title/meta/OG tags present) on first response, verified via curl or "View Source" — not just in the rendered DOM.
• sitemap.xml validates in Google Search Console with 0 errors and reflects newly added courses without manual intervention.
• Invalid course routes return HTTP 404.
• Each course page has a unique, keyword-targeted title and meta description.
• Google's Rich Results Test passes for the Course schema on a sample page.
• Facebook Sharing Debugger and Twitter Card Validator show correct title, description, and image for a sample course link.
• Lighthouse SEO score ≥ 90 on a sample course page.

Out of Scope / Follow-up

• General Core Web Vitals / page speed optimization (affects ranking but is a separate effort).
• Content strategy (blog posts, backlinks, etc.) for off-page SEO.