Refactor kalvidres LTI extractor to improve URL extraction and direct Kaltura HLS download

[chmanny]

Summary of changes

• Generalize the LTI launch page form parser by extracting the first available form dynamically and parsing hidden inputs via _hidden_inputs, instead of strictly matching the ID "ltiLaunchForm".
• Broaden the JavaScript redirection regex to safely capture variations like window.top.location.replace or location.href across different Moodle configurations.
• Introduce comprehensive regex fallback patterns across the launch, submit, and redirect stages to scrape partner_id, entry_id, and ks tokens if standard embedding extraction fails.
• Bypass standard KalturaIE API flavor resolution completely. Instead, construct a direct signed HLS playManifest URL using the extracted metadata, routing the download directly through the CloudFront CDN stream and resolving API access/extraction errors.

---

Summary of changes

• Bypass Kaltura API Restrictions: Modifies KalvidresLtiIE to bypass standard KalturaIE API flavor resolution. It now constructs a direct signed HLS playManifest URL using partner_id, entry_id, and the extracted Kaltura Session (ks) token, routing the download directly through the CloudFront CDN stream.
• Generalize LTI Launch Form Parsing: Instead of strictly matching the form ID "ltiLaunchForm", the extractor now grabs the first available form and parses inputs dynamically using common hidden input utilities.
• Robust JavaScript Redirection Handling: Broadens the redirection regex to catch variations in syntax (e.g., window.top.location.replace or location.href) utilized by various Moodle configurations.
• Fallback Metadata Scraping: Introduces fallback regex patterns to scan the combined source text of the launch, submit, and redirect stages for tokens when standard KalturaIE embed extraction fails.

Detailed Breakdown

1. LTI Launch Form Extraction

Example HTML:

<form action="https://<GENERIC_KAF_ID>[.kaf.kaltura.com/lti/launch](https://.kaf.kaltura.com/lti/launch)" method="POST">
  <input type="hidden" name="custom_entry_id" value="0_placeholder">
  </form>

• Old Behavior: Strictly looked for the ID ltiLaunchForm. Threw Unable to extract ltiLaunchForm form exception when missing.
• New Behavior: Grabs the first available form regardless of ID and utilizes global hidden input scraping:

launch_form_str = self._search_regex(r'(?P<form><form[^>]*>)', launch_webpage, ..., default=None)
if launch_form_str:
    action_url = extract_attributes(launch_form_str).get('action') or launch_url
    launch_inputs = self._hidden_inputs(launch_webpage)

2. JavaScript Redirection Handling

Example HTML:

<script>
    window.top.location.replace("https://<KAF_SUBDOMAIN>[.kaltura.com/browseandembed/](https://.kaltura.com/browseandembed/)...");
</script>

• Old Behavior: Failed to match double quotes, .replace(), or window wrappers (window.top), throwing an Unable to extract kalvidres redirect url error.
• New Behavior: Uses a resilient regex pattern to match multiple browser context wrapper styles:

mobj = re.search(r'window(?:(?:\.top|\.parent))?\.location\.(?:href|replace)\s*(?:=|)\s*\(?[\'"](?P<url>[^\'"]+)[\'"]\)?', submit_page)

3. Kaltura API vs. Direct CDN Stream Bypass

• Old Behavior: Passed the URL off to KalturaIE._extract_url(), which frequently failed due to strict API multirequest rule updates or restricted API access.
• New Behavior: Extracts tokens directly from all available HTML layers to manually construct a valid manifest pipeline:

[https://cdnapisec.kaltura.com/p/](https://cdnapisec.kaltura.com/p/)<PARTNER_ID>/sp/<PARTNER_ID>00/playManifest/entryId/<ENTRY_ID>/format/applehttp/protocol/https/ks/<KS_TOKEN>

The server successfully issues an HLS .m3u8 playlist which routes directly to the CloudFront CDN source (https://cfvod.kaltura.com/scf/hls/...), bypassing the API limitations entirely.

[C0D3D3V]

I'm not really happy with the manual construction of cdnapisec.kaltura.com. But since I have no way to test this I hope it improves the extractor.

Did you make the changes yourself or did you use an AI?

I will merge it later :)

[chmanny]

Hi, yes, I used an LLM to write the fix.
Can I contact you via telegram or email to send you more detailed information (i.e. a link so you can try yourself)?

EDIT
I have made some changes since before:

• 1. URL Smuggling via KalturaIE

  • Direct HLS manifest construction is fragile and bypasses yt-dlp's built-in format sorting, subtitles, and CDN fallback logic. Passing kaltura:PARTNER_ID:ENTRY_ID via smuggle_url lets KalturaIE do the heavy lifting natively.

• 2. Dynamic Service Host Detection

  • This removes the hardcoding of cdnapisec.kaltura.com and automatically accommodates custom Kaltura domain hosting.

• 3. Tiered LTI Launch Form Handling

  • Strict parsing (id="ltiLaunchForm") prevents the extractor from accidentally grabbing the wrong hidden inputs (like search bars or CSRF tokens elsewhere on the page), while the fallback ensures backward compatibility.

• 4. Regex Refinements ({8,10} and \b)

  • Kaltura entry IDs have indeed grown over time. Restricting partner IDs with word boundaries (\b) is excellent defensive programming to avoid greedy regex mismatches.