Implementing the Media Fragments URI Specification

Implementing the Media Fragments URI Specification

• 1.
  Implementing the Media Fragments URI Specification Raphaël Troncy <raphael.troncy@eurecom.fr>, Davy Van Deursen, Erik Mannens Silvia Pfeiffer, Yves Lafon
• 2.
  Use Case  Aidemreceived on her Facebook wall a status message containing a Media Fragment URI  Use a ‘#’ !  Highlight a video sequence  Highlight a region to pay attention to 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -2
• 3.
  Media Fragments URI Provide URI-based mechanisms for uniquely identifying fragments for media objects on the Web, such as video, audio, and images. Photo credit: Robert Freund 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -3
• 4.
  Requirements  r01: Temporalfragments:  a clipping along the time dimension from a start to an end time that are within the duration of the media resource  r02: Spatial fragments:  a clipping of an image region, only consider rectangular regions  r03: Track fragments:  a track as exposed by a container format of the media resource  r04: Named fragments:  a media fragment - either a track, a time section, or a spatial region - that has been given a name through some sort of annotation mechanism 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -4
• 5.
  Side Conditions  Restrictto what the container format (encapsulating the compressed media content) can express (and expose), thus no transcoding  Protocol covered: HTTP(S), FILE, RTSP, RTMP https://www.w3.org/TR/media-frags-reqs/ 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -5
• 6.
  Media Fragments processing General principle:  Smart UA will strip out the fragment definition and encode it into custom http headers ...  (Media) Servers will handle the request, slice the media content and serve just the fragment while old ones will serve the whole resource  Four recipes proposed 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -6
• 7.
  Recipe 1: UAmapped byte ranges  The User Agent knows how to map a custom unit into bytes and sends a normal Range request expressed in bytes 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -7
• 8.
  Recipe 1: UAmapped byte ranges 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -8
• 9.
  Recipe 2: Servermapped byte ranges  The UA sends a Range request expressed in a custom unit (e.g. seconds), the server answers directly with a 206 Partial Content and indicates the mapping between bytes and the custom unit 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -9
• 10.
  Recipe 2: Servermapped byte ranges 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 10
• 11.
  Recipe 3: Proxycacheable variant  The UA sends a Range request expressed in a custom unit (e.g. seconds), the server answers first with just a Head and the mapping between the custom unit and bytes so that the UA issues another normal Range request expressed this time in bytes making the answer cacheable. 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 11
• 12.
  Recipe 3: Proxycacheable variant 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 12
• 13.
  Recipe 4: Servingplayable resources  The UA sends a Range request expressed in a custom unit (e.g. seconds), the server provides a multipart message body reply (multipart/byte-ranges) containing not only the bytes corresponding to requested media fragment but also the media header data making the resource playable. 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 13
• 14.
  Recipe 4: Servingplayable resources 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 14
• 15.
  Implementation  Media Fragmentserver (4 recipes supported):  Ninsuna: https://ninsuna.elis.ugent.be/MediaFragmentsServer  Media Fragment user agents:  Ninsuna Flash player: https://ninsuna.elis.ugent.be/MediaFragmentsPlayer Supports recipe 1  Silvia Pfeiffer's experiment with HTML5 + JS: https://annodex.net/~silvia/itext/mediafrag.html Supports recipe 1 (for .ogg files and time dimension)  Firefox pluggin development in order to support all recipes (HTML5 + XMLHttpRequest) 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 15
• 16.
  Issues  Communicating withthe media player  HTML5 browser: patch submitted for Firefox  Flash: need to build a custom player  Media Fragments rendering  Spatial dimension?  Names/Tracks discovery  Media Multitrack (JS) API: https://www.w3.org/WAI/PF/HTML/wiki/Media_MultitrackAPI  Get involved? https://www.w3.org/TR/media-frags/ https://lists.w3.org/Archives/Public/public-media-fragment/ 29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 16