
{"id":10226,"date":"2022-06-21T09:11:39","date_gmt":"2022-06-21T09:11:39","guid":{"rendered":"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/"},"modified":"2024-12-04T14:10:50","modified_gmt":"2024-12-04T14:10:50","slug":"gstreamer_analysis","status":"publish","type":"page","link":"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/","title":{"rendered":"GStreamer Analysis"},"content":{"rendered":"<!DOCTYPE html PUBLIC \"-\/\/W3C\/\/DTD HTML 4.0 Transitional\/\/EN\" \"http:\/\/www.w3.org\/TR\/REC-html40\/loose.dtd\">\n <html><body><div class=\"toc-macro client-side-toc-macro  conf-macro output-block\" data-headerelements=\"H1,H2,H3,H4,H5,H6,H7\" data-hasbody=\"false\" data-macro-name=\"toc\"> <\/div><h3 class=\"wp-sub-header\">Introduction<\/h3><p>The purpose of this document is to simplify some of the terms, descriptions, and mechanisms of the GStreamer core. It will,&nbsp;initially,&nbsp;serve as a foundation to help model and test the GStreamer implementation in the RDK\/HAL system.<\/p><p>    \n\n<\/p><img decoding=\"async\" src=\"\/wp-content\/uploads\/sites\/16\/2023\/09\/Gstreamer.drawio.png\"><p>This document describe the details of the following,<\/p><ul><li>API for multimedia applications<\/li><li><strong>Plugin<\/strong>&nbsp;architecture<\/li><li><strong>Pipeline<\/strong>&nbsp;architecture<\/li><li>Mechanism for media type handling, negotiation, synchronization<\/li><\/ul><h3 class=\"wp-sub-header\">Performance<\/h3><ul><li>Data passing between plugins is lightweight (pointers reduce overhead)<\/li><li>Shared memory mechanism<\/li><li>Sub-buffers split buffers into manageable pieces (blocking)<\/li><li>Dedicated streaming threads with scheduling handled by the kernel<\/li><\/ul><h3 class=\"wp-sub-header\">Plugin<\/h3><ul><li>Protocol handler (loaded at runtime)<\/li><li>Sources: audio\/video (protocol plugins)<\/li><li>Formats: parsers, formaters, muxers, demuxers, metadata, subtitles<\/li><li>Codecs: coders\/decoders<\/li><li>Filters: converters, mixers, effects<\/li><li>Sinks: audio\/video (protocol plugins)<\/li><\/ul><h3 class=\"wp-sub-header\">Object Model<\/h3><ul><li>Adheres to GObject (Glib 2.0)<\/li><li>Uses signals and object properties<\/li><\/ul><h3 class=\"wp-sub-header\">Architecture<\/h3><p>    \n\n<\/p><img decoding=\"async\" src=\"\/wp-content\/uploads\/sites\/16\/2023\/09\/GStreamer%20Architecture.drawio.png\"><h3 class=\"wp-sub-header\">GST-Element<\/h3><ul><li>Has one specific function (read, decode, &#8230;)<\/li><li>Has two&nbsp;<strong>Pads:<\/strong><ol><li><span>source (output)<\/span><\/li><li><span>sink (input)<\/span><\/li><\/ol><\/li><li>GStreamer core views elements as blocks of bytes<\/li><li>Linked or chained elements create a pipeline that performs a specific task<\/li><\/ul><p>    \n\n<\/p><img decoding=\"async\" src=\"\/wp-content\/uploads\/sites\/16\/2023\/09\/GST%20Element%20Link.drawio.png\"><h3 class=\"wp-sub-header\">GST-Element Pad<\/h3><p>Is defined by two properties:<\/p><ol><li><strong>direction<\/strong>: source, sink<\/li><li><strong>availability<\/strong>: always, sometimes (dynamic), on request<\/li><\/ol><p><em>ghost pad<\/em>: a pad from some element in the bin that can be accessed directly from the bin as well.<\/p><h3 class=\"wp-sub-header\">GST-Bin<\/h3><ul><li>A container for a collection of elements (manages the state of all elements within it)<\/li><li>Forwards bus messages from contained elements (errors, tags, EOS)<\/li><li>Simplifies implementing complex pipelines by allowing the break up of the pipeline into smaller blocks<\/li><\/ul><h3 class=\"wp-sub-header\">GST-Pipeline<\/h3><ul><li>A top-level bin<\/li><li>A generic container that manages the synchronization and bus messages of the contained elements.<\/li><li>Contains a bus by default<\/li><\/ul><h3 class=\"wp-sub-header\">GST-Bus<\/h3><ul><li>A bus is a simple system that forwards messages from the streaming thread to an application in its own thread context.<\/li><\/ul><h3 class=\"wp-sub-header\">GST-Bus Message Types<\/h3><p>All messages contain a message&nbsp;<strong><em>source<\/em><\/strong>,<strong><em>&nbsp;type<\/em><\/strong>, and&nbsp;<strong><em>time-stamp<\/em><\/strong>.<\/p><ul><li><strong>Error<\/strong>: fatal message that terminate data-passing<\/li><li><strong>Warning<\/strong>: non-fatal message, but user-visible problems will happen<\/li><li><strong>Information<\/strong>: non-problem notification<\/li><li><strong>End of Stream<\/strong>: emitted when the stream had ended<\/li><li><strong>Tags<\/strong>: emitted when metadata is found in the stream<\/li><li><strong>State-changes<\/strong>: emitted after a successful state change<\/li><li><strong>Buffering<\/strong>: emitted during caching of network-streams<\/li><li><strong>Element<\/strong>: special messages that are unique to certain elements and usually represent additional features.<\/li><li><strong>Application-specific<\/strong>: primarily meant for internal use in applications in case the application needs to marshal information from some thread into the main thread<\/li><\/ul><h3 class=\"wp-sub-header\">GST Communication Mechanisms<\/h3><ol><li><span><strong>Buffer<\/strong><\/span>: objects for passing streaming data between elements in pipeline<br><ol><li>travel downstream<\/li><\/ol><\/li><li><span><strong>Events<\/strong><\/span>: objects sent between elements or from the application to elements<br><ol><li>travel downstream and upstream<\/li><\/ol><\/li><li><span><strong>Messages<\/strong><\/span>: objects posted by elements on pipeline&rsquo;s message bus (element to application)<br><ol><li>handled synchronously from pipeline to bus; asynchronously from bus to application<\/li><\/ol><\/li><li><span><strong>Queries<\/strong><\/span>: allow applications to request information from pipeline<br><ol><li>travel downstream and upstream; always handled synchronously<\/li><\/ol><\/li><\/ol><h3 class=\"wp-sub-header\">Communication Control Flow<\/h3><ol><li><strong>Downstream&nbsp;<\/strong>: src element to sink element<\/li><li><strong>Upstream<\/strong>: sink element to src element<\/li><\/ol><p>    \n\n<\/p><img decoding=\"async\" src=\"\/wp-content\/uploads\/sites\/16\/2023\/09\/Communication%20Control%20Flow.drawio.png\"><div class=\"table-wrap\"> <table class=\"wrapped confluenceTable\"><colgroup><col><col><col><col><\/colgroup><tbody><tr><th class=\"confluenceTh\"><p><span><u><strong>Buffer<\/strong><\/u><\/span><\/p><p><span>Actual media data.<\/span><\/p><p><span>A simple buffer consists of:<\/span><\/p><ul><li><strong>Pointers to memory objects.<\/strong><\/li><li><strong>T<\/strong><span>imestamp<\/span><\/li><li><span>Reference count<\/span><\/li><li><strong>Flags<\/strong><\/li><\/ul><\/th><th class=\"confluenceTh\"><p><span><strong><u>Events<\/u><\/strong><\/span><\/p><p><span>Control information.<\/span><\/p><\/th><th class=\"confluenceTh\"><p><span><u>Messages<\/u><\/span><\/p><p><span>Information on elements or pipeline.<\/span><\/p><p><span>A message consist of:<\/span><\/p><ul><li><span>Source<\/span><\/li><li>Type<\/li><li>Timestamp<\/li><\/ul><\/th><th class=\"confluenceTh\"><p><span><u>Queries<\/u><\/span><\/p><p>Request for a specific stream property related to progress tracking.<\/p><\/th><\/tr><\/tbody><\/table> <\/div><p><br><\/p><h3 class=\"wp-sub-header\">GST-Element Types<\/h3><ol><li><strong>Source<\/strong>: generates data<\/li><li><strong>Filter<\/strong>: performs task on input data to send proper output data (convertors, demuxers, muxers, codecs etc)<\/li><li><strong>Sink<\/strong>: receives data<\/li><\/ol><p>These 3 types of elements create a simple GST-Pipeline.<\/p><p>    \n\n<\/p><img decoding=\"async\" src=\"\/wp-content\/uploads\/sites\/16\/2023\/09\/pipeline.drawio.png\"><p>Pipelining is an implementation technique whereby multiple instructions are overlapped in execution; it takes advantage of parallelism that exists among the actions needed to execute an instruction.<em>&nbsp;<\/em><\/p><h3 class=\"wp-sub-header\">GST-Element States<\/h3><ol><li><strong>GST_STATE_NULL<\/strong>: default state; no resources are allocated in this state<\/li><li><strong>GST_STATE_READY<\/strong>: an element has allocated all of its global resources within the stream, but the stream is NOT open yet.<\/li><li><strong>GST_STATE_PAUSED<\/strong>: an element has opened the stream, but no actively processing it. (clock does NOT run)<\/li><li><strong>GST_STATE_PLAYING<\/strong>: an element maintains the open stream while processing it. (clock starts)<\/li><\/ol><h3 class=\"wp-sub-header\">Porting<\/h3><p>The following is a list of components that might be needed to properly test implementation of the HAL GST components<\/p><ol><li><span>aesdecyrpt<\/span><\/li><li><span>aesencrypt<\/span><\/li><li><span>dtcpdecrypt<\/span><\/li><li><span>dtcpencrypt<\/span><\/li><li>httpsink<\/li><li><span>rbifilter<\/span><\/li><\/ol><p><span><br><\/span><\/p><\/body><\/html>\n","protected":false},"excerpt":{"rendered":"<p>Introduction The purpose of this document is to simplify some of the terms, descriptions, and [&hellip;]<\/p>\n","protected":false},"author":28,"featured_media":0,"parent":10158,"menu_order":8,"comment_status":"open","ping_status":"closed","template":"","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"class_list":["post-10226","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.9 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>GStreamer Analysis - RDK Documentation Portal | Documentation<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"GStreamer Analysis - RDK Documentation Portal | Documentation\" \/>\n<meta property=\"og:description\" content=\"Introduction The purpose of this document is to simplify some of the terms, descriptions, and [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/\" \/>\n<meta property=\"og:site_name\" content=\"RDK Documentation Portal | Documentation\" \/>\n<meta property=\"article:modified_time\" content=\"2024-12-04T14:10:50+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/\",\"url\":\"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/\",\"name\":\"GStreamer Analysis - RDK Documentation Portal | Documentation\",\"isPartOf\":{\"@id\":\"https:\/\/developer.rdkcentral.com\/documentation\/#website\"},\"datePublished\":\"2022-06-21T09:11:39+00:00\",\"dateModified\":\"2024-12-04T14:10:50+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/developer.rdkcentral.com\/documentation\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Documentation\",\"item\":\"https:\/\/developer.rdkcentral.com\/documentation\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"RDK Video\",\"item\":\"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Sub-Systems\",\"item\":\"https:\/\/developer.rdkcentral.com\/documentation\/?page_id=10158\"},{\"@type\":\"ListItem\",\"position\":5,\"name\":\"GStreamer Analysis\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/developer.rdkcentral.com\/documentation\/#website\",\"url\":\"https:\/\/developer.rdkcentral.com\/documentation\/\",\"name\":\"RDK Documentation Portal | Documentation\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/developer.rdkcentral.com\/documentation\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"GStreamer Analysis - RDK Documentation Portal | Documentation","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/","og_locale":"en_US","og_type":"article","og_title":"GStreamer Analysis - RDK Documentation Portal | Documentation","og_description":"Introduction The purpose of this document is to simplify some of the terms, descriptions, and [&hellip;]","og_url":"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/","og_site_name":"RDK Documentation Portal | Documentation","article_modified_time":"2024-12-04T14:10:50+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/","url":"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/","name":"GStreamer Analysis - RDK Documentation Portal | Documentation","isPartOf":{"@id":"https:\/\/developer.rdkcentral.com\/documentation\/#website"},"datePublished":"2022-06-21T09:11:39+00:00","dateModified":"2024-12-04T14:10:50+00:00","breadcrumb":{"@id":"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/sub-systems\/gstreamer_analysis\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/developer.rdkcentral.com\/documentation\/"},{"@type":"ListItem","position":2,"name":"Documentation","item":"https:\/\/developer.rdkcentral.com\/documentation\/"},{"@type":"ListItem","position":3,"name":"RDK Video","item":"https:\/\/developer.rdkcentral.com\/documentation\/documentation\/rdk_video_documentation\/"},{"@type":"ListItem","position":4,"name":"Sub-Systems","item":"https:\/\/developer.rdkcentral.com\/documentation\/?page_id=10158"},{"@type":"ListItem","position":5,"name":"GStreamer Analysis"}]},{"@type":"WebSite","@id":"https:\/\/developer.rdkcentral.com\/documentation\/#website","url":"https:\/\/developer.rdkcentral.com\/documentation\/","name":"RDK Documentation Portal | Documentation","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/developer.rdkcentral.com\/documentation\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/developer.rdkcentral.com\/documentation\/wp-json\/wp\/v2\/pages\/10226","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/developer.rdkcentral.com\/documentation\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/developer.rdkcentral.com\/documentation\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/developer.rdkcentral.com\/documentation\/wp-json\/wp\/v2\/users\/28"}],"replies":[{"embeddable":true,"href":"https:\/\/developer.rdkcentral.com\/documentation\/wp-json\/wp\/v2\/comments?post=10226"}],"version-history":[{"count":1,"href":"https:\/\/developer.rdkcentral.com\/documentation\/wp-json\/wp\/v2\/pages\/10226\/revisions"}],"predecessor-version":[{"id":10232,"href":"https:\/\/developer.rdkcentral.com\/documentation\/wp-json\/wp\/v2\/pages\/10226\/revisions\/10232"}],"up":[{"embeddable":true,"href":"https:\/\/developer.rdkcentral.com\/documentation\/wp-json\/wp\/v2\/pages\/10158"}],"wp:attachment":[{"href":"https:\/\/developer.rdkcentral.com\/documentation\/wp-json\/wp\/v2\/media?parent=10226"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}