Video Contract
Applications play video using GStreamer. Video pipelines are created using playbin and rely on auto-plugging. Applications must not create specific elements. Applications may specify appsrc as the pipeline source element by setting the uri
property of playbin to appsrc://
and use the playbin source-setup
signal to obtain and configure the appsrc element.
Video Sink
The video sink element is selected by playbin via auto-plugging. The application may obtain the video sink element selected by playbin for the pipeline using the playbin video-sink
property:
GstElement *videoSink; g_object_get( playbin, “video-sink”, &videoSink, NULL);
Alpha Hole Punching
Video is presented beneath graphics. The application is responsible for performing alpha hole punching so the video is visible. This involves drawing graphics pixels with zero alpha (for full video visibility) or pixels with alpha values less than opaque (for translucent graphics blended with video). See the rectangle
property for video positioning.
Video Sink Contract
QoS
The video sink will supply QoS messages that the application can monitor to obtain information such as total frames and dropped frames. The application can obtain this information within a GStreamer bus watch function using:
gst_message_parse_qos_stats( GstMessage msg, GstFormat format, quint64 *processed, quint64 *dropped );
HDR
For HDR content where HDR meta-data is at the container level rather than the elementary stream level, for example with VP9, the video sink will require and accept HDR meta-data via GstCaps
. The following pseudo code illustrates how the sink would access this data:
GGstStructure *structure; const char *colorimetry; const char *masteringDisplayMetadata; const char *contentLightLevel; structure= gst_caps_get_structure(caps, 0); if ( gst_structure_has_field(structure, "colorimetry") ) { colorimetry= gst_structure_get_string(structure,"colorimetry") } if ( gst_structure_has_field(structure, "mastering-display-metadata") ) { masteringDisplayMetadata= gst_structure_get_string(structure,"mastering-display-metadata"); } if ( gst_structure_has_field(structure, "content-light-level") ) { contentLightLevel= gst_structure_get_string(structure,"content-light-level"); }
Video Sink Interface
The application may interact with the video sink using properties and signals.
Properties
Properties | Read/Write | Description |
---|---|---|
rectangle | Write Only | Sets the video size and position in application display coordinates where (0,0) is the top left of the application display and (width, height) is the bottom right of the application display. The application must perform alpha hole punching to reveal video beneath graphics.g_object_set( videoSink, “rectangle”, “100,100,640,360”, NULL); |
force-aspect-ratio | Read/Write | When enabled, scaling will respect original aspect ratio. Valid values are true of false . |
async | Read/Write | Go asynchronously to PAUSED . Valid values are true of false . |
Signals
Signals | Description |
---|---|
first-video-frame-callback | Signals the display of the first video frame |
Audio Contract
Applications play audio using GStreamer. Audio pipelines are created using playbin and rely on auto-plugging. Applications must not create specific elements. Applications may specify appsrc as the pipeline source element or set the uri
property of playbin.
Audio Sink
The audio sink element is selected by playbin via auto-plugging. The application may obtain the video sink element selected by playbin for the pipeline using playbin’s audio-sink
property:
GstElement *audioSink; g_object_get( playbin, “audio-sink”, &audioSink, NULL);
Audio Sink Contract
Audio Sink Interface
The application may interact with the audio sink using properties and signals.
Properties
Properties | Read/Write | Description |
---|---|---|
mute | Read/Write | Sets whether audio is muted or unmuted. Valid values are true or false . |
volume | Read/Write | Sets the volume level. Valid range is 0.0 to 1.0. |
async | Read/Write | Go asynchronously to PAUSED . Valid values are true or false . |
Signals
Signals | Description |
---|---|
first-audio-frame-callback | Signals the rendering of the first audio frame |