Customize Consent Preferences

We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.

The cookies that are categorized as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... 

Always Active

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

No cookies to display.

Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

No cookies to display.

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

No cookies to display.

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

No cookies to display.

Advertisement cookies are used to provide visitors with customized advertisements based on the pages you visited previously and to analyze the effectiveness of the ad campaigns.

No cookies to display.

Log Upload

Created on December 8, 2023


Introduction

Logging and Log Upload are critical aspects when considering large-scale RDK deployment. CPEs have a space constraint to store device logs for future use so either we store them for a limited number of days in CPE or in cloud servers.

Logs play a crucial role in RDK deployments by providing data for deriving field matrices and assessing the criticality of situations, such as determining whether an issue needs immediate attention or can wait until the next day. Controlled log uploads, ranging from 100% to very small numbers (2-3%) based on device maturity, help manage the volume of data. Configurable upload intervals, typically set to once a day, and the right level of abstraction ensure a balance between available storage space and the logs generated. Additionally, rotate-backup-upload policies are implemented to maintain efficient log management. Log Upload uses XConf as the primary endpoint, allowing the configuration to enable or disable log upload for a device or group of devices. During bootup, the CPE retrieves configuration information from XConf, and based on rules defined in the XConf server, it can control whether a particular device needs to perform log uploads. This setup allows for the sampling of devices from a larger population. The configuration server is responsible for retrieving and delivering settings to any client, and it supports configurable upload methods such as HTTPS, HTTP, and TFTP.


Architecture

RDK Log Upload Mechanism

Log_Upload

Log uploader scripts are responsible for gathering local logs, packaging them, and uploading them to a remote URL. A client-side scheduler would control when it executes, but it could also be activated via immediate triggers from the log upload checker or other local actions.

The Uploader is responsible for sending CPE log files to a remote file store. It performs the following:

  • Responds to triggers (i.e. from cron)
  • Checks local Device Settings for the URL of the file store, the logs it should send, and the repository it should use
  • Gets connection credentials
  • Makes a connection to the remote server and sends the files

How CPEs Upload Logs and Metrics

CPEs check with XConf for their logging and telemetry configuration through a query. Part of their query string includes several (about four) parameters of this particular CPE, such as the MAC address, model, current firmware version, and environment. The XConf server looks at these parameters and runs them through a rules engine that finds the closest targeting rule match to the CPE’s profile. Once it finds a match, it retrieves the configuration for that rule. The configuration includes things like an upload schedule, repository URL, upload protocol, path to the file server, protocol indicator, upload time smear, and more. Once the CPE gets the response, it parses it and sets the specified random schedule that delays uploading the files. At that time, the CPE packages up all of its log files, compresses the bundle and uploads the archive to the destination.

The DCM service, which can be checked using the command $systemctl status dcm-log, is responsible for log upload in CPE. This service runs as a system service and invokes /lib/rdk/StartDCM.sh on start-up. The StartDCM.sh script internally calls /lib/rdk/DCMscript.sh for log upload. Once the logs are prepared, the script /lib/rdk/uploadSTBLogs.sh takes over the actual uploading task. To monitor the log upload process, you can use the command $tail -f /opt/logs/dcmscript.og.


Configuration Details

⁠To know how XConf server-side configuration and feature validation are done please refer to XConf Server – User guide for configuration and feature validation .