Custom logs play a pivotal role in the Oracle Cloud Infrastructure (OCI) Logging Service, providing insights into various aspects of applications and workloads. Let’s delve into the intricacies of setting up and configuring custom logs within OCI.
Understanding Custom Logs
Custom logs encompass logs generated by bespoke applications or workloads, whether they operate within OCI, on-premises environments, or external cloud providers. It’s essential to configure custom logs comprehensively, even for OCI compute instances, to ensure the collection of application-specific or system-level logs.
Methods of Ingesting Custom Logs
There are two primary methods for ingesting custom logs: utilizing the “Put logs” API or employing the Unified Monitoring Agent, also known as the FluentD agent. Our focus will be on the FluentD agent due to its significance in log ingestion within OCI.
Unified Monitoring Agent: An Overview
The Unified Monitoring Agent, built on CNCF specification version 1, serves as an open-source data collector facilitating a unified logging layer between data sources and backend systems. Deployment of the agent varies depending on the environment.
Enabling the Agent on OCI Compute Instances
For OCI compute instances, the agent can be enabled conveniently through the Oracle Cloud Agent plugin UI. By accessing the Oracle Cloud Agent tab from the instance page, users can enable the Unified Agent or the custom logs monitoring plugin effortlessly.
Manual Installation for External Systems
Alternatively, manual installation of the agent is preferred for external systems, such as workloads in on-premises or different cloud infrastructure environments. While recent OCI instances feature the Oracle Cloud Agent plugin, manual installation may be necessary for specific requirements.
Supported Operating Systems for the Agent
The FluentD agent is compatible with various operating systems, including Oracle Linux, CentOS, Ubuntu, and Windows Server versions, ensuring flexibility in deployment across diverse environments.
Workflow of the Agent
Enabling custom logs entails a two-step process. First, install the agent binary on the desired instances or workloads. Second, configure the agent within the OCI Logging Service, specifying the hosts or instances for log collection, desired log types, and parsers for data interpretation.
Log Parsing for Enhanced Analysis
Log parsing facilitates the breakdown of large log volumes into interpretable data segments. Users can opt for parsers tailored to specific formats like XML, syslog, or JSON, streamlining data analysis.
Defining Log Destinations
Lastly, define the log destination within the OCI Logging Service, specifying the compartment, log group, and log object for storing custom logs.
Managing Agent Services
On compute instances, services such as the Unified Monitoring Agent service and associated services on Linux-based instances oversee the agent’s running state, installation, and updates, ensuring seamless log collection.
Configuring Host Groups
Host groups play a crucial role in specifying hosts for log collection. Dynamic groups, defined through IAM policies, are ideal for OCI compute instances, while User Groups cater to instances outside the OCI environment.
Selecting Log Inputs
The agent configuration allows users to specify the type of logs for ingestion, including Windows event logs and directory paths for log files, with support for advanced parsers like Auditd, JSON, CSV, or syslog.
Wrapping Up
In conclusion, enabling custom logs in OCI involves a structured approach encompassing the deployment of the FluentD-based Unified Monitoring Agent, meticulous configuration of agent services, and definition of log collection parameters. By following these steps, users can harness the power of custom logs to gain valuable insights into their OCI environments.