Fluent Bit Configuration Guide on Ubuntu for Efficient Log Forwarding

In software reliability engineering and system administration, efficient log management is not just a practice but a necessity. It aids in system monitoring, troubleshooting, and resource optimization. For Site Reliability Engineers (SREs) and engineers tasked with ensuring system reliability and efficiency, Fluent Bit represents a significant ally. This guide provides a comprehensive walkthrough for setting up Fluent Bit on Ubuntu for efficient log forwarding. It also covers how to configure Fluent Bit to forward logs to your chosen destination, such as OpenObserve , and how to verify that your logs are being processed.

Introduction to Fluent Bit

Before we dive into the setup process, let's familiarize ourselves with Fluent Bit. Fluent Bit is an open-source and multi-platform Log Processor and Forwarder. It's designed to collect data/logs from different sources, unify them, and send them to multiple destinations. It shines in Docker and Kubernetes environments but is equally capable on bare metal instances and virtual machines. It collects logs from various sources like Linux kernel, application logs, file logs, and logs from the standard input.

Fluent Bit operates as a robust pipeline for handling log data, involving stages where logs are collected, parsed (transformed from unstructured to structured data), filtered (enriched, excluded, or modified), buffered (temporarily stored), and finally routed to various destinations for analysis and storage. This modular approach makes Fluent Bit a versatile tool in log management.

Preparing for Fluent Bit Setup

Before you install Fluent Bit on Ubuntu, make sure you meet the following prerequisites:

• Ubuntu System: Fluent Bit supports a wide range of Ubuntu versions. Verify that your system version is compatible.
• Server GPG Key: Necessary for verifying the integrity and authenticity of the Fluent Bit package.
• APT Repository Configuration: You must configure your APT sources to include Fluent Bit's repository for easy installation and updates.
• Superuser Privileges: Installation and configuration steps need to be executed with root or sudo privileges to modify system settings and files.

Setting Up Fluent Bit on Ubuntu

Here are the steps to get Fluent Bit up and running on your Ubuntu system:

For the comprehensive instructions on installing Fluent Bit on Ubuntu, refer to the official documentation.

1. Setup Server GPG Key

To ensure the authenticity of the Fluent Bit package, configure the server GPG key with the following command:

sudo sh -c 'curl https://packages.fluentbit.io/fluentbit.key | gpg --dearmor > /usr/share/keyrings/fluentbit-keyring.gpg'        

2. Export Your Ubuntu Release Name

Identify your Ubuntu release name to configure the APT repository correctly. There are a couple of ways to find out the release name of your Ubuntu system. Here's how you can do it:

Using the lsb_release Command

lsb_release -a        

Example output:

No LSB modules are available.
Distributor ID: Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:    	22.04
Codename:   	jammy        

Or?

Using the /etc/os-release File

cat /etc/os-release        

Example output:

ubuntu@ip-172-31-6-194:~$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy        

After identifying the release name, export the release name (e.g., jammy for Ubuntu 22.04) as an environment variable:

export CODENAME="jammy"        

3. Update Your Sources Lists

Add the Fluent Bit APT repository to your system:

echo "deb [signed-by=/usr/share/keyrings/fluentbit-keyring.gpg] https://packages.fluentbit.io/ubuntu/$CODENAME/ $CODENAME main" | sudo tee /etc/apt/sources.list.d/fluentbit.list        

4. Install Fluent Bit

Update your package lists and install Fluent Bit:

sudo apt-get update
sudo apt-get install fluent-bit        

5. Enable Fluent Bit Service with Systemd

To ensure Fluent Bit starts automatically at server boot, enable its service using systemd.

a. To start the Fluent Bit service, execute the following command:

sudo systemctl start fluent-bit        

b. To verify that the Fluent Bit service is running as expected, execute the status command:

systemctl status fluent-bit        

Example output:

● fluent-bit.service - Fluent Bit
 	Loaded: loaded (/lib/systemd/system/fluent-bit.service; disabled; vendor preset: enabled)
 	Active: active (running) since Tue 2023-07-25 00:19:53 UTC; 20s ago
   	Docs: https://docs.fluentbit.io/manual/
   Main PID: 2755 (fluent-bit)
  	Tasks: 4 (limit: 2307)
 	Memory: 3.0M
    	CPU: 33ms
 	CGroup: /system.slice/fluent-bit.service
         	└─2755 /opt/fluent-bit/bin/fluent-bit -c //etc/fluent-bit/fluent-bit.conf        

c. To exit the status view, enter q. This command quits the status screen and returns you to the command prompt.

FYI: By default, Fluent Bit is configured to gather CPU usage metrics and sends these records to the standard output found in /var/log/syslog. To review the data, check the syslog file.

6. Configuring Fluent Bit to Forward Logs to OpenObserve

Configuring Fluent Bit for log forwarding involves defining input and output plugins within its configuration file. The input plugin specifies the log source, while the output plugin determines the destination where logs will be forwarded. This setup enables Fluent Bit to collect logs from various sources and send them to a centralized log management system, like OpenObserve, for analysis and storage.

Forwarding Logs to OpenObserve

Follow the steps below to forward logs to OpenObserve:

a. Default Configuration File Location

The default configuration file for Fluent Bit is located at:

/etc/fluent-bit/fluent-bit.conf        

Edit this file, use the vi editor by running the following command:

sudo vi /etc/fluent-bit/fluent-bit.conf        

b. Configuring Log Input

Fluent Bit can tail log files from a specified path. Configure it to read logs stored in the /var/log directory, and add the following configuration under the INPUT section:

[INPUT]
	Name          	tail
	Path          	/var/log/*.log
	Parser        	docker
	Tag           	logs.*
	Refresh_Interval  5
	Rotate_Wait   	5
	Mem_Buf_Limit 	5MB
	Skip_Long_Lines   On        

c. Configuring Log Output

To forward logs to OpenObserve, you'll need to copy the output section configuration from the OpenObserve UI. An example configuration might look like this:

[OUTPUT]
	Name http
	Match *
	URI /api/justprabhat_organization_1010/default/_json
	Host api.openobserve.ai
	Port 443
	tls On
	Format json
	Json_date_key	_timestamp
	Json_date_format iso8601
	HTTP_User [email protected]
	HTTP_Passwd qatj1wfZ890Q5Sk123k1
	compress gzip        

Replace the URI, HTTP_User, and HTTP_Passwd with the details provided by OpenObserve.

d. Restarting Fluent Bit Service

After configuring the input and output, restart the Fluent Bit service to apply the changes:

sudo systemctl restart fluent-bit        

Verifying the Service Status

To ensure Fluent Bit is running without issues, check the status of the service:

systemctl status fluent-bit        

If everything is configured correctly, you should see the service status as active (running). Example output might look like this:

● fluent-bit.service - Fluent Bit
 	Loaded: loaded (/lib/systemd/system/fluent-bit.service; disabled; vendor preset: enabled)
 	Active: active (running) since Tue 2023-07-25 01:28:14 UTC; 38s ago
   	Docs: https://docs.fluentbit.io/manual/
   Main PID: 2926 (fluent-bit)
  	Tasks: 6 (limit: 2307)
 	Memory: 6.0M
    	CPU: 95ms
 	CGroup: /system.slice/fluent-bit.service
         	└─2926 /opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf        

Troubleshooting

If you encounter any errors, you can check the logs of the Fluent Bit service using the following command:

journalctl -u fluent-bit.service        

Check the Logs in OpenObserve

To verify that your Fluent Bit configuration is successfully forwarding logs to OpenObserve, navigate to the OpenObserve User Interface (UI). Here, you can explore and analyze the logs that have been forwarded.?

Fluent Bit – An Essential Tool for Kubernetes Logging

Fluent Bit is a pivotal tool in the Kubernetes ecosystem due to its lightweight footprint and powerful log processing capabilities. It's perfect for capturing logs from Kubernetes pods and nodes. Then it forwards them to various destinations for analysis and monitoring. Here are a few tips for leveraging Fluent Bit in Kubernetes:

• DaemonSet Deployment: Deploy Fluent Bit as a DaemonSet within your Kubernetes cluster. This ensures an instance of Fluent Bit is running on each node, enabling comprehensive log collection from all pods.
• Custom Parsing: Utilize Fluent Bit's custom parsing features to interpret the diverse log formats found in Kubernetes environments. This aids in transforming unstructured log data into a structured format, enhancing log analysis.
• Filtering and Tagging: Use filters and tagging to manage logs. Filtering allows you to modify or enrich log messages before forwarding. Tagging helps route logs to the appropriate output.

FluentD vs Fluent Bit: Understanding the Differences

FluentD and Fluent Bit are both part of the Fluent ecosystem and share similar goals. However, key differences influence their use cases.

• Performance: Fluent Bit is designed with performance in mind. It boasts a smaller footprint and higher efficiency, especially in CPU and memory usage. This makes it ideal for edge devices, low-resource environments, and as a log forwarder in more extensive systems.
• Use Case: FluentD is more feature-rich, offering a broader plugin ecosystem and more extensive capabilities. This makes it suitable for complex log processing pipelines where flexibility and plugin availability are crucial.
• Integration: Fluent Bit offers better native integration with Kubernetes. This makes it the preferred choice for Kubernetes log management. In a FluentD-Fluent Bit hybrid architecture, it can serve as a log aggregator. Fluent Bit handles log collection and forwarding, while FluentD performs more complex processing.

Conclusion

Throughout this guide, we've explored the significance of Fluent Bit as a pivotal tool in log management, particularly for those managing Ubuntu systems. From the basics and installation steps to configuring Fluent Bit for log forwarding and verifying that logs are correctly being forwarded by checking them in OpenObserve, this guide aims to provide a comprehensive overview to empower SREs and engineers.

With the knowledge and steps outlined, you're well-equipped to implement Fluent Bit on your Ubuntu systems, leading to more efficient log management, improved system monitoring, and better troubleshooting capabilities.

Remember, efficient log management is a cornerstone of system reliability and observability. Fluent Bit provides a powerful yet manageable solution to harness the full potential of your logging infrastructure.

Get Started Now

Don't let the complexities of log management hinder your system's potential. Implement the steps outlined in this guide to enhance your log management process. Dive into the world of efficient log forwarding with Fluent Bit on Ubuntu and discover how OpenObserve can transform your log management strategy. Start today and witness the impact of optimized logging on your systems.