[Solution] Missing logstash-plain.log File in Logstash

Logstash is a vital part of the ELK (Elasticsearch, Logstash, Kibana) stack, responsible for gathering and processing logs before pushing them to Elasticsearch for indexing. However, you may occasionally run into issues where the logstash-plain.log file is missing, which can complicate troubleshooting and monitoring.

In this article, we’ll cover potential causes, steps for diagnosis, and how to fix this missing log file issue.

Possible Reasons for Missing logstash-plain.log

The logstash-plain.log file may be missing due to several reasons, including:

1. Logstash may lack permissions to create or write to the logstash-plain.log file.
2. The logging path may be incorrect in the Logstash configuration file, preventing Logstash from logging.
3. If log rotation has not been configured properly, older logs may be removed without new logs being created.
4. Logstash service may not start or may be disabled, resulting in no log output.

Let’s go through each potential solution to get the log file working again.

Step 1: Check Logstash Service Status

The first step is to verify if the Logstash service is running properly. Use the following command to check the status of the Logstash service:

sudo systemctl status logstash.service

This command provides information about whether the Logstash service is running or if there are any errors. If the service is inactive or failed, there may be an issue with the service startup, which could contribute to the missing log file. Look for error messages in the output that may indicate why the service failed to start.

Step 2: Manually Create the logstash-plain.log File

If the service is active, the log file may simply be missing due to an issue in the log directory structure or permissions. To create the log file manually, follow these steps:

Run the following command to manually create the logstash-plain.log file:

sudo touch /var/log/logstash/logstash-plain.log

Ensure that Logstash has the necessary permissions to write to this file. Run the following commands to set the ownership and permissions:

sudo chown logstash:logstash /var/log/logstash/logstash-plain.log
sudo chmod 755 /var/log/logstash/logstash-plain.log

Here, chmod 755 sets read, write, and execute permissions for the owner and read/execute permissions for others, which should suffice for Logstash.

After creating the file and setting the permissions, restart the Logstash service to initiate logging:

sudo systemctl restart logstash.service

Step 3: Verify Log Path in Logstash Configuration

Logstash’s configuration files typically reside in /etc/logstash. Open the Logstash configuration file (e.g., logstash.yml) to confirm that the logging path is correctly specified.

Open Logstash Configuration:

sudo nano /etc/logstash/logstash.yml

Look for the path.logs parameter, which defines where Logstash writes its log files. Ensure it’s pointing to the correct directory, usually /var/log/logstash.

path.logs: /var/log/logstash

If you made changes, restart the Logstash service to apply the updated configuration.

Step 4: Check Log Rotation Configuration

If the log file is periodically deleted or rotated improperly, inspect the log rotation settings in /etc/logrotate.d/logstash. Adjust the settings to ensure the logs are not deleted before they are re-created.

Once you have taken these steps, verify that `logstash-plain.log` is being populated by:

1. Checking the log file size with ls -lh /var/log/logstash/logstash-plain.log.
2. Tailing the log output with:

tail -f /var/log/logstash/logstash-plain.log

If the file is now receiving log entries, the issue is resolved. Otherwise, re-check service status, permissions, and configuration file settings.

Conclusion

The missing `logstash-plain.log` file can often be fixed with a few permission adjustments, manual file creation, and service restarts. By following these troubleshooting steps, you should be able to restore logging functionality for Logstash and continue monitoring your system’s logs efficiently.

You may also like: