hdhr_disk_space_monitor - Automated Disk Space and Recording Management

Help and support with third-party TV software.
Windows Media Center, J River Media Center, Next PVR, MythTV, InstaTV, etc.
Post Reply
youzernaym
Posts: 14
Joined: Sat Jun 08, 2019 9:18 pm

hdhr_disk_space_monitor - Automated Disk Space and Recording Management

Post by youzernaym »

This is a continuation of the Maintaining Hard Drive Free Space thread. Version 2.0.0 is a big change and deserves a new thread.

hdhr_disk_space_monitor is a Python program that can be easily downloaded and installed using regular Python installation tools. It is hosted on GitHub, where you can find:
I hope others can make use of it.

Description/Purpose
Monitor disk space utilization of HDHomeRun SCRIBE, SERVIO, and RECORD devices. Optionally delete recordings to stay above a specified free space minimum, get rid of recordings older than a maximum age, or keep only a certain number of episodes.

Use Cases
  1. Disk Space Reporting - "I want to see how much space is being used by my storage devices."

    If not told to do anything else, the monitor will simply report disk space utilization every 10 minutes for all HDHomeRun storage devices found on the network.

    Code: Select all

    $ hdhr_monitor_disk_space
    2020-06-19 22:08:51,967 [HDHomeRun SCRIBE QUATRO 12345678] Total: 999.71 GB; Used: 402.27 GB (40.2%); Free: 597.45 GB (59.8%)
    2020-06-19 22:08:51,983 [HDHomeRun RECORD 192.168.1.100] Total: 1.07 TB; Used: 218.65 GB (20.4%); Free: 854.56 GB (79.6%)
    
  2. Disk Space Maintenance - "I want to make sure that my storage devices do not fill up."

    Tell the monitor the amount of free space to maintain, either by percentage (-p/--percent-free) or absolute gigabytes (-g/--gigabytes-free), and it will delete the oldest recording (configurable) when that amount is no longer free.

    Code: Select all

    $ hdhr_monitor_disk_space --gigabytes-free 10
    2020-06-19 22:08:51,967 [HDHomeRun SCRIBE QUATRO 12345678] Total: 999.71 GB; Used: 402.27 GB (40.2%); Free: 597.45 GB (59.8%); Minimum Free: 10.00 GB (1.0%)
    2020-06-19 22:08:51,983 [HDHomeRun RECORD 192.168.1.100] Total: 1.07 TB; Used: 218.65 GB (20.4%); Free: 854.56 GB (79.6%); Minimum Free: 10.00 GB (0.9%)
    
    Whenever a recording is deleted, it is reported, along with the reason.

    Code: Select all

    2020-05-01 23:53:50,637 [HDHomeRun SCRIBE QUATRO 12345678] Deleting "Keeping Up Appearances" recorded Sun Jul 28 22:30:00 2019 to free space
    
  3. Disk Space Maintenance - "I want to make sure that my storage devices do not fill up, but I want to use different settings for each device."

    The configuration file can be used for this. If one device needs to maintain 10GB free, and the other needs to maintain 25% free, there are a few options.
    1. Set 10GB as the default and override one device with 25%.

      Code: Select all

      [DEFAULT]
      gigabytes_free: 10
      
      [device:192.168.1.100]
      # Override the default
      gigabytes_free:
      percent_free: 25
      
    2. Configure each device individually.

      Code: Select all

      [device:12345678]
      gigabytes_free: 10
      
      [device:192.168.1.100]
      percent_free: 25
      
      In either case above, if the configuration file is named /etc/hdhr_disk_space_monitor.conf, then the following would get it started:

      Code: Select all

      $ hdhr_monitor_disk_space --conf-file /etc/hdhr_disk_space_monitor.conf
      2020-06-19 22:08:51,967 [HDHomeRun SCRIBE QUATRO 12345678] Total: 999.71 GB; Used: 402.27 GB (40.2%); Free: 597.45 GB (59.8%); Minimum Free: 10.00 GB (1.0%)
      2020-06-19 22:08:51,983 [HDHomeRun RECORD 192.168.1.100] Total: 1.07 TB; Used: 218.65 GB (20.4%); Free: 854.56 GB (79.6%); Minimum Free: 267.50 GB (25.0%)
      
    3. Independent processes without configuration file.

      Code: Select all

      $ hdhr_monitor_disk_space --device-id 12345678 --gigabytes-free 10
      2020-06-19 22:08:51,967 [HDHomeRun SCRIBE QUATRO 12345678] Total: 999.71 GB; Used: 402.27 GB (40.2%); Free: 597.45 GB (59.8%); Minimum Free: 10.00 GB (1.0%)
      

      Code: Select all

      $ hdhr_monitor_disk_space --device-id 192.168.1.100 --percent-free 25
      2020-06-19 22:08:51,983 [HDHomeRun RECORD 192.168.1.100] Total: 1.07 TB; Used: 218.65 GB (20.4%); Free: 854.56 GB (79.6%); Minimum Free: 267.50 GB (25.0%)
      
  4. Recording Maintenance - "I want 'news' category recordings to be deleted after 2 days, 'sport' category recordings to be protected from any automatic deletion, and no more than 5 episodes of 'The Masked Singer'.

    Recording maintenance is configured entirely in the configuration file. See the example configuration file for a description of all options. The recordings are maintained as a whole across all storage devices, so it's best not to split this across multiple processes as in Option 3 above.

    Code: Select all

    [category:news]
    max_age_days: 2
    
    [category:sport]
    protected: yes
    
    [series:The Masked Singer]
    max_episodes: 5
    
Much more information is available in the links above. Please have a look there, if interested.

youzernaym
Posts: 14
Joined: Sat Jun 08, 2019 9:18 pm

Re: hdhr_disk_space_monitor - Automated Disk Space and Recording Management

Post by youzernaym »

A word of caution for RECORD users who have multiple search paths configured as documented here: Don't use the free space maintenance part of hdhr_disk_space_monitor if those paths are on different file systems. Here's why...

Imagine the following configured in hdhomerun.conf:

Code: Select all

RecordPath=/var/lib/hdhomerun/path_1;/var/lib/hdhomerun/path_2
New recordings go into path_1, but both path_1 and path_2 can have recordings in them.

The API reports total and free space for the RECORD instance as only that of path_1. If the mount point is /var/lib/hdhomerun or higher, then there's no problem because those figures will represent the space for both paths. But if path_1 and path_2 are individual mount points, there is a problem.

The API doesn't indicate which path any given recording is stored on. So if the file system mounted on path_1 fills up beyond the specified threshold, it's possible that the recording deleted to free up space will be on path_2. In a worst-case scenario, all recordings on path_2 will be deleted trying to free space on path_1.

To continue to use hdhr_disk_space_monitor but disable free space maintenance on a specific RECORD instance, put the following in the config file, assuming the RECORD is running on 192.168.1.100:

Code: Select all

[device:192.168.1.100]
gigabytes_free:
percent_free:

Post Reply