DVR prune episodes python script

Want to write your own code to work with a HDHomeRun or work with the HDHomeRun DVR? We are happy to help with concepts, APIs, best practices.
Post Reply
bwsd19
Posts: 6
Joined: Fri Dec 18, 2020 5:55 am

DVR prune episodes python script

Post by bwsd19 »

The only feature I miss on the Quatro compared to my DirecTV DVR is the ability to control the number of episodes recorded. I wrote this python script to handle that until the feature is native in the SD device. I have the script run once a day in a cron job on a Raspberry Pi using python3. I am not a python programmer. I coded in C a few decades ago (and put two spaces after the period at the end of a sentence) and that, plus Google, makes me basically a python script kiddie. I had to Google anything pythonish that varies from C, which is a lot, and in particular list sorting and GET/POST methods. So, for anyone who knows python, apologies in advance.

USAGE:
The key script variables you'll want to edit are:
SDIP - the IP address of your SiliconDust device that will be queried with GET and POST commands
MaxEpisodesN - maximum number of episodes to save for a show; older ones are deleted
prunelist - The list of shows you want to prune on your DVR. These are cut/paste from show titles provided by querying the device's StorageURL.

If prunelist is empty, [], then all recordings on the SD device will be pruned.
If you just want to see what shows are on your device, set prunelist to an empty string ['']. It will query the device, give you the titles of shows currently on it, and you can cut/paste these as desired into prunelist in the script to prune some shows and not others.
If you want to prune specific shows, put them in prunelist exactly as they appear cut/paste from running with prunelist = [''] (that's two apostrophes, not a single quote) per the example in the code. The list can be as long or short as you like. If you specify shows in prunelist, shows on your SD device not listed in it will be ignored by the pruning. Alternately, you can put all your shows in and just alter the title, e.g., by putting a leading 'x' in it, to keep it from being pruned as a crude way of toggling pruning on or off for a show.

The script uses print() to show what it's doing and its output is verbose so you can see what it does. I have my cron job send that to a log file so I can see what its doing. You can also run it from a command line. It's been working well since bug noted on 12/22 was fixed that day; I just didn't get around to reposting until 1/22.

The pruning algorithm does two things:

First it looks to see if there is more than 1 show recorded. If there is, it will prune the show's episodes so that there is only 1 appearing per day. I have a news program that records under the same title three times a day, so this will leave me with just the most recent episode recorded on any given day.

The next thing it does, after deleting single day duplicates, is eliminate all but the MaxEpisodesN most recent episodes for each show from the DVR.

That's it. There's lots of functionality I may work on next, such as controlling episodes per day and total maximum episodes per show rather than for all shows, but it will be built off of this base.

I think a lot of people are wishing there were a way to specify this in the SD interface directly when setting up recording shows as you can do with other DVRs, e.g., my DirecTV, but until it is available, this is a kludge to use in the meantime.

EDIT 12/22/20 - there is a bug. Number of episodes isn't updated after same day episodes are removed. This causes too many episodes to be pruned if the initial count of episodes, including daily duplicates, was over the number to prune, even if the number of episodes remaining on different days was less than the number to prune after daily duplicates were removed. Fix is simple. Testing now and will replace code below when tested.

EDIT 1/20/21 - bug fixed; updated code inserted below.

Code: Select all

# V1.0 20201221 - original
# V1.1 20201222 - Fixed for not updating episode count after calling PruneToOnePerDay() when that call removed some episodes.  This
#                 subsequently resulted in the episode count being too high and thus removing too many eiposdes when PruneToNTotal()
#                 was called.
# V1.1.1 20201228 - changed program list so some are not pruned by placing
#                   an 'x' in front of program name.  Moved the definition of prunelist that is non-destructive to last place so it is default if
#                   script is run without modification by an eager implementer.

import urllib.request
import urllib.parse
import json
import time
import datetime
from operator import itemgetter

def postURL(url):
# for SiliconDust device, post of URL with no data is required for success
#    url = urllib.parse.quote(url,encoding='ascii')
    url = str(url)
    parameters = b""
    print('postURL: ' + url)
    
    operURL = urllib.request.urlopen(url, data=parameters)
    if(operURL.getcode()==200):
        return 0
    else:
        print('POST url Error',operURL.getcode())
        print('failed url:' + cmd)
        print(operUrl)
        return -1
    return 0

def getURL(url):
#    print('getURL: ' + str(url))
    operUrl = urllib.request.urlopen(url)
    if(operUrl.getcode()==200):
        data = operUrl.read()
#        print(data)
#        print('==================')
        jsonData = json.loads(data.decode('utf-8'))
    else:
        print("Error retrieving data",operUrl.getcode())
    return jsonData

def PruneToOnePerDay(RecordedEpisodes):
    print(' PruneToOnePerDay')
#    print(RecordedEpisodes)
    safexit = 100
    RecordedEpisodesN = len(RecordedEpisodes)

# sort the episodes so they appear most recent to least recent
    sorted_list = sorted(RecordedEpisodes, key=itemgetter('RecordStartTime'), reverse=True)
    episode_keep_day = "1970-01-01"
    for i in range(0, RecordedEpisodesN):
# this method has problems with time zone and putting different day recordings on same day
#        episode_day = int(RecordedEpisodes[i]["RecordStartTime"]/86400 - utcOffsetSec)
# fromtimestamp() only works thorugh 2038?
        episode_day = datetime.datetime.fromtimestamp(RecordedEpisodes[i]["RecordStartTime"])
        episode_day = str(episode_day)
        episode_day = episode_day[0:10]
        if episode_keep_day != episode_day:
            episode_keep_day = episode_day
#            print("  keep day [" + str(episode_day) + "]: " + str(RecordedEpisodes[i]["RecordStartTime"])
#              + " -> "
#              + str(sorted_list[i]["RecordStartTime"])
#              + " : "
#              + str(sorted_list[i]["CmdURL"]))
            print("  keep day [" + str(episode_day) + "]")
        else:
#            print("  delete day [" + str(episode_day) + "]: " + str(RecordedEpisodes[i]["RecordStartTime"])
#              + " -> "
#              + str(sorted_list[i]["RecordStartTime"])
#              + " : "
#              + str(sorted_list[i]["CmdURL"]))
            print("  delete day [" + str(episode_day) + "]")

            DelCmdURL = sorted_list[i]["CmdURL"] + '&cmd=delete'
            postURL(DelCmdURL)

    return 0

#keep newest N episodes and remove the rest.
def PruneToNTotal(N,RecordedEpisodes):
    print(' PruneToNTotal')
    RecordedEpisodesN = len(RecordedEpisodes)
    
    if RecordedEpisodesN <= N:
        return 0
    else:
#create a list sorted from newest to oldest
        sorted_list = sorted(RecordedEpisodes, key=itemgetter('RecordStartTime'), reverse=True)
#delete all excess older shows
        for i in range(N,RecordedEpisodesN):
            episode_day = datetime.datetime.fromtimestamp(RecordedEpisodes[i]["RecordStartTime"])
            episode_day = str(episode_day)
            episode_day = episode_day[0:10]
            print("   delete day [" + str(episode_day) + "]: " + str(RecordedEpisodes[i]["RecordStartTime"])
              + " -> "
              + str(sorted_list[i]["RecordStartTime"])
              + " : "
              + str(sorted_list[i]["CmdURL"]))
            RecordedEpisodeCmdURL = RecordedEpisodes[i]["CmdURL"]
            DelCmdURL = RecordedEpisodeCmdURL + '&cmd=delete'
            postURL(DelCmdURL)
    return 0
    
def main():
#configuration variables

#IP address of device
    SDIP = "x.x.x.x"
    
# hard coded prune to N episodes (after one-per-day pruning)
    MaxEpisodesN = 5

# pick the prunelist definition below that meets your needs and turn off anything you don't want that appears after it (delete or comment out).  Last one present wins.

# to prune all existing recordings:
    prunelist = []

# To prune list of shows:
    prunelist = [
        'x60 Minutes',
        'CBS News Sunday Morning',
        'Channel 2 News',
        'NBC Nightly News With Lester Holt',
        'PBS NewsHour',
        'Survivor',
        'The Amazing Race']

# To see shows on your SD device listed in output but not prune anything:
    prunelist = ['']

# ---------------------- begin code
# query DVR for its base information
    urlData = "http://"+SDIP+":80/discover.json"
    DiscoverData = getURL(urlData)
#    print(DiscoverData)
#    print('-------------------')
# calculate free space

#    print(DiscoverData["FreeSpace"])
#    print(DiscoverData["TotalSpace"])

    pctfree = 100*DiscoverData["FreeSpace"]/DiscoverData["TotalSpace"]
    print("<=========================================================================>")
    print("SD DVR prune on [" + SDIP + "] at " + str(datetime.datetime.now()))
    print('DVR free space: %.1f' % pctfree + '%')
    print("------")

#get base storage URL
    StorageURL = DiscoverData["StorageURL"]
    RecordedShows = getURL(StorageURL)
    
#print recorded shows (not episodes)
    RecordedShowsN = len(RecordedShows)
    if RecordedShowsN == 0:
        print ("No recorded shows found.")
        return 0
    print("Recorded shows N=" + str(RecordedShowsN) + ":")
    for i in range(RecordedShowsN):
        print(RecordedShows[i]["Title"])
              
    prunelistN = len(prunelist)
    print("------")
# if prunelist is a single empty string, we just want to see recordes shows so exit
    if prunelistN == 1 and prunelist[0]=='':
        return 0

# if prunelist is an empty list then populated it with all recorded shows
# so all are pruned    
    if prunelistN == 0:
            for i in range(len(RecordedShows)):
                prunelist.append(RecordedShows[i]["Title"])
 
# show the prunelist
    prunelistN = len(prunelist)
    print("Prune shows N=" + str(prunelistN) + ":")
    for i in range(len(prunelist)):
        print(prunelist[i])
    print("======")

# begin pruning
    print("Pruning to " + str(MaxEpisodesN) + " episodes max")
    for i in range(len(prunelist)):
        print('prune chk: ' + prunelist[i])
        match = False
        for j in range(len(RecordedShows)):
            if (RecordedShows[j]["Title"] == prunelist[i]):
                match = True
                EpisodesURL = RecordedShows[j]["EpisodesURL"]
                RecordedEpisodes = getURL(EpisodesURL)
                RecordedEpisodesN = len(RecordedEpisodes)
                print(' ' + str(RecordedEpisodesN) + ' episodes found')
# Send to one-per-day prune
                if(RecordedEpisodesN > 1):
                    retval = PruneToOnePerDay(RecordedEpisodes)
# Get new eipsode count as some may have been deleted
                    RecordedEpisodes = getURL(EpisodesURL)
                    RecordedEpisodesN = len(RecordedEpisodes)

                    if(RecordedEpisodesN > MaxEpisodesN):
# Send to max episodes prune
                        print (' ' + str(RecordedEpisodesN) + ' episodes remain')
                        retval = PruneToNTotal(MaxEpisodesN,RecordedEpisodes)
        if match == False:
            print(" 0 episodes found")
        

if __name__ == '__main__':
    main()
Last edited by bwsd19 on Wed Jan 20, 2021 3:00 pm, edited 6 times in total.

titantractor271
Posts: 45
Joined: Mon Jan 18, 2021 9:27 am

Re: DVR prune episodes python script

Post by titantractor271 »

Outstanding. Thank you for the script. It worked for me!

Note: I already built it into a BAT script and scheduled it with a log so I can diagnose problems. The above program is running a little after midnight for me daily. My DVR space got really trim and the system is working the way I've been using a DVR for years.

bwsd19
Posts: 6
Joined: Fri Dec 18, 2020 5:55 am

Re: DVR prune episodes python script

Post by bwsd19 »

Thanks. You reminded me I hadn't posted the bug fix. Glad it works for you.

titantractor271
Posts: 45
Joined: Mon Jan 18, 2021 9:27 am

Re: DVR prune episodes python script

Post by titantractor271 »

bwsd19 wrote: Wed Jan 20, 2021 2:41 pm Thanks. You reminded me I hadn't posted the bug fix. Glad it works for you.
I works really well. I'll test this one - thank you.

Note: An enhancement might be to auto prune the oldest shows in the DVR as space is required to always keep 3 or X hours of recording space available. Also to have a list which says don't touch that for certain shows. I may experiment with that if that need arises.

titantractor271
Posts: 45
Joined: Mon Jan 18, 2021 9:27 am

Re: DVR prune episodes python script

Post by titantractor271 »

It works!

My batch file: (so I record date and time of run and keep the log the program makes)
echo "starting HDHomerun purge at %myDateTime%" > purge.txt
cmd /c "C:\Users\backg\AppData\Local\Programs\Python\Python39\python" purge.py >> purge.txt


My Task Scheduler Action: (My action, the trigger is daily)
"C:\Users\Google Drive\tasks\purge.bat"

I'll see a nightly report on Google Drive :)

titantractor271
Posts: 45
Joined: Mon Jan 18, 2021 9:27 am

Re: DVR prune episodes python script

Post by titantractor271 »

I thought I found a bug, but in reality... I have to run the purge code it seems against both units now that I have two Homerun units. I am not sure if the code is going to work right with two units... I am not sure if the algorithm will work with two units... I need to assess this... I've setup the code to keep one episode and run that on both units. It does have strange results that way. It may work...

bwsd19
Posts: 6
Joined: Fri Dec 18, 2020 5:55 am

Re: DVR prune episodes python script

Post by bwsd19 »

Easiest thing may be to just duplicate the script and configure a different script for each device. That's the no code approach. But, if you like to code, not the "elegant" approach. Have fun.

titantractor271
Posts: 45
Joined: Mon Jan 18, 2021 9:27 am

Re: DVR prune episodes python script

Post by titantractor271 »

bwsd19 wrote: Mon Feb 01, 2021 9:16 am Easiest thing may be to just duplicate the script and configure a different script for each device. That's the no code approach. But, if you like to code, not the "elegant" approach. Have fun.
Yes, that is what I did and it's working... except it's looking quirky.

I may take a crack at coding some of the features I'd like. Thinking I'd want an episode keep count individually for shows. Example would be all the news would be one day, but perhaps I am watching a series and only want the last five of that show. As I mentioned when I get to the maximum space, I may want to start purging the oldest files. Then I may want to have a flag to skip that file from the oldest files list... like "Keep until I delete". All of these features I had on the XYZ system I was using before, so I can't claim to be creative... just use to features I don't have....

Post Reply