Recording metadata - TPeterson

Want to write your own code to work with a HDHomeRun or work with the HDHomeRun DVR? We are happy to help with concepts, APIs, best practices.
TPeterson
Posts: 138
Joined: Thu May 31, 2007 8:29 pm
Device ID: 10123716, 10157425, 1039FE2B, 103AEA6C, 1075D4B1, 1076C3A7, 1080F19F
Location: San Carlos, CA
Contact:

Re: Recording metadata - TPeterson

Post by TPeterson »

Thanks for the additional info. I have to apologize for misleading folks about the progress bar's mis tracking. It turns out that is only an issue on the LG TV-hosted app, which is a shame since that's the one that does AC4 audio!

Ken.F
Posts: 2584
Joined: Fri Apr 05, 2013 9:20 am
Device ID: 1041A706, 1043EB32, 104BAD9E, 13168DC5, 1322A7AC
Location: West Rockhill, PA
x 8

Re: Recording metadata - TPeterson

Post by Ken.F »

nickk wrote: Wed Apr 10, 2024 11:16 am Suggest not changing the EndTime as EndTime is the scheduled EndTime and not related to the length of the recording. In practice we only use StartTime in the UI.
Good to know, thanks. My OCD dictates everything must match so I'll keep doing it my way even if it's wrong. ;)

nickk
Silicondust
Posts: 20428
Joined: Tue Jan 13, 2004 9:39 am
x 272

Re: Recording metadata - TPeterson

Post by nickk »

Ken.F wrote: Wed Apr 10, 2024 11:29 am
nickk wrote: Wed Apr 10, 2024 11:16 am Suggest not changing the EndTime as EndTime is the scheduled EndTime and not related to the length of the recording. In practice we only use StartTime in the UI.
Good to know, thanks. My OCD dictates everything must match so I'll keep doing it my way even if it's wrong. ;)
RecordStartTime and RecordEndTime should be different to StartTime and EndTime anyway due to padding.

StartTime and EndTime is the TV guide... if you change the EndTime it will no longer match the TV guide and won't make sense.

If you are editing the length of the file set RecordStartTime to StartTime (the original start padding is no longer relevant) and set the RecordEndTime to StartTime + edited length. Think of the recording as having negative end padding.

Ken.F
Posts: 2584
Joined: Fri Apr 05, 2013 9:20 am
Device ID: 1041A706, 1043EB32, 104BAD9E, 13168DC5, 1322A7AC
Location: West Rockhill, PA
x 8

Re: Recording metadata - TPeterson

Post by Ken.F »

nickk wrote: Wed Apr 10, 2024 11:37 am RecordStartTime and RecordEndTime should be different to StartTime and EndTime anyway due to padding.

StartTime and EndTime is the TV guide... if you change the EndTime it will no longer match the TV guide and won't make sense.

If you are editing the length of the file set RecordStartTime to StartTime (the original start padding is no longer relevant) and set the RecordEndTime to StartTime + edited length. Think of the recording as having negative end padding.
Yes, that is exactly what I do, but I also change the EndTime to match the RecordEndTime since the end padding has been edited out. After the recording has been completed it doesn't matter (to me) if the EndTime no longer matches the TV guide since I can't look back at past programs in the guide.

nickk
Silicondust
Posts: 20428
Joined: Tue Jan 13, 2004 9:39 am
x 272

Re: Recording metadata - TPeterson

Post by nickk »

Ken.F wrote: Wed Apr 10, 2024 11:56 am but I also change the EndTime to match the RecordEndTime since the end padding has been edited out.
Ok, just know that doing so is wrong as per our schema.

Ken.F
Posts: 2584
Joined: Fri Apr 05, 2013 9:20 am
Device ID: 1041A706, 1043EB32, 104BAD9E, 13168DC5, 1322A7AC
Location: West Rockhill, PA
x 8

Re: Recording metadata - TPeterson

Post by Ken.F »

nickk wrote: Wed Apr 10, 2024 1:09 pm Ok, just know that doing so is wrong as per our schema.
Got it.

TPeterson
Posts: 138
Joined: Thu May 31, 2007 8:29 pm
Device ID: 10123716, 10157425, 1039FE2B, 103AEA6C, 1075D4B1, 1076C3A7, 1080F19F
Location: San Carlos, CA
Contact:

Re: Recording metadata - TPeterson

Post by TPeterson »

I see from the logs that the DVR engine creates that it rather frequently "phones home" to find out, among other things, whether I've subscribed to the DVR service. Given that I only intend to use the HDHomerun viewer app to play local recordings, that seems unnecessary O/H for both Silicondust and me. Is there any setting available to mitigate these communications?

Also, I know that your DVR paradigm requires the recording machines to stay powered 24/7, but our method only powers them up for the actual recording, scheduling, and playback sessions with Power Options set to put them to sleep otherwise. I've found however that playback via the DVR engine does not honor the PC's setting "do not sleep during media streaming". Could you change its behavior to honor that setting, please?

rikd
Silicondust
Posts: 48
Joined: Thu Mar 02, 2023 10:48 am
Device ID: 108042A1, 10814D8E
Location: Portugal
x 3

Re: Recording metadata - TPeterson

Post by rikd »

TPeterson wrote: Mon Apr 15, 2024 3:24 pm I see from the logs that the DVR engine creates that it rather frequently "phones home" to find out, among other things, whether I've subscribed to the DVR service. Given that I only intend to use the HDHomerun viewer app to play local recordings, that seems unnecessary O/H for both Silicondust and me. Is there any setting available to mitigate these communications?
The phone home is to ensure the record engine syncs up with the guide, time sync, notifications for recordings, etc..
i.e. if recording of an event is required, the client will request the cloud service to add the scheduled event - and the DVR engine then phones home periodically to get notification of that event. It doesn't require the HDHomerun Viewer App to be the creator of that event - pretty much any authorized client can add scheduled events to the cloud service - thus the DVR engine must stay in sync with the cloud service to get ALL scheduled events when necessary.
Also, I know that your DVR paradigm requires the recording machines to stay powered 24/7, but our method only powers them up for the actual recording, scheduling, and playback sessions with Power Options set to put them to sleep otherwise. I've found however that playback via the DVR engine does not honor the PC's setting "do not sleep during media streaming". Could you change its behavior to honor that setting, please?
I think there may be some confusion on that power management setting.
Do not sleep during media streaming/sharing ensures that when using the GPU for displaying (sharing/streaming) a video to the screen the device will not go to sleep. It has nothing to do with the streaming of a file via IP over your network - that is generally the CPU/storage/network power management settings - and typically device will not sleep if the CPU is active (reading from disk, packetizing and sending over network).

But am a little confused - you say you put the device to sleep via power options, but then are wondering why it does go to sleep?
Are you expecting the device to wake up if someone decides to stream?
If so - that is definitely not the intention of that power management setting - instead you would need some 'wake on lan' setting to wake up the PC once some client tries to attach to the DVR engine.. then assuming the device wakes up in time and services the request then it should do so and will go to sleep once the CPU, storage and network go idle. But be warned - that time to wake can be more considerable than you may think and could result in the API request to the engine timing out.

TPeterson
Posts: 138
Joined: Thu May 31, 2007 8:29 pm
Device ID: 10123716, 10157425, 1039FE2B, 103AEA6C, 1075D4B1, 1076C3A7, 1080F19F
Location: San Carlos, CA
Contact:

Re: Recording metadata - TPeterson

Post by TPeterson »

@rikd, thanks for your reply. Since you're new to this conversation, some background seems to be in order. CW_EPG, my HDHR PVR app (user manual here), has been using HDHR tuners (and others before there were HDHR tuners) for many years. It handles scheduling and recording sessions via Windows waketimers and setting the Windows do-not-sleep flag (ES_SYSTEM_REQUIRED) for the duration of those events. Otherwise, it clears that flag so that the system goes to sleep after the user-designated interval of inactivity. If the system has the "do not go to sleep while media streaming" flag set, the system will stay awake if the usual SMB file requests have been made. This works for streaming using VLC, etc., but does not work; e.g., with LG's DLNA server which somehow is not visible to that watchdog, so the system falls asleep after the Power Options interval has passed.

Silicondust's original recording app hdhomerun_config.exe clearly also used ES_SYSTEM_REQUIRED to stay awake for recordings. However, it appears that the DVR engine, like the LG DLNA server, does not set it when serving recorded media, presumably because it's assumed to be running on a 24/7 host. I'm asking that you change that, please.

Regarding the "phone home" activity, CW_EPG is not going to request EPG data, time sync, or recording schedules from Silicondust's server because its user is not subscribed to the DVR service. It only needs the DVR engine's involvement to provide the HDHR viewer app access to locally recorded files. Therefore, all of the phone-home CPU cycles are wasted motion for both Silicondust and CW_EPG's user.

rikd
Silicondust
Posts: 48
Joined: Thu Mar 02, 2023 10:48 am
Device ID: 108042A1, 10814D8E
Location: Portugal
x 3

Re: Recording metadata - TPeterson

Post by rikd »

TPeterson wrote: Tue Apr 16, 2024 9:09 am @rikd, thanks for your reply. Since you're new to this conversation, some background seems to be in order. CW_EPG, my HDHR PVR app (user manual here), has been using HDHR tuners (and others before there were HDHR tuners) for many years. It handles scheduling and recording sessions via Windows waketimers and setting the Windows do-not-sleep flag (ES_SYSTEM_REQUIRED) for the duration of those events. Otherwise, it clears that flag so that the system goes to sleep after the user-designated interval of inactivity. If the system has the "do not go to sleep while media streaming" flag set, the system will stay awake if the usual SMB file requests have been made. This works for streaming using VLC, etc., but does not work; e.g., with LG's DLNA server which somehow is not visible to that watchdog, so the system falls asleep after the Power Options interval has passed.
Ahhh - the request for ES_SYSTEM_REQUIRED to prevent system sleep makes more sense to me.
Silicondust's original recording app hdhomerun_config.exe clearly also used ES_SYSTEM_REQUIRED to stay awake for recordings. However, it appears that the DVR engine, like the LG DLNA server, does not set it when serving recorded media, presumably because it's assumed to be running on a 24/7 host. I'm asking that you change that, please.
so set this for the thread once the engine spins it up for streaming to the client. Otherwise the system will just drop back to sleep anyways if no active client attaches.
Makes sense.
Regarding the "phone home" activity, CW_EPG is not going to request EPG data, time sync, or recording schedules from Silicondust's server because its user is not subscribed to the DVR service. It only needs the DVR engine's involvement to provide the HDHR viewer app access to locally recorded files. Therefore, all of the phone-home CPU cycles are wasted motion for both Silicondust and CW_EPG's user.
so you really just want a DVR playout engine - not record engine.
i.e. something to parse the metdata on the TS files in a folder structure, provide a recordings list, and stream requested recordings (and stay awake ;)) to the client, all through HDHomeRun APIs to the client.
Makes sense.

TPeterson
Posts: 138
Joined: Thu May 31, 2007 8:29 pm
Device ID: 10123716, 10157425, 1039FE2B, 103AEA6C, 1075D4B1, 1076C3A7, 1080F19F
Location: San Carlos, CA
Contact:

Re: Recording metadata - TPeterson

Post by TPeterson »

rikd wrote: Tue Apr 16, 2024 9:58 amso you really just want a DVR playout engine - not record engine.
i.e. something to parse the metdata on the TS files in a folder structure, provide a recordings list, and stream requested recordings (and stay awake ;)) to the client, all through HDHomeRun APIs to the client.
Makes sense.
Now we're on the same page. :)

Any chance of seeing these changes?

TPeterson
Posts: 138
Joined: Thu May 31, 2007 8:29 pm
Device ID: 10123716, 10157425, 1039FE2B, 103AEA6C, 1075D4B1, 1076C3A7, 1080F19F
Location: San Carlos, CA
Contact:

Re: Recording metadata - TPeterson

Post by TPeterson »

nickk wrote: Sun May 05, 2024 11:52 am
TPeterson wrote: Sun May 05, 2024 11:24 am
nickk wrote: Sun May 05, 2024 7:00 amOpen to feedback - what features/areas do you think there is room to improve on?
I hope that you have this already on the to-do list?
The record engine already supports playout only operation (ie disabling recording) by setting the RecordStreamsMax to 0 in the conf file.

We can't disable internet API handling completely because our guide servers need to know what images to keep around.
I've set RecordStreamsMax to 0 in the record engine host's Registry; however, I still see in its log entries every 10 minutes (when the host is awake) the same notices about checking for subscribed status and new recording events. IIUC, it cannot execute any recordings while RecordStreamsMax is 0, so why does it continue these checks? It seems to me that it should forgo such checks as superfluous until RecordStreamsMax is nonzero. Hmm?

As far as the guide servers are concerned, the record engine with RecordStreamsMax=0 would seem to have no more need for checking those than when it's not running (i.e., when the "use this computer for recordings" checkbox is cleared). What am I missing?

nickk
Silicondust
Posts: 20428
Joined: Tue Jan 13, 2004 9:39 am
x 272

Re: Recording metadata - TPeterson

Post by nickk »

It should not be checking every 10 minutes... that suggests the record engine thinks the sync failed triggering a faster than normal retry.

The sync is required because the guide servers need to know what images to keep around for recordings to have images.

TPeterson
Posts: 138
Joined: Thu May 31, 2007 8:29 pm
Device ID: 10123716, 10157425, 1039FE2B, 103AEA6C, 1075D4B1, 1076C3A7, 1080F19F
Location: San Carlos, CA
Contact:

Re: Recording metadata - TPeterson

Post by TPeterson »

nickk wrote: Mon May 06, 2024 2:43 pm It should not be checking every 10 minutes... that suggests the record engine thinks the sync failed triggering a faster than normal retry.

The sync is required because the guide servers need to know what images to keep around for recordings to have images.
Here's a section of the log showing the 10-minute polling with "recorded sync success" and "no timer events planned" entries. If there are no recording events, there will be no images to save, hmm? Why do I see "event download failed" when there are no events? What am I doing wrong?

Code: Select all

20240506-18:01:08 Status: Resource: nbk=2 dmk=429
20240506-18:01:21 Recording: sending discover using local ip [fe80::62d4:2540:2154:b612] (ifindex 19)
20240506-18:01:21 Recording: sending discover using local ip 192.168.1.5 (ifindex 19)
20240506-18:01:21 Recording: sending discover using local ip [fe80::62d4:2540:2154:b612] (ifindex 19)
20240506-18:01:21 Recording: sending discover using local ip 192.168.1.5 (ifindex 19)
20240506-18:01:22 Recording: discover response from 585DF01D-33BD-4AAD-9C1C-08D94361A44D http://[::1]:49680
20240506-18:01:22 Recording: discover response from 585DF01D-33BD-4AAD-9C1C-08D94361A44D http://127.0.0.1:49680
20240506-18:01:22 Recording: discover response from 585DF01D-33BD-4AAD-9C1C-08D94361A44D http://[fe80::62d4:2540:2154:b612]:49680
20240506-18:01:22 Recording: discover response from 585DF01D-33BD-4AAD-9C1C-08D94361A44D http://192.168.1.5:49680
20240506-18:01:22 Recording: discover response from 1075D4B1 http://[fe80::218:ddff:fe07:5d4b]:80
20240506-18:01:22 Recording: discover response from 1075D4B1 http://192.168.1.16:80
20240506-18:01:22 Recording: discover response from 1080F19F http://[fe80::218:ddff:fe08:f19]:80
20240506-18:01:22 Recording: discover response from 1080F19F http://192.168.1.18:80
20240506-18:01:22 Recorded: recorded sync to record-api.hdhomerun.com
20240506-18:01:22 Recording: event download from record-api.hdhomerun.com
20240506-18:01:22 Recording: 1075D4B1 lineup request success (found 166 channels)
20240506-18:01:22 Recording: 1080F19F lineup request success (found 164 channels)
20240506-18:01:22 System: server time = Mon May  6 18:01:22 2024 (correction of 0s)
20240506-18:01:22 Recorded: recorded sync success
20240506-18:01:22 System: server time = Mon May  6 18:01:22 2024 (correction of 0s)
20240506-18:01:22 Recording: myhdhomerun_record authorization error or not subscribed to dvr service
20240506-18:01:22 Recording: event download failed
20240506-18:01:22 Recording: disk space available = 216GB
20240506-18:01:22 Recording: current time = Mon May  6 18:01:22 2024 (correction of 0s)
20240506-18:01:22 Recording: no timer events planned
20240506-18:03:08 Status: Resource: nbk=2 dmk=429
20240506-18:05:08 Status: Resource: nbk=2 dmk=429
20240506-18:07:08 Status: Resource: nbk=2 dmk=429
20240506-18:09:08 Status: Resource: nbk=2 dmk=429
20240506-18:11:08 Status: Resource: nbk=2 dmk=429
20240506-18:11:40 Recording: sending discover using local ip [fe80::62d4:2540:2154:b612] (ifindex 19)
20240506-18:11:40 Recording: sending discover using local ip 192.168.1.5 (ifindex 19)
20240506-18:11:40 Recording: sending discover using local ip [fe80::62d4:2540:2154:b612] (ifindex 19)
20240506-18:11:40 Recording: sending discover using local ip 192.168.1.5 (ifindex 19)
20240506-18:11:40 Recording: discover response from 574CBC44-9C22-4F65-BF22-07C01AE6A414 http://[fe80::22a2:28fa:5423:6b76]:49695
20240506-18:11:40 Recording: discover response from 574CBC44-9C22-4F65-BF22-07C01AE6A414 http://192.168.1.2:49695
20240506-18:11:40 Recording: discover response from 585DF01D-33BD-4AAD-9C1C-08D94361A44D http://[::1]:49680
20240506-18:11:40 Recording: discover response from 585DF01D-33BD-4AAD-9C1C-08D94361A44D http://127.0.0.1:49680
20240506-18:11:40 Recording: discover response from 585DF01D-33BD-4AAD-9C1C-08D94361A44D http://[fe80::62d4:2540:2154:b612]:49680
20240506-18:11:40 Recording: discover response from 585DF01D-33BD-4AAD-9C1C-08D94361A44D http://192.168.1.5:49680
20240506-18:11:40 Recording: discover response from 1075D4B1 http://[fe80::218:ddff:fe07:5d4b]:80
20240506-18:11:40 Recording: discover response from 1075D4B1 http://192.168.1.16:80
20240506-18:11:40 Recording: discover response from 1080F19F http://[fe80::218:ddff:fe08:f19]:80
20240506-18:11:40 Recording: discover response from 1080F19F http://192.168.1.18:80
20240506-18:11:40 Recorded: recorded sync to record-api.hdhomerun.com
20240506-18:11:40 Recording: event download from record-api.hdhomerun.com
20240506-18:11:40 Recording: 1075D4B1 lineup request success (found 166 channels)
20240506-18:11:40 Recording: 1080F19F lineup request success (found 164 channels)
20240506-18:11:41 System: server time = Mon May  6 18:11:41 2024 (correction of 1s)
20240506-18:11:41 Recorded: recorded sync success
20240506-18:11:41 System: server time = Mon May  6 18:11:41 2024 (correction of 1s)
20240506-18:11:41 Recording: myhdhomerun_record authorization error or not subscribed to dvr service
20240506-18:11:41 Recording: event download failed
20240506-18:11:41 Recording: disk space available = 216GB
20240506-18:11:41 Recording: current time = Mon May  6 18:11:41 2024 (correction of 1s)
20240506-18:11:41 Recording: no timer events planned

TPeterson
Posts: 138
Joined: Thu May 31, 2007 8:29 pm
Device ID: 10123716, 10157425, 1039FE2B, 103AEA6C, 1075D4B1, 1076C3A7, 1080F19F
Location: San Carlos, CA
Contact:

Re: Recording metadata - TPeterson

Post by TPeterson »

nickk wrote: Mon May 06, 2024 2:43 pm It should not be checking every 10 minutes... that suggests the record engine thinks the sync failed triggering a faster than normal retry.

The sync is required because the guide servers need to know what images to keep around for recordings to have images.
Nick, do you need more information than the above to answer my question about what is wrong such that the DVR engine is making excessively frequent calls home?

Post Reply