Support running multiple record engines as a pool

Downloads & Instructions
Online
nickk
Silicondust
Posts: 20163
Joined: Tue Jan 13, 2004 9:39 am
x 376

Re: NEW - support running multiple record engines as a pool

Post by nickk »

Beerman wrote: Fri May 17, 2019 6:51 pm I need to do a little checking about but I noticed that after installing the multiple record engines I'm getting duplicate recordings in both NAS's. I obviously did something wrong.
Go to http://my.hdhomerun.com/

1) Check that the version is 20190516beta2 (or newer) on all record engines.

2) Click on each record engine and go to the System page - check there is a StorageID shown and check each record engine has a different StorageID.

Nick

Beerman
Posts: 1642
Joined: Thu May 21, 2015 7:05 pm
x 5

Re: NEW - support running multiple record engines as a pool

Post by Beerman »

nickk wrote: Sat May 18, 2019 6:59 am
Beerman wrote: Fri May 17, 2019 6:51 pm I need to do a little checking about but I noticed that after installing the multiple record engines I'm getting duplicate recordings in both NAS's. I obviously did something wrong.
Go to http://my.hdhomerun.com/

1) Check that the version is 20190516beta2 (or newer) on all record engines.

2) Click on each record engine and go to the System page - check there is a StorageID shown and check each record engine has a different StorageID.

Nick
1) Yes
2)They have the same ID.
What did I do wrong? I'm relatively sure they had different ID's when I first installed the beta but perhaps after I installed demonrik's DVR Manager something changed?????

Online
nickk
Silicondust
Posts: 20163
Joined: Tue Jan 13, 2004 9:39 am
x 376

Re: NEW - support running multiple record engines as a pool

Post by nickk »

Beerman wrote: Sat May 18, 2019 7:06 am 2)They have the same ID.
What did I do wrong? I'm relatively sure they had different ID's when I first installed the beta but perhaps after I installed demonrik's DVR Manager something changed?????
Try going to the NAS using Windows Explorer to open the share and deleting the hdhomerun.conf file from the HDHomeRun folder. Then run NAS install again so it re-creates the conf file.

You may need to restart the NAS for the new conf file to take effect.

gtb
Expert
Posts: 4224
Joined: Thu Oct 06, 2011 1:00 pm
Location: Sunnyvale, CA USA
x 16

Re: NEW - support running multiple record engines as a pool

Post by gtb »

nickk wrote: Thu May 16, 2019 5:41 pm To limit the number of concurrent recordings set RecordStreamsMax in the conf file (registry on Windows). Once this limit is reached another record engine will take over additional concurrent recordings. The default/max is 16.
While I personally would be highly unlikely to exceed 16 concurrent streams, I can easily see a robust engine environment being easily able to successfully exceed 16 concurrent streams for recording without breaking a sweat. I think the default is probably good, but the max should be considered to be able to be set higher (perhaps with a warning message in the logs about possibly exceeding your engine capacity?). And while I could run multiple VMs or containers on the host to exceed that 16, that really is a workaround for an artificial limit. Thanks for the consideration.

Beerman
Posts: 1642
Joined: Thu May 21, 2015 7:05 pm
x 5

Re: NEW - support running multiple record engines as a pool

Post by Beerman »

nickk wrote: Sat May 18, 2019 8:02 am
Beerman wrote: Sat May 18, 2019 7:06 am 2)They have the same ID.
What did I do wrong? I'm relatively sure they had different ID's when I first installed the beta but perhaps after I installed demonrik's DVR Manager something changed?????
Try going to the NAS using Windows Explorer to open the share and deleting the hdhomerun.conf file from the HDHomeRun folder. Then run NAS install again so it re-creates the conf file.

You may need to restart the NAS for the new conf file to take effect.
Thanks, all I needed was to stop and restart the service and it gave separate ID's.
Odd that even though I have my system set to show all hidden files, I can't find the config file anywhere. No worries, things look to be working like they should.

Online
nickk
Silicondust
Posts: 20163
Joined: Tue Jan 13, 2004 9:39 am
x 376

Re: NEW - support running multiple record engines as a pool

Post by nickk »

gtb wrote: Sat May 18, 2019 8:13 am While I personally would be highly unlikely to exceed 16 concurrent streams, I can easily see a robust engine environment being easily able to successfully exceed 16 concurrent streams for recording without breaking a sweat. I think the default is probably good, but the max should be considered to be able to be set higher (perhaps with a warning message in the logs about possibly exceeding your engine capacity?). And while I could run multiple VMs or containers on the host to exceed that 16, that really is a workaround for an artificial limit. Thanks for the consideration.
Agreed. The next beta will default to 16 with a configurable max of 128.

Online
nickk
Silicondust
Posts: 20163
Joined: Tue Jan 13, 2004 9:39 am
x 376

Re: NEW - support running multiple record engines as a pool

Post by nickk »

Beerman wrote: Sat May 18, 2019 8:42 am Thanks, all I needed was to stop and restart the service and it gave separate ID's.
Odd that even though I have my system set to show all hidden files, I can't find the config file anywhere. No worries, things look to be working like they should.
Can you please run one more quick test - check the StorageID, restart the record engine, then check the StorageID again... make sure it is the same ID. If it is different then the conf file is being lost.

Thunderthud
Posts: 704
Joined: Sun Apr 12, 2015 6:59 am
Device ID: 1080B565, 1322B8A6, 13257C3D, 1041CFDB, 10514020, 101886A, 10120815
Location: Virginia
x 1

Re: NEW - support running multiple record engines as a pool

Post by Thunderthud »

Tried installing on my original MyCloud (Single Drive Ver. 1, which I got from SD Kickstarter), along with my currently in use MyCloud (Single Drive Ver. 2). I had re-initialized the original MyCloud to wipe clean and was attempting to install the record engine and got the error messages below... (Record engine installed okay on my existing V2 MyCloud)

Model: WDMyCloud
SSH Enabled: Ok
SSH Session: Ok
NAS Architecture: Unknown
HDHomeRun Share: Error
smbd not found or not running
Unable to install


I can see and access the HDHomeRun share on my PC over the network, and am able to create files there, with Windows Explorer, with no problem.


Any suggestions? Thanks.

Beerman
Posts: 1642
Joined: Thu May 21, 2015 7:05 pm
x 5

Re: NEW - support running multiple record engines as a pool

Post by Beerman »

Looks good to me!


BEFORE

Version 20190516beta2
Storage ID c9c84fd3-e824-8e1a-4e17-45b15509d58c
Record Path /share/Multimedia/HDHomeRunDVR/
Search Path /share/Multimedia/HDHomeRunDVR/
Free Space 972 GB
IP Address 192.168.1.71

Version 20190516beta2
Storage ID 9fe3eb3b-44fc-dbb0-f7b3-862433cd0650
Record Path /volume1/HDHomeRunDVR/
Search Path /volume1/HDHomeRunDVR/
Free Space 12202 GB
IP Address 192.168.1.121


AFTER

Version 20190516beta2
Storage ID c9c84fd3-e824-8e1a-4e17-45b15509d58c
Record Path /share/Multimedia/HDHomeRunDVR/
Search Path /share/Multimedia/HDHomeRunDVR/
Free Space 972 GB
IP Address 192.168.1.71


Version 20190516beta2
Storage ID 9fe3eb3b-44fc-dbb0-f7b3-862433cd0650
Record Path /volume1/HDHomeRunDVR/
Search Path /volume1/HDHomeRunDVR/
Free Space 12202 GB
IP Address 192.168.1.121

demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: NEW - support running multiple record engines as a pool

Post by demonrik »

Beerman wrote: Odd that even though I have my system set to show all hidden files, I can't find the config file anywhere. No worries, things look to be working like they should.
If using my packages, the NAS system usually hides the location. synology can only be found with ssh, and you really need to know where to look. ps -ef helps ;)

working perfectly for my needs across 3 NASes here
set RecordStreamsMax to 0 on the boxes i don't want recording and they didn't last night..

Have simple script running on Synology to move any recording > 15 days to my Qnap and movies to the Asustor
Clients can play all recordings

if i was to set RecordStreamsMax to >0 is there a way to set preferred first engine?
i.e. i want to use my synology which is on the same GbE switch first

signcarver
Expert
Posts: 11083
Joined: Wed Jan 24, 2007 1:04 am
Device ID: 10A05954 10802091 131B34B7 13231F92 1070A18E 1073ED6F 15300C36
x 39

Re: NEW - support running multiple record engines as a pool

Post by signcarver »

demonrik wrote: Sat May 18, 2019 11:15 am if i was to set RecordStreamsMax to >0 is there a way to set preferred first engine?
i.e. i want to use my synology which is on the same GbE switch first
I believe nickk stated it would generally use the one with most space... I can see that I also would like to set a preference and perhaps another preference for LiveTV.

gtb
Expert
Posts: 4224
Joined: Thu Oct 06, 2011 1:00 pm
Location: Sunnyvale, CA USA
x 16

Re: NEW - support running multiple record engines as a pool

Post by gtb »

nickk wrote: Thu May 16, 2019 5:41 pm To limit the number of concurrent recordings set RecordStreamsMax in the conf file (registry on Windows). Once this limit is reached another record engine will take over additional concurrent recordings. The default/max is 16.
I have not tried this (but I was thinking of edge cases), but if you have a number of record engines, and set *all* of them to have a RecordStreamsMax to zero (or the only currently reachable ones all have the setting of zero, or a NULL first record path), will "Live TV" work properly without any available record capable DVRs?

signcarver
Expert
Posts: 11083
Joined: Wed Jan 24, 2007 1:04 am
Device ID: 10A05954 10802091 131B34B7 13231F92 1070A18E 1073ED6F 15300C36
x 39

Re: NEW - support running multiple record engines as a pool

Post by signcarver »

I have actually had "null" allow live tv in the past (my extra engines each have nullLiveTV as the path) but such behavior may have been fixed. I would like an option to prefer one for live tv while another for recording (i.e. a MaxLiveStreams setting) but most likely this would also need to be handled by the client as to which to prefer.

jjm1982
Posts: 433
Joined: Wed Sep 21, 2011 5:07 am

Re: NEW - support running multiple record engines as a pool

Post by jjm1982 »

What is the expected interaction between recording engines if you have more than one recording engine saving recordings to the same path? I have two Linux servers running debian. On the primary file server, the recordings are being saved to folder 'A'. Folder 'A' is configred to be a shared folder accessible by user permission. The second server has the share of folder 'A' mapped in fstab to a directory on folder 'B' where the recording engine on the second server can read/write to. Would both recording engines write to the same log file without any issues?

jasonl
Silicondust
Posts: 16810
Joined: Sun Oct 28, 2007 9:23 pm
x 60

Re: NEW - support running multiple record engines as a pool

Post by jasonl »

The DVR holds a write lock on the log file. If you try to do that it will probably not end well. I'm not sure why you would even want to do that in the first place.

Post Reply