unofficial Docker Container for Record Engine

Downloads & Instructions
Post Reply
demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: unofficial Docker Container for Record Engine

Post by demonrik »

AnalogBill wrote: Fri Jun 24, 2022 7:39 am Stuck with this error:
DVRMgr: ** Starting the DVR Engine as user dvr
ash: /HDHomeRunDVR/data/hdhomerun_record: Permission denied
2022-06-23 15:01:10,003 INFO exited: dvr (exit status 126; not expected)

Gotta be a permissions problem, but I'm not understanding where.
-rwxrwxrwx+ 1 dvr 1000 80 Jun 18 09:17 dvr.conf
---x------ 1 dvr 1000 1983318 Mar 3 13:39 hdhomerun_record
the permissions on the hdhomerun_record are really weird.
It should be set to -rwx-------
---x------- is not right.
as root - delete the hdhomerun_record file and restart the container.

AnalogBill
Posts: 22
Joined: Wed Aug 11, 2021 6:23 am
Location: Fort Wayne, Indiana

Re: unofficial Docker Container for Record Engine

Post by AnalogBill »

after deleting and restarting the container I got the same result. --x------
Could the dvr user be the problem? Originally I created a normal user account, with no password, then modified the /etc/passed file:
dvr:x:1000:100:default user for HDHomerunDVR:/var/services/homes/dvr:/bin/false
After changing the UID to 1000 the dvr user no longer appears under Users in the Control Panel.

demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: unofficial Docker Container for Record Engine

Post by demonrik »

New version pushed to dockerhub https://hub.docker.com/r/demonrik/hdhrdvr-docker

Changes
  • Fixed the logfile parsing
  • Added support for changing the NGINX port to one of your choosing through environment var DVRUI_PORT
  • Moved the configuration files for nginx and php out to the same data folder as the dvr configuration
Still to do
  • user UID/GID overrides
  • ARM builds
With 2 above, you can now move the port by setting an environment variable on the container in your container manager of choice.
Just make sure it's free and clear of any firewall rules.

demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: unofficial Docker Container for Record Engine

Post by demonrik »

AnalogBill wrote: Sun Jun 26, 2022 8:13 am after deleting and restarting the container I got the same result. --x------
Could the dvr user be the problem? Originally I created a normal user account, with no password, then modified the /etc/passed file:
dvr:x:1000:100:default user for HDHomerunDVR:/var/services/homes/dvr:/bin/false
After changing the UID to 1000 the dvr user no longer appears under Users in the Control Panel.
Apologies - I got distracted with work all last week
I think you're right and there is an issue on synology with UID of 1000

Honestly, my recommendation for Synology at this point is to do a build yourself for now.
Don't be too worried - it is actually really easy and you've already demonstrated more than enough skill.

You can remove the bad user by SSH to the NAS and executing

Code: Select all

/usr/syno/sbin/synouser --del dvr
So first things first - you need to grab the docker sources from https://github.com/demonrik/HDHR-DVR-docker
See the green button saying 'Code' on the above page
You can either download a zip, copy to NAS and unzip to a temporary folder, or install GIT on the nas and clone using one of the clone options provided.
For simplicity I would say do the ZIP

Once you have the ZIP decompressed to a folder on the NAS
Create a new DVR user and look up it's UID/GID (and make sure that user has access to the volumes we're going to use)
Now you need to edit the 'Dockerfile' in the source
Look for

Code: Select all

# Create default user and group & patch up permissions
RUN addgroup -g 1000 dvr
RUN adduser -HDG dvr -u 1000 dvr
And replace the 1000 for the group dvr with the right GID, and the 1000 for the user with the UID, e.g. for a UID of 2300 and GID of 400

Code: Select all

# Create default user and group & patch up permissions
RUN addgroup -g 400 dvr
RUN adduser -HDG dvr -u 2300 dvr
And save

If I remember rightly on Synology you will need to use 'sudo' for most docker commands
SSH to the NAS and cd to the folder above where the sources are.
First we build the Image

Code: Select all

sudo docker build -t localdvr:latest .
Notice we've named tagged this image differently than demonrik....
This is so you can't mistakenly update :)

Now we run it
You can run it manually on command line with

Code: Select all

 sudo docker run --name dvr --network host -d -v <setpath>:/dvrdata -v <RecordPath>:/dvrrec -e DVRUI_PORT=59080 localdvr:latest
replacing <setpath> and <RecordPath> with the right paths.

Or if happier with something like portainer, make sure to click on 'advanced mode' under 'Image Configuration'
Set the Image to our 'localdvr:latest', and disable 'always pull the image'
And proceed to setup the container.
Remember to now add the new Environment variable DVRUI_PORT too (you add in the Env tab of 'advanced container settings')
The reason you want to use the advanced mode is your don't want portainer going off querying dockerhub - you already have the image.

AnalogBill
Posts: 22
Joined: Wed Aug 11, 2021 6:23 am
Location: Fort Wayne, Indiana

Re: unofficial Docker Container for Record Engine

Post by AnalogBill »

Thank you! Big learning curve...my most recent software course was in 1971, Fortran. This will be interesting. :)
Update: Build failed at Step 6: Can't find php7 package.

Code: Select all

Step 6/36 : RUN apk add --no-cache php7...ERROR: unable to select packages:
  php7 (no such package)...
  The command '/bin/sh -c apk add --no-cache php7         php7-common  ... returned a non-zero code: 11
Php7 should be in the Community repository, but I found only php81
Portainer shows the partially completed image (with no name) while Docker shows only a new Apline:latest image.

May have found problem. Build script using alpine V3.16.
fetch https://dl-cdn.alpinelinux.org/alpine/v ... DEX.tar.gz
php7 in not in the repository after V3.15 How to remedy?

demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: unofficial Docker Container for Record Engine

Post by demonrik »

AnalogBill wrote: Sat Jul 09, 2022 5:54 pm Thank you! Big learning curve...my most recent software course was in 1971, Fortran. This will be interesting. :)
Update: Build failed at Step 6: Can't find php7 package.

Code: Select all

Step 6/36 : RUN apk add --no-cache php7...ERROR: unable to select packages:
  php7 (no such package)...
  The command '/bin/sh -c apk add --no-cache php7         php7-common  ... returned a non-zero code: 11
Php7 should be in the Community repository, but I found only php81
Portainer shows the partially completed image (with no name) while Docker shows only a new Apline:latest image.

May have found problem. Build script using alpine V3.16.
fetch https://dl-cdn.alpinelinux.org/alpine/v ... DEX.tar.gz
php7 in not in the repository after V3.15 How to remedy?
interesting - good catch.
For some reason my docker hadn't spotted to pull latest alpine and was still using the olde pulled 3.15 version.

Simplest method is what I'll be checking in (should be hitting github now)
In the Dockerfile replace line 1

Code: Select all

FROM alpine:latest
with

Code: Select all

FROM alpine:3.15.4
This will force to older 3.15.4

Pushing new build to dockerhub too which has the slight change to the latest alpine linux.

AnalogBill
Posts: 22
Joined: Wed Aug 11, 2021 6:23 am
Location: Fort Wayne, Indiana

Re: unofficial Docker Container for Record Engine

Post by AnalogBill »

Thx, that fixed the build. I can get the container running with no log errors if I chown dvr:dvr the record engine after a restart. Restarts set the engine back to root:root and random permissions, usually -x--------. Still befuddled by networking. I can get a system status page from mynasip:59090 but nothing from the DVRUI_PORT, 8080. Network is bridge not host and I haven't figured out how to fix that. My tuners are on a 192.168.0.x network and 172.17.0.3 is all docker sees.

demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: unofficial Docker Container for Record Engine

Post by demonrik »

AnalogBill wrote: Mon Jul 11, 2022 4:39 pm Thx, that fixed the build. I can get the container running with no log errors if I chown dvr:dvr the record engine after a restart. Restarts set the engine back to root:root and random permissions, usually -x--------. Still befuddled by networking. I can get a system status page from mynasip:59090 but nothing from the DVRUI_PORT, 8080. Network is bridge not host and I haven't figured out how to fix that. My tuners are on a 192.168.0.x network and 172.17.0.3 is all docker sees.
Use Host network.
You must use host for the DVR to find the tuners.
My bad - I think I sent you down that rabbit hole.
Once you bring back to host it should all work (fingers crossed)

demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: unofficial Docker Container for Record Engine

Post by demonrik »

New builds posted to dockerhub.com
Now has builds for armv7, aarch64 (sometimes called arm64) and x86_64 (AMD64)
Should cover most supported NAS architectures.

AnalogBill
Posts: 22
Joined: Wed Aug 11, 2021 6:23 am
Location: Fort Wayne, Indiana

Re: unofficial Docker Container for Record Engine

Post by AnalogBill »

created new container to fix networking:
docker run -d --name dvr --restart=unless-stopped --network host -e DVRUI_PORT=8080 -v /volume1/HD/data:/dvrdata -v /volume1/HD/recordings:/dvrrec localdvr:latest
1) Got the permission denied error due to rec engine owned by root

Code: Select all

drwxrwxrwx+ 1 dvr   400       0 Jul 11 07:58 dvr
-rwxrwxrwx+ 1 dvr   400     128 Jul 11 11:04 dvr.conf
-rwxrwxrwx+ 1 root root 1112605 Jul 12 13:29 hdhomerun_record_beta
-rwxrwxrwx+ 1 root root 1983318 Mar  3 13:39 hdhomerun_record_rel
drwxrwxrwx+ 1 dvr   400      52 Jul 12 13:15 http
drwxrwxrwx+ 1 dvr   400      54 Jul 12 13:15 php
2) chown dvr:dvr to fix

Code: Select all

-rwxrwxrwx+ 1 dvr 400 1112605 Jul 12 13:29 hdhomerun_record_beta
-rwxrwxrwx+ 1 dvr 400 1983318 Mar  3 13:39 hdhomerun_record_rel
3) restarted container, got different filename & permissions, but runs without errors.

Code: Select all

-rwx------  1 dvr 400 1983318 Mar  3 13:39 hdhomerun_record
Not understanding the DVRUI_PORT=8080. Doesn't seem to do anything. On <mynasip>:59090 I get a status page. Should I expect to see the same DVR Manager UI as I get on my .spk installation?

demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: unofficial Docker Container for Record Engine

Post by demonrik »

AnalogBill wrote: Tue Jul 12, 2022 1:15 pm created new container to fix networking:
docker run -d --name dvr --restart=unless-stopped --network host -e DVRUI_PORT=8080 -v /volume1/HD/data:/dvrdata -v /volume1/HD/recordings:/dvrrec localdvr:latest
1) Got the permission denied error due to rec engine owned by root

Code: Select all

drwxrwxrwx+ 1 dvr   400       0 Jul 11 07:58 dvr
-rwxrwxrwx+ 1 dvr   400     128 Jul 11 11:04 dvr.conf
-rwxrwxrwx+ 1 root root 1112605 Jul 12 13:29 hdhomerun_record_beta
-rwxrwxrwx+ 1 root root 1983318 Mar  3 13:39 hdhomerun_record_rel
drwxrwxrwx+ 1 dvr   400      52 Jul 12 13:15 http
drwxrwxrwx+ 1 dvr   400      54 Jul 12 13:15 php
2) chown dvr:dvr to fix

Code: Select all

-rwxrwxrwx+ 1 dvr 400 1112605 Jul 12 13:29 hdhomerun_record_beta
-rwxrwxrwx+ 1 dvr 400 1983318 Mar  3 13:39 hdhomerun_record_rel
3) restarted container, got different filename & permissions, but runs without errors.

Code: Select all

-rwx------  1 dvr 400 1983318 Mar  3 13:39 hdhomerun_record
Not understanding the DVRUI_PORT=8080. Doesn't seem to do anything. On <mynasip>:59090 I get a status page. Should I expect to see the same DVR Manager UI as I get on my .spk installation?
Apologies for delay in responding.
interesting on the root ownership of the engine - I will investigate.. possibly an issue with that script I need to go work.
the DVRUI_PORT will make the DVR Manager UI available.
If you can't get to it - could be that port is already in use by the Synology. 8080 is a popular port for moving a web server to for security, etc.
Try to set to something above 32000.. I use 59080 for example. Then you should be able to go to <mynasip>:59080 and see the Manager UI.

AnalogBill
Posts: 22
Joined: Wed Aug 11, 2021 6:23 am
Location: Fort Wayne, Indiana

Re: unofficial Docker Container for Record Engine

Post by AnalogBill »

While testing the root problem by starting/stopping the container last night I did a chown -R in a very wrong place. Almost finished rebuilding the nas. Have to rebuild the image as I never figured out how to push it to the repository. Should go faster this time :D

Success! 21 hours from a trashed NAS to a DVR running without errors! Thanks for the coaching, I learned a lot this week.
Last edited by AnalogBill on Thu Jul 14, 2022 3:49 pm, edited 1 time in total.

demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: unofficial Docker Container for Record Engine

Post by demonrik »

AnalogBill wrote: Thu Jul 14, 2022 12:43 pm While testing the root problem by starting/stopping the container last night I did a chown -R in a very wrong place. Almost finished rebuilding the nas. Have to rebuild the image as I never figured out how to push it to the repository. Should go faster this time :D
I feel for you - easily done
You shouldn't have to rebuild the container unless you removed it before you rebuilt
you won't be able to push it to dockerhub unless you tag it to a repo you create on there - is pretty easy to do though
I will look in to the root issue today and see if I can see what's going wrong.

demonrik
Posts: 1591
Joined: Mon May 04, 2015 10:03 am
Device ID: 108042A1, 10814D8E
x 38

Re: unofficial Docker Container for Record Engine

Post by demonrik »

AnalogBill wrote: Tue Jul 12, 2022 1:15 pm 1) Got the permission denied error due to rec engine owned by root

Code: Select all

drwxrwxrwx+ 1 dvr   400       0 Jul 11 07:58 dvr
-rwxrwxrwx+ 1 dvr   400     128 Jul 11 11:04 dvr.conf
-rwxrwxrwx+ 1 root root 1112605 Jul 12 13:29 hdhomerun_record_beta
-rwxrwxrwx+ 1 root root 1983318 Mar  3 13:39 hdhomerun_record_rel
drwxrwxrwx+ 1 dvr   400      52 Jul 12 13:15 http
drwxrwxrwx+ 1 dvr   400      54 Jul 12 13:15 php
2) chown dvr:dvr to fix

Code: Select all

-rwxrwxrwx+ 1 dvr 400 1112605 Jul 12 13:29 hdhomerun_record_beta
-rwxrwxrwx+ 1 dvr 400 1983318 Mar  3 13:39 hdhomerun_record_rel
3) restarted container, got different filename & permissions, but runs without errors.

Code: Select all

-rwx------  1 dvr 400 1983318 Mar  3 13:39 hdhomerun_record
ok - I did some looking.
It could be just cleanup of issues that needed to happen to get it move forward. or something went wrong. Let me walk through each point

(1)
The actual problem here shouldn't have been a permission denied. The script within the container to download the engines runs as root. Once it downloads the files from the beta or the release link it compares the time stamps to see which is newer and then moves the rel or beta to be the record engine and chown it to the dvr user.
The fact that you still had the beta and rel files but no final file suggests something broke before the mv to make one of the engines the engine we want.
So something else went wrong.
For future - If you can find where in your setup to view the logs from the container it would be useful. I know Portainer allows for this, and am guessing Synology has something too. But look there and you should see something like:

Code: Select all

************************************************

           Starting DVR Container

************************************************

INFO: DVR data folder exists already
INFO: DVR Data folder is linked
INFO: DVR Folder linked to correct folder /dvrdata
INFO: DVR recordings folder exists already
INFO: DVR recordings folder is linked
INFO: DVR recordings Folder linked to correct folder /dvrrec
INFO: PHP WWW Server config file is missing, pulling from defaults...
INFO: Updating the NGINX Port for UI to 59080
2022-07-14 10:01:34,009 INFO Set uid to user 0 succeeded
2022-07-14 10:01:34,010 INFO supervisord started with pid 18
2022-07-14 10:01:35,013 INFO spawned: 'dvr' with pid 19
2022-07-14 10:01:35,014 INFO spawned: 'nginx' with pid 20
2022-07-14 10:01:35,016 INFO spawned: 'php-fpm' with pid 21
DVRMgr: ** Validating the Config File is available and set up correctly
DVRMgr: Config File exists and is writable - is record path and port correct
DVRMgr: ** Installing the HDHomeRunDVR Record Engine
DVRMgr: Lets remove any existing engine - we're going to take the latest always.... 
DVRMgr: Checking it was deleted - if we can't remove it we can't update
DVRMgr: Downloading latest release
DVRMgr: Downloading latest beta
DVRMgr: Comparing which is newest
DVRMgr: Both versions are same - using the Release version
DVRMgr: Engine Updated to...  20220303
DVRMgr: ** Starting the DVR Engine as user dvr
2022-07-14 10:01:36,702 INFO success: dvr entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-07-14 10:01:36,702 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-07-14 10:01:36,702 INFO success: php-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
The DVRMgr lines are the ones we want to watch. We can see that it gets the engine version at the end - so here it actually attempts to run the record engine to jsut get the version. Doing that will extract the real binary into /tmp in the container and then execute it. So should have no impact unless for some reason something on the filesystem is preventing root from executing anything off your share - which shouldn't be the case since we see the --x and it works later.

So why does changing to dvr user fix it.. I've no idea - it shouldn't have mattered.
But that said - there is really no reason to not run this script as the dvr user anyway. There's a few changes to be made, but should be clean enough and I'll do over weekend. Should mean if there is something causing it to crash between downloading and mv the final that is fixed by chowning to dvr it will never need to do the chown and thus will work.

(2)
as I said, that shouldn't have made a difference and that it did makes me think I should implement the last bit

(3)
The filename changed because the 'mv' happened as it should.

AnalogBill
Posts: 22
Joined: Wed Aug 11, 2021 6:23 am
Location: Fort Wayne, Indiana

Re: unofficial Docker Container for Record Engine

Post by AnalogBill »

Here's the initial log, created when I first ran my Docker run command

Code: Select all

date	stream	content
2022-07-14T22:32:56.034008765Z	stdout	************************************************
2022-07-14T22:32:56.034116463Z	stdout	
2022-07-14T22:32:56.034172332Z	stdout	           Starting DVR Container
2022-07-14T22:32:56.034238561Z	stdout	
2022-07-14T22:32:56.034291510Z	stdout	************************************************
2022-07-14T22:32:56.034346379Z	stdout	
2022-07-14T22:32:56.034404798Z	stdout	INFO: DVR data folder exists already
2022-07-14T22:32:56.034475147Z	stdout	INFO: DVR Data folder is linked
2022-07-14T22:32:56.034649434Z	stdout	INFO: DVR Folder linked to correct folder /dvrdata
2022-07-14T22:32:56.034804901Z	stdout	INFO: DVR recordings folder exists already
2022-07-14T22:32:56.034880989Z	stdout	INFO: DVR recordings folder is linked
2022-07-14T22:32:56.035567877Z	stdout	INFO: DVR recordings Folder linked to correct folder /dvrrec
2022-07-14T22:32:56.035652216Z	stdout	INFO: DVR config folder doesn't exist, creating...
2022-07-14T22:32:56.036781265Z	stdout	INFO: NGINX config folder doesn't exist, creating...
2022-07-14T22:32:56.037733138Z	stdout	INFO: NGINX config file is missing, pulling from defaults...
2022-07-14T22:32:56.048130182Z	stdout	INFO: NGINX DVR Server config file is missing, pulling from defaults...
2022-07-14T22:32:56.066918416Z	stdout	INFO: PHP config folder doesn't exist, creating...
2022-07-14T22:32:56.067926118Z	stdout	INFO: PHP FPM config file is missing, pulling from defaults...
2022-07-14T22:32:56.084368514Z	stdout	INFO: PHP WWW Server config file is missing, pulling from defaults...
2022-07-14T22:32:56.101122894Z	stdout	INFO: PHP ini file is missing, pulling from defaults...
2022-07-14T22:32:56.114300178Z	stdout	INFO: Updating the NGINX Port for UI to 59080
2022-07-14T22:32:57.459956246Z	stdout	2022-07-14 22:32:57,459 INFO Set uid to user 0 succeeded
2022-07-14T22:32:57.468480874Z	stdout	2022-07-14 22:32:57,468 INFO supervisord started with pid 25
2022-07-14T22:32:58.472328342Z	stdout	2022-07-14 22:32:58,471 INFO spawned: 'dvr' with pid 26
2022-07-14T22:32:58.476339490Z	stdout	2022-07-14 22:32:58,475 INFO spawned: 'nginx' with pid 27
2022-07-14T22:32:58.481158294Z	stdout	2022-07-14 22:32:58,480 INFO spawned: 'php-fpm' with pid 28
2022-07-14T22:32:58.511835455Z	stdout	DVRMgr: ** Validating the Config File is available and set up correctly
2022-07-14T22:32:58.512752258Z	stdout	DVRMgr: Config is missing - creating initial version
2022-07-14T22:32:58.513747130Z	stdout	DVRMgr: ** Creating Initial Config File
2022-07-14T22:32:58.517384005Z	stdout	DVRMgr: ** Installing the HDHomeRunDVR Record Engine
2022-07-14T22:32:58.517560612Z	stdout	DVRMgr: Lets remove any existing engine - we're going to take the latest always.... 
2022-07-14T22:32:58.526289576Z	stdout	DVRMgr: Checking it was deleted - if we can't remove it we can't update
2022-07-14T22:32:58.526530331Z	stdout	DVRMgr: Downloading latest release
2022-07-14T22:32:58.922523462Z	stderr	[14-Jul-2022 22:32:58] NOTICE: fpm is running, pid 28
2022-07-14T22:32:58.923629302Z	stderr	[14-Jul-2022 22:32:58] NOTICE: ready to handle connections
2022-07-14T22:32:59.538185630Z	stderr	sh: out of range
2022-07-14T22:32:59.538475925Z	stdout	2022-07-14 22:32:59,537 INFO success: dvr entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-07-14T22:32:59.538765170Z	stdout	2022-07-14 22:32:59,538 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-07-14T22:32:59.538854768Z	stdout	2022-07-14 22:32:59,538 INFO success: php-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-07-14T22:32:59.539011505Z	stdout	DVRMgr: Not using Beta Versions - defaulting to Release Version
2022-07-14T22:32:59.628825627Z	stdout	DVRMgr: Engine Updated to...  20220303
2022-07-14T22:32:59.629080993Z	stdout	DVRMgr: ** Starting the DVR Engine as user dvr
Changed DVRUI_PORT=59080, now getting DVR Manager.

Post Reply