What kind of hardware required for HDHR Recording computer?

Help and support for HDHomeRun DVR and HDHomeRun software for Windows 10, Mac, Android, XBox, etc.
averyfreeman
Posts: 217
Joined: Sun Apr 09, 2017 11:12 am
Device ID: 1326E235 1313788A
Location: Olympia, WA
Contact:

What kind of hardware required for HDHR Recording computer?

Post by averyfreeman » Mon Jul 29, 2019 4:26 pm

Hey,

Just curious what kind of amd64 hardware would be necessary for recording 6+ streams if I was using dedicated units? Processor, ram, etc.

What's a prime's most throughput-hogging stream spec, is it 1080px24bit@30fps MPEG-2?

Thanks :)

gtb
Expert
Posts: 3985
Joined: Thu Oct 06, 2011 1:00 pm
Location: Sunnyvale, CA USA

Re: What kind of hardware required for HDHR Recording computer?

Post by gtb » Mon Jul 29, 2019 4:43 pm

Moving transport stream packets (from the network to the disk for recording, and back again for playback) requires very little CPU resources (unless the design is software bitbanging for all the interfaces). What is far more important is the I/O (network, disk) restrictions/limitations. That is why most NAS devices which have either an ARM processor, or a low end x86, are more than sufficient for running the record engine because those solutions have the necessary I/O ports and bandwidth. Some of the "consumer" SoCs (the RPi, etc.) that are targeted as being more consumer educational/playthings with very limited I/O are typically a problem (when you try to push everything through a slow USB port, things go badly).

jasonl
Silicondust
Posts: 14067
Joined: Sun Oct 28, 2007 9:23 pm

Re: What kind of hardware required for HDHR Recording computer?

Post by jasonl » Mon Jul 29, 2019 4:48 pm

Depends on the OS. If it's something lightweight like Linux without a GUI, I don't think you could find an AMD64 system that couldn't do 6. A dual-core ARM with a gig of RAM can handle that.

The highest data rates seen in real-world usage are Canadian OTA broadcasts and a few Fios HD channels, which get up to about 18.5Mb/s. Fios has a 4K channel that runs closer to 22, but it's only active for a handful of events each months so it's probably not worth planning around. Most US HD broadcasts are more like 14, and some are a lot less. Most other countries have moved to H.264 for HD and are like 8. Comcast cable channels are around 6.

averyfreeman
Posts: 217
Joined: Sun Apr 09, 2017 11:12 am
Device ID: 1326E235 1313788A
Location: Olympia, WA
Contact:

Re: What kind of hardware required for HDHR Recording computer?

Post by averyfreeman » Tue Jul 30, 2019 8:52 am

jasonl wrote:
Mon Jul 29, 2019 4:48 pm
Depends on the OS. If it's something lightweight like Linux without a GUI, I don't think you could find an AMD64 system that couldn't do 6. A dual-core ARM with a gig of RAM can handle that.

The highest data rates seen in real-world usage are Canadian OTA broadcasts and a few Fios HD channels, which get up to about 18.5Mb/s. Fios has a 4K channel that runs closer to 22, but it's only active for a handful of events each months so it's probably not worth planning around. Most US HD broadcasts are more like 14, and some are a lot less. Most other countries have moved to H.264 for HD and are like 8. Comcast cable channels are around 6.
Wait a sec

4k? I didn't think they were capable of 4k (?)

So does that mean the prime just delivers the data in whatever format it's broadcast in?

Is there any documentation about the way the HW works? I've been searching and the best I found was a PDF from I think it was 2011

rpcameron
Posts: 845
Joined: Fri Mar 25, 2016 9:55 am

Re: What kind of hardware required for HDHR Recording computer?

Post by rpcameron » Tue Jul 30, 2019 8:59 am

averyfreeman wrote:
Tue Jul 30, 2019 8:52 am
jasonl wrote:
Mon Jul 29, 2019 4:48 pm
Depends on the OS. If it's something lightweight like Linux without a GUI, I don't think you could find an AMD64 system that couldn't do 6. A dual-core ARM with a gig of RAM can handle that.

The highest data rates seen in real-world usage are Canadian OTA broadcasts and a few Fios HD channels, which get up to about 18.5Mb/s. Fios has a 4K channel that runs closer to 22, but it's only active for a handful of events each months so it's probably not worth planning around. Most US HD broadcasts are more like 14, and some are a lot less. Most other countries have moved to H.264 for HD and are like 8. Comcast cable channels are around 6.
Wait a sec

4k? I didn't think they were capable of 4k (?)

So does that mean the prime just delivers the data in whatever format it's broadcast in?

Is there any documentation about the way the HW works? I've been searching and the best I found was a PDF from I think it was 2011
Yes, that's exactly what the tuners do; they simply pass whatever stream they receive onto the network.

jasonl
Silicondust
Posts: 14067
Joined: Sun Oct 28, 2007 9:23 pm

Re: What kind of hardware required for HDHR Recording computer?

Post by jasonl » Tue Jul 30, 2019 12:53 pm

It's not that they can't, it's just that there is almost none out there that isn't streaming. Verizon added the Fox 4K channel for a NASCAR race in May, then used it for the Women's World Cup, and after that there have been 2 baseball games, and apparently the MLS All-Star game vs. Atlético tomorrow will be in 4K. Optimum has the same Fox 4K channel, but I have not been able to confirm if it's a real channel that the PRIME can access or if it's delivered as VOD and can only be accessed through their box. All of Comcast's 4K content is VOD so that's inaccessible. That's it as far as I know for traditional cable providers in the US.

averyfreeman
Posts: 217
Joined: Sun Apr 09, 2017 11:12 am
Device ID: 1326E235 1313788A
Location: Olympia, WA
Contact:

Re: What kind of hardware required for HDHR Recording computer?

Post by averyfreeman » Tue Jul 30, 2019 3:36 pm

jasonl wrote:
Tue Jul 30, 2019 12:53 pm
It's not that they can't, it's just that there is almost none out there that isn't streaming. Verizon added the Fox 4K channel for a NASCAR race in May, then used it for the Women's World Cup, and after that there have been 2 baseball games, and apparently the MLS All-Star game vs. Atlético tomorrow will be in 4K. Optimum has the same Fox 4K channel, but I have not been able to confirm if it's a real channel that the PRIME can access or if it's delivered as VOD and can only be accessed through their box. All of Comcast's 4K content is VOD so that's inaccessible. That's it as far as I know for traditional cable providers in the US.
I guess on one hand that's cool that it's starting to proliferate, and the device is capable

On the other hand, I don't have any UHD devices and I'd have to record huge files and wreck my network throughput for no benefit

Thank god all I record is news. That'll probably look like crap forever ;)

Just curious, is it the cable co in the area generally that pushes out the resolution, or the primary network source?

Also, is there any additional strain / limit that starts to occur when generating these UHD streams? Like, in terms of my prime's ability to tune all channels, etc.

Like, let's say there's a time when 4K content is the norm, would the prime which was developed like a decade ago be able to handle all 3 streams at once still?

gtb
Expert
Posts: 3985
Joined: Thu Oct 06, 2011 1:00 pm
Location: Sunnyvale, CA USA

Re: What kind of hardware required for HDHR Recording computer?

Post by gtb » Tue Jul 30, 2019 3:46 pm

averyfreeman wrote:
Tue Jul 30, 2019 3:36 pm
Like, let's say there's a time when 4K content is the norm
Not to worry, it is not expected to be the norm on linear QAM.

jasonl
Silicondust
Posts: 14067
Joined: Sun Oct 28, 2007 9:23 pm

Re: What kind of hardware required for HDHR Recording computer?

Post by jasonl » Tue Jul 30, 2019 4:14 pm

Yeah, that's never going to happen because cable providers don't have the capacity for it. But if it did, it's just data, and the PRIME doesn't really care what's in it. It's just a bit more data than it would have sent before.

averyfreeman
Posts: 217
Joined: Sun Apr 09, 2017 11:12 am
Device ID: 1326E235 1313788A
Location: Olympia, WA
Contact:

Re: What kind of hardware required for HDHR Recording computer?

Post by averyfreeman » Tue Jul 30, 2019 6:46 pm

gtb wrote:
Tue Jul 30, 2019 3:46 pm
averyfreeman wrote:
Tue Jul 30, 2019 3:36 pm
Like, let's say there's a time when 4K content is the norm
Not to worry, it is not expected to be the norm on linear QAM.
jasonl wrote:
Tue Jul 30, 2019 4:14 pm
Yeah, that's never going to happen because cable providers don't have the capacity for it. But if it did, it's just data, and the PRIME doesn't really care what's in it. It's just a bit more data than it would have sent before.
What do you anticipate they'll replace QAM with, WAM?

I know for data homes can use up to 32 bonded downstream channels for modems now, but I'm not sure if that has anything to do with TV.

I figured out I could use ffmpeg -i <recording> for getting specs on recordings. Here's one of the shows I record regularly:

Code: Select all

Input #0, mpegts, from 'The Rachel Maddow Show 20190729 [20190730-0100].mpg':
  Duration: 01:01:01.13, start: 34271.381289, bitrate: 3856 kb/s
  Program 4
    Stream #0:0[0x123f]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
    Stream #0:1[0x1240](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 384 kb/s
[truncated for brevity]
So apparently, yeah, 720p. I'm surprised to see that it's 60FPS, that seems kind of high.

Should I expect most content on this channel will be the same format?

Is it the local broadcaster or the network (e.g. NBC) that has control over the content format?

Thanks :D

jasonl
Silicondust
Posts: 14067
Joined: Sun Oct 28, 2007 9:23 pm

Re: What kind of hardware required for HDHR Recording computer?

Post by jasonl » Tue Jul 30, 2019 8:55 pm

DOCSIS 3.1 does away with the concept of channels and moves to OFDM subcarriers spread over a given bandwidth (similar to the move from 11b to 11g wifi), and also supports denser QAM constellations, up to 4096 (12 bits per symbol) instead of the 256 (8 bits per symbol) used for DOCSIS 3. Getting these benefits requires running 3 and 3.1 side by side until all the legacy modems are replaced, so that means more bandwidth has to be taken away from TV at least temporarily, and probably permanently so that when 3 is shut off they can reallocate that bandwidth to 3.1.

60fps is normal for 720p. 1080i runs at 60 fields per second.

Could be either the station or the cable provider converting to 720p. The NBC network is 1080i, but there are some affiliates who convert to 720p since it compresses better and lets them fit more services in their limited bandwidth with less noticeable degradation. Comcast is also known to convert almost everything to 720p H.264 so they can fit more channels in less bandwidth. They don't even have enough bandwidth for 1080i, so it's unlikely they're going to be able to come up with double that for 4K.

averyfreeman
Posts: 217
Joined: Sun Apr 09, 2017 11:12 am
Device ID: 1326E235 1313788A
Location: Olympia, WA
Contact:

Re: What kind of hardware required for HDHR Recording computer?

Post by averyfreeman » Wed Jul 31, 2019 2:28 pm

jasonl wrote:
Tue Jul 30, 2019 8:55 pm
DOCSIS 3.1 does away with the concept of channels and moves to OFDM subcarriers spread over a given bandwidth (similar to the move from 11b to 11g wifi), and also supports denser QAM constellations, up to 4096 (12 bits per symbol) instead of the 256 (8 bits per symbol) used for DOCSIS 3. Getting these benefits requires running 3 and 3.1 side by side until all the legacy modems are replaced, so that means more bandwidth has to be taken away from TV at least temporarily, and probably permanently so that when 3 is shut off they can reallocate that bandwidth to 3.1.
Interesting. I know I have a Docsis 3.1 compatible modem already (I think, it's a motorola mb8600) is there a timeline on this transition yet?
jasonl wrote:
Tue Jul 30, 2019 8:55 pm
60fps is normal for 720p. 1080i runs at 60 fields per second.
Now, is that because 1080I is interlaced? It has to run at 2x 30FPS?
jasonl wrote:
Tue Jul 30, 2019 8:55 pm
Could be either the station or the cable provider converting to 720p. The NBC network is 1080i, but there are some affiliates who convert to 720p since it compresses better and lets them fit more services in their limited bandwidth with less noticeable degradation. Comcast is also known to convert almost everything to 720p H.264 so they can fit more channels in less bandwidth. They don't even have enough bandwidth for 1080i, so it's unlikely they're going to be able to come up with double that for 4K.
Yeah I'll look at some other recordings, too. I'm guessing it's because I have comcast.

I'm in an awful area for DSL, which is ~4Mbps still, but Comcast started offering us gig speed cable internet a year and a half ago, so even though they're the only game in town like a lot of smaller cities, they're at least upgrading the backbone to some extent.

Maybe the change is coming sooner than I thought. I didn't think I'd have gig down speed now, either, honestly. I wish it were fiber, but in a town of 50K people I can't really complain.

averyfreeman
Posts: 217
Joined: Sun Apr 09, 2017 11:12 am
Device ID: 1326E235 1313788A
Location: Olympia, WA
Contact:

Re: What kind of hardware required for HDHR Recording computer?

Post by averyfreeman » Wed Jul 31, 2019 4:14 pm

Noticed one of my recordings was 5.8G for a 1hr recording :shock:

ffmpeg -i :

Code: Select all

Input #0, mpegts, from 'The Late Show With Stephen Colbert S04E184 20190729 [20190730-0635].mpg':
  Duration: 01:03:00.28, start: 84956.744733, bitrate: 13262 kb/s
  Program 2
    Stream #0:0[0x87d]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x87e](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Stream #0:2[0x87f](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 128 kb/s
So 30FPS but not h.264 and it's 1080p (am I reading that right?)

So would a 4K recording be 4x this size? :?

The smallest file was 1.0GB for a 1hr recording,

Code: Select all

Input #0, mpegts, from 'CIA Declassified S01E03 20140213 [20190124-1600].mpg':
  Duration: 01:01:00.20, start: 61458.153933, bitrate: 2218 kb/s
  Program 9
    Stream #0:0[0x118f]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 528x480 [SAR 40:33 DAR 4:3], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x1190](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 128 kb/s
    Stream #0:2[0x1191](spa): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 96 kb/s
Quick tip for nix users (I'm on FreeBSD 12), I found this by invoking this command in my recordings directory:

Code: Select all

$ find . -type f -exec du -ah {} + | sort -n  
It finds files, displays how much disk space they use, and then sorts them in descending order (can put an r after the n at the end to reverse that and get smallest files last).

jasonl
Silicondust
Posts: 14067
Joined: Sun Oct 28, 2007 9:23 pm

Re: What kind of hardware required for HDHR Recording computer?

Post by jasonl » Wed Jul 31, 2019 7:35 pm

It's up to each provider to decide when they want to start offering 3.1 service, and again when they want to discontinue legacy service. As I understand it, Comcast is done with their rollout of 3.1, and Charter and Cox are close if not complete. Basically, if you have gigabit as an option to you without needing fiber, you have DOCSIS 3.1 in your area. I don't know if it's actually available to every customer, or they've just reached as far as they care to go, and some people will just never get it because it's not worth the expense of upgrading. In order to shut down legacy service, they have to replace all legacy equipment, which is obviously a pain. I haven't seen anyone announce they're planning to do it.

Progressive TV delivers a full frame every 1/60th of a second (in the US system; it's 1/50th in most of the rest of the world). Interlaced TV delivers a field (half a frame, either the even lines or the odd lines), every 1/60th of a second. The fields are offset from each other in both time and space. This was done both to save bandwidth and due to limits on what old CRTs were capable of. Since they're offset in space, any areas of the picture that aren't moving effectively get the full resolution (on a completely still shot, you get the full 1920x1080). In areas with motion, you end up with half resolution (1920x540), but this isn't as bad as it sounds since human vision can't see as much detail in a moving area anyway, and a good quality deinterlacer can reconstruct some of that missing data to improve the perceived quality. The two fields are considered to be part of the same frame, so there are 30 frames per second, but each half of the frame comes from a different point in time.

That recording is 1080i, not 1080p. If the 4K was MPEG2, it would end up 8x that size if the same number of bits per pixel were used. However, 4K is usually encoded with HEVC, which is roughly 4X as efficient as MPEG2, so it would be more like twice that size, though it depends on the encoding settings. Some of the more important improvements in AVC and HEVC were improving the way the picture degrades. When you starve MPEG2, the picture ends up noticeably degraded, usually with lots of visible blocks on the screen. The newer codecs will eliminate detail on the screen while preserving the overall picture.

averyfreeman
Posts: 217
Joined: Sun Apr 09, 2017 11:12 am
Device ID: 1326E235 1313788A
Location: Olympia, WA
Contact:

Re: What kind of hardware required for HDHR Recording computer?

Post by averyfreeman » Fri Aug 02, 2019 4:15 pm

jasonl wrote:
Wed Jul 31, 2019 7:35 pm
Basically, if you have gigabit as an option to you without needing fiber, you have DOCSIS 3.1 in your area. I don't know if it's actually available to every customer, or they've just reached as far as they care to go, and some people will just never get it because it's not worth the expense of upgrading. In order to shut down legacy service, they have to replace all legacy equipment, which is obviously a pain. I haven't seen anyone announce they're planning to do it.
That's weird, I thought I had read that 3.0 is good for up to gig speed and 3.1 is required for anything higher than that.

My modem says its 32 downstream channels are all QAM256 so that's still 3.0 right? My real-world speed tests using speedtest.py on the gateway have been 640-720 Mbps, with or without dual-port LACP from the modem.

The standard rental modem in my area is the Arris TG3482G, according to wikidevi it's DOCSIS 3.1, so I suppose if they wanted to upgrade the backbone it looks like they already have most their userbase compatible with it.

Would be a lot easier if they wouldn't force people to enter contracts for VOIP to get other subscription features, because I know a lot of people keep old eMTA modems just to keep their phones working (self included).
jasonl wrote:
Wed Jul 31, 2019 7:35 pm
Progressive TV delivers a full frame every 1/60th of a second . . . The two fields are considered to be part of the same frame, so there are 30 frames per second, but each half of the frame comes from a different point in time.
Do I understand this correctly in that ffmpeg is identifying the recording as being 60 FPS, but since it's a progressive-scan broadcast it is only half the data for each frame?

Is it fairly safe to assume the 60 FPS recordings are progressive-scan and the 30 FPS ones are non-interlaced? Sorry if I got that all wrong, it's pretty new terminology for me.
jasonl wrote:
Wed Jul 31, 2019 7:35 pm
That recording is 1080i, not 1080p. If the 4K was MPEG2, it would end up 8x that size if the same number of bits per pixel were used. However, 4K is usually encoded with HEVC, which is roughly 4X as efficient as MPEG2, so it would be more like twice that size, though it depends on the encoding settings. Some of the more important improvements in AVC and HEVC were improving the way the picture degrades. When you starve MPEG2, the picture ends up noticeably degraded, usually with lots of visible blocks on the screen. The newer codecs will eliminate detail on the screen while preserving the overall picture.
Good to know a 4K recording wouldn't be 8x the size. I mean, I have plenty of space to work with, but I just don't need recordings taking up that much room.

On a practical level, what happens if I record something that's broadcast in 4K - will I be able to view it even without a video card capable of 4K, on a 1080P televison? Do I need to see if I can find HEVC codecs for FreeBSD?

Post Reply