ATSC 3.0 audio upgrade - DSperber

ATSC 3.0 Forum
Post Reply
DSperber
Posts: 163
Joined: Wed Dec 15, 2021 3:23 am
x 9

Re: ATSC 3.0 audio upgrade - DSperber

Post by DSperber »

VERY BIG BREAKTHROUGH!! HUGE!!! Zeroing in on what it seems to be... not in Windows itself, but rather in HDHomeRun RECORD service running on Windows.

Tonight I decided to try and rearrange the cabling in my LAN, so that everything must pass through the router itself. Previously there was only one cable connecting the router to switch #1, and everything else connected to switch #1 either directly or through secondary switches. So the router wasn't really directly involved with inter-device communication (which was handled by the switches) once the IP address for any given device had been assigned by the router. Also, the router provided a gateway to the connected modem and hence to the internet. But within the LAN itself everything was handled by just the switches.

I really didn't have any problem with this arrangement, and I thought it made for a more efficient inter-device communication with the router not present in inter-device communication pathway. I figured one less device for traffic to traverse can't be a bad thing. And everything worked perfectly. Until this multicast issue with the LG app, and its bizarre symptom of working properly on the first launch after a Windows boot, but failing for the second and subsequent launches. So tonight I decided to see if actually having everything pass through the router when going from every device to any other device (almost) might make a difference. Perhaps the router needed to be in the middle of things if multicast is going on, which it wasn't before.

So I changed things (in the above schematic) so that switches #1, #2 and #3 now go to ports 1-3 of the router. And Win7/10 HTPC that hosts HDHR DVR now goes to port 4 of the router. Switch #4 is still connected to switch #3. So all of the relevant devices in the above schematic now actually have to pass through the router when going from device A to device B. Hence if the router must be involved in multicast between LG TV and HDHR DVR, it now physically is whereas before it schematically wasn't.

And the results? NO CHANGE. I accomplished absolutely nothing. Yes, everything still works fine, but just as it did before regarding the LG app and its ability to find the DVR device only on the first launch after a Windows boot. So I accomplished nothing. And being forced to reboot the HTPC every time I wanted to play a recording clearly makes no sense. I couldn't live with this "solution".

What, then, you ask, is the HUGE BREAKTHROUGH???

Well, not being at all pleased with having accomplished nothing I decided to try yet one more "who knows if it will make any difference" gimmick. I decided to STOP the HDHomeRun RECORD service running on the HTPC, and then START it again. No idea what if anything that might cause, good or bad, helpful or not, but I gave it a try.

And... ASTONISHINGLY... after STOP/START of the HDHomeRun RECORD service sure enough launching the LG app once again PRODUCED TWO DEVICES and RECORD/DVR functionality!!! I didn't have to re-boot Windows to achieve that. I only needed to STOP/START the HDHomeRun RECORD service.

I repeated this umpteen times, just to be sure. 100% repeatable. Boot Windows, launch LG app, works fine. Power off the TV, power on the TV, re-launch LG app a second time, fails (i.e. one device). Power off the TV, STOP the HDHomeRun RECORD service on the HTPC, START the service, power on the TV, re-launch LG app again, and WORKS FINE (i.e. two devices).

==> Whatever is going on inside of HDHomeRun RECORD service, it appears to get initialized when the service starts... either because of a Windows reboot or because I've manually STOPPed and STARTed the service. Either of these sequences clears things out and/or initializes whatever table or data it needs to initialize. And now, the very first time the LG app does its multicast to try and find the DVR server (or whatever it's trying to accomplish when launching), it is successful with the initialized empty table inside of HDHomeRun RECORD service. And then let's say one table entry gets populated. Apparently the now non-empty table, with one table entry populated, this seems to be the cause of the failure on the next launch of the LG app to do whatever it had successfully done on the first launch (with the empty table in RECORD service).

Look. I don't really know what's going on here. So I'm making up a story. But all I can tell you is that I can now "force success manually" not by re-booting Windows (which is actually another way to accomplish success again, because of the obvious reinitializing of that table the first time the service runs after boot or STOP/START) but just by manually STOP'ing and START'ing the RECORD service.

So, we continue to get closer to the real culprit here. Not BitDefender. Not Win7/Win10. Not even some multi-cast table in Windows. But apparently something inside of RECORD service itself that performs properly when run for the very first time after the service is started... either post-boot, of following manual STOP/START.

What else can I say? I believe this is a detailed description of what has to be considered a BUG in RECORD service code. Something is obviously broken in RECORD service code. It is obviously not working properly, due to whatever it populates in whatever tables as a result of the first handshake with the LG app when all tables are initialized and empty. On subsequent handshakes something different is clearly happening, no doubt as a result of whatever got stored in whatever tables as a result of the first handshake and what data got remembered in whatever tables. It clearly isn't working properly and it is repeatable. Should be easy to chase down and find this bug.

Can you say that you don't see this behavior in your world? You ALWAYS see the DVR on your LG app tests, no matter how many times in a row you launch it (after powering off the TV and then powering it back on). Or can you truly duplicate my results now that I've precisely defined the failing scenario, and that the second and subsequent launch after boot (or STOP/START) will fail even though the first launch was successful.

Even if you can't duplicate my results, just use my configuration as the laboratory. Code some debug trace logic to diagnose what that particular routine is doing and put out some log output you can examine. I will run that debug version of RECORD service, send the diagnostic output to you, and we can make some progress in getting this fixed. First time with empty tables, works perfectly, and populates table entries. Second time with non-empty tables, broken.

No question in my mind the culprit is HDHomeRun RECORD in terms of how it handshakes with LG app and deals with multicast.

Any feedback?

Online
nickk
Silicondust
Posts: 18222
Joined: Tue Jan 13, 2004 9:39 am
x 166

Re: ATSC 3.0 audio upgrade - DSperber

Post by nickk »

The record engine doesn't announce anything, just listens and responds to discover requests.

If restarting the record engine fixes the problem it suggests that the multicast group registration might be timing out somehow either by Windows or by a IGMP aware network switch.

I will try to reproduce here.

DSperber
Posts: 163
Joined: Wed Dec 15, 2021 3:23 am
x 9

Re: ATSC 3.0 audio upgrade - DSperber

Post by DSperber »

nickk wrote: Sun Jan 16, 2022 9:57 am The record engine doesn't announce anything, just listens and responds to discover requests.

If restarting the record engine fixes the problem it suggests that the multicast group registration might be timing out somehow either by Windows or by a IGMP aware network switch.

I will try to reproduce here.
I only have unmanaged switches: Netgear GS108 and Netgear GS105. Nothing "smart" or 'managed" or IGMP-aware.

And besides, I would bet I can demonstrate this failure with no switches involved at all if that's what it takes to convince you. I could run a test with just the LG TV and Windows PC plugged directly into two ports of the router, with no switches involved whatsoever. That would be an interesting test and would also totally eliminate the relevance of network switches in any way. Only the RECORD engine effectively directly connected to the LG TV. If it still fails will you believe that there's a bug? I will set this test up, just to confirm. A bit of work, because I also will have to relocate the Flex 4K from where it is to down next to the router, plugging it into a third port.

Also, "timing out" doesn't make sense. You can cause the problem within 10 seconds, just by powering on the TV, launching LG app successfully (i.e. getting DVR functionality), power off the TV, then power on the TV and re-launch the LG app a second time... this time with no DVR functionality, all happening within under a minute.

Does not the RECORD engine "remember" anything about the client app that just connected to it? Is there a table or information about IP address? Perhaps it's cleared out when started (i.e. from Windows boot or the STOP/START sequence), then populated on that first connection, and because it is now non-empty somehow impacts what gets done for the second connection (from ANY client, not just the same one that was seen first)?

There is clearly some difference in behavior by the RECORD engine when it performs right after getting started, vs. how it performs after having already been "used" once to connect with a client. This is the logic that needs to be evaluated as the likely "culprit", in my opinion. That's just what my evidence points to.

DSperber
Posts: 163
Joined: Wed Dec 15, 2021 3:23 am
x 9

Re: ATSC 3.0 audio upgrade - DSperber

Post by DSperber »

Just one more piece of information, that perhaps might be relevant. I don't think it's relevant, but it's still information you might want to think about.

The Windows PC used is one of my two Windows Media Center HTPCs, and as such contains a PCIe Ceton InfiniTV 6-tuner card inside. It works as its own network gateway (192.168.200.1) and then each of the 6 tuners is accessed as 192.168.200.2:8000, 192.168.200.2:8001, 192.168.200.2:8002, etc.

Over on the NextPVR forum I was working with them to try and determine why their Android app running on my Samsung S21 Ultra phone was not able to connect with the NextPVR DVR at 192.168.1.26:8866 (which is the IP of the Windows PC), and we had done lots of data gathering. They use "broadcast", not "multicast".

Anyway I was asked to provide their own LOG output (running in "verbose" mode and generating additional diagnostic information) and this particular snippet is provided here just to show you that the separate Ceton 192.168.200.2 network shows up in the LOG. As it turns out that had no relevance to why the S21 was not connecting to their NextPVR server running on that machine, but it was at least interesting to observe.

So I'm providing just that little portion of their NextPVR DVR server LOG output that also shows the Ceton tuner "participating" in a response to the "broadcast" handling. It might be completely irrelevant, or it might be of interest.

Code: Select all

2021-12-29 09:49:06.184	[DEBUG][10]	Received broadcast from 192.168.1.18:58175 : DISCOVER_REQUEST
2021-12-29 09:49:06.184	[DEBUG][10]	 (fe80::a97a:6f51:6720:d3a3%14, 192.168.1.18)
2021-12-29 09:49:06.184	[DEBUG][10]	 (fe80::dd05:a7f:a5cc:33d8%51, 192.168.1.18)
2021-12-29 09:49:06.184	[DEBUG][10]	 (192.168.1.26, 192.168.1.18)
2021-12-29 09:49:06.184	[DEBUG][10]	 (192.168.200.2, 192.168.1.18)
2021-12-29 09:49:06.184	[DEBUG][10]	address: 192.168.1.26
2021-12-29 09:49:06.199	[DEBUG][10]	Waiting for broadcast
2021-12-29 09:49:16.870	[DEBUG][30]	Got request [192.168.1.18]: /services/service (session.initiate)
2021-12-29 09:49:16.885	[DEBUG][30]	method=session.initiate
2021-12-29 09:49:16.885	[DEBUG][30]	parameters: 
2021-12-29 09:49:16.885	[DEBUG][30]	   method: session.initiate
2021-12-29 09:49:16.885	[DEBUG][30]	   ver: 1.0
2021-12-29 09:49:16.885	[DEBUG][30]	   device: android
2021-12-29 09:49:16.885	[DEBUG][30]	   client_ip: 192.168.1.18
2021-12-29 09:49:16.885	[DEBUG][30]	   user_agent: Dalvik/2.1.0 (Linux; U; Android 12; SM-G998U Build/SP1A.210812.016)
2021-12-29 09:49:16.885	[DEBUG][30]	   host_callback: ...
2021-12-29 09:49:16.885	[DEBUG][30]	   sid: default
2021-12-29 09:49:16.885	[INFO][30]	InitiateSession (device=android)
2021-12-29 09:49:16.885	[DEBUG][30]	SetSessionObject(a59cbb86146a4323b00d241b11d890a9, 'child', NON-null)

DSperber
Posts: 163
Joined: Wed Dec 15, 2021 3:23 am
x 9

Re: ATSC 3.0 audio upgrade - DSperber

Post by DSperber »

Also, I believe I mentioned that both of my LG TVs are connected to my network via ethernet cable, not Wifi... if that is of any relevance to "multicast" handling by RECORD.

But I believe you said it was NOT relevant.

Online
nickk
Silicondust
Posts: 18222
Joined: Tue Jan 13, 2004 9:39 am
x 166

Re: ATSC 3.0 audio upgrade - DSperber

Post by nickk »

Test I am running:
LG TV and Windows 10 PC connected to the same unmanaged switch.
1) Start record engine on a Windows 10 PC.
2) Power on the TV and run HDHomeRun app. Verify the record engine is detected.
3) Power off the TV.
4) Power on the TV and run HDHomeRun app. Verify the record engine is detected.

This works every time.

Edit: test repeated with Windows 7 - works as well.

Do you see the same problem with both models of LG TV you have?

DSperber
Posts: 163
Joined: Wed Dec 15, 2021 3:23 am
x 9

Re: ATSC 3.0 audio upgrade - DSperber

Post by DSperber »

I've found three long ethernet cables, as well as another spare GS105 switch. So it wasn't too much trouble to put ALL THREE devices on the one switch: LG C7, Flex 4K, and Win10 PC with HDHR DVR RECORD service running on it. The switch then has a direct ethernet cable path to the router, i.e. it doesn't go through any other switches to reach the router.

Repeated my test, and it FAILED 100% of the time. First launch of LG client app shows TWO DEVICES as well as DVR functionality. Power the TV off, power the TV on, re-launch the LG client app -> ONE DEVICE and NO DVR functionality.

Then go to Windows SERVICES.MSC and STOP the HDHR RECORD service. Then START the service. Then repeat the LG client app test. And once again, first launch after STOP/START of the RECORD service once again shows TWO DEVICES as well as DVR functionality. Power the TV off, power the TV on, re-launch the LG client app -> ONE DEVICE and NO DVR functionality.

Fails every time. I didn't waste time trying this booted to Win7, since this Win10 test saw no change in earlier results when everything was moved onto one single switch.

And yes, it fails not only on this LG C7 but also on my other LG C9.

Again, all my evidence points not to a networking issue but rather to a problem inside of HDHR RECORD. There clearly is a group of variables and tables that that get initialized when the program gets started, either as a result of a Windows boot or as a result of STOP/START. And then the program enters "steady state" listening and waiting for clients to connect, be they from Android or iOS, or from LG, or from Windows, etc. When the LG client connects for the first time it is successful (proving that multicast seems to work fine on my configuration), obviously because all relevant variables and tables are still empty from initialization. And now, I assume RECORD stores some information (in variables and table entries) to keep track of the "conversation in progress" that it is having at this moment with the LG client.

Then, all of a sudden, since there is no "graceful EXIT" from the LG client app on the TV, there is a power off on the TV and the LG client app disappears instantly. RECORD which had been in a "conversation" with the client app is suddenly not talking to anyone. Perhaps it goes into an error recovery loop. Or maybe it is just waiting for the other end to return to life. Or, maybe it rightly decides to vaporize all history of that conversation which is suddenly no longer. Or maybe none of the above, but it's got to be in some state at this point having just had the ongoing conversation suddenly vanish.

Anyway, whatever state RECORD entered when the TV powered off, when the next connection request arrives from the second launch of the LG app RECORD no longer has initialized variables and empty tables. It has whatever it stored when the previous conversation was initiated. And it is in some state yet to be determined, that surely is not the same state it was in immediately post-startup when that very first knock on the door from the LG client wanting to connect was first fielded. I am of the opinion that it is this "residual data" and mystery state of the program, with what data was validly stored when the first handshake completed and also what data was left behind as residual data when the TV suddenly powered off and vanished. Things look quite different when the second connection handshake arrives and is supposed to be fielded by RECORD.

I am convinced this is where the problem lies, the combination of non-initialized variables and non-empty tables, along with RECORD entering an unknown state when the TV got powered off and could no longer receive the live TV data that RECORD was sending out. This combination of circumstances must be causing RECORD to not accept the second connection attempt from the LG client. There has to be some explanation for what I'm seeing.

Of course this is all just my theory. And we also have to factor in your experience, which is that none of my symptoms occur in your test. There must be some explanation for that 100% success in contrast to my 100% failure. But only when we find the bug will we be able to explain the two different results.

In the meantime I think you need to provide me with some kind of debug version of RECORD, that puts out some information you would find helpful. Or, maybe you connect to my machine (TeamViewer or UltraViewer or RealVNC, all of which I support, or maybe something else that you use) install Wireshark (which I do not know anything about) and under your control use it to watch what goes on during my two launch scenario.

There must be an explanation for my failures, and your non-failures.

Online
nickk
Silicondust
Posts: 18222
Joined: Tue Jan 13, 2004 9:39 am
x 166

Re: ATSC 3.0 audio upgrade - DSperber

Post by nickk »

The discovery process is stateless other than asking the OS to join the multicast group.

Does unplugging the Ceton tuner change the behavior at all?

Nick

DSperber
Posts: 163
Joined: Wed Dec 15, 2021 3:23 am
x 9

Re: ATSC 3.0 audio upgrade - DSperber

Post by DSperber »

nickk wrote: Mon Jan 17, 2022 10:42 am Does unplugging the Ceton tuner change the behavior at all?

Nick
Not really possible. It's a PCIe card inside the PC.

Is there a special testing version of RECORD that you have for development use, that puts out a verbose LOG or something that you can then analyze after a test of something you're chasing, in order to see the output of whatever special debug logic you've added?

DSperber
Posts: 163
Joined: Wed Dec 15, 2021 3:23 am
x 9

Re: ATSC 3.0 audio upgrade - DSperber

Post by DSperber »

I see that there is now A SECOND THREAD on the forum that has been posted and that you have responded to, that could be complaining of the same problem I'm having:

"I, too, have a new (2021) LG and am having problems with the HDHomeRun APP. I was hoping that the "fix" would solve my problems. I am unable to access any of the recorded material stored on my Synology NAS. Do I need to reinstall the APP?"

I assume he's still got the DVR software running on Windows but storing recording storage located on the NAS?

There's not much real detail in that post, but is he actually complaining that he doesn't have the DVR functionality on the LG app on his TV,? When he says "I am unable to access any recorded material" could he really be saying he really doesn't have a connection to the DVR RECORD engine, which is of course exactly the symptom we've been chasing on my issue?

Could he maybe just not be describing his problem in proper technical terms and with sufficient words? But could he, too, be experiencing what I'm experiencing (for all connections after the very first one after Windows boot):

Image

Online
nickk
Silicondust
Posts: 18222
Joined: Tue Jan 13, 2004 9:39 am
x 166

Re: ATSC 3.0 audio upgrade - DSperber

Post by nickk »

For the test you should be able to go into Device Manager and disable the Ceton from there.

Nick

DSperber
Posts: 163
Joined: Wed Dec 15, 2021 3:23 am
x 9

Re: ATSC 3.0 audio upgrade - DSperber

Post by DSperber »

nickk wrote: Mon Jan 17, 2022 4:12 pm For the test you should be able to go into Device Manager and disable the Ceton from there.

Nick
True enough. Actually I could for that matter just uninstall the drivers and supporting Ceton software for the test. A quick Macrium Reflect system image backup before that will only take a few minutes, and the restore after the test another few minutes. So I can burn WMC to the ground for the test, and not really impact the real world.

I'll give that a go, at least just to see if it makes any difference in how multicast and RECORD then functions.

ON THE OTHER HAND (he says, after thinking for a moment)... the test I ran earlier was in Win10, not Win7. So there is no WMC in Win10, and there are no drivers installed for Ceton in Win10. In fact the Ceton card is shown in Device Manager in "Other Devices" - > multimedia video controller, because it has no driver associated with it.

So it is in effect non-existent and certainly non-functional in Win10. So there's no way it could be interfering in any way with what RECORD is doing.

That "snippet" I'd posted earlier, which showed 192.168.200.2 in the feedback from "broadcast discover" was actually done when I was booted to Win7, where the Ceton card is truly installed so that WMC can use it. But in Win10, which is where last night's test was run, it is in effect non-existent.

So, back to the drawing board. What about a "debug verbose" version of RECORD, to produce some kind of LOG output or "send diagnostic output real time to SD" (much as NextPVR has built into it, to allow their developers to really see what's going on)? That way you'd have something tangible to look at. There has to be an explanation... and therefore a solution.

Online
nickk
Silicondust
Posts: 18222
Joined: Tue Jan 13, 2004 9:39 am
x 166

Re: ATSC 3.0 audio upgrade - DSperber

Post by nickk »

Simply disabling the device in Device Manager will work. Don't uninstall the drivers.

DSperber
Posts: 163
Joined: Wed Dec 15, 2021 3:23 am
x 9

Re: ATSC 3.0 audio upgrade - DSperber

Post by DSperber »

nickk wrote: Mon Jan 17, 2022 5:30 pm Simply disabling the device in Device Manager will work. Don't uninstall the drivers.
But that would only be for Win7 where it actually was an operating device.

The Win10 test doesn't even have the Ceton in Device Manager to disable! It's not installed with a driver, and doesn't need disabling. It's just an "other device" (yellow exclamation mark) that's in the machine but not in use.

So haven't I already run exactly the test you want me to perform... just in Win10 (like your setup) rather than in Win7 (where I would need to first disable the device)? Hasn't the test already been run (just in Win10) and concluded that it made no difference, since the failure occurs with RECORD operating on Win10 as well?


One more unrelated question. A Shield-related discussion on the AVS Forum regarding HDHR client app running on the Shield has uncovered the fact that one user is using a Flex 4K with the HDHR app running on his Roku Ultra 2020 (4800 model, which claims to have AC-4 capability built in, like the LG TVs do). Is this true? Do you know if it's true or not?

I was told by "Andrew" (of Silicon Dust located in Texas USA) when I was still pre-purchase for my Flex 4K and needed some technical questions answered that I should NOT use the Roku app for the Flex 4K. It apparently was "unfinished" (last touched in 2020?) and abandoned because the Roku had too little horsepower. Of course that might have been previous models of the Roku, and the Roku Ultra 2020 (4800) is a much stronger beast. And, may even truly contain built-in AC-4 decoding like the LG TV's do.

And that means theoretically it, too, could be enlisted by the HDHR app for Roku, to use the "Roku Media Player" to provide sound, similar to how the 'LG player" does it on the LG TV.

Can you please summarize the status of the HDHR client app for Roku, and in particular for the Roku Ultra 2020 (4800) model which is their latest and greatest piece of hardware that supposedly includes native AC-4 support. Should I try to use it? Is it workable? Stable? Usable? Does it deliver sound? Does it even support ATSC 3.0 channels in an acceptable way? Or not?

Thanks for any Roku Ultra insights you can provide.

Online
nickk
Silicondust
Posts: 18222
Joined: Tue Jan 13, 2004 9:39 am
x 166

Re: ATSC 3.0 audio upgrade - DSperber

Post by nickk »

Roku - we have confirmed AC4 audio support on the latest model Roku Ultra. I don't know yet about older Ultra models.

Ceton disable test - I didn't realize it wasn't running with Win10. No need to run that test.

If you run "ipconfig" from a cmd prompt does it show a single Ethernet interface in Win10?

Post Reply