Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New IR decoding capabilities and revamped MCE Vista + IR handling. #409

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

kdschlosser
Copy link
Member

No description provided.

There are 147 decoders and they all function properly. I still have to finish up the encoding side of things and also the code storage.
There are a slew of tweaks that are going to ned to be done to get everything to work properly.
Copy link

@codeclimate codeclimate bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 36485 lines exceeds the maximum allowed for the inline comments feature.

I made the tolerance operate in a more variable way. If a protocol has a lead in and a start bit I find out the percentage difference between the 2 and if that percentage is < 50 I then add in the user set tolerance. If the user set tolerance and the calculated one is < 40 i use that. if not I add the calculated tolerance and the user tolerance / 2. I only use this calculated tolerance for checking the data part of the code and also the lead out. the lead in is checked using the tighter user supplied tolerance.
Copy link

@codeclimate codeclimate bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 36547 lines exceeds the maximum allowed for the inline comments feature.

Copy link

@codeclimate codeclimate bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 36560 lines exceeds the maximum allowed for the inline comments feature.

@AppVeyorBot
Copy link

WIP-2020.07.29-04.44.20 failed (commit 07dfd49436)
Artifacts:

…ce, repeat code timing

I also improved upon the repeat code speed and duplicate code speed. I now have it down to 500 millionths of a second (1/2 a millisecond!!). There is no way to improve upon the new code protocol detection which is currently running at about 9 milliseconds. This is because it is a process of hunt and peck in order to find the proper decoder. That is really the only way that it can be done.

The MCE receiver timings are a HUGE pain because of how sloppy they are allowed to be. This is going to make it impossible to decode some of the protocols without disabling all of the other first. I believe I can make this process easier for the user if they use the database I am going to set up. If the code's friendly name does not match then they know that the remote is not going to be that protocol and thus can disable it. I am also going to attempt to institute a priority setting for the protocols once I manage to collect more information.

I did also add 2 new features. The user is now able to get the repeat count of a code and also the number of times a code has been used.

I created a new problem that I have to sort out. This deals with the saved named of a code.
Copy link

@codeclimate codeclimate bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 36768 lines exceeds the maximum allowed for the inline comments feature.

Corrected a starting issue with the plugin. I forgot to remove a file write from the ioctrl.py file.
Cleans up the code_wrapper.CodeWrapper.__init__ method.
Fixes the name for an IRCode when decoded by the MCE decoder
Changes the order of the decoders.
Adds some additional fine tuned tolerances.
fixes the naming of some of the properties in the ir_code.IRCode class
corrects issue with xml_handler and text fields when the field is multi line.
Copy link

@codeclimate codeclimate bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 72267 lines exceeds the maximum allowed for the inline comments feature.

Corrects in proper handling of unicode characters.
Copy link

@codeclimate codeclimate bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 72277 lines exceeds the maximum allowed for the inline comments feature.

@codeclimate
Copy link

codeclimate bot commented Aug 4, 2020

Code Climate has analyzed commit 137f9ae and detected 126 issues on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 87
Duplication 37
Bug Risk 2

View more on Code Climate.

@Crowley007
Copy link

[WIP-2020.08.04-08.13.25 completed]

Still has to be run as admin when Installing the plugin. After that, it seems to work without elevation also. But if you open up the plugin settings this error pops up:

6:59:51 AM Exception in thread Thread-6: 6:59:51 AM Traceback (most recent call last): 6:59:51 AM File "threading.pyc", line 801, in __bootstrap_inner 6:59:51 AM File "threading.pyc", line 754, in run 6:59:51 AM File "C:\Program Files (x86)\EventGhost\eg\WinApi\ir_class_ioctl\ioctl.py", line 1039, in __receive_loop 6:59:51 AM _io_control(IOCTL_IR_EXIT_PRIORITY_RECEIVE, hDevice, NULL, NULL) 6:59:51 AM File "C:\Program Files (x86)\EventGhost\eg\WinApi\ir_class_ioctl\ioctl.py", line 665, in _io_control 6:59:51 AM raise ctypes.WinError(err) 6:59:51 AM WindowsError: [Error 22] The device does not recognize the command.

Can't say how well it works otherwise, I don't get any events in the log when pressing buttons. Cursor seems to move with navigation keys, but not very well.

@kdschlosser
Copy link
Member Author

kdschlosser commented Aug 11, 2020

@Crowley007

what is the manufacturers name and the model number of your receiver?
do you know the protocol that your remote control is using? if not what is the brand of remote?

Cursor seems to move with navigation keys, but not very well.

What do you mean by this statement?

I need you to be testing this plugin without anything else installed into EG. so create a new file and save it then close EG and start it back up. Then add the plugin. Also make 100% sure that you have stopped and also disabled the alternatemceir service if it is installed. This needs to be done before installing the plugin.

I am not sure why your receiver is not allowing the IOCTL_IR_EXIT_PRIORITY_RECEIVE command It accepted the IOCTL_IR_ENTER_PRIORITY_RECEIVE command. or maybe it didn't.. I have to look at the code again and see if maybe I have some kind of error catching in place.

@Crowley007
Copy link

Crowley007 commented Aug 11, 2020

I have HP IR receiver:
https://www.amazon.com/HP-5188-1667-Center-Remote-Transmitter/dp/B000Z7XJME
Like that.

And I use Logitech Harmony to send MCE IR codes. Device name there is "Media Center PC SE"

There is nothing else on my testing tree, and the service is stopped an disabled while testing. The "selection/cursor" moves but seems to skip several lines when I press navigation/arrow key and seems delayed. But I do not get any events on the eg log.

In theory, I can send other protocols with harmony as well, if something else would work better than MCE. I would just need to know the device make/model to use. Or does the plugin only read mce compatible ir commands?

Edit: Just as a test, I tried adding the HP IR receiver to harmony directly, and it worked. Now the navigation buttons work better. I still do not get any events though, but this might be a better device to use with Harmony.

@gibman
Copy link

gibman commented Aug 11, 2020

WIP-2020.08.04-08.13.25 completed (commit f08adde)
Artifacts:
Build.log
CHANGELOG.md
CHANGELOG_THIS_RELEASE.bb
CHANGELOG_THIS_RELEASE.md
EventGhost_WIP-2020.08.04-08.13.25_Setup.exe

running win10 64 bit version 2004, latest.
mce usb receiver + harmony hub remote
empty EG tree.
MCE transceiver is listed in the MCE trans dropdown. Just nice :)
But clicking OK results in the exception below

10:20:58 PM Exception in thread Thread-40:
10:20:58 PM Traceback (most recent call last):
10:20:58 PM File "threading.pyc", line 801, in __bootstrap_inner
10:20:58 PM File "threading.pyc", line 754, in run
10:20:58 PM File "C:\Program Files (x86)\EventGhost\eg\WinApi\ir_class_ioctl\ioctl.py", line 910, in __receive_loop
10:20:58 PM NULL
10:20:58 PM File "C:\Program Files (x86)\EventGhost\eg\WinApi\ir_class_ioctl\ioctl.py", line 665, in _io_control
10:20:58 PM raise ctypes.WinError(err)
10:20:58 PM WindowsError: [Error 22] The device does not recognize the command.
10:20:58 PM

@Crowley007
Copy link

Crowley007 commented Aug 13, 2020

I did some more testing, I missed the part where you said you have to disable the old service BEFORE installing the plugin. I do now get events in the log.

Edit: Now that I have it working, it really does seem to work better, no unknown codes. Have to do some more testing later.

I still needed to start EG as admin or the receiver wont show. And when I tried to start it normally, it stopped working, and now will not work even as admin I just get this:

Exception in thread Thread-6: 3:33:27 PM Traceback (most recent call last): 3:33:27 PM File "threading.pyc", line 801, in __bootstrap_inner 3:33:27 PM File "threading.pyc", line 754, in run 3:33:27 PM File "C:\Program Files (x86)\EventGhost\eg\WinApi\ir_class_ioctl\ioctl.py", line 1039, in __receive_loop 3:33:27 PM _io_control(IOCTL_IR_EXIT_PRIORITY_RECEIVE, hDevice, NULL, NULL) 3:33:27 PM File "C:\Program Files (x86)\EventGhost\eg\WinApi\ir_class_ioctl\ioctl.py", line 665, in _io_control 3:33:27 PM raise ctypes.WinError(err) 3:33:27 PM WindowsError: [Error 22] The device does not recognize the command.

I have to remove and reinstall the plugin as admin to get it working again.

@kdschlosser
Copy link
Member Author

@Crowley007

Thanks for the diligent testing you have done thus far. you are the only person other then myself that has gotten it working. I am not sure how else to tell people how to install the thing and for them to follow that direction. You found step you skipped and corrected how to install it and now it runs.

There are going to be bugs in it, it is going to take some time to locate them all. This update is some 25K lines of code so it's massive. But there is nothing else that provides real time decoding for as many protocols and also is able to decode the raw IR stream as fast as a half a millisecond. And it does this while maintaining an high decode accuracy. It also properly handles repeat codes for all 162 protocols. It also take into consideration variations of a protocol that may exist because of odd learning that has been done. An example is the denon protocol.

This is the bit layout for the stream.
D:5,F:8,0:2,1,-165,D:5,~F:8,3:2,1,-165

Now look carefully.
D:5, F:8,0:2,1,-165,
D:5,~F:8,3:2,1,-165

The single stream can actually get learned incorrectly because of the -165 lead out. so a learned code may only be the first portion of the entire code. Now in the strictest terms the code should fail to be decoded. because of the missing back half. well I am a tricky one and what I did was handle the back half as a repeat code. so the front half I use as the "code" and I start a repeat timer if the back half does not come in before that timeout expires I know that the code an incomplete learned code and I still generate an event for it, I build the back half of the code so there is now a complete code available to the user if they want to transmit it.

With the denon protocol the back half is a checksum. see the ~ before the F. that means that F:8 from the first half needs to have all of it's bits logically flipped. so if the binary for F was 10101010. then flipping it would be 01010101. and that flipped set of bits would need to match ~F on the back half in order to know that the code was received without error.

I did also want to mention that the Universal "Unknown" decoder is turned off by default. with supporting 162 protocols it should only need to be turned on for special use cases. This is because an invalid checksum is actually a decode error and cannot be handled in a special way. This is because of how close some of the protocols are to each other in their format. I need to treat a checksum failure in a manner that will let the program continue to try and decode it by other protocols. So if there is an actual checksum failure we don't want to do anything and let the code simply fall off. If the Universal decoder is enabled then the code will not fall off it ends up getting normalized and then it gets analysed to determine the type of bit encoding that is sued in the protocol. There are a few types of bit encodins (modulation), you have pulse distance, pulse width and bi phase, then there are different variations of the bit encodings. With bi phase you have space as the leading edge or you have a mark as the leading edge. with the other 2 you can have a mark and a space to define 0 and a mark and a space to define 1. this would represent a 1 or a 0 in binary. some protocols will define 4 pairs of timings. so a single "burst pair" in the IR stream can actuallt decode to 2 bits instead of one. It took me a while to get this mentally handled if ya catch my drift.

@Crowley007
Copy link

Crowley007 commented Aug 14, 2020

Seems like a massive undertaking, thanks for the effort. The potential really is there, when it works, it works really well.

About the service, perhaps you should add a check when installing the plugin. ;)

@kdschlosser
Copy link
Member Author

@Crowley007

About the service, perhaps you should add a check when installing the plugin. ;)

I am thinking the same thing. I knew the 2 would bump heads, I didn't know it would cause this much headache.

Now that I am thinking about it I am using low level hardware calls to get the information I want from the device. So what I believe is happening is the service is accessing the device and then the plugin does. This causes an error to happen. In the plugin I have it ignore the error and then try the last command again. where as the service tells the device to completely exit receiving and then tries to start the whole process over again. So now the plugin attempts to use a command it's not supposed to use because that "mode" has been exited by the service. The device doesn't have a clue where the commands are coming from it just carries them out.

So now the "unknown command" fits better. The error could have been been a tad more descriptive like. "incorrect mode to carry out command". that would fit better.

@Crowley007
Copy link

To be clear, the error above happens when you restart eg, even if the old service is disabled.

@kdschlosser
Copy link
Member Author

@Crowley007
Can you do me a favor and replicate the error again except this time before you start EG open up the Task Manager and make sure that the EventGhost process has terminated before you start EG up again. If you can also approximate the length of time it takes for the process to end and let me know what it is.

I am wondering if it is a bonking of heads if you either use the restart from inside of EG and also if there is an issue with the thread for the receiving not exiting properly. If you find the issue only happens when you use the restart I am going to have to code the restart in EG to close all of the plugins then start the new process to pass in the restart parameter. This way they do not bump heads. It is a complicated way that EG restarts. because of how command line parameters need to be passed to an already existing session. It's actually kind of batty how I had to set the thing up. I really do wish I could get the thing working correctly with named pipes. I have alot more knowledge with respect to Python and the Windows API then I had 2 years ago. I might have another crack at it to see if I can get it working. It's really a pain because of UAC and RunAs.

@Crowley007
Copy link

Crowley007 commented Aug 15, 2020

Ok, the problem is indeed with the eg menu restart option. If I restart manually, it will keep working. When you close eg down, it does not stay open for long according to task manager.

But eg still needs to be run as admin or it will not see the IR Receiver.

Another thing I noticed, I can't get the options dialog to open:

2:54:36 PM ---> Welcome to EventGhost <---
2:54:38 PM Traceback (most recent call last) (WIP-2020.08.04-08.13.25):
2:54:38 PM File "C:\Program Files (x86)\EventGhost\eg\Classes\TaskletDialog.py", line 162, in ProcessingTask
2:54:38 PM self.Configure(*args, **kwargs)
2:54:38 PM File "C:\Program Files (x86)\EventGhost\eg\Classes\OptionsDialog.py", line 198, in Configure
2:54:38 PM languageChoice.SetSelection(languageList.index(config.language))
2:54:38 PM ValueError: 'en_EN' is not in list

My Windows Display language is English, but location and keyboard is Finnish. But it wont open with keyboard set to English either. But I guess you still have more important things to look at at this point.

@kdschlosser
Copy link
Member Author

@Crowley007

So my suspicions are confirmed about the restart inside of EG. I am going to have to fix that in the core code. It can also cause the same kind of an issue with other plugins. TY for testing that out for me.

I am not sure what is up with the whole administrator thing I have to do quite a bit more digging into that one to see what is up with having to be an administrator. If it's not even listing the device then it has something to do with one of the functions I am using to collect information about the attached receivers. There may be a different way to go about it then what I am doing and that way might be permissions friendly.

The options dialog problem is a known problem and has been fixed. I probably forgot to merge the master branch into my local copy before I created the branch for this PR. That is my fault and I am sorry for that.

As a side note. I have started tinkering about with the ESP32 micro controllers to be used as an alternative to the MCE Receivers . This has many benefits.

WiFi
runs Python
super fast
1us accuracy for IR codes (MCE is 50us).
Super cheap. as low as 5.00 USD for the micro controller and another 2.00USD for the IR receiver and an amplified IR transmitter.

I will be creating a plugin for using these things that will take care of flashing the firmware and uploading needed files to it to get it up and running. I will also provide direction for connecting the blaster(s) and receiver(s) to it. It will be plug and play.

Now that I have spent as much time as I have dealing with the MCE receivers I am really starting to dislike them. They have to high an error margin and the API is a complete nightmare. If I can't get the thing to attach to the receiver without needing to be logged in as administrator I am going to have to update the service so it doesn't create ascii characters from the RLC. It should have been written so it just streams data as it comes in converting the integers to numeric text. I am not sure what the thought was in converting the bursts to ascii was, that process is almost guaranteed to have problems at some point. and the flooding of the Windows Log is not necessary either.

@Crowley007
Copy link

It makes me wonder if the old service wasn't originally made to solve this problem.

But yeah, Windows Media Center hasn't been a thing in years and the related software/apis' probably haven't seen any development for much longer. At some point you probably can't find any of these devices outside of china and second hand sellers, so a better alternative would definitely be interesting.

Going seriously off topic here, but would the device handle long IR receiver cable (10m / 30 ft) or multiple blasters well? I could see replacing my old IR extender/repeater with something like this as well.

@kdschlosser
Copy link
Member Author

With how the new MCE plugin is coded (and if i can get past the administrative privileges thing) it now supports more then 1 MCE receiver plugged into the same PC. Why that is important... You can go out and pick up a Rasperry Pi B+ and install a program in it called VirtualHere. Then install VirtualHere on the machine that is running EG and connect the Raspberry Pi and the PC together. What this does is USB over IP. so anything you plug into the Raspberry Pi is going to get attached to the PC as if you were plugging the device directly into the PC.

The thing with USB is there is a distance limitation of 3 meters (15 feet), This is a theoretical limitation and there could be variances in the actual distance depending on cable shielding and resistance, and also nearby EMI. So with only having 15 feet to play with for distance it does not make it a viable solution.

While the Raspberry Pi will work it also is not a viable solution to me because of the cost.
MCE Receiver = 20.00USD
Raspberry Pi B+ = 30.00USD
USB WiFi for the Raspberry = 10.00USD

I do not consider 60.00USD +Shipping to be a low cost solution. It is a solution but not one that I consider to be apart of the "DIY" Home automation which is what EG is all about.

now a price under 10.00 USD.. that's what I am talking about!
and on top of it the footprint is small.. about the size of a cigarette lighter if using the development board. and if using the chip directly it is as small as a CR 2032 Battery.

Not to mention there are other things we can do with the ESP at the same time as the IR. Temperature, humidity, light, sound and motion just to name a few. A HUGE thing that can be done since there are some TV's that don't have a way to handle whether they are on or off is if the TV has a USB port 9 times out of 10 that port will get powered on when the TV does. we can use that power signal from the USB as a trigger to know when the TV is on.

@Crowley007
Copy link

Sounds very interesting indeed.

@kdschlosser
Copy link
Member Author

This evening I am going to put some time into working on the Windows API binding of the code to see if I can locate the specific piece that is causing the permissions issue.

@kdschlosser
Copy link
Member Author

I am not so sure about the service being used as a solution for the permissions problem. I have to check WinLiRC and see if it requires administrative privileges

@gibman
Copy link

gibman commented Sep 25, 2020

lately been running build : EventGhost_WIP-2020.08.04-08.13.25_Setup.exe

it seems to generally work just nice.
I do see the above mentioned issue as well when restarting E.G from within itself.

But another issue I see once in a while is when coming out of standby then no remote control events are being registered,
Clicking on the config/options for the plugin shows an empty dropdown list (meaning no MCE receivers found) - this cant be good. The same symptoms I see when restarting E.G from within itself btw.

when this problem occurs with resume from standby I just fix this by restarting E.G. manually.

@kdschlosser
Copy link
Member Author

Holy COW! someone that is running it and not having all kinds of cooky problems!!!.. (besides myself)..

I am aware of the waking from sleep issue and this is easily handle I simply have not fixed it yet.

Do you have the administrative privileges issue at all?? meaning you have to run EG as administrator to get the thing to work.

@gibman
Copy link

gibman commented Sep 26, 2020

I just tested this now.
closing E.G.. then double clicking on the desktop icon (non admin mode)

it says:
7:48:13 PM MCE IR: Unable to locate MCE receiver "eHome Infrared Receiver (USBCIR)"
and remote obviously does not work.

running as admin works though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants