Port config request.

[MattWestb]

I can see in the code that parameter p = TCP port and can being overridden then loading the application.
I like trying using the /dev/ttyS0 that is being used as local console for one IKEA ICC-A-1 module for sniffing Zigbee trafic.

Is it possible for you to implanting one parameter like "-S0" for starting Socat using /dev/ttyS0 ?

I think i can (with not so much Linux compiling experience) downloading toolchain and compiling one separate binary for it but then i also must have 2 95% identical version installed in the ZBGW.

Do you have some good advice to "muting" the log / debug out pot on the local console as much as possible ? Redirecting logs to dev null ?

The Zigbee sniffing is never starting at boot so that is not one problem (if not getting the NCP in blocking state beaus of boot logging) but its very nice done then having all that i need on J1 for doing good sniffings in full speed also in large networks.

PS: All EZSP NCP firmware (if not being stripped) is having one working mode for sniffing zigbee.

[banksy-git]

Here you go: https://github.com/banksy-git/lidl-gateway-freedom/releases/download/Release-1.1/serialgateway

To select serial port use option -d.

You can make it quiet by redirecting its output, e.g.:

/tuya/serialgateway -d/dev/ttyS0 -f 2>/dev/null

Have fun!

[MattWestb]

Thanks you very much !!

I trying it out if i can getting it working and informing you how it working so you can using the knowledge if you like.

Divertiti ! !

[MattWestb]

I have not putting one IKEA module inside but its looks working OK in the ZBGW side:

# /tuya/serialgateway -p8989 -d/dev/ttyS0 -f
serialgateway: serial=/dev/ttyS0 port 8989, flow=sw
Connect from host 192.168.2.10
Closing existing connection

Or i taking one LIDL LED stripe controller that is having on EFR32MG21 but then i need some cooking one new firmware for it and i have abusing our firmware cooker to much the last mounts :-((

[MattWestb]

I was possible one time getting one IKEA ICC-A-1 module booting in to bootloader but its nearly impossible.

# cat /dev/ttyS0
▒

R~▒T}:~B▒▒\(▒▒.▒~▒D;~C▒▒▒▒▒~
Gecko Bootloader v1.A.3
1. upload gbl
2. run
3. ebl info
BL >

But normally the system is taking the port and opening the login prompt and distorting the sequence.
I have pinpointing it to one process that is starting the login prompt:

  152 root      1288 S    /bin/login
  247 root      1292 S    init


ls -la /bin/login
lrwxrwxrwx    1 1000     1000            7 Apr 30  2020 /bin/login -> busybox

ls -la /bin/init
lrwxrwxrwx    1 1000     1000            7 Apr 30  2020 /bin/init -> busybox

Normally its one init PID and then sending somthing to the port and the NCP is sending somthing back its starting one /bin/login PID = its the login prompt.

Killing them is not helping then somthing is restarting them.
If killing the PID 1 that is also init the system is rebooting.

Was seeing somthing in the tuya app logs but i cant finding wot is starting the service and kicking he 2 processes thru busybox commands that i listening to the comport and triggering the login screen on it.

Do you have some where i can finding the service and deleting the login part of it ??

Thanks for help to Linux novice hardware users.

[banksy-git]

There's a couple of things that might interfere:

1. The login (getty) processes. These are started automatically by init according to rules in this file: /etc/inittab. Normally you could just edit that file and comment out the line but it's on a read-only file system. You can change it by copying the entire root filesystem off the flash, extracting it, modifying it, re-packing it and putting it back. (That's what I initially did to get access.). Given you have root access already, it's a bit easier because you can use the /dev/mtd2 device. It's a little involved but roughly speaking you can do something like this :

ssh [email protected] "cat /dev/mtd2" >mtd2.original
unsquashfs mtd2.original
edit squashfs-root/etc/inittab
mksquashfs squashfs-root mtd2.new ... (I forget the args... I'll look them up later)
attach realtek header + checksum (I hacked together a script for this, will try to find it later)
cat "mtd2.new" | ssh [email protected] "cat >/dev/mtd2""

If you get it wrong your device won't boot. Provided you used mtd2 you should still be able to get into the bootloader and fix it from there (TFTP the original back and then write to flash.).

2. The kernel console. The kernel itself outputs its own messages to the serial port. You see them when it boots but they can happen afterwards too. Normally for embedded devices this is configured at build time for the kernel. Sometimes it's possible to set a kernel command-line argument too but I'm not sure how to do that on the Realtek bootloader - would need to dig in further. What you can do is disable nearly all its output by using dmesg, e.g.

dmesg -n 1

That'll probably be sufficient for a quick hack. 😄

[MattWestb]

I think only comment out the /bin/login is enough if the kernel is not spamming the comport (its normally not doing then tuya app is deactivated and having network connected OK) .

# Start an "askfirst" shell on the console (whatever that may be)
ttyS0::askfirst:/bin/login
#ttyS0::askfirst:/bin/sh

Braking the link to /bin/login shall also working but then i think i cant login from SSH :-((

I have making one DD for all 5 partitions from SSH shell and uploaded them with TFTP to my laptop so shall being safe if the DD was making the bin files OK.

Is the /dev/mtd2 "neutral" or is it having part of the encrypted root password ?
If its "neutral" then its no problem making one patched version and sharing it but not recommended if its have the root password in it.

I was baying one LIDL ZBGW yesterday (My first is one "Revolt" = one white label one) and its have the same HW version as yours so shall being identical.

[MattWestb]

Do you think its possible changing the rights for accessing /dev/ttyS0 so the system cant accessing it and only the Socat ca do it ??

[MattWestb]

One silly question: the /sys/devices/platform/serial8250/tty/ttyS0/ is one RW mount.
Is it possible "tricking" the kernel backing out / deleting the ttyS0 and initiating the hardware to ttyS2 ??
I have not compiling one "real" Linux kernel for around 30 yer and its have being little evolution on that part.

[banksy-git]

MTD2

MTD2 is the same across devices. (Or at least it was for the two images I've seen.) That said, for a couple of reasons, I think it would be better to share methodology than actual images.

Using access controls

The kernel doesn't use the filesystem to access the device. The init process runs as root so it'll bypass all the DACLs on the /dev/ttyS0 device node anyway. (The D in DACLs being discretionary.)

Tell the kernel to release

It might be possible to ask the kernel through some API to close its internal reference to the serial port. I'm not sure but I'm curious so I'll dig in the source later to see. Gut feeling is that it's probably not possible but worth a look! There's potentially some more hacky things we could do to bring about the same result.

I'd like to build a fresh kernel at some point - if all the drivers (particularly networking) are present.

[MattWestb]

If you have your method for modding and repacking the MTD2 i like testing it if you have it documented so its easy to do and redo if its not working so well.

In witch of the dumped partitions is the encrypted root password ?

One new kernel you can implanting wot you like and then its no problem overriding the inits you like and its one god way getting control of the HW and the SW.

If you is getting some more or less crazy idea and like some to trying it out i can trying it out then i heaving one LIDL and one tuya clone too trying with.

[MattWestb]

I think i have messing the local console up.
Was restarting my new LIDL ZBGW and it was stopping starting the ziigbee network and refusing connecting with SSH also the local console is strange its giving:

tuya-linux login: root
Password:
Tuya Linux versi
Please press Enter to activate this console.
tuya-linux login: root 

I think i have writing some bad in the system partition that is making the console being messed up by the serial server.

I have doing one backup with DD of all 5 MTDs so shall being possible restoring the system but i dont knowing how.

Can you pleas hinting how to writing back the system partition from the bootloader.
The bootloader is working and the extracted root password have being working and if trying login with wrong root password the Linux is complaining.

Thanks in advance !!

Mattias

[MattWestb]

The restarting the ZBGW the console is writing:

Please press Enter to activate this console. serialgateway: serial=/dev/ttyS1 port 8888, flow=HW

So i think i have putting the wrong config in the start file.

[banksy-git]

Disclaimer for those late to our party: You can brick your device with this so double check all things before pressing ENTER!

If you have a backup you can restore with boot loader and TFTP:

1. Enter bootloader mode.

2. Discover IP address by issuing this command:

ipconfig

Likely it's 192.168.1.6

3. The bootloader runs a TFTP server so use a TFTP client to send up the partition you want to restore. e.g. (from a Linux box)

tftp 192.168.1.6 -m octet -c put mtd2

The bootloader will output something like this:

**TFTP Client Upload File Size = 00200000 Bytes at 80500000

Which indicates that your file was placed into RAM at address 80500000.

4. Write the data you just copied into RAM into the flash.

a) You need to select the address of the partition you're restoring. Helpfully these are output by the device when it boots as follows:

0x000000000000-0x000000020000 : "boot+cfg"
0x000000020000-0x000000200000 : "linux"
0x000000200000-0x000000400000 : "rootfs"
0x000000400000-0x000000420000 : "tuya-label"
0x000000420000-0x000001000000 : "jffs2-fs"

First one is mtd0 and they go up.

So if we're restoring mtd2 we select our address as 0x200000

DANGER: Select the proper address and double check you typed it correctly! Restoring mtd1, 2, 3, 4 and 5 should be fine but I recommend NOT trying to flash mtd0!

b) Write the data in RAM to flash by issuing this bootloader command:

FLW 200000 80500000 200000

Where 200000 is the SPI flash address to write to, 80500000 is the RAM address to read from and the last 200000 is the size of the file in RAM.

Good luck!

(edit: tftp client transfers in ASCII mode.)

[MattWestb]

Thanks for joining the party late or not !! ;-)

The device is working OK with ZHA and have using it for sniffing little zigbee traffic.

I trying flashing the rootfs in the weekend or in the next week.
Looks being easy testing that all is going OK until doing the flashing that is the critical doing OK.

I informing you how its going so you knowing if more "normal" user can doing your deep hacking working :-))

And one more time thanks !!

[MattWestb]

If you is interested getting the commands for booting to bootloder so have Silabs one python utility that can do it locally and i was using the commands from it with SSH for reboting the NCP.
Silabs posting https://silabs-prod.adobecqms.net/community/wireless/zigbee-and-thread/forum.topic.html/zigbee_and_uart_howtostart-nbEi
and the NCP.py (renamed to zip from rar) ncp.zip.

You is only need using v7 commands with original 6.5.0 or the 6.5.1 firmware that i have getting for going back to tuya app, and 6.7.8 you need using v8 commands.

Its easy only some string to sending and you have rebooted the NCP but the format is different for the protocol versions.

[chaisaeng]

I found something interesting in /tuya directory.
there is a binary named tuyamtd in my device
I tried executed it without argument the output seem to very interesting

# /tuya/tuyamtd
argc err.for example:
tuyamtd read auzkey/uuid/master_mac/slave_mac

# /tuya/tuyamtd read auzkey
jQjxu6K1ANAva9fV0mWW9nWd96760rTW
 
# /tuya/tuyamtd read uuid  
6157802868572d116432

it is not very useful if you do not have root access to the device . also i'm not sure uuid is the same as kek key or not

[MattWestb]

I think tuya is having one "default" firmware on the ZBG that is being updated then connecting to the cloud but its must having some default root password for using those tools then first connecting and setting the keys in the flash (as your is ff and not connected to the cloud).

I think its very interesting for @banksy-git looking in your original firmware for getting ore clues how tuya have cooking its ZBGW !!

I have one of the square clones that is working OK but i was soft braking on newer LIDL with configurating the second com port and its interference with login both SSH and local on the terminals (login is OK but is being kicked out to login prompt then coming in the shell) and im too lazy re flashing the device with the dumped images. It working on the normal 8888 port for the normal com port but is having problems and loosing contact with ZHA and also is using it for sniffing zigbee traffic with bellows over the network.

Thanks for sharing great information !!!

[MattWestb]

I have working / testing ESPHome on MwMos D1 Mini with serial server to HA with one IKEA ICC-A-1 Zigbee module.

I have implanted software commands in ESPHome with scrips for rebooting i to bootloader and its working great but must still using one terminal that can sending firmware with XModem and i was using ExtraPuTTY before and it was working OK with real serial ports but not with socket connected ports and one day later i was trying ZOC and it was working great.

Then have trying your update script and its also working great to updating Zigbee modules in ESPHome !!

But I is missing the software commands for rebooting the Zigbee module of all versions and not needing have bellows installed for doing it.

First added: import time
And then one new section:

def ncp_blm(ip, port):

    s = socket.create_connection((ip, port))
    s.settimeout(5)
    s.send(OOB_HW_FLOW_OFF, socket.MSG_OOB)

    s.send(b'\x1A\xC0\x38\xBC\x7E') # EZSP Configuration Frame: version ID: 0x00
    while s.recv(7) != b'\x1A\xC1\x02\x0B\x0A\x52\x7E': # Wait for RSTACK FRAME (Reset ACK)
        pass
    print("NCP reseted")

# Ver 8:
    s.send(b'\x00\x42\x21\xA8\x5C\x2C\xA0\x7E') # EZSP V8 Configuration Frame: version ID: 0x00
    while s.recv(11) [1:5] != b'\x42\xA1\xA8\x5C': # Wait for Data Response { protocolVersion, stackType, stackVersion } this must be ACK'd

# Ver 7:
#    s.send(b'\x00\x42\x21\xA8\x53\xDD\x4F\x7E') # EZSP V7 Configuration Frame: version ID: 0x00
#    while s.recv(11) [1:5] != b'\x42\xA1\xA8\x53': # Wait for Data Response { protocolVersion, stackType, stackVersion } this must be ACK'd

# Ver 6:
#    s.send(b'\x00\x42\x21\xA8\x52\xCD\x6E\x7E') # EZSP V6 Configuration Frame: version ID: 0x00
#    while s.recv(11) [1:5] != b'\x42\xA1\xA8\x52': # Wait for Data Response { protocolVersion, stackType, stackVersion } this must be ACK'd

# Ver 5:
#    s.send(b'\x00\x42\x21\xA8\x51\xFD\x0D\x7E') # EZSP V5 Configuration Frame: version ID: 0x00
#    while s.recv(11) [1:5] != b'\x42\xA1\xA8\x51': # Wait for Data Response { protocolVersion, stackType, stackVersion } this must be ACK'd

# Ver 4:
#    s.send(b'\x00\x42\x21\xA8\x50\xED\x2C\x7E') # EZSP V4 Configuration Frame: version ID: 0x00
#    while s.recv(11) [1:5] != b'\x42\xA1\xA8\x50': # Wait for Data Response { protocolVersion, stackType, stackVersion } this must be ACK'd
        pass
    print("EZSP Configuration Version X sent")

    s.send(b'\x81\x60\x59\x7E') # DATA ACK response frame
    print("DATA ACK response sent")

    s.send(b'\x7D\x31\x43\x21\x27\x55\x6E\x90\x7E') # Sending rebooot in to the bootloader
    while s.recv(1) != b"\x00":
        pass
    print("Reboot in to bootloader command sent")
    time.sleep(1)

And in the end before your run statement:
ncp_blm(args.ip, args.port)

Its one 110% copy and past work as im not one programmer but its working.
I have not finding how to do the selecting of the protocol version so i have comment out that i is not using for the moment.

I have trying updating EZSP 6.7.9.0, 6.9.2.0 and 6.4.0.0 (its V7) and changing protocol version for going to and from V7 and its working great.

If you like you can using what is is good and fixing all bad things i have doing.

Attaching one "updated" version of your script (ncp_blm.py) plus the ncp.py that is updated of ZHA devs and is the reference from Sillabs for sending commands without one real host system that is doing all the magic with the frames received / sent.
ncp_blm.zip

[MattWestb]

One good thing can being sending one "2" before starting the reset of the NCP so if its in bootloader is restarting and running the EZSP app or better "detecting" if its in bootloader mode by sending one "enter" and looking if getting the bootloader prompt (that is done in the ncp.py) and dont need restarting the chip and also its working if the EZSP app cant running the APP and is starting the bootloader then hardware resetting / power on of the chip.

Also one check if the file is one GBL file or not and exiting if its not one GBL file can being good.

[MattWestb]

Thanks for for the great working restore instruction !!

I was restoring MTD02 = "rootfs" = not helping
MTD03 = "tuya-label" = not helpping
MTD04 = "jffs2-fs" = starting but cant finding /tuya/tuya_start.sh

Looking the the tuya folder and its was looking little interesting:

# ls -la
drwxr-xr-x    7 root     0               0 Jan  1 00:00 .
drwxrwxr-x   14 1000     1000          186 Apr 30  2020 ..
-rwxrwxr-x    1 1000     1000       189774 Apr 21  2020 NcpUpgrade.ota
-rwxrwxr-x    1 1000     1000         9226 Apr 21  2020 app_upgrade.sh
drwxrwxr-x    2 1000     1000            0 Jan  1 00:00 config
-rwxrwxr-x    1 1000     1000          453 Apr 21  2020 def.cfg
-rw-r--r--    1 root     0              15 Jan  1 00:02 dump.txt
-rwxrwxr-x    1 1000     1000         2161 Apr 21  2020 json_parser.sh
drwxr-xr-x    2 root     0               0 Jan  1 00:00 log_dir
drwxr-xr-x    2 root     0               0 Jan  1 00:00 ssh
-rwxrwxr-x    1 1000     1000          936 Apr 21  2020 ssh_monitor.sh
-rw-r--r--    1 root     0              32 Jan  1 00:00 start.conf
-rwxrwxr-x    1 1000     1000          639 Apr 21  2020 start_record.sh
-rw-r--r--    1 root     0             496 Jan  1 00:00 start_record_file
-rwxrwxr-x    1 1000     1000          272 Apr 21  2020 tuya_net_start.sh
-rwxr-xr-x    1 root     0            4266 Jan  1 00:01 tuya_start.original.sh
drwxrwxr-x    4 1000     1000            0 Jan  1 00:00 tuya_user1
-rwxrwxr-x    1 1000     1000       296748 Apr 21  2020 tuyadropbear
-rwxrwxr-x    1 1000     1000        39823 Apr 21  2020 tuyamtd
-rwxrwxr-x    1 1000     1000          883 Apr 30  2020 udhcpc.script
-rw-r--r--    1 root     0             307 Jan  1 00:00 zigbeeNetInfo.txt
-rw-r--r--    1 root     0               2 Jan  1 00:00 zigbee_net_valid_flag
# 

Its have tuya_start.original.sh = moded then hacking it first time but its not having the tuya_start.sh.
Also serialgateway is not there so its not original and not the hacked FS but its looks working OK.
Installing serialgateway and its looks working OK but i have not testing if its stabile butt its on other thing.

One more time THNAKS for helping and getting our ZBGW working with our community systems !!

[MattWestb]

24 hour test passed !!

With with original EZSP 6.5.0.0 firmware it was not possible sniffing with bellows more then some minutes before the NCP was going i fault state and needed reset if little traffic on the zigbee network.
After flashing EZSP 6.7.8.0 its working great and have sniffing / dumping Zigbee traffic from my production network in 24 hours without problems !!