Sync of 2 ESP32 to display WS2812FX effects at the same time

[EherXtrem]

Hello,

I have connected 2 ESP32-S3-DevKitC-1 v1.1 via ESPNow and both play the same effects at the touch of a button. Unfortunately they are not in sync. Is there a way to keep the display of the effects in sync at the start and over a longer period of time?

Please see my short Demo-Video: https://drive.google.com/file/d/1u-8pZSvynUaM558xhD7Z02Gr1ibFc6BI/view?usp=sharing

[moose4lord]

Are you using the start() and stop() functions to start and stop the effects? Start() runs the resetSegmentRuntimes() function which should reset all the timing parameters and keep the two strips in sync, as long as the two strips have to have the same number of LEDs and the "speed" parameter is the same.

[EherXtrem]

Big THX, it works like a charm. At startup they are absolutely synchronous. As the runtime progresses, LED patterns deviate from each other, but this is probably due to the component tolerances.

You made my day. Like always :)

[moose4lord]

Great! If the two devices get too far out of sync, you can probably use ESPNow to periodically resync them.

[EherXtrem]

Yes, I thought about that too, but the resync would have to come at a suitable point so that you don't notice it. Do you have any idea when that moment might be and how to identify it?

[moose4lord]

I'm not sure how you would resync without having a noticeable glitch in the animation. You might be able to use isFrame() to count animation frames, but it would be tricky.

[moose4lord]

You got me thinking about running two ESPs simultaneously and having them gradually get out of sync. Crystal oscillators are supposed to be pretty precise, so I was thinking it would take quite a while for the timing to drift significantly. So I ran an experiment starting two ESPs running the comet effect at the same time and it didn't take very long at all for them to get out of sync. After only an hour the two animations were noticeably out of sync. I was surprised.

Still haven't thought of a good way to resync them without at least one of them glitching.

[EherXtrem]

The same here. It would be nice if there was a way to do the resync at a time when it wouldn't be noticeable - for example when the Comet has just finished a run. Isn't there a possibility?

[moose4lord]

Well, I took a stab at a sync feature. It's not pretty, but it kind'a works.

Each ESP counts animation frames and when it reaches a certain threshold, sends an ESP-NOW sync message to it's partner. Whichever ESP is running slightly faster be the first to send the sync message, then pause it's animation and wait for the sync message from the other ESP to signal it to resume the animation. If the two ESPs aren't too far out of sync, the animation pause should not be noticeable.

#include <WS2812FX.h>
#include <esp_now.h>
#include "WiFi.h"

#define LED_PIN    22  // digital pin used to drive the LED strip
#define LED_COUNT  64  // number of LEDs on the strip
#define MOSFET_PIN 23

esp_now_peer_info_t peerInfo;

bool isESP1 = false;
uint8_t esp1MAC[] = {0x94, 0x3C, 0xC6, 0x33, 0x02, 0x74};
char *esp1MACstr = "94:3C:C6:33:02:74";
uint8_t esp2MAC[] = {0x0C, 0x8B, 0x95, 0x76, 0x39, 0x10};
char *esp2MACstr = "0C:8B:95:76:39:10";

int frameCnt = 0;
bool syncReceived = false;

WS2812FX ws2812fx = WS2812FX(LED_COUNT, LED_PIN, NEO_GRB + NEO_KHZ800);

void OnDataSent(const uint8_t *mac_addr, esp_now_send_status_t status) {
  if(status == ESP_NOW_SEND_SUCCESS) {
    Serial.println("Sync message delivery success.");
  } else {
    Serial.println("Sync message delivery failed");
  }
}

void OnDataRecv(const uint8_t * mac, const uint8_t *incomingData, int len) {
//Serial.print("data received: "); Serial.println((char *)incomingData);
  if(strcmp((char *)incomingData, "sync") == 0) {
    Serial.println("syncReceived");
    syncReceived = true;
    ws2812fx.resume();
  }
}

void setup() {
  Serial.begin(115200);
  delay(500);
  Serial.println("\n");

  WiFi.mode(WIFI_MODE_STA);
  Serial.println(WiFi.macAddress());

  if(WiFi.macAddress().equals(esp1MACstr)) isESP1 = true;
  Serial.print("isESP1: "); Serial.println(isESP1 ? "true" : "false");

  pinMode(MOSFET_PIN, OUTPUT);     // MOSFET GPIO
  digitalWrite(MOSFET_PIN, HIGH);  // MOSFET off

  ws2812fx.init();
  ws2812fx.setBrightness(16);

  if(isESP1) {
    ws2812fx.setSegment(0, 0, 63, FX_MODE_COMET, GREEN, 5000);
  } else {
    ws2812fx.setSegment(0, 0, 63, FX_MODE_COMET, BLUE, 5000);
  }

  digitalWrite(MOSFET_PIN, LOW);   // MOSFET on
  delay(10);

  ws2812fx.start();

  if (esp_now_init() != ESP_OK) {
    Serial.println("Error initializing ESP-NOW");
    return;
  }

  // register send and recv callbacks
  esp_now_register_recv_cb(OnDataRecv);
  esp_now_register_send_cb(OnDataSent);

  // Register peer
  uint8_t *recvMAC = isESP1 ? esp2MAC : esp1MAC;
  memcpy(peerInfo.peer_addr, recvMAC, 6);
  peerInfo.channel = 0;  
  peerInfo.encrypt = false;
    
  // Add peer        
  if (esp_now_add_peer(&peerInfo) != ESP_OK){
    Serial.println("Failed to add peer");
    return;
  }
}

void loop() {
  bool newFrame = ws2812fx.service();
  if(newFrame) frameCnt++;

  // periodically send sync message via ESP-NOW
  if(frameCnt > (LED_COUNT * 20)) {
    Serial.println("Timer expired");
    sendSync();
    if(!syncReceived) {
      ws2812fx.pause();
    }
    frameCnt = 0;
    syncReceived = false;
  }
}

void sendSync() {
  uint8_t *recvMAC = isESP1 ? esp2MAC : esp1MAC;
  esp_err_t result = esp_now_send(recvMAC, (uint8_t *) "sync", sizeof("sync"));
  if (result == ESP_OK) {
    Serial.println("Sending sync message");
  } else {
    Serial.println("Error sending sync message");
  }
}

[EherXtrem]

I love your solution but after a lot of testing I'm afraid it can't be 100% sync, right? In particular when the sketches are changed quickly (e.g. multi_strobe, blink_rainbow) it is optically not synchronous. The two LED strips are usually already out of sync when you start them for the first time. So sade :(

But THX for your support !

[moose4lord]

Yeah, if you're running a quickly changing animation, the sync is probably never going to be good enough.

Sorry it didn't work out for you.

I found an interesting approach to LED syncing using a clock obtained from a mesh network.
https://github.com/costyn/LEDswarm/tree/master
I did not try it out and I'm not sure it would work any better for fast animations, but it might be worth pursuing.

[EherXtrem]

The sync process would need a kind of regular "restart" in order to start again from the beginning and reasonably synchronously after each run. That should work with ws2812fx.stop() and ws2812fx.start() in connection with newframe, right?

[moose4lord]

I think that would also work for slowly changing animations. But for fast animations, I don't think it would work any better than the code I had been playing with.

The mesh network idea synchronizes the call to ws2812fx.service() to a mesh network clock signal, so the two ESP32s would run in lock-step. But it would require a fast network clock signal (on the order of milliseconds), so I don't know if you could keep a reliable, steady stream of clock ticks at that rate.

[tobi01001]

I was following this thread/issue and it got me thinking. ;-)

On your test setups, did you drive the LED-stripes with bitbanging? Because I do believe that much of running out of sync is caused by interrupts (and interrupts being paused during bitbanging). So especially in combination with WiFi and stuff this may cause significant differences between.

So my Idea would be to:

• use DMA or alike for driving the LEDs (bit banging never did me any good anyways) which does not disable the interrupts and runs more or less in parallel and independent.
• only update / write the leds based on a defined slow and fast enough framerate
• maybe inbetween sync these framerate with the internal clock (i.e. clock cycles)

If the frames would still run out of sync, one could wait for the other frame to be finished and fade/blend between end of animation and start of animation to prevent stuttering animations.

Don't know if that helps (and I do not have that much time to try on my own at the moment).

[moose4lord]

I used two ESP32s for my sync testing, which use the RMT and DMA hardware to create the pulse train, so no bit-banging involved. They were still noticeably out of sync after an hour. My bandwidth is limited too, so I'll have to put this off for a rainy day.

[kitesurfer1404]

Idea (maybe?): Count frames and/or clock cycles on each device and share them between the devices every x seconds/minutes. Artificially slow down the faster devices based on the difference over time by adding some delay/extra cycles for every frame/loop/whatever.
How bad is the drift in practice? Might depend on the network traffic as well as on the different clock speeds.

[tobi01001]

I used two ESP32s for my sync testing, which use the RMT and DMA hardware to create the pulse train, so no bit-banging involved. They were still noticeably out of sync after an hour. My bandwidth is limited too, so I'll have to put this off for a rainy day.

OK, but hourly timeframes / drifts are probably easier to sync then minutes or seconds...

Idea (maybe?): Count frames and/or clock cycles on each device and share them between the devices every x seconds/minutes. Artificially slow down the faster devices based on the difference over time by adding some delay/extra cycles for every frame/loop/whatever.

Or (another idea) just slow down the "server" by the (mean) time it takes to sent the sync message:

• delay the frame start (and service routine) by xDelay ms.
• note the current clock cycles / microseconds SendMicroSeconds
• send a frame start signal to the "client"
• start the frame on the "client" and sent a confirmation
• on reception of the confirmation calculate xDelay by xDelay = (CurrentMicroSeconds - SendMicroseconds) / 2 ; - or a moving average over the last 5 to 10 values (so it does not stutter / jump)

So the sync message is used to start and sync the animation on the client and the answer is used to calculate the necessary delay on the "server" side.

This way, both ESP should start the frames at quite the same time as the delay is dynamically calculated.
As long as the connection is stable and there is not too much difference in the timing, this could work.
And one could use websockets to sync the devices.

Finally, if the devices aren too far apart (physically) one could simply add a sync wire to one of the I/Os?

[EherXtrem]

Thanks for the many tips and support. I'll think about it even though most of it is beyond my skills as a hobbyist programmer.

The two ESP32 are installed in costumes by different artists, so unfortunately a cable is not possible.

For a little impression you can look here: https://www.youtube.com/watch?v=KXyNkO_nh_Q&t=11s