HardwareSerial delivering stray data on serial.read()

[Miq1]

We have a couple of users of our eModbus library that are experiencing additional data being delivered to serial.read(). A logic analyzer or scope will see the correct data on the RX and TX wires, while serial.read() will bring more than are seen on the outside.

The data looks like remainders from previous transmissions and is one to about 250 bytes long. The read loop is basically

int bufptr = 0;
uint8_t buffer[512];

if (serial.available()) {
  int b = serial.read();
  while (b >= 0)  {
    buffer[bufptr++] = b;
    if (bufptr >=511) break;
    b = serial.read();
  }
}

(shortened to the essential)

To get the data as soon it arrives, we do a

UART1.conf1.rxfifo_full_thrhd = 1;

(or UART0, UART2) upon initialization, if that matters.

Is something known to be causing such effects?

[mrengineer7777]

Interesting. Perhaps you should create an issue for this? What version of arduino-esp32 are you targeting?

[Miq1]

2.0.2 is the version AFAIK. I am reluctant opening an issue as I am not able to reproduce it myself yet and have only the users' reports (and logs of course) so far. But it were reports of unrelated users, so I tend to believe the effect does exist in fact.
The logs are showing the data must have come from the serial.read(), while the logic analyzer dumps taken at the same time on the RX pin do not have it.

[Miq1]

There was an issue in IDF (this one) resulting in wrong buffer sizes being returned by uart_get_buffered_data_len() (hence: available()) in multithreaded applications. It was fixed mid-October 2021. Does the arduino-esp32 core have that fix?

[mrengineer7777]

v2.0.3 is based on IDF 4.4.1. https://github.com/espressif/arduino-esp32/releases/tag/2.0.3
Looking at the local copy of IDF on my computer: C:\Users\David\.platformio\packages\framework-espidf\components\driver\uart.c.
uart.c:258:

esp_err_t uart_get_stop_bits(uart_port_t uart_num, uart_stop_bits_t *stop_bit)
{
    ESP_RETURN_ON_FALSE((uart_num < UART_NUM_MAX), ESP_FAIL, UART_TAG, "uart_num error");
    UART_ENTER_CRITICAL(&(uart_context[uart_num].spinlock));
    uart_hal_get_stop_bits(&(uart_context[uart_num].hal), stop_bit);
    UART_EXIT_CRITICAL(&(uart_context[uart_num].spinlock));
    return ESP_OK;
}

Yes, it contains the critical wrapper fix. Compare this to commit espressif/esp-idf@6c363a4

esp_err_t uart_get_stop_bits(uart_port_t uart_num, uart_stop_bits_t* stop_bit)
{
    UART_CHECK((uart_num < UART_NUM_MAX), "uart_num error", ESP_FAIL);
    UART_ENTER_CRITICAL(&(uart_context[uart_num].spinlock));
    uart_hal_get_stop_bits(&(uart_context[uart_num].hal), stop_bit);
    UART_EXIT_CRITICAL(&(uart_context[uart_num].spinlock));
    return ESP_OK;
}

[mrengineer7777]

You can download the archive of 2.0.0 and see if that version has the fix, if you wish: https://github.com/espressif/arduino-esp32/releases/tag/2.0.0

[Miq1]

Thanks, good to see it has been incorporated. Unfortunately that would have been handy to explain the things our users are experiencing. Well, continue hunting it is again...

[Miq1]

Looks like it was something else, the issues reportedly are gone for now.

Thanks for the insights anyway!