Memory leak with Tasmota 14.1.0 ?

[wjohn007]

A few weeks ago I created a Bluetooth application with Berry.
Since Bluetooth requires a lot of memory, I created an image with the Tasmo-compiler to have more memory available. After about 14 days, the controller's WEB server was no longer accessible.
I suspected low memory.
I recorded the memory history via Node-Red using following command
http://tasmota-635c38-7224/cm?cmnd=br%20tasmota.memory()

Properties heap_free and frag should be tracked.

The comments below describe the changes to the previous phase

Phase 1:
The controller runs with my Berry application.
As soon as the free memory falls below 20k, instabilities occur (web page can no longer be accessed)

Phase 2:
My Berry application does not start in this scenario.
However, the free memory decreases over time.

Phase 3:
BLE has been disabled. Here too, the free memory is reduced over time.

Phase 4:
The official release 14.1.0 (tasmota32.bin) was flashed to a NodeMCU32. Here too, the free memory is reduced.

As a workaround, I configured an auto-restart after 7 days.

Is this a memory leak?

What is the expected behavior?

Thank you for your hints.
John

[sfromis]

Expected behavior depends a lot on what you're doing on it. However, due to garbage collection, tracking heap_free gives a lot of ambiguity, as the garbage collection may not have been executed within the same time frame each time, and that varying amounts of RAM could be pending collection.

Here's an example of a garbage collection message (with Weblog 4) from a system subscribing to a lot of MQTT payloads, and maintaining a simple HASPmota display:

BRY: GC from 77869 to 39221 bytes, objects freed 502/714 (in 9 ms) - slots from 1705/1734 to 768/1493

No memory issues on that system, while running for weeks without booting.

With a "normal memory leak", I'd not expect to see the fragmentation going down gradually. Obviously, it looks suspicious with the heap_free gradually falling, meaning "reasons to investigate", rather than having basis for making any conclusion. "Risk zones" might include things like your Berry application performing operations in a way causing new resources to be allocated each time, of course this is quite speculative.

[stefanbode]

Checked on my esp32 controlling shutter with latest 14.1. no issues visible. But not using berry on this device

[sfromis]

That's one small step towards narrowing down what your device is doing when the free heap falls gradually.

[wjohn007]

@sfromis
The comments for each phase describe the changes from the previous phase.
Berry was therefore only active in phase 1, except for the HTTP request mentioned.

@stefanbode
The period you observed is too short.
The beginning of phase 4 does not seem critical; the memory loss only becomes apparent over a longer period of time

[sfromis]

As I said "Expected behavior depends a lot on what you're doing on it", and even your method of collecting data means that you have some Berry on the device.

Anyway, I have a lot of ESP32 devices running with current Tasmota, and I'm not seeing anything like dwindling free heap, which I have data on due to it already being a field in the telemetry period STATE payloads. This includes devices basically doing nothing but sitting there.

Therefore I can only maintain that any dwindling of memory depends on what you're doing on the device. And do remember that I pointed out the issue of your collection method not working well without ensuring garbage collection.

[Jason2866]

Your question about a possible memory hole is to unspecific. It needs a more detailed look.
Which part exactly is responsible for loosing memory. It could be just your Berry code

The controller runs with my Berry application.

Currently there is just a statistic showing a memory decreases. Not more not less.
It is expected that with low memory functions start to behave incorrectly.

[wjohn007]

@Jason2866
Thank you for your answer.

Which part exactly is responsible for loosing memory

Custom bery code was only active in phase1.

Currently there is just a statistic showing a memory decreases. Not more not less.

In phase 4 I use the regular firmware tasmota32.bin “naked” without any “rules” or any Berry code.
Isn't the reduction in memory a symptom of a memory leak?

[Jason2866]

Something is different, can't see any memory decrease with the standard binary tasmota32.bin

[sfromis]

There may be a memory leak, but when others can't reproduce, things are still up to what is happening on your device(s).

[wjohn007]

I understand that really.
Any suggestions on how I can isolate my personal memory leak with my application-less, freshly flashed Tasmota device?

Also, nmap found UDP port 64673 on the Tasmota device. What is this used for?

PORT      STATE         SERVICE
80/tcp    open          http
64673/udp open|filtered unknown

[sfromis]

Answers to such questions again depends on what you're doing on the device. One possibility is that you're running some feature using UDP and getting a random local port assigned for that.

[wjohn007]

I changed the command for tracking memory from (using berry)

http://tasmota-635c38-7224/cm?cmnd=br%20tasmota.memory()

to (without berry)
http://tasmota-635C38-7224/cm?cmnd=status%204

with following result (Phase 5)

Memory seems to be stable now.

Is this all a Berry-Issue ?

[sfromis]

As I repeatedly mentioned, you need to ensure garbage collection if you are picking up Berry memory data. This is no issue, as there is no requirement to make automatic garbage collections before needed.

Of course, if you have an issue where Tasmota fails with memory exhaustion, that's another thing.

[wjohn007]

As I repeatedly mentioned, you need to ensure garbage collection if you are picking up Berry memory data

I'm not convinced of that. Maybe my idea of ​​GC is different from yours.

GC runs constantly in the background. There is initially no reason to actively initiate this.

I tried following test

class test
  def init()
  	print("class created")
	end
	
  def deinit()
    print("garbage collection done")
  end
end

xxx = test()
xxx = nil

Console-Output

17:20:27.549 class created
17:20:43.001 garbage collection done

Unnecessary instances are automatically cleaned up after 20 seconds at the latest.

On a controller with tasmota32.bin without an application and a working GC,
I expect that this request

http://tasmota-635c38-7224/cm?cmnd=br%20tasmota.memory()

will not result in a permanent loss of memory .

Am I wrong ?

[stefanbode]

Hmm, let's see where this ends. Same device like last image.

[wjohn007]

@stefanbode
Is there any news about memory history?

The command
http://tasmota-635c38-7224/cm?cmnd=br%20tasmota.memory()
in my case it runs every 5 minutes.

[stefanbode]

I strongly believe we have an issue. There is a konsistent decrease of memory on the shutter just from being online.

[Noschvie]

Which field is of interest?

{'heap_free': 132, 'program_free': 878, 'program': 2001, 'stack_low': 3.75, 'iram_free': 35, 'frag': 19, 'flash': 4096, 'stack_size': 8, 'flash_real': 4096}

[wjohn007]

heap_free is the tracked value.