System status reported as Unknown after HA restart

[FredericMa]

DESCRIPTION

I have an automation that sends out a notification if EMS-ESP is not online.
That's how I've noticed that after a HA restart, the binary_sensor.system_status sensor becomes unknown after a HA restart.
That's because it is not retained and is only published when EMS-ESP connects to the broker and isn't sent periodically.
So HA will never be able to receive the status topic unless I restart the EMS-ESP or the MQTT broker.
I noticed that this behavior was changed in this commit: 189ea6b

What was the reasoning behind it?
I was also a bit confused that it wasn't retained since I have the "Always set retain flag" setting turned on.

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

• Searched the issue in issues
• Searched the issue in discussions
• Searched the issue in the docs
• Searched the issue in the chat
• Provide the System information in the area below, taken from http://<IP>/api/system

{"System Info":{"version":"3.6.5","uptime":"062+19:05:32.882","uptime (seconds)":5425532,"platform":"ESP32","arduino":"Tasmota Arduino v2.0.14","sdk":"4.4.6.240105","free mem":158,"max alloc":103,"used app":1591,"free app":393,"partition":"app0","reset reason":"Power on reset / APP CPU reset by PRO CPU"},"Network Info":{"network":"WiFi","hostname":"ems-esp","RSSI":-37,"TxPower setting":78,"static ip config":false,"enable IPv6":true,"low bandwidth":false,"disable sleep":false,"enable MDNS":true,"enable CORS":false,"AP provision mode":"disconnected","AP security":"wpa2","AP ssid":"ems-esp"},"NTP Info":{"NTP status":"connected","enabled":true,"server":"time.google.com","tz label":"Europe/Brussels"},"OTA Info":{"enabled":false,"port":8266},"MQTT Info":{"MQTT status":"connected","MQTT publishes":2804860,"MQTT queued":0,"MQTT publish fails":0,"MQTT connects":10,"enabled":true,"client id":"ems-esp","keep alive":60,"clean session":false,"entity format":1,"base":"ems-esp","discovery prefix":"homeassistant","discovery type":0,"nested format":1,"ha enabled":true,"mqtt qos":0,"mqtt retain":true,"publish time heartbeat":60,"publish time boiler":10,"publish time thermostat":10,"publish time solar":10,"publish time mixer":10,"publish time other":10,"publish time sensor":10,"publish single":false,"publish2command":false,"send response":false},"Syslog Info":{"enabled":false},"Sensor Info":{"temperature sensors":0,"temperature sensor reads":0,"temperature sensor fails":0},"API Info":{"API calls":0,"API fails":0},"Bus Info":{"bus status":"connected","bus protocol":"HT3","bus telegrams received (rx)":7148952,"bus reads (tx)":994039,"bus writes (tx)":308,"bus incomplete telegrams":1,"bus reads failed":970,"bus writes failed":15,"bus rx line quality":100,"bus tx line quality":100},"Settings":{"board profile":"NODEMCU","locale":"en","tx mode":3,"ems bus id":11,"shower timer":false,"shower alert":false,"hide led":true,"notoken api":false,"readonly mode":false,"fahrenheit":false,"dallas parasite":false,"bool format":1,"bool dashboard":1,"enum format":1,"analog enabled":false,"telnet enabled":true,"max web log buffer":50,"web log buffer":50},"Devices":[{"type":"boiler","name":"Condens 2500/Logamax/Logomatic/Cerapur Top/Greenstar/Generic HT3","device id":"0x08","product id":95,"version":"23.04","entities":69,"handlers received":"0x10 0x11 0x15 0x1C 0x18 0x19 0x34 0x04","handlers fetched":"0x14 0x16 0x33","handlers pending":"0xBF 0xC2 0x1A 0x35 0x26 0x2A","handlers ignored":"0x1E"},{"type":"thermostat","name":"FW200","device id":"0x10","product id":106,"version":"12.14","entities":31,"handlers received":"0x06","handlers fetched":"0x016F 0x0170 0x0165 0x0166 0xBB 0x01D3","handlers pending":"0xA3 0xA2 0x12 0x13 0x0171 0x0172 0x0167 0x0168 0x23 0x0123","handlers ignored":"0xBE"},{"type":"mixer","name":"IPM","device id":"0x20","product id":102,"version":"20.08","entities":2,"handlers received":"0x010C","handlers pending":"0x011E","handlers ignored":"0x23 0xBE 0x010B"},{"type":"mixer","name":"IPM","device id":"0x21","product id":102,"version":"20.08","entities":4,"handlers received":"0x010C","handlers pending":"0x011E","handlers ignored":"0x23 0xBE 0x010B"},{"type":"solar","name":"ISM1","device id":"0x30","product id":101,"version":"23.04","entities":8,"handlers fetched":"0x0103 0x0101","handlers pending":"0x0104","handlers ignored":"0xBE"},{"type":"controller","name":"HT3","device id":"0x09","product id":95,"version":"23.04","entities":0}]}

TO REPRODUCE

1. Make sure EMS-ESP is started.
2. Observe the binary_sensor.system_status sensor being shown as "Connected"
3. Restart HA.
4. Observe the binary_sensor.system_status sensor being shown as "Unknown"

To fix:

• Restart the EMS-ESP.
• Disable MQTT and enable afterwards.
• Restart the MQTT broker.

EXPECTED BEHAVIOUR

I would expect the binary_sensor.system_status sensor to be shown as "Connected" after a HA restart.

SCREENSHOTS

[proddy]

That was the old way of detection. The trouble with that method is that it would give false positives, e.g. if the MQTT broker or the WiFi restarted/switched channels. There is a new method described in https://emsesp.org/Home-Assistant/#alert-when-ems-esp-boots. I would also recommend upgrading to 3.7.0.

[FredericMa]

Thanks for the reply.
I actually want to reach the opposite; I want to be notified when EMS-ESP goes offline, not when it is booted. What would be the recommended way to do that since I can't rely on the binary_sensor.system_status?

Would it be possible to make the binary_sensor.system_status follow the "Always set retain flag"?
At least for me the value of that sensor is almost always "Unknown" (and I guess for the majority of the people to be honest).

[proddy]

ok, I see now. There is no reliable way of getting EMS-ESP to notify when it goes offline or crashes. If there is a network interruption or MQTT server restart, EMS-ESP is still technically alive. There is an MQTT last will topic, but that doesn't say much and usually fired too late. This is why we had so many false positives in HA thinking EMS-ESP crashed when in fact it didn't. Tehnically EMS-ESP is only offline if it can't connect to the network.

I'd be happy to explore other ways, but this needs careful thinking through all different scenarios. You may want to look at other vendors (sonoff, shelly) and OSS (tasmota, esphome) to see how they do it. You could try using the ping platform in HA. Or checking the state of sensors when they become unavailable, but I'm not sure of the rules around that.

[proddy]

Moving to a discussion.

[MichaelDvP]

I think this is a special case. The status message is set as "last will" retain on the broker. On a emsesp crash the broker will set this message as "offline". On each connect emsesp sends this message as "online", but not retain. Now we have different scenarios:

• emsesp crashes -> broker sets last will
• emsesp restarts -> "online" is send again
• emsesp closes connection normally (mqtt switched off) -> status "online" stays
• broker is down -> emsesp timeouts and tries to reconnect -> sends "online" again on reconnect
• broker restarts -> if it's up again before emsesp timeouts emsesp does not see the broker was down und just contiues to send messages and do not send this message. This is case here and status is not set on the broker.

I think if we set the "online" message to retain this will help in this case and does not break anything. Last will will still overwrite the status on a abnormal disconnect.

[FredericMa]

I happen to be working in the industrial automation industry and we use the VDA5050 protocol which is built on MQTT.
It also has a "connection" topic which relies on the LWT to indicate an unexpected disconnect.
In VDA5050:

• a vehicle sets the LWT to publish "CONNECTIONBROKEN" on unexpected disconnect.
• a vehicle publishes:
  • "ONLINE" on connect.
  • "OFFLINE" on graceful disconnect.

All these messages are published with the retain flag on.
For reference: https://github.com/VDA5050/VDA5050/blob/main/VDA5050_EN.md#614-topic-connection:~:text=connectionState,has%20unexpectedly%20ended.

For EMS-ESP I think the system status sensor should clearly indicate that this is referring to the connection status and that it doesn't relate to the overall EMS-ESP system status (the device_class is already set to connectivity though)
I guess the only reliable way to know if the EMS-ESP is still alive is to check the UI, ping the device, get the system state via the API, ...

In my opinion the following config would be a correct representation of the connection status towards HA:

• On connect: publish "online".
• On graceful disconnect (disable MQTT, EMS-ESP restart, ...): publish "offline".
• Last will by broker; publish "offline".
  All these messages should be retained so HA can receive the last known connection state of EMS-ESP at any time.

I do understand that those false positive reports can be frustrating but I suppose that if a user can provide the data he receives from the http://<IP>/api/system call, we can consider EMS-ESP being alive and running, no?

[proddy]

Ok, so we keep the <base>/status topic as is, which is either offline or online and make it MQTT retain.

The line

    queue_publish_retain("status", "online", false); // with retain off

in Mqtt::on_connect() goes to ..., true)

so don't use anything from the heartbeat (binary_sensor.system_status)

[proddy]

in 3.7.0-dev.45

[FredericMa]

Thanks, it works like a charm!
I do notice that the sensor.boiler_setflowtemp became unavailable after upgrading.
Is this expected? Otherwise I'll create a bug for that.

[proddy]

It's not a bug but go ahead and create a GH issue with your system info. A lot has changed since 3.6.5

[proddy]

It's not a bug but go ahead and create a GH issue with your system info. A lot has changed since 3.6.5

[NongJoWo]

I have the same behavior after HA Restart with the ems-esp shower device. All entities are shown as:

• Shower Active: unavailable
• Shower Duration: unavailable
• Shower Timestamp: unavailable

As soon as the next shower is completed all entities are fine again.
I would expect the last status, same as for all other ems-esp entities.

Do I need to raise an own issue or is there already a solution available?

The Firmware on my EMS-ESP: v3.7.0-dev.17

Many Thanks!

[proddy]

This has been issue for a long time, and known. I'll see if I can fix it, it's a ticky one as I don't want to 'retain' the shower mqtt topics

[proddy]

#2124