DON'T UPDATE FIRMWARE!!! Firmware 14 stops reporting data

[dacarson]

I recently updated from Firmware 10 to Firmware 14. Firmware 10 was mostly working fine, though I wanted to test my NUT UPS driver with the newer firmware.
I am running a script to read the data from the i2c bus every 10 seconds to see which fields are actually updated by the sample interval. The script was working fine with FW 10.
FW 14 runs for a while (approx 35-39 minutes) then stops reporting new data. I can read the registers ok, but they stop updating. I found that I have to reboot the device to get it running again (remove power and batteries, then re-insert power and batteries). When I do so, I see that the Running Time counters are all reset to zero.
I can set values in the registers but they don't stick.

I tried downloading firmware 10, but it says the firmware version not found.

[NightFlowed]

Ohhh, this seems to be the same bug we reported with @deHarro .
If you leave the system running and regularly send requests over I²C, the firmware eventually crashes and stops charging the batteries. The time before it crashes is a bit random. When it does crash, the battery is no longer charging even though USB still provides 9V.

I tried a workaround: performing an on/off sequence on the power input whenever this occurs.
It works and allows the system to recover and function normally again.

BUT after some time (between 23 hours and 3 days), the firmware crashes again. As you mentioned, the values stop updating, but the UPS continues to work “fine” and charges the battery correctly. However, if you unplug the USB, it will drain the battery without any limit. When you plug USB back in, it resumes charging, but the firmware itself doesn’t recover — it stays frozen.

So I came up with another workaround: I soldered a transistor onto the microcontroller to trigger a reset whenever the values stop updating.

Honestly, I’m getting a bit tired of the bugs on this UPS, so I decided to fix it myself since no one at Geeekpi seems willing to acknowledge that there is a firmware bug causing the crashes. If they made the software open source, we could fix it — but they keep it as some kind of “open secret.”

I hope you’ll get an answer and a solution… If not, you can use the same workaround I did. It will hard-reset the microcontroller regularly and bring it back to normal operation.

Regards

[dacarson]

Thanks for all your insights. I had no issues with FW 10, I never experienced this crash. I wish they made FW 10 available so I could roll back to that version.
The way the device functions, it looks the same as when you put it into "OTA mode" via registers, ie
i2cset -y 1 0x17 50 127 b
I put it in quotes as in my experience, it doesn't enable 0x18. I had to do the other method to update the firmware.
I wouldn't be surprised if it was somehow falling into that state. Everything continues to work "fine" and charges the battery, powers the RPi, but values stop updating.

Can you give me more details on the transistor path?

FWIW I have also asked for the software to be made open source, I emailed them directly. The issue I was facing with FW 10 and I suspect the same with FW 14, was that it powers the RPi back on too soon, it draws too much power and it shuts it down again.
It turns OFF the power to the 5V pin when battery_voltage < protection_voltage, but it turns it back ON when Battery_voltage + 100mV > protection_voltage. The RPi causes more than 100mV drop on the battery while powering on :-( I request to make the 100mV configurable, and I was told no. I should just not use the auto-restart flag.

[dacarson]

I found an old unit that had FW 10 and it I have been able to get up and running.

Retitled this bug report to hopefully let other people know before they update.

[markkkx000]

I am really regretting updating the firmware... I wish i saw this sooner. I hope someone in geeekpi becomes reasonable and make the firmware open-source.

[NightFlowed]

They don't care...
The firmware is fully bugged and nobody care about it, they will never make it open source.
Throw it to trash and buy a real ups.

[dacarson]

They are now working on a new UPS, UPSv6:
https://github.com/geeekpi/upsv6_pub
https://wiki.52pi.com/index.php?title=EP-0250#Description

Their Wiki says that the firmware is customizable. I posted a question about that:
geeekpi/upsv6_pub#14

[markkkx000]

If you leave the system running and regularly send requests over I²C, the firmware eventually crashes

@NightFlowed did you confirm whether it happens on both I²C reads and write requests? i was running a script before that just reads battery percentage every 60 seconds and it doesn't freeze the firmware (although i only experimented for about 12 hrs). I only noticed mine crashing after i enabled the crontab tasks that regularly executes the scripts in $HOME/bin (upsPlus.py and upsPlut_iot.py). Those scripts have 'bus.write_byte_data()' lines so i suspect that it only crashes on constant I²C writes. If this is the case, I could still potentially salvage the automatic shutdown function by using a script that just reads the battery voltage and executes a shutdown command if voltage drops to a certain point, so instead of writing to the registers like this:

# Reset Protect voltage
bus.write_byte_data(DEVICE_ADDR, 17, PROTECT_VOLT & 0xFF)
bus.write_byte_data(DEVICE_ADDR, 18, (PROTECT_VOLT >> 8)& 0xFF)
print("Successfully set the protection voltage to: %d mV" % PROTECT_VOLT)

we just handle it all on the raspberry pi, for the BACK-TO-AC feature, we could just set it to 1 right before the pi executes shutdown... this works under the assumption that regularly reading from the registers is safe and preventing firmware freezes by minimizing the write operations. this eliminates a lot of the possible use cases of this UPS but if i just want the automatic shutdown and BACK-TO-AC function this could work. i'll experiment again for longer durations and see if it still crashes on purely read operations.

[deHarro]

Hi @markkkx000!

I suppose, writing the registers is not planned to be made over and over again.
Set the protection voltage once and forget.

The supplied demo scripts are for reference only and describe, what you can do with the registers.
But let's wait, what the authors are saying about it.

[deHarro]

Hi @dacarson!

The up-to-date version of the wiki for UPSv6 is EP-0245.

[dacarson]

Unfortunately I'm not home at the moment otherwise I would share my script that I use to monitor the UPSPlus. It does not write to i2c but just reads the registers every second (I was trying to work out which registers are updated on the 'sample' interval).
The device does stop reporting new numbers after a while. I have not been able to work out the interval - sometimes it stops within minutes other times it stops after a day or so.
I have only experienced the issue with the newest firmware (14). The other unit that I have firmware 10 on does not stop. I don't know the difference between 10 and 14.

[yoyojacky]

They are now working on a new UPS, UPSv6: https://github.com/geeekpi/upsv6_pub https://wiki.52pi.com/index.php?title=EP-0250#Description

Their Wiki says that the firmware is customizable. I posted a question about that: geeekpi/upsv6_pub#14

Sorry, the customizing function still not open for customers due to this new UPS Gen 6 's customized function needs more test and documentation works, so it will be opened in future, it will be a LTS verison of UPS series, and it will support multiple external devices such as TFT OLED display, SATA Power supply board, and multiple battery pack support and so on. so we are going to upload the app source file of UPS Plus (Gen 5, UPS Plus is v5 ) and just for reference with no tech support and documentations.

[yoyojacky]

I recently updated from Firmware 10 to Firmware 14. Firmware 10 was mostly working fine, though I wanted to test my NUT UPS driver with the newer firmware. I am running a script to read the data from the i2c bus every 10 seconds to see which fields are actually updated by the sample interval. The script was working fine with FW 10. FW 14 runs for a while (approx 35-39 minutes) then stops reporting new data. I can read the registers ok, but they stop updating. I found that I have to reboot the device to get it running again (remove power and batteries, then re-insert power and batteries). When I do so, I see that the Running Time counters are all reset to zero. I can set values in the registers but they don't stick.

I tried downloading firmware 10, but it says the firmware version not found.

I2c data may be lost, you can just modify the python script to read one more time, or just make sure the script can read the i2c's data and then print it out, and do not read the register too quick to prevent it stack the data transfering .

[yoyojacky]

Ohhh, this seems to be the same bug we reported with @deHarro . If you leave the system running and regularly send requests over I²C, the firmware eventually crashes and stops charging the batteries. The time before it crashes is a bit random. When it does crash, the battery is no longer charging even though USB still provides 9V.

I tried a workaround: performing an on/off sequence on the power input whenever this occurs. It works and allows the system to recover and function normally again.

BUT after some time (between 23 hours and 3 days), the firmware crashes again. As you mentioned, the values stop updating, but the UPS continues to work “fine” and charges the battery correctly. However, if you unplug the USB, it will drain the battery without any limit. When you plug USB back in, it resumes charging, but the firmware itself doesn’t recover — it stays frozen.

So I came up with another workaround: I soldered a transistor onto the microcontroller to trigger a reset whenever the values stop updating.

Honestly, I’m getting a bit tired of the bugs on this UPS, so I decided to fix it myself since no one at Geeekpi seems willing to acknowledge that there is a firmware bug causing the crashes. If they made the software open source, we could fix it — but they keep it as some kind of “open secret.”

I hope you’ll get an answer and a solution… If not, you can use the same workaround I did. It will hard-reset the microcontroller regularly and bring it back to normal operation.

Regards

I have uploaded the firmware app source file in repository, please check it out.

[yoyojacky]

If you leave the system running and regularly send requests over I²C, the firmware eventually crashes

@NightFlowed did you confirm whether it happens on both I²C reads and write requests? i was running a script before that just reads battery percentage every 60 seconds and it doesn't freeze the firmware (although i only experimented for about 12 hrs). I only noticed mine crashing after i enabled the crontab tasks that regularly executes the scripts in $HOME/bin (upsPlus.py and upsPlut_iot.py). Those scripts have 'bus.write_byte_data()' lines so i suspect that it only crashes on constant I²C writes. If this is the case, I could still potentially salvage the automatic shutdown function by using a script that just reads the battery voltage and executes a shutdown command if voltage drops to a certain point, so instead of writing to the registers like this:

# Reset Protect voltage
bus.write_byte_data(DEVICE_ADDR, 17, PROTECT_VOLT & 0xFF)
bus.write_byte_data(DEVICE_ADDR, 18, (PROTECT_VOLT >> 8)& 0xFF)
print("Successfully set the protection voltage to: %d mV" % PROTECT_VOLT)

we just handle it all on the raspberry pi, for the BACK-TO-AC feature, we could just set it to 1 right before the pi executes shutdown... this works under the assumption that regularly reading from the registers is safe and preventing firmware freezes by minimizing the write operations. this eliminates a lot of the possible use cases of this UPS but if i just want the automatic shutdown and BACK-TO-AC function this could work. i'll experiment again for longer durations and see if it still crashes on purely read operations.

yes, that is demo code, just for you to reference, everybody has their own requirements, so everybody can modify the python script s to fit for their own requirements. thus, your new script is functional and great jobs ~ have a nice day~