GEM data error - values truncation?

Ask questions about the GEM here.
jbai
Posts: 6
Joined: Fri Feb 04, 2022 12:41 pm

GEM data error - values truncation?

Post by jbai » Fri Feb 04, 2022 1:16 pm

We recently installed our GEM and started logging data every 15 seconds via HTTP GET.

Reviewing our logs we find intermittent errors in values where one or two digits seem to be missing from an otherwise consistently populated value.

Below is an example of log data highlighting two examples bad values interspersed with consistent (and presumably accurate) values.
Any suggestions for the possible cause or fix?

GEM-data-error-example.jpg
GEM-data-error-example.jpg (162.95 KiB) Viewed 1110 times
ben
Site Admin
Posts: 4262
Joined: Fri Jun 04, 2010 9:39 am

Re: GEM data error - values truncation?

Post by ben » Fri Feb 04, 2022 2:17 pm

jbai wrote:
Fri Feb 04, 2022 1:16 pm
We recently installed our GEM and started logging data every 15 seconds via HTTP GET.

Reviewing our logs we find intermittent errors in values where one or two digits seem to be missing from an otherwise consistently populated value.

Below is an example of log data highlighting two examples bad values interspersed with consistent (and presumably accurate) values.
Any suggestions for the possible cause or fix?


GEM-data-error-example.jpg
What COM firmware is the GEM running?
Ben
Brultech Research Inc.
E: ben(at)brultech.com
jbai
Posts: 6
Joined: Fri Feb 04, 2022 12:41 pm

Re: GEM data error - values truncation?

Post by jbai » Fri Feb 04, 2022 2:30 pm

COM firmware ver: 5.23
ENG firmware ver: 1.49
RealTime Status: ON
Packet Send Interval: 15
Pri. Packet Format: 3
Sec. Packet Format: 0
Sys Status: OK
ben
Site Admin
Posts: 4262
Joined: Fri Jun 04, 2010 9:39 am

Re: GEM data error - values truncation?

Post by ben » Fri Feb 04, 2022 2:45 pm

jbai wrote:
Fri Feb 04, 2022 2:30 pm
COM firmware ver: 5.23
ENG firmware ver: 1.49
RealTime Status: ON
Packet Send Interval: 15
Pri. Packet Format: 3
Sec. Packet Format: 0
Sys Status: OK
Give COM 5.39 a try (issue in the thread isn't related but has upgrade instructions) https://brultech.com/community/viewtopic.php?f=3&t=2220

We had another customer with similar HTTP Get issues that Paul patched. It may solve the issue.
Ben
Brultech Research Inc.
E: ben(at)brultech.com
jbai
Posts: 6
Joined: Fri Feb 04, 2022 12:41 pm

Re: GEM data error - values truncation?

Post by jbai » Fri Feb 11, 2022 1:38 pm

Thank you, Ben. We updated the firmware to v5.39 but the issue persists.

We also appear to have at least one record pulled with an incorrect channel assignment or something.
The flagged meter reading below lines up better with other channels, and is shares the timestamp with a valid entry for this channel (see last screenshot below).

Please let us know if you have any troubleshooting guidance. We are concurrently assessing our data polling code & methods for possible errors.
Attachments
gem-data-error-truncation-2022-02-11.jpg
gem-data-error-truncation-2022-02-11.jpg (193.7 KiB) Viewed 1058 times
GEM-data-error-truncation-2-2022-02-11.jpg
GEM-data-error-truncation-2-2022-02-11.jpg (190.84 KiB) Viewed 1058 times
GEM-Data-error-wrong-channel-2022-02-11.jpg
GEM-Data-error-wrong-channel-2022-02-11.jpg (152.56 KiB) Viewed 1058 times
ben
Site Admin
Posts: 4262
Joined: Fri Jun 04, 2010 9:39 am

Re: GEM data error - values truncation?

Post by ben » Fri Feb 11, 2022 2:36 pm

jbai wrote:
Fri Feb 11, 2022 1:38 pm
Thank you, Ben. We updated the firmware to v5.39 but the issue persists.

We also appear to have at least one record pulled with an incorrect channel assignment or something.
The flagged meter reading below lines up better with other channels, and is shares the timestamp with a valid entry for this channel (see last screenshot below).

Please let us know if you have any troubleshooting guidance. We are concurrently assessing our data polling code & methods for possible errors.
Can you send your Status Page on the GEM?
Ben
Brultech Research Inc.
E: ben(at)brultech.com
jbai
Posts: 6
Joined: Fri Feb 04, 2022 12:41 pm

Re: GEM data error - values truncation?

Post by jbai » Sat Feb 12, 2022 11:50 am

Screenshot of our current GEM status page:
Attachments
GEM-setup-20220212.jpg
GEM-setup-20220212.jpg (371.43 KiB) Viewed 1044 times
ben
Site Admin
Posts: 4262
Joined: Fri Jun 04, 2010 9:39 am

Re: GEM data error - values truncation?

Post by ben » Tue Feb 15, 2022 11:51 am

Trying to simulate it here, what type of server are you sending to?
Ben
Brultech Research Inc.
E: ben(at)brultech.com
ben
Site Admin
Posts: 4262
Joined: Fri Jun 04, 2010 9:39 am

Re: GEM data error - values truncation?

Post by ben » Tue Feb 15, 2022 4:24 pm

While we're trying to get it to act up, can you try dropping both COM1 Baud and the baud on the WiFi/Ethernet module to 19200? Wondering if it's a buffer issue.

- In the GEM Setup Webpage under Adv you can change the baud rate GEM-side. Do this first.
- Then on the WiFi/Ethernet Port 80 config page you can change the baud under Application Setup.
Ben
Brultech Research Inc.
E: ben(at)brultech.com
jbai
Posts: 6
Joined: Fri Feb 04, 2022 12:41 pm

Re: GEM data error - values truncation?

Post by jbai » Wed Feb 16, 2022 6:20 pm

Thank you, Ben. A colleague or I will post updates/questions in the coming days as we dig into this and collect data.

To answer your prior question, our GEM data is being sent to an Apache/mod_wsgi server on Ubuntu 20 on an Oracle cloud VM.
Post Reply