Firmware Update Failure and failed recovery

Ask questions about the GEM here.
Post Reply
rjsmith0
Posts: 13
Joined: Wed Feb 19, 2014 11:45 pm

Firmware Update Failure and failed recovery

Post by rjsmith0 » Wed Jul 01, 2015 5:05 pm

I have been running my Ethernet/Wifi GEM for over a year now without any issues until recently. I noticed that my GEM would go offline and I would get an email from SEG stating such. When this happens, it would be unresponsive to pings or other network communication but a manual reboot would always resolve the problem. I have been polling the GEM via wifi from two separate systems, btmon for historical data and CQC driver for live data on my HA system. They work fine for a while, but go offline after 8-12 hours. If I disable one of the two polling systems, the system seems to be rock solid.

I thought I would try setting it up with the wired lan connection in case the problem was wifi related. Unfortunately I can never get a link using the LAN port. Today I noticed 3.24 is out and thought I would update with the hope there might be a fix to my problem. I was previously running 3.18 COM and 1.49 ENG. I first tried to run the update using the web interface. That failed, so I tried again using the Network Utility. It got to about 30% complete and appeared to hang. After hanging on the firmware update, I can connect to port 80, but not on port 8000.

I attempted to place it into recovery mode, but it doesn't appear to let me set the baud rate to 19200. When I apply the change to the port, the site appears to hang and the GEM disappears from the network (won't respond to ping) and doesn't come back. When I reset it, I am able to reconnect on port 80 and it is back to 115200 again. I have tried to force the firmware update at 115200 via the network utility and it doesn't get past 0%. I have tried to establish a direct connection via the serial ports and am unable to establish a connection. I have pulled the battery and tried every solution I could find on the forum.

Does anyone have any other suggestions?

Thanks!

Joe
rjsmith0
Posts: 13
Joined: Wed Feb 19, 2014 11:45 pm

Re: Firmware Update Failure and failed recovery

Post by rjsmith0 » Wed Jul 01, 2015 8:38 pm

I have had some success. I completely removed the wifi/ethernet module and I was then able to connect and recover the firmware. Prior to that, any action I took on the web interface, such as changing the baud rate, resulted in the COMM light going out and me losing connectivity to the device. With it update, I added back the wifi/ethernet interface and now when the unit starts up, the COMM light comes on and I am able to ping the device for a few seconds confirming connectivity and then the COMM light turns off and I lose my connection. I have had nothing but problems with this wifi/ethernet interface since day one and have never been able to successfully able to establish a wired ethernet connection (I am a network engineer by trade).

I am open to any suggestions. In the meantime, I am going to try and flash back to 3.18 and see if the problem is 3.24.

thanks,

Joe
rjsmith0
Posts: 13
Joined: Wed Feb 19, 2014 11:45 pm

Re: Firmware Update Failure and failed recovery

Post by rjsmith0 » Wed Jul 01, 2015 10:09 pm

I reverted back 3.18 and it didn't make any difference. What appears to be happening now is that the unit will startup and both LEDs will be green indicating a link to COMM. The unit will respond to a ping, and when I connect to the web interface, I am prompted for authentication. After authentication, it attempts to load the page and then the COMM light immediately goes out and will not come back on until I power cycle the unit. This is reminiscent of the problems I have been having for over a month now, but instead of happening after 8-12 hours of uptime, it happens immediately.

I have tried to reset the comm interface by holding the reset button for 6 seconds and it appears to reset, but then immediately connects to the wireless and responds via the old IP, so I know it isn't resetting. Is there a way to jumper-reset the wifi/ethernet interface? I would love to hear from someone with more experience with this module. I am sure someone from Brultech will be along shortly. After spending 6+ hours on this today I am about ready to take a hammer to it and just go serial. I assume there is a good reason there is a V2 combo interface available now...

Joey
rjsmith0
Posts: 13
Joined: Wed Feb 19, 2014 11:45 pm

Re: Firmware Update Failure and failed recovery

Post by rjsmith0 » Thu Jul 02, 2015 12:03 am

One more finding before I finally give up and turn in. If I don't try to connect to the wifi/ethernet interface management page directly, I AM able to poll the device with btmon and CQC at port 8000 since my previous "Server" config is still present, but the instant I log into the communication board management page, it loses connectivity and the COMM light drops out. So it appears to be functioning for now as a pass thru device only. I don't know how long that will last though. I suspect if I was able to successfully reset it, it could either help or completely stop working. altogether.
rjsmith0
Posts: 13
Joined: Wed Feb 19, 2014 11:45 pm

Re: Firmware Update Failure and failed recovery

Post by rjsmith0 » Thu Jul 02, 2015 7:46 am

Dropped offline after less than 4 hours. :(

With yesterday being Canada Day, I suspect I may not hear back from anyone until Monday. I may attempt setting up a Lantronix serial server in the meantime unless anyone has other suggestions.

Joe
ben
Site Admin
Posts: 4262
Joined: Fri Jun 04, 2010 9:39 am

Re: Firmware Update Failure and failed recovery

Post by ben » Thu Jul 02, 2015 9:47 am

Joe,

How old is the unit? Do you have a Mini-USB and a 5VDC USB adapter around?

Try ditching the 5V adapter and using a Mini-USB or a different 5V adapter (500mA-2A, center positive barrel plug, 2.1mm * 5.5mm). I would suspect a bad power supply if removing the WiFi/Ethernet module allowed you to do things to the GEM (and the COMM LED dropping out).
Ben
Brultech Research Inc.
E: ben(at)brultech.com
rjsmith0
Posts: 13
Joined: Wed Feb 19, 2014 11:45 pm

Re: Firmware Update Failure and failed recovery

Post by rjsmith0 » Thu Jul 02, 2015 2:33 pm

Thanks for the reply. Mine was purchased in february of last year. My serial number is 875. I do not think I have a Mini USB adapter on my unit. I probably do have a different 5V adapter in my stockpiles. I will check when I get back home and give it a go.

Thanks!
ben
Site Admin
Posts: 4262
Joined: Fri Jun 04, 2010 9:39 am

Re: Firmware Update Failure and failed recovery

Post by ben » Thu Jul 02, 2015 3:20 pm

rjsmith0 wrote:Thanks for the reply. Mine was purchased in february of last year. My serial number is 875. I do not think I have a Mini USB adapter on my unit. I probably do have a different 5V adapter in my stockpiles. I will check when I get back home and give it a go.

Thanks!
Every GEM has the mini-USB port, it's between the PT and 5V inputs.
Ben
Brultech Research Inc.
E: ben(at)brultech.com
rjsmith0
Posts: 13
Joined: Wed Feb 19, 2014 11:45 pm

Re: Firmware Update Failure and failed recovery

Post by rjsmith0 » Thu Jul 02, 2015 11:45 pm

I found a USB power supply and after connecting it, it seems to have resolved all of my issues. Not only can I connect to the network management interface successfully without losing the COMM interface, but I also have a link when using a wired connection (first time since I have owned the GEM). I suspect this will resolve all my problems, but time will tell. Thanks so much for the help!
Post Reply