Best way to monitor 2 ECM-1240s w/ btmon?

[sandeen]

I'm about to get my 2nd ECM-1240 set up (yep - should have just gotten the GEM instead, but water under the bridge!)

I have my current one on an rPi running btmon/mtools in realtime mode, and it's working great.

Any idea what the best way to add the 2nd might be? I figure 3 choices:

• Multiplex serial cable, realtime, with different intervals set (say, 10 and 11 seconds) to avoid ongoing collisions.
  Multiplex serial, polling, so that btmon can orchestrate accesses.
  Use two serial ports, both ECM's in realtime mode.

If I did the last one, I suppose I'd need to have two copies of btmon running, with two different config files, to define channels and serial port. (And then I'd need to get some usbserial naming persistence, but I can work that out).

What's best practice here?

[ben]

I'm about to get my 2nd ECM-1240 set up (yep - should have just gotten the GEM instead, but water under the bridge!)

I have my current one on an rPi running btmon/mtools in realtime mode, and it's working great.

Any idea what the best way to add the 2nd might be? I figure 3 choices:

• Multiplex serial cable, realtime, with different intervals set (say, 10 and 11 seconds) to avoid ongoing collisions.
  Multiplex serial, polling, so that btmon can orchestrate accesses.
  Use two serial ports, both ECM's in realtime mode.

If I did the last one, I suppose I'd need to have two copies of btmon running, with two different config files, to define channels and serial port. (And then I'd need to get some usbserial naming persistence, but I can work that out).

What's best practice here?

Can two instances of btmon.py run on the same database? If so, using a USB hub and 2 separate serial-to-USB cables would likely be the easiest solution to setup.

If you don't mind running an extra power cable, an active multiplexer would likely be the best bet if you want them both running within the same instance of btmon.py. Would completely avoid possible collisions.

Ben

[sandeen]

Can two instances of btmon.py run on the same database? If so, using a USB hub and 2 separate serial-to-USB cables would likely be the easiest solution to setup.

I would think it should work; if so, I'll just do that. I got my 2nd usb-serial device today, and got the rules set up so that they stay consistently named.

[sandeen]

For what it's worth, I ended up running 2 btmon.py instances on the rPi, each collecting data from its own serial port; they both send data only to a MySQL database on my main server, which runs a 3rd (!) btmon.py instance to send the data out to various services.

This seems to work just fine, and saves write cycles on the SD card in the rPi, too.

[mark_anderson_us]

For what it's worth, I ended up running 2 btmon.py instances on the rPi, each collecting data from its own serial port; they both send data only to a MySQL database on my main server, which runs a 3rd (!) btmon.py instance to send the data out to various services.

This seems to work just fine, and saves write cycles on the SD card in the rPi, too.

Hi Sandeen

My two ECM-1240's have been idle for months, as I've had nothing but problems with ECM gateway software. Sound like you're using a Pi (I have a spare one). Would you mind sharing what you've done. I'd like to store info in MS SQL server and or send to web service it possible

Thanks

Mark

[sandeen]

My two ECM-1240's have been idle for months, as I've had nothing but problems with ECM gateway software. Sound like you're using a Pi (I have a spare one). Would you mind sharing what you've done. I'd like to store info in MS SQL server and or send to web service it possible
Mark

Hi Mark - sorry, missed this reply (subscribed to topic now).

I basically installed btmon on the pi, as outlined at http://lancet.mit.edu/mwall/projects/po ... howto.html, and run 2 instances of it, with 2 different config files, talking to 2 different USB adapters. Note that in the config files they are identified by serial number, so they never get mixed up.

The initscript I use on the pi is here, and the 2 configuration files they run with are here and here. (Hm, note that those "here's" are links).

The 2 configfiles on the pi simply tell it to send data to another host running mysql, in the [mysql] section. All other destinations are set to false.

On the 2nd host, which is running the MySQL server, btmon is running with the config file here. It has one source listed (the database), and then other destinations configured as you wish. The actual config file I posted is truncated, I removed all my service keys & usernames etc, so you'll need to use it as a guide, vs. dropping it right in.

[mark_anderson_us]

My two ECM-1240's have been idle for months, as I've had nothing but problems with ECM gateway software. Sound like you're using a Pi (I have a spare one). Would you mind sharing what you've done. I'd like to store info in MS SQL server and or send to web service it possible
Mark

Hi Mark - sorry, missed this reply (subscribed to topic now).

I basically installed btmon on the pi, as outlined at http://lancet.mit.edu/mwall/projects/po ... howto.html, and run 2 instances of it, with 2 different config files, talking to 2 different USB adapters. Note that in the config files they are identified by serial number, so they never get mixed up.

The initscript I use on the pi is here, and the 2 configuration files they run with are here and here. (Hm, note that those "here's" are links).

The 2 configfiles on the pi simply tell it to send data to another host running mysql, in the [mysql] section. All other destinations are set to false.

On the 2nd host, which is running the MySQL server, btmon is running with the config file here. It has one source listed (the database), and then other destinations configured as you wish. The actual config file I posted is truncated, I removed all my service keys & usernames etc, so you'll need to use it as a guide, vs. dropping it right in.

Thanks a lot. Actually got it up and running with 2 serial connections to my home automation server using two instances of BTMon

Regards

mark

[Dax333]

Hi Sandeen.

I'm trying to implement your setup and I'm getting stuck on the 2nd host part. For first implementation I'm using just one rPi running 2 instances of btmon. 1st script is logging to localhost mysql without errors. Second instance connecting to database (even if the database doesn't exist) gives me this:

./btmon.py --config-file=btmon-upload.cfg -p
2014/12/02 22:32:53 device type: ecm1240
2014/12/02 22:32:53 device list: ['fc']
2014/12/02 22:32:53 packet format: ecm1240bin
2014/12/02 22:32:53 schema: counters
Traceback (most recent call last):
File "./btmon.py", line 3880, in <module>
options.mysql_poll_interval or DB_POLL_INTERVAL)
File "./btmon.py", line 2273, in __init__
super(MySQLCollector, self).__init__(database+'.'+table, poll_interval)
File "./btmon.py", line 2228, in __init__
infmsg('DB: polling interval: %d seconds' % self._poll_interval)
TypeError: %d format: a number is required, not str

If I comment out the POLL line:
mysql_read = true
#mysql_poll_interval = 60
mysql_src_host = localhost
mysql_src_user = ecmuser
mysql_src_passwd = ecmpass
mysql_src_database = ecm
mysql_src_table = ecmdata

I get this:
pi@rpi1 ~/btmon $ ./btmon.py --config-file=btmon-upload.cfg -p
2014/12/02 22:59:31 device type: ecm1240
2014/12/02 22:59:31 device list: ['fc']
2014/12/02 22:59:31 packet format: ecm1240bin
2014/12/02 22:59:31 schema: counters
2014/12/02 22:59:31 DB: polling interval: 60 seconds
2014/12/02 22:59:31 DB: table: ecm.ecmdata
2014/12/02 22:59:31 MYSQL: host: localhost
2014/12/02 22:59:31 MYSQL: username: ecmuser
2014/12/02 22:59:31 MYSQL: database: ecm
2014/12/02 22:59:31 SEG: upload period: 60
2014/12/02 22:59:31 SEG: url: http://api.smartenergygroups.com/sites/hidden.xml
2014/12/02 22:59:31 SEG: token: hidden
2014/12/02 22:59:31 SEG: map:

No Data is making it to SEG (api trace) and it appears the script is just hanging. Verified SEG does work if I send straight from serial to SEG using btmon (avoiding MYSQL)

I've also verified that changing the sql password on btmon-upload.cfg and collect1.cfg both cause access failures.

Testing using btmon 301 and 303.

Any ideas on what I may be doing wrong?
Thanks,
Dax

[mark_anderson_us]

Hi Sandeen.

I'm trying to implement your setup and I'm getting stuck on the 2nd host part. For first implementation I'm using just one rPi running 2 instances of btmon. 1st script is logging to localhost mysql without errors. Second instance connecting to database (even if the database doesn't exist) gives me this:

./btmon.py --config-file=btmon-upload.cfg -p
2014/12/02 22:32:53 device type: ecm1240
2014/12/02 22:32:53 device list: ['fc']
2014/12/02 22:32:53 packet format: ecm1240bin
2014/12/02 22:32:53 schema: counters
Traceback (most recent call last):
File "./btmon.py", line 3880, in <module>
options.mysql_poll_interval or DB_POLL_INTERVAL)
File "./btmon.py", line 2273, in __init__
super(MySQLCollector, self).__init__(database+'.'+table, poll_interval)
File "./btmon.py", line 2228, in __init__
infmsg('DB: polling interval: %d seconds' % self._poll_interval)
TypeError: %d format: a number is required, not str

If I comment out the POLL line:
mysql_read = true
#mysql_poll_interval = 60
mysql_src_host = localhost
mysql_src_user = ecmuser
mysql_src_passwd = ecmpass
mysql_src_database = ecm
mysql_src_table = ecmdata

I get this:
pi@rpi1 ~/btmon $ ./btmon.py --config-file=btmon-upload.cfg -p
2014/12/02 22:59:31 device type: ecm1240
2014/12/02 22:59:31 device list: ['fc']
2014/12/02 22:59:31 packet format: ecm1240bin
2014/12/02 22:59:31 schema: counters
2014/12/02 22:59:31 DB: polling interval: 60 seconds
2014/12/02 22:59:31 DB: table: ecm.ecmdata
2014/12/02 22:59:31 MYSQL: host: localhost
2014/12/02 22:59:31 MYSQL: username: ecmuser
2014/12/02 22:59:31 MYSQL: database: ecm
2014/12/02 22:59:31 SEG: upload period: 60
2014/12/02 22:59:31 SEG: url: http://api.smartenergygroups.com/sites/hidden.xml
2014/12/02 22:59:31 SEG: token: hidden
2014/12/02 22:59:31 SEG: map:

No Data is making it to SEG (api trace) and it appears the script is just hanging. Verified SEG does work if I send straight from serial to SEG using btmon (avoiding MYSQL)

I've also verified that changing the sql password on btmon-upload.cfg and collect1.cfg both cause access failures.

Testing using btmon 301 and 303.

Any ideas on what I may be doing wrong?
Thanks,
Dax

Hi Dax

I got this working (but with serial). My batch file that starts it looks like this:

c:
cd \btmon
btmon.py -c config.cfg

[source]
device_type = ecm1240
device_list = fc
serial_read = true
serial_port = COM13
serial_poll_interval = 30
[seg]
smartenergygroups_out = true
seg_upload_period = 60
seg_token = YOUR_TOKEN_HERE

You have to leave the name SEG creates during auto discovery (e.g. XXXX123) for it to work

Hope this helps

Regards

mark

[Dax333]

For this to work I assume I have to take my ecm's out of real time mode?

I tested the settings but get:
2014/12/03 03:22:35 wrong response 0x0, expected 0xfc
2014/12/03 03:22:35 wrong response 0x41, expected 0xfc
2014/12/03 03:22:35 wrong response 0x0, expected 0xfc
2014/12/03 03:22:35 3 requests failed for device fc

The SQL approach from Sandeen looked useful because I could gather real time info into MySQL and then disseminate it as needed.