I managed to figure it out (I think) by walking through the btmon3 Python code. The trick is to create a massive map that maps the values coming from the GEM streams into fields that you want in the InfluxDB database. I made an assumption that if there was not a map, then the field name would just be the same as the stream name coming in.
Well, you know what they say about "assume".
Through trial and error (and reading the code, although my Python is elementary), I figured out the mapping syntax. I have four GEMs, each with 32 channels (aws and pws), plus 4 pulses, 8 temperatures, and voltage. In all, 77 data points per GEM. That's a lot to deal with, and so I created a spreadsheet with all of the channels and the names of the channels according to my circuit breaker listing. I already had a spreadsheet for the purpose of identifying all of the breakers and which GEM device/channel each one goes to. The purpose of the spreadsheet is to create a really long map string.
Then the task was to identify what was coming over in the stream, and what btmon3 was looking for. This was straightforward using the --debug flag on btmon3.
The spreadsheet looks something like this:
Code: Select all
01021554 ch1 aws HouseL C01 Mistress_attic 01021554_ch1_aws HouseL_C01_Mistress_attic_aws 01021554_ch1_aws,HouseL_C01_Mistress_attic_aws,
01021554 ch1 pws HouseL C01 Mistress_attic 01021554_ch1_pws HouseL_C01_Mistress_attic_pws 01021554_ch1_pws,HouseL_C01_Mistress_attic_pws,
01021554 ch2 aws HouseL C02 Garage_wall_outlets 01021554_ch2_aws HouseL_C02_Garage_wall_outlets_aws 01021554_ch2_aws,HouseL_C02_Garage_wall_outlets_aws,
01021554 ch2 pws HouseL C02 Garage_wall_outlets 01021554_ch2_pws HouseL_C02_Garage_wall_outlets_pws 01021554_ch2_pws,HouseL_C02_Garage_wall_outlets_pws,
01021554 ch3 aws HouseL C03 Hallway_outlets 01021554_ch3_aws HouseL_C03_Hallway_outlets_aws 01021554_ch3_aws,HouseL_C03_Hallway_outlets_aws,
01021554 ch3 pws HouseL C03 Hallway_outlets 01021554_ch3_pws HouseL_C03_Hallway_outlets_pws 01021554_ch3_pws,HouseL_C03_Hallway_outlets_pws,
The last column contains all of the maps. It was just a matter of pasting that into a code editor and replacing all newlines with nothing.
Putting these together, I was able to create a massive influxdb_map field that had all fields mapped. So my configuration file now looks something like this:
Code: Select all
[source]
device_type=gem
ip_read=true
ip_port=8003
ip_mode=server
[influxdb]
influxdb_out=true
influxdb_host=localhost
influxdb_port=8086
influxdb_upload_period=60
influxdb_username=admin
influxdb_password=pword
influxdb_database=btmon
influxdb_measurement=energy
influxdb_mode=row
influxdb_map=01021554_ch1_aws,HouseL_C01_Mistress_attic_aws,
01021554_ch1_pws,HouseL_C01_Mistress_attic_pws,
01021554_ch2_aws,HouseL_C02_Garage_wall_outlets_aws,
01021554_ch2_pws,HouseL_C02_Garage_wall_outlets_pws,
01021554_ch3_aws,HouseL_C03_Hallway_outlets_aws,
01021554_ch3_pws,HouseL_C03_Hallway_outlets_pws,
...etc
01021578_t7,Barn_T07_Temp_7,
01021578_t8,Barn_T08_Temp_8,
01021578_volts,Barn_V_volts
influxdb_db_schema=counters
I broke up this example to show the mapping pairs, but it's important that there be no whitespace (space, tab, newline) in the map. That screws up the parsing of the map when it loads. Also, make sure that the last entry does not have a trailing comma.
Also, there was a bug in the btmon3 software the caused a failure-to-insert-record error if a field came along that wasn't matched. That would explain the "Exception in InfluxDBProcessor: 'ch1_a'" error in the first post of this tread.
The error is in the process_calculated function around line 4280. I added a check that the key exists in the map:
Code: Select all
for p in packets:
dev_serial = obfuscate_serial(p['serial'])
for c in PACKET_FORMAT.channels(self.db_schema):
key = mklabel(p['serial'], c)
if self.map and key not in self.map: # BT: added to avoid error later on
continue
I haven't tested this with other configurations, but it seems to work with InfluxDB.
