Smarthome data logging with RPi/domoticz/influxdb/grafana
(Updated: )
When we got a smart meter and solar panels I wanted to record this data. Along the way I added more sensors and I’ve extended the software
Goals ¶
- Record (smart) meter data and graph it (now including: electricity smart meter via USB, PV via bluetooth, heat meter via machine vision, water meter via line sensor, temperature via EvoHome)
- Log data in generic, long-lasting format locally (i.e. no externally hosted solution)
- Robust
- Interactive graphs on smartphone (domoticz, plot.ly, Highcharts, dygraphs)
- Customisable graphs (like grafana)
- Cheap/low-power solution (i.e. no Intel NUC)
- Versatile wrt interfaces (e.g. bluetooth/wifi/GPIO/USB)
- Not used to control or actuate. For that I use COTS products that work and don’t crash :p
Architecture ¶
Data model ¶
Units ¶
I use only SI units for storing meter data so I can combine e.g. heat meter (native unit MJ) with smart meter (native unit Wh) with PV (native unit kWh).
- Power: W, J/s
- Energy: J
- Temperature: deg C
- Wind: m/s
- Wind direction: deg
- Voltage: V
- Water: liter
- Rain: mm
Database scheme ¶
Influxdb can only combine / add / subtract data from the same measurement. Keep this in mind when designing your scheme.
Generic format: <unit>, type=<quantity>, device=<datasource>, subtype=<device qualifier> value1=X value2=Y value3=Z <timestamp>
.
Fully worked out, this looks like:
Power,type=elec,device=<kaifa/sma/smartplug> energy=X power=Y
Power,type=heat,device=<landisgyr> energy=X
Power,type=radiation,device=solar energy=X
temperature,type=heating,device=livingroom actual=X setpoint=Y
temperature,type=device,device=sma actual=X
temperature,type=weather,device=XXweatherloc actual=X
wind,type=weather,device=XXweatherloc wind=X
winddirection,type=weather,device= XXweatherloc winddir=X
voltage,type=elec,device=sma, DC=X AC=Y
water,type=potable,device=sensus water=X
rain,type=weather,device=XXweatherloc rain=X
Optionally this could all be in one table (called ‘measurement’ in influxdb parlance)
Polling cadence ¶
- Temperature rooms: 1x/5 minutes, resolution: 0.5 deg C
- Power: 1x/min
- PV: 1x/min
- Water: on-demand incremental
- Heat: 1x/minute Data usage at 1 measurement/minute is: 1440min/day * 2byte/measurement ±3kb/day/metric = 1 MB/yr/metric.
Software architecture ¶
- Easy access visualization: domoticz
- Database: influxdb
- Full visualization only accessibly locally: grafana
- Selected visualizations remotely accessible on smartphone: Dygraphs (influxdata.com)
- Data collection: domoticz with pushers to influxdb, as well as custom scripts to push directly to influxdb.
Implementation ¶
Implementation occurred more or less chronologically in this order. I’ve written the text including hindsight to fix initial mistakes:
- Installing domoticz on Rpi (vanwerkhoven.org)
- Extending domoticz with influxdb/grafana
- Connecting sensors to Rpi/domoticz
Alternatives ¶
Before settling on the above solution I’ve considered (and rejected) the following alternative solutions for various reasons:
- hass.io + influxdb: NOK hass.io has no influxdb export, and appears too heavy (never actually tried)
- OpenHab: NOK extremely slow on RPi0, briefly tried on RPi3B but still slow.
- prometheus: not as widely supported as influxdb. Might actually be better because of more powerful query language
- rpi-monitor / pimatic: appears not mature enough
- vis.js / Dygraphs / Highcharts / Rickshaw / Plotly: too complicated
Sources ¶
- Domoticz part 5: Grafana, InfluxDB, Telegraf – beautiful and easy graphs (jokielowie.com) (Archive (archive.org))
- OH webserver (openhab.org)
- How to install Grafana+InfluxDB on the Raspberry Pi (circuits.dk) (Archive (archive.org))
- Comparison to alternatives - Prometheus vs. Graphite (prometheus.io)
- Honeywell API documentation (dirkgroenen.nl)
- Smart meter (videgro.net)
- Custom Widget: Weather Chart (Meteogram) (openhab.org)
- Visualize (and push) your IOT data (thethingsnetwork.org)