Smarthome data logging with RPi/domoticz/influxdb/grafana
(Updated: )
When we got a smart meter and solar panels I wanted to record this data. Along the way I added more sensors and I’ve extended the software
Goals
- Record (smart) meter data and graph it (now including: electricity smart meter via USB, PV via bluetooth, heat meter via machine vision, water meter via line sensor, temperature via EvoHome)
- Log data in generic, long-lasting format locally (i.e. no externally hosted solution)
- Robust
- Interactive graphs on smartphone (domoticz, plot.ly, Highcharts, dygraphs)
- Customisable graphs (like grafana)
- Cheap/low-power solution (i.e. no Intel NUC)
- Versatile wrt interfaces (e.g. bluetooth/wifi/GPIO/USB)
- Not used to control or actuate. For that I use COTS products that work and don’t crash :p
Architecture
Data model
Units
I use only SI units for storing meter data so I can combine e.g. heat meter (native unit MJ) with smart meter (native unit Wh) with PV (native unit kWh).
- Power: W, J/s
- Energy: J
- Temperature: deg C
- Wind: m/s
- Wind direction: deg
- Voltage: V
- Water: liter
- Rain: mm
Database scheme
Influxdb can only combine / add / subtract data from the same measurement. Keep this in mind when designing your scheme.
Generic format: <unit>, type=<quantity>, device=<datasource>, subtype=<device qualifier> value1=X value2=Y value3=Z <timestamp>
.
Fully worked out, this looks like:
Power,type=elec,device=<kaifa/sma/smartplug> energy=X power=Y
Power,type=heat,device=<landisgyr> energy=X
Power,type=radiation,device=solar energy=X
temperature,type=heating,device=livingroom actual=X setpoint=Y
temperature,type=device,device=sma actual=X
temperature,type=weather,device=XXweatherloc actual=X
wind,type=weather,device=XXweatherloc wind=X
winddirection,type=weather,device= XXweatherloc winddir=X
voltage,type=elec,device=sma, DC=X AC=Y
water,type=potable,device=sensus water=X
rain,type=weather,device=XXweatherloc rain=X
Optionally this could all be in one table (called ‘measurement’ in influxdb parlance)
Polling cadence
- Temperature rooms: 1x/5 minutes, resolution: 0.5 deg C
- Power: 1x/min
- PV: 1x/min
- Water: on-demand incremental
- Heat: 1x/minute Data usage at 1 measurement/minute is: 1440min/day * 2byte/measurement ±3kb/day/metric = 1 MB/yr/metric.
Software architecture
- Easy access visualization: domoticz
- Database: influxdb
- Full visualization only accessibly locally: grafana
- Selected visualizations remotely accessible on smartphone: Dygraphs
- Data collection: domoticz with pushers to influxdb, as well as custom scripts to push directly to influxdb.
Implementation
Implementation occurred more or less chronologically in this order. I’ve written the text including hindsight to fix initial mistakes:
- Installing domoticz on Rpi
- Extending domoticz with influxdb/grafana
- Connecting sensors to Rpi/domoticz
Alternatives
Before settling on the above solution I’ve considered (and rejected) the following alternative solutions for various reasons:
- hass.io + influxdb: NOK hass.io has no influxdb export, and appears too heavy (never actually tried)
- OpenHab: NOK extremely slow on RPi0, briefly tried on RPi3B but still slow.
- prometheus: not as widely supported as influxdb. Might actually be better because of more powerful query language
- rpi-monitor / pimatic: appears not mature enough
- vis.js / Dygraphs / Highcharts / Rickshaw / Plotly: too complicated
Sources
- https://blog.jokielowie.c…f-latwe-i-piekne-wykresy/
- https://community.openhab.org/t/oh-webserver/36756/10
- https://www.circuits.dk/install-grafana-influxdb-raspberry/
- https://prometheus.io/doc…/#prometheus-vs.-influxdb
- http://dirkgroenen.nl/pro…-endpoints-documentation/
http://blog.videgro.net/2017/09/smart-meter/ - https://community.openhab…her-chart-meteogram/39324
- https://www.thethingsnetw…d-push-your-iot-data/1788