Looping thru sensor time series

[jkandasa]

@daniele Yes, you may need more than one rule here.

Another idea, You can create a dummy sensor variable in a node and change that variable value from a rule with dampening option(of course two rules to ON and OFF), in your regular script(with another rule) do action based on the dummy sensor status.

[Daniele]

@jkandasa , your ideas are always very useful, I'm really thankful.
At the same time, I understand from your answers that there's no easy way to get the last value of a sensor at a certain point in the past, right?

[jkandasa]

@daniele it is easy to get the last two values of a sensor variable,

I just removed some portation of data, You can get the last data timestamp, value, previousValue. When your node sends data, value will be moved to previousValue.

{
  "id": 1108,
  "variableType": "V_VAR3",
  "metricType": "NONE",
  "timestamp": 1537261706537,
  "value": "100.92.189.23",
  "previousValue": "100.98.193.11",
  "unitType": "U_NONE",
  "readOnly": true,
  "offset": 0,
  "priority": 100,
  "name": "IP Address"
}

If you want to see all the available options in a sensor, node, gateway, or sensor variable, just execute a script from script location, as shown below,

[Daniele]

@jkandasa said in Looping thru sensor time series:

If you want to see all the available options in a sensor, node, gateway, or sensor variable, just execute a script from script location, as shown below,

Yes, I know how to look at node / sensor properties.
I was just struggling if there was some method to get a sensor value at a given time.

[jkandasa]

@daniele

I was just struggling if there was some method to get a sensor value at a given time.

• For binary data you can get based on the data retention settings (Settings >> Metrics >> Data retention settings ). you can adjust binary data retention duration.
• For double values maximum raw data availability will be 1 minute. Beyond this period will be aggregated.

I think you are doing actions with binary data.

we can get metric data in multiple ways, one of the easy ways,

var myImports = new JavaImporter(java.io, java.lang, java.util, java.text, org.mycontroller.standalone.metrics.DATA_TYPE);

with(myImports) {
  var currentTimestamp = Date.now();
  // getMetricData(String uid, Long start, Long end, String duration, String bucketDuration, DATA_TYPE dataType)
  // duration: mn, h, d, m (minute, hour, day, month)
  // bucketDuration: mn, h, d, m, raw (for binary always use "raw")
  // dataType: SENSOR_VARIABLE, NODE_BATTERY_USAGE
  var metricData = mcApi.metric().getMetricData("hall-motion", null, null, "30mn", "raw", DATA_TYPE.SENSOR_VARIABLE);
}

^^ This script returns binary data of the UUID, sensor variable for the last 30 minutes.

The response will be something like this,

{
  "myImports": {
    "DATA_TYPE": {
      "representedClass": "org.mycontroller.standalone.metrics.DATA_TYPE"
    }
  },
  "currentTimestamp": 1537267200721,
  "metricData": [
    {
      "timestamp": 1537265432037,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537265449283,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537265480367,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537265495422,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537265511740,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537265527848,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537265572986,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537265594730,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537265662062,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537265684702,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537265866356,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537265883293,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537266104600,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537266123576,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537266393200,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537266428781,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537266543199,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537266589976,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537266610921,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537266640202,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537266680558,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537266699762,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537266943799,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537266961912,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537266982785,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537266997397,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537267009561,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537267033716,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537267048097,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537267062941,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537267099985,
      "empty": false,
      "state": true
    },
    {
      "timestamp": 1537267116460,
      "empty": false,
      "state": false
    },
    {
      "timestamp": 1537267116479,
      "empty": false,
      "state": false
    }
  ]
}

You can use timestamp and state properties.

Supported API's by script: https://www.mycontroller.org/files/docs/javadoc/apidocs/org/mycontroller/standalone/scripts/api/McScriptApi.html

[Daniele]

This is really interesting, thanks!
I looked at the class documentation, but without any example / comment it's really hard to understand how to use those methods (at least for me, I'm not a very good Java programmer).

I guess that the getMetricsDoubleData method allows me to access the aggregated data.
Could you point me to an usage example?

Thank a lot!

[jkandasa]

@daniele

I looked at the class documentation, but without any example / comment it's really hard to understand how to use those methods (at least for me, I'm not a very good Java programmer).

These are all core metric API's used in MyController. all the graphs are using these API's.

I guess that the getMetricsDoubleData method allows me to access the aggregated data.

Yes, getMetricsDoubleData specific to double data. I recommend to use getMetricData for everything, will be switched automatically in the backend.
NOTE: There is no aggregation happens for binary data. binary data are always raw data.

Could you point me to an usage example?

Can you give a use case, I can write a program based on use case.

[Daniele]

First example I can think about: I'd like to use atmospheric pressure data to build a weather forecast, similar to what has been done here.
Basically the agorithm is based on pressure changes between points at a certain time in the past, so my starting point would be to retrieve the pressure value at (now - x minutes).

[jkandasa]

@daniele you can call as mentioned below,

var myImports = new JavaImporter(java.io, java.lang, java.util, java.text, org.mycontroller.standalone.metrics.DATA_TYPE);

with(myImports) {
  var currentTimestamp = Date.now();
  // getMetricData(String uid, Long start, Long end, String duration, String bucketDuration, DATA_TYPE dataType)
  // duration: mn, h, d, m (minute, hour, day, month)
  // bucketDuration: mn, h, d, m, raw (for binary always use "raw")
  // dataType: SENSOR_VARIABLE, NODE_BATTERY_USAGE
  var metricData = mcApi.metric().getMetricData("water-level", null, null, "30mn", "5mn", DATA_TYPE.SENSOR_VARIABLE); // double data
  var index;
  for(index = 0;index<metricData.length;index++) {
    var _data = metricData[index];
    // your code
  }
}

my starting point would be to retrieve the pressure value at (now - x minutes).

getMetricData("water-level", null, null, "30mn", "5mn", DATA_TYPE.SENSOR_VARIABLE),

Here I have mentioned duration as 30mn, it means, now - 30 minutes. And I have mentioned bucketDuration like 5 minutes (5mn). If you just want to do min, max and avg, just keep bucketDuration as your duration. So that you will get only one point.

result:

{
  "_data": {
    "timestamp": 1537287892414,
    "empty": false,
    "min": 51.7,
    "max": 53.8,
    "avg": 52.75000000000001,
    "samples": 8
  },
  "myImports": {
    "DATA_TYPE": {
      "representedClass": "org.mycontroller.standalone.metrics.DATA_TYPE"
    }
  },
  "currentTimestamp": 1537287892347,
  "index": 6,
  "metricData": [
    {
      "timestamp": 1537286392414,
      "empty": false,
      "min": 49.6,
      "max": 50.3,
      "avg": 49.879999999999995,
      "samples": 10
    },
    {
      "timestamp": 1537286692414,
      "empty": false,
      "min": 49.6,
      "max": 50.3,
      "avg": 50.02,
      "samples": 10
    },
    {
      "timestamp": 1537286992414,
      "empty": false,
      "min": 49.6,
      "max": 50.3,
      "avg": 49.68750000000001,
      "samples": 8
    },
    {
      "timestamp": 1537287292414,
      "empty": false,
      "min": 49.6,
      "max": 50.3,
      "avg": 49.81,
      "samples": 10
    },
    {
      "timestamp": 1537287592414,
      "empty": false,
      "min": 49.6,
      "max": 51.7,
      "avg": 50.455555555555556,
      "samples": 9
    },
    {
      "timestamp": 1537287892414,
      "empty": false,
      "min": 51.7,
      "max": 53.8,
      "avg": 52.75000000000001,
      "samples": 8
    }
  ]
}

[Daniele]

Amazing!
Really thanks