Looping thru sensor time series



  • Hi,
    I have some PIRs that controls fans thru a relay; I'd like to shut down the fans when nobody is in the room for more than 10 minutes.

    In my first release, I realized it thru 2 condition scripts (one for activate and one to shut down) + 2 rules + 2 operations (to send the 1 or 0 payload to the relay), and it works perfectly.

    Now I'd like to add more variables (on / off, cold / heat, use / don't use PIRs, etc.), and the system of objects to create / maintain becomes very large with my previous approach.

    So I tried to do everything in one script, I managed to do it except for the "dampening" behavior. In this moment I'm using the PIR previous value to check if people are away for a while, it's working but relays on the node script that re-sends the PIR value after some minutes even if the value didn't change.

    var myImports = new JavaImporter(java.io, java.lang, java.util);
    
    with(myImports) {
    
    var sensCaldaia = mcApi.uidTag().getByUid("P0_Relay_Caldaia").getResource();
    var sensVentole = mcApi.uidTag().getByUid("P0_Relay_Ventole").getResource();
    
    var sMode = mcApi.uidTag().getByUid("P0_Mode").getResource().value;
    var sPirMode = parseFloat(mcApi.uidTag().getByUid("P0_PIR_Mode").getResource().value);
    var sCaldaia = parseFloat(mcApi.uidTag().getByUid("P0_Relay_Caldaia").getResource().value);
    var sVentole = parseFloat(mcApi.uidTag().getByUid("P0_Relay_Ventole").getResource().value);
    var sTemp = parseFloat(mcApi.uidTag().getByUid("P0_Temperatura").getResource().value);
    var sTarget = parseFloat(mcApi.uidTag().getByUid("P0_Temp_Target").getResource().value);
    
    var sPirC = parseFloat(mcApi.uidTag().getByUid("P0_PIR_Caldaia").getResource().value);
    var sPirT = parseFloat(mcApi.uidTag().getByUid("P0_PIR_Tavolo").getResource().value);
    var sPirF = parseFloat(mcApi.uidTag().getByUid("P0_PIR_Finestra").getResource().value);
    var sPirS = parseFloat(mcApi.uidTag().getByUid("P0_PIR_Studio").getResource().value);
    
    var sPPirC = parseFloat(mcApi.uidTag().getByUid("P0_PIR_Caldaia").getResource().previousValue);
    var sPPirT = parseFloat(mcApi.uidTag().getByUid("P0_PIR_Tavolo").getResource().previousValue);
    var sPPirF = parseFloat(mcApi.uidTag().getByUid("P0_PIR_Finestra").getResource().previousValue);
    var sPPirS = parseFloat(mcApi.uidTag().getByUid("P0_PIR_Studio").getResource().previousValue);
    
    var vCaldaia = 0;
    var vVentole = 0;
    var vResult = false;
    
    if ( sMode == "CoolOn" ) {
        vCaldaia = 0;
        if ( sPirMode == 1 && (sPirC == 1 || sPirT == 1 || sPirF == 1) )
            vVentole = 1;
        else if ( sPirMode == 0 )
            vVentole = 1;
    }
    
    if ( sMode == "HeatOn" && sTemp  <= sTarget -0.5 ) {
        if ( sPirMode == 1 
            && (
                sPirC == 1 || sPirT == 1 || sPirF == 1 || sPirS == 1 ||
                sPPirC == 1 || sPPirT == 1 || sPPirF == 1 || sPPirS == 1
            ) ) {
            vVentole = 1;
            vCaldaia = 1;
        }
        else if ( sPirMode == 0 ) {
            vVentole = 1;
            vCaldaia = 1;
        }
    }
    
    if ( sMode == "Off" ) {
        vCaldaia = 0;
        vVentole = 0;
    }
    
    if ( vCaldaia != sCaldaia || vVentole != sVentole ) {
        vResult = true;
        sensCaldaia.setValue(vCaldaia);
        sensVentole.setValue(vVentole);
        mcApi.sensor().sendPayload(sensCaldaia);
        mcApi.sensor().sendPayload(sensVentole);
    }
    
    mcResult = vResult;
    
    }
    

    Is there an easy way to loop thru sensor values on a time basis?
    Thank you so much for your support (and sorry for the long explanation!)
    Daniele


  • ADMIN

    @daniele You may set up a normal rule to check PIR status and execute a script via operation, there you can write all your logic.



  • But I should have anyway 2 rules, one to turn on and one to turn off, right?
    And at least one other when I want to turn on fans or heating (next step in my project) even if nobody is present in the room.


  • ADMIN

    @daniele Yes, you may need more than one rule here.

    Another idea, You can create a dummy sensor variable in a node and change that variable value from a rule with dampening option(of course two rules to ON and OFF), in your regular script(with another rule) do action based on the dummy sensor status.



  • @jkandasa , your ideas are always very useful, I'm really thankful.
    At the same time, I understand from your answers that there's no easy way to get the last value of a sensor at a certain point in the past, right?


  • ADMIN

    @daniele it is easy to get the last two values of a sensor variable,

    I just removed some portation of data, You can get the last data timestamp, value, previousValue. When your node sends data, value will be moved to previousValue.

    {
      "id": 1108,
      "variableType": "V_VAR3",
      "metricType": "NONE",
      "timestamp": 1537261706537,
      "value": "100.92.189.23",
      "previousValue": "100.98.193.11",
      "unitType": "U_NONE",
      "readOnly": true,
      "offset": 0,
      "priority": 100,
      "name": "IP Address"
    }
    

    If you want to see all the available options in a sensor, node, gateway, or sensor variable, just execute a script from script location, as shown below,
    0_1537262866254_57641e82-de60-4f68-b89d-02bb5f11c518-image.png



  • @jkandasa said in Looping thru sensor time series:

    If you want to see all the available options in a sensor, node, gateway, or sensor variable, just execute a script from script location, as shown below,

    Yes, I know how to look at node / sensor properties.
    I was just struggling if there was some method to get a sensor value at a given time.


  • ADMIN

    @daniele

    I was just struggling if there was some method to get a sensor value at a given time.

    • For binary data you can get based on the data retention settings (Settings >> Metrics >> Data retention settings ). you can adjust binary data retention duration.
    • For double values maximum raw data availability will be 1 minute. Beyond this period will be aggregated.

    I think you are doing actions with binary data.

    we can get metric data in multiple ways, one of the easy ways,

    var myImports = new JavaImporter(java.io, java.lang, java.util, java.text, org.mycontroller.standalone.metrics.DATA_TYPE);
    
    with(myImports) {
      var currentTimestamp = Date.now();
      // getMetricData(String uid, Long start, Long end, String duration, String bucketDuration, DATA_TYPE dataType)
      // duration: mn, h, d, m (minute, hour, day, month)
      // bucketDuration: mn, h, d, m, raw (for binary always use "raw")
      // dataType: SENSOR_VARIABLE, NODE_BATTERY_USAGE
      var metricData = mcApi.metric().getMetricData("hall-motion", null, null, "30mn", "raw", DATA_TYPE.SENSOR_VARIABLE);
    }
    

    ^^ This script returns binary data of the UUID, sensor variable for the last 30 minutes.

    The response will be something like this,

    {
      "myImports": {
        "DATA_TYPE": {
          "representedClass": "org.mycontroller.standalone.metrics.DATA_TYPE"
        }
      },
      "currentTimestamp": 1537267200721,
      "metricData": [
        {
          "timestamp": 1537265432037,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537265449283,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537265480367,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537265495422,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537265511740,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537265527848,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537265572986,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537265594730,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537265662062,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537265684702,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537265866356,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537265883293,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537266104600,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537266123576,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537266393200,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537266428781,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537266543199,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537266589976,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537266610921,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537266640202,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537266680558,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537266699762,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537266943799,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537266961912,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537266982785,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537266997397,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537267009561,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537267033716,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537267048097,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537267062941,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537267099985,
          "empty": false,
          "state": true
        },
        {
          "timestamp": 1537267116460,
          "empty": false,
          "state": false
        },
        {
          "timestamp": 1537267116479,
          "empty": false,
          "state": false
        }
      ]
    }
    

    You can use timestamp and state properties.

    Supported API's by script: https://www.mycontroller.org/files/docs/javadoc/apidocs/org/mycontroller/standalone/scripts/api/McScriptApi.html



  • This is really interesting, thanks!
    I looked at the class documentation, but without any example / comment it's really hard to understand how to use those methods (at least for me, I'm not a very good Java programmer).

    I guess that the getMetricsDoubleData method allows me to access the aggregated data.
    Could you point me to an usage example?

    Thank a lot!


  • ADMIN

    @daniele

    I looked at the class documentation, but without any example / comment it's really hard to understand how to use those methods (at least for me, I'm not a very good Java programmer).

    These are all core metric API's used in MyController. all the graphs are using these API's.

    I guess that the getMetricsDoubleData method allows me to access the aggregated data.

    Yes, getMetricsDoubleData specific to double data. I recommend to use getMetricData for everything, will be switched automatically in the backend.
    NOTE: There is no aggregation happens for binary data. binary data are always raw data.

    Could you point me to an usage example?

    Can you give a use case, I can write a program based on use case.



  • First example I can think about: I'd like to use atmospheric pressure data to build a weather forecast, similar to what has been done here.
    Basically the agorithm is based on pressure changes between points at a certain time in the past, so my starting point would be to retrieve the pressure value at (now - x minutes).


  • ADMIN

    @daniele you can call as mentioned below,

    var myImports = new JavaImporter(java.io, java.lang, java.util, java.text, org.mycontroller.standalone.metrics.DATA_TYPE);
    
    with(myImports) {
      var currentTimestamp = Date.now();
      // getMetricData(String uid, Long start, Long end, String duration, String bucketDuration, DATA_TYPE dataType)
      // duration: mn, h, d, m (minute, hour, day, month)
      // bucketDuration: mn, h, d, m, raw (for binary always use "raw")
      // dataType: SENSOR_VARIABLE, NODE_BATTERY_USAGE
      var metricData = mcApi.metric().getMetricData("water-level", null, null, "30mn", "5mn", DATA_TYPE.SENSOR_VARIABLE); // double data
      var index;
      for(index = 0;index<metricData.length;index++) {
        var _data = metricData[index];
        // your code
      }
    }
    

    my starting point would be to retrieve the pressure value at (now - x minutes).

    getMetricData("water-level", null, null, "30mn", "5mn", DATA_TYPE.SENSOR_VARIABLE),

    Here I have mentioned duration as 30mn, it means, now - 30 minutes. And I have mentioned bucketDuration like 5 minutes (5mn). If you just want to do min, max and avg, just keep bucketDuration as your duration. So that you will get only one point.

    result:

    {
      "_data": {
        "timestamp": 1537287892414,
        "empty": false,
        "min": 51.7,
        "max": 53.8,
        "avg": 52.75000000000001,
        "samples": 8
      },
      "myImports": {
        "DATA_TYPE": {
          "representedClass": "org.mycontroller.standalone.metrics.DATA_TYPE"
        }
      },
      "currentTimestamp": 1537287892347,
      "index": 6,
      "metricData": [
        {
          "timestamp": 1537286392414,
          "empty": false,
          "min": 49.6,
          "max": 50.3,
          "avg": 49.879999999999995,
          "samples": 10
        },
        {
          "timestamp": 1537286692414,
          "empty": false,
          "min": 49.6,
          "max": 50.3,
          "avg": 50.02,
          "samples": 10
        },
        {
          "timestamp": 1537286992414,
          "empty": false,
          "min": 49.6,
          "max": 50.3,
          "avg": 49.68750000000001,
          "samples": 8
        },
        {
          "timestamp": 1537287292414,
          "empty": false,
          "min": 49.6,
          "max": 50.3,
          "avg": 49.81,
          "samples": 10
        },
        {
          "timestamp": 1537287592414,
          "empty": false,
          "min": 49.6,
          "max": 51.7,
          "avg": 50.455555555555556,
          "samples": 9
        },
        {
          "timestamp": 1537287892414,
          "empty": false,
          "min": 51.7,
          "max": 53.8,
          "avg": 52.75000000000001,
          "samples": 8
        }
      ]
    }
    


  • Amazing!
    Really thanks


 

Suggested Topics

3
Online

375
Users

354
Topics

2.1k
Posts