Posts made by Avamander

Happy holidays to everyone

I too like source and data.

The demo is also looking rather nice.

2021 the year of MyController 2.0? - I can't wait

Same.

@skywatch said in Frequent errors related to Ethernet Gateways:

There it is said that watchdog is not working in standard 'Chinese' bootloader.

Yes, some products ship with the old bootloader.

You can either use an existing Arduino or an USBASP as an ISP programmer to flash a new bootloader, not tremendously difficult if you just wire it up and press the button in Arduino IDE.

@Daniele
I still have this question though, instead of watchdog, did you check that your ethernet module/code just renews its DHCP lease? I forgot to do that on one of my projects and had very similar issues. You should also maybe add a piece of code that sends ICMP ping to your gateway every 1h to make sure ARP cache doesn't time out - avoids that issue too.

@Daniele

That's the same thing I thought, but having 4 of them all failing sounds a bit odd....

Maybe the gateways' DHCP lease runs out, they lose their IP and don't ask it back soon enough?

@jkandasa said in Forum logo transparency:

NodeBB

OH, this is NodeBB not Discourse, they're very similar... I unfortunately don't know if NodeBB has such features.

Though, it's possible to migrate: https://meta.discourse.org/t/thinking-of-switching-from-nodebb-to-discourse/85658/6

@skywatch Hm, I'll try splitting it once again:

1. Preliminary requirement, when you have your domain and nginx, follow a tutorial to set up LetsEncrypt with nginx, there are plethora of guides way better than what I can explain

2. If you have your configuration (probably in /etc/nginx/sites-available/default) then you find the server { section that has your TLS (SSL) configured in step 1, then you just paste in the described content

3. Find the folder where you installed mycontroller, the folder has a www folder in, find [mycontroller_install_directory] with that path (e.g. /home/mycontroller/www)

4. The same www folder, run the find ... commands in the folder, it automatically precompresses the folders's contents

5. Make a new configuration file in /etc/nginx (e.g. mycontroller_http2_push.conf) and place the lines that start with http2_push in the file. Make sure the filename in the location / block matches with the configuration file name you made.

6. Restart nginx and it should now display you mycontroller with encrypted connection and really rather fast.

I was also wondering, would it be possible to update this instance, it doesn't seem to have browser push notification feature and DiscourseHub support, that would make it more possible to be more responsive in various threads and would probably help to liven up the forum.

When one changes the Discourse theme, the header logo doesn't look nice because it has a white background (when the header is dark blue for example).

I downloaded the logo and made the background transparent, it should now work somewhat better with other themes, the very least it doesn't stick out:

@skywatch A-ha, I understood that you had problems with the MySensors code

@skywatch

First thing is that delays are a bad thing, they slow your code down, make it impossible to multi-task and at some point the delays definitely get unmanageable. Instead you could do this:

void setup() {
   // Set your pinModes and everything
}


uint64_t timer = 0;
uint8_t mode = 0;
void loop() {
    // You can do a lot more here now, check your sensors every 100 milliseconds if you wish
    // Because this loop runs really really fast unless the timer below triggers
    // Doing a moving average of a brightness sensor wouldn't be a bad idea :)
    if (millis() - timer >= 500) {
        // It has been ~500ms since the last time this ran
        if (mode == 0 && sensor_a <= value && sensor_b == 1) { // If it's the first time sensor values match lights on
            mode = 1;
        } else if (mode == 1 && sensor_a <= value && sensor_b == 1) { // It is the second time sensor values match lights on
            sendUpdateToGateway();
            mode = 0;
        } else { // Reset to mode 0 because lights weren't confirmed
            mode = 0;
        }
        timer = millis();
    }
}

It's worth keeping in mind that if you copy the contents of www to some other folder, make sure you also create _configurations folder in it and allow mycontroller to write into it, otherwise the UI doesn't load.

@skywatch What part did you get stuck on?

If you're still getting junk data I suggest you take heavy countermeasures against any interference.

First of all, if you have access to an oscilloscope, connect it to the TX and RX lines, see if you can turn the square-ish signal more square with a pull-up (or pull-down between data wire and 5V (or GND), this is a super common issue. There are massive amounts of videos about it online and how it can solve a lot of problems.

Second thing you should definitely do is decouple all power rails, funky power has caused so many issues I can't even count. Attach like ~1mF to RPi's 5V rail, ~1mF to the Pro Mini's 3.3V rail and ~1-5mF electrolytic + ~50uF ceramic directly soldered to the nRF24L01+'s power pins. I'm one of the maintainers on an nRF24 library and a heavy user of these modules it's incredible how many super weird issues just poor wires or power can cause.

I've tested InfluxDB + MyController on my Raspberry Pi 3 for a few months now (thank you so much again for your work @jkandasa). The setup offered super easy way to submit readings into a database but I noticed weird anomalies and after some time I started seeing corrupt data. I'm going to switch everything back to PostgreSQL and see if that could provide more stability.

Although (memory) heavy, I do feel Java runtime provides significant and important cross-platform support with better performance than say Python or Node. What I mean is that current choice to use Java is not a bad one.

The WebUI is as well a bit heavy on browsers, but it can be sped up significantly with modern tech such as HTTP2 (+PUSH) and TLSv1.3 making it absolutely okay to use - some good CSS could fix pretty much everything wrong with it. What I fear is that going with the latest tech here, expecting good cross-platform support and easy code, will end up worse than what we have now, for example HA's webgui. I propose that MyController provides a good API, something uses that API to render a webGUI (the webgui could use say Grafana by-default to display graphs instead of reinventing the wheel) or to display data in a native Android application.

To increase security, place nginx under a subdomain and to get rid of "insecure" https warning I set up nginx as a proxy. The downside was that this significantly increased the load on the Raspberry Pi making mycontroller load really slowly, so I found a workaround and even managed to make the proxied version load a bit faster (about 33% (1sec)) than mycontroller alone did and that all with stronger crypto and smaller download sizes :D.

Nginx configuration:

   location / {
       # I'm sending HSTS header with this config
       include security_headers.conf;
       # Should contain pushed files
       include mycontroller_http2_push.conf;
       # CPU becomes a bottleneck with this many files or is just ineffective, workaround for that is below
       gzip off; 
       tcp_nopush on;
       tcp_nodelay on;
       # This would benefit if mycontroller sent `Link` headers
       http2_push_preload on;
       # You'll get smaller overhead if this is disabled 
       proxy_buffering off;

       # Just as a courtesy
       proxy_set_header        X-Real-IP $remote_addr;
       proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

      # Port and IP of mycontroller
       proxy_pass              http://127.0.0.1:8443;
       charset                 utf-8;
   }

   location ~ ^/(app.js|app.css|services|partials|libs|languages|images|_configurations|controllers){
       gzip on;
       gzip_static on;
       root /[mycontroller_install_directory]/www/;
   }

In order to actually make gzip_static on work you should compress all the files in mycontroller's installation's subdirectory www before with this script, it should be ran every time you update mycontroller:

find . -type f -name "*.css" -exec sh -c "touch {} && \
    gzip -9v < {} > {}.gz" \;

find . -type f -name "*.js" -exec sh -c "touch {} && \
    gzip -9v < {} > {}.gz" \;

find . -type f -name "*.html" -exec sh -c "touch {} && \
    gzip -9v < {} > {}.gz" \;

In order to speed up downloads I recommend you utilize http2 push too, a separate configuration file for that is a good idea:

http2_push /libs/angular-resource/angular-resource.min.js?mcv=20;
http2_push /libs/bootstrap-duallistbox/dist/jquery.bootstrap-duallistbox.min.js?mcv=20;
http2_push /controllers/resources-group.js?mcv=20;
http2_push /controllers/sensors.js?mcv=20;
http2_push /libs/patternfly/dist/css/patternfly.min.css?mcv=20;
http2_push /libs/weather-icons/css/weather-icons.min.css?mcv=20;
http2_push /libs/angular-bootstrap-datetimepicker/css/datetimepicker.css?mcv=20;
http2_push /libs/kubernetes-topology-graph/topology-graph.css?mcv=20;
http2_push /libs/d3/d3.min.js?mcv=20;
http2_push /libs/bootstrap-switch/dist/js/bootstrap-switch.min.js?mcv=20;

I tried pushing more but either nginx or chrome does not accept any other files than these, further improvement could be made by making the controller push resources with Link and nginx having http2_push_preload on; but it doesn't matter much on LAN because round trip takes so little time.

For a few additional milliseconds shaved off you can add fastopen=100 to your listen lines (make sure TCP fast open is also enabled by your OS).

If you want you can also compile nginx with brotli support and further compress all the required files, my tests have shown it's about 5% increase in speed compared to static gzip.