Speeding up nginx mycontroller proxy

[Avamander]

To increase security, place nginx under a subdomain and to get rid of "insecure" https warning I set up nginx as a proxy. The downside was that this significantly increased the load on the Raspberry Pi making mycontroller load really slowly, so I found a workaround and even managed to make the proxied version load a bit faster (about 33% (1sec)) than mycontroller alone did and that all with stronger crypto and smaller download sizes :D.

Nginx configuration:

   location / {
       # I'm sending HSTS header with this config
       include security_headers.conf;
       # Should contain pushed files
       include mycontroller_http2_push.conf;
       # CPU becomes a bottleneck with this many files or is just ineffective, workaround for that is below
       gzip off; 
       tcp_nopush on;
       tcp_nodelay on;
       # This would benefit if mycontroller sent `Link` headers
       http2_push_preload on;
       # You'll get smaller overhead if this is disabled 
       proxy_buffering off;

       # Just as a courtesy
       proxy_set_header        X-Real-IP $remote_addr;
       proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

      # Port and IP of mycontroller
       proxy_pass              http://127.0.0.1:8443;
       charset                 utf-8;
   }

   location ~ ^/(app.js|app.css|services|partials|libs|languages|images|_configurations|controllers){
       gzip on;
       gzip_static on;
       root /[mycontroller_install_directory]/www/;
   }

In order to actually make gzip_static on work you should compress all the files in mycontroller's installation's subdirectory www before with this script, it should be ran every time you update mycontroller:

find . -type f -name "*.css" -exec sh -c "touch {} && \
    gzip -9v < {} > {}.gz" \;

find . -type f -name "*.js" -exec sh -c "touch {} && \
    gzip -9v < {} > {}.gz" \;

find . -type f -name "*.html" -exec sh -c "touch {} && \
    gzip -9v < {} > {}.gz" \;

In order to speed up downloads I recommend you utilize http2 push too, a separate configuration file for that is a good idea:

http2_push /libs/angular-resource/angular-resource.min.js?mcv=20;
http2_push /libs/bootstrap-duallistbox/dist/jquery.bootstrap-duallistbox.min.js?mcv=20;
http2_push /controllers/resources-group.js?mcv=20;
http2_push /controllers/sensors.js?mcv=20;
http2_push /libs/patternfly/dist/css/patternfly.min.css?mcv=20;
http2_push /libs/weather-icons/css/weather-icons.min.css?mcv=20;
http2_push /libs/angular-bootstrap-datetimepicker/css/datetimepicker.css?mcv=20;
http2_push /libs/kubernetes-topology-graph/topology-graph.css?mcv=20;
http2_push /libs/d3/d3.min.js?mcv=20;
http2_push /libs/bootstrap-switch/dist/js/bootstrap-switch.min.js?mcv=20;

I tried pushing more but either nginx or chrome does not accept any other files than these, further improvement could be made by making the controller push resources with Link and nginx having http2_push_preload on; but it doesn't matter much on LAN because round trip takes so little time.

For a few additional milliseconds shaved off you can add fastopen=100 to your listen lines (make sure TCP fast open is also enabled by your OS).

If you want you can also compile nginx with brotli support and further compress all the required files, my tests have shown it's about 5% increase in speed compared to static gzip.

[jkandasa]

@avamander This is great sharing! Thank you so much!
Let me know If I have any action item on this task.

[skywatch]

@Avamander It would be great to have a tutorial on how to do this in a step-by-step type instruction.

I tried it today and got to a dead end. I maybe will try again later.

[Avamander]

@skywatch What part did you get stuck on?

[Avamander]

It's worth keeping in mind that if you copy the contents of www to some other folder, make sure you also create _configurations folder in it and allow mycontroller to write into it, otherwise the UI doesn't load.

[skywatch]

@Avamander said in Speeding up nginx mycontroller proxy:

@skywatch What part did you get stuck on?

Everything after you said "To increase security,"

I get the idea, I just don't know what commands to follow to make it all happen. I have a DDNS domain I can use and can install nginx, then I get lost. !

But lets wait for ver 2.0 of MyController before you put any time into helping on this, it might not be needed....

[Avamander]

@skywatch Hm, I'll try splitting it once again:

1. Preliminary requirement, when you have your domain and nginx, follow a tutorial to set up LetsEncrypt with nginx, there are plethora of guides way better than what I can explain

2. If you have your configuration (probably in /etc/nginx/sites-available/default) then you find the server { section that has your TLS (SSL) configured in step 1, then you just paste in the described content

3. Find the folder where you installed mycontroller, the folder has a www folder in, find [mycontroller_install_directory] with that path (e.g. /home/mycontroller/www)

4. The same www folder, run the find ... commands in the folder, it automatically precompresses the folders's contents

5. Make a new configuration file in /etc/nginx (e.g. mycontroller_http2_push.conf) and place the lines that start with http2_push in the file. Make sure the filename in the location / block matches with the configuration file name you made.

6. Restart nginx and it should now display you mycontroller with encrypted connection and really rather fast.