Your advice is good and I have already done a lot of what you suggested. I didn't put a scope on it, I may make another portable system and try that then as I do not see any pullups on the pro mini nor on the pi3 schematics. That could well be the centre of my problems.
This also seems to pinpoint issues with serial on the Raspberry Pi 3B, and why they happen. Seems the foundation made changes that have had a detrimental effect on serial comms stability. It would explain why this happened after 'upgrading' from Pi 2B to Pi 3B. It seems conclusive to me that the issue is with the RPi 3B serial port not working correctly.....Live and learn I guess
Still, for the last 4 days I have been running with a serial to usb adapter and no ghost nodes and no data spikes on the graphs. I wonder if there are pullups on the board? It's difficult to see clearly with the plastic wrapper on it!
Things are looking very hopeful right now though...
The only issue I still face is random loss of data from some nodes. The outages can be just one value report missing (Most nodes send data once every 5 minutes), right up to over 6 hours of data loss. Then suddenly it all starts working again. I might have more time now to look into that a bit morenow the system is stable again, but it is so random.
I did also find quite a few 'less than good' dupont cables (from China) - So I have soldered directly a lot of connections now or modified/re-crimped the cables that were 'iffy'.
Thanks again to you and Jkandasa for the support and help. I am nearly there with this now and can build some more nodes with confidence.
Can you post the top of your sketch (the mysensors stuff)...
I had the same issue some time ago where the sensor names that I had set in mycontroller got overwritten, most probably when I upgraded to a new release..
I fixed this by hard coding the sensor description names into the sensors presentation loop:
childSensorId - The unique child id you want to choose for the sensor connected to this Arduino. Range 0-254. sensorType - The sensor type you want to create. description An optional textual description of the attached sensor. ack - Set this to true if you want destination node to send ack back to this node. Default is not to request any ack.
There is one minor difference - when using the serial gateway all my repeaters were interconnected with lines, now they are all islands. Before I believe that only non-repeater showed as unconnected. I kind of liked that distinction - Why is this so ?
There is no difference in MyController side for MQTT or Serial. On gateway, page select the gateway and execute Discover.
Discover job will be executed 30 minutes once I guess(default).
Your MQTT gateway RF may be weaker than serial gateway RF. When enabling ack for your gateway might resolve this issue. If the message failed to send, MyController retries N(in this image 5 times) times in the interval of 1 second(1000 ms).