Do Android proxy settings apply to all apps on the device
No, they do not apply globally and without root there is no way to force a proxy to be used by all applications. The reason the message you found is worded that way is that it is up to the app creator to respect the proxy settings and use them or do the wrong thing and ignore them.
The reason that this is not the default behavior is because it could present some security risks. If all traffic could be redirected, users could have all of their traffic be going through some bad proxy server that snoops on them so they default to only allowing apps to use proxy if they explicitly ask for it.
So make sure that all devices connected on wifi must allowed to the direct internet connection for specific ports.
For example :
For WhatsApp you need to allow direct internet connection on below ports for all clients
TCP/UDP: 59234, 50318