Benefit of the LLVM compiler

I just use Inno Setup which installs all the DLLs on the users system
and creates the link to the program. I found this to be the easiest
and safest way.

Exactly. A single EXE would nice, but in the end, having DLLs in a folder that nothing but your own program sees is a lot easier than jumping through hoops trying to get it down to a single EXE.