Nevermind, I've fixed it
This is how I do it:
On the Outliner, I've seearched for the Fkgroup of the controller where the weapon is connected and moved it under the FKgroup controller of the joint where I want it to be connected wihtout adding weighs to it.