Bryan Singer -- 'He Had Every Chance to Rape Me and Didn't' ... Says Gay Disney Star

A former Disney actor is calling BS on allegations Bryan Singer once sexually assaulted an underage actor ... telling us the "X-Men" director could have raped him plenty of times -- but never laid a hand on him.Matthew Smith -- a gay actor who…