Japanese translation help

[lamperouge_0]

In the sentence 「なんと言おうが、俺の自由にさせてもらうぜ」I am having trouble understanding what 「させてもらう」means. Could someone please help?