by all rights; properly speaking; legally speaking
Used to state what should be the case according to rules, principles, or common sense, often implying that reality differs. The 'legally speaking' gloss is a specific application of this general meaning.
By all rights, he should take responsibility.
Properly speaking, this document should have been submitted by yesterday.
A fixed expression composed of 本来 (originally, essentially), から (from), 言う (to say), and the conditional ば. Literally 'if speaking from the original standpoint', it has become a set phrase meaning 'by all rights'.