in addition; furthermore; moreover
Used to add information, often with a nuance of 'on top of that' or 'what's more'. Common in both spoken and written Japanese.
彼は頭がいい。その上、努力家だ。
He is smart. Moreover, he is a hard worker.
この店は安い。その上、味もいい。
This restaurant is cheap. On top of that, the food is good.
on top of that; above that
Literally 'on top of that', used to refer to something physically above or in addition to a previously mentioned thing. Often used in explanatory or descriptive contexts.
机の上に本がある。その上にペンが置いてある。
There is a book on the desk. On top of that, a pen is placed.
This restaurant is cheap. What's more, it's delicious.
それに is a casual, everyday way to add a reason or fact, similar in tone to the spoken use of その上 but slightly lighter.
おまけに is a colloquial 'and on top of that', often used for an unwelcome extra circumstance, while その上 is more neutral and can be positive or negative.
そのうえ means 'moreover' and links two full clauses, whereas に加えて attaches to a noun phrase to add it to what precedes.