ear bauble, hilt
珥 centers on small decorative objects attached to the ear, like earrings or baubles, and by extension the hilt of a sword, which can be seen as a decorative or functional attachment.
珥 combines 王 (jade) and 耳 (ear), suggesting an ear ornament made of jade. The hilt meaning may derive from a similar decorative or attached object.
The left side 王 (jade) and right side 耳 (ear) together picture a jade ear bauble. Imagine a decorative jade piece hanging from an ear, and by extension, the ornamental hilt of a sword.
For ジ, imagine a jeweled ear bauble that makes a soft 'ji' sound when it sways: ji -> ジ.