Japanese nutmeg, plum-yew
榧 refers to a specific evergreen tree: the Japanese nutmeg or plum-yew, valued for its wood and seeds.
榧 combines 木 (tree) with 匪, which likely contributes the sound ヒ. The character was created to name the Japanese nutmeg tree.
A tree 木 with the right side 匪 (which looks like a box 匚 filled with wings 非) — imagine a tree whose seeds look like little winged boxes: the Japanese nutmeg.
For ヒ, picture a tree that makes you say 'hee!' in surprise when you see its unusual winged seeds: hee -> ヒ.