Take a roll of cardboard such as the centre piece of a toilet roll - just as an example.
Set the tube over a corner of a box or cube imagine the 'roll' as a flat ended cutting tool machining along the line between vertices.
At first a pyramid shape will be removed - until the full diametre is cutting on the three cube edges, at this point the tool begins to bury itself into the 'box'.
After a while the tool fully enters the cube visible by the curved line along three faces of the cube, at the point where the tool is completely buried - the cut is merely a cylinder until the tool appears at the opposite faces.
The final shapes at the corners in question will be crown like.
This can be modelled with Google sketchup, set a solid cube to dimensions given, add a solid cylindrical shape on common centre being line through opposite vertices of cube.
Extract the cylindrical shape - find the volume of cube remaining.
Can also be calculated - need volume of pyramid shape - maybe use
http://mathworld.wolfram.com/CylindricalWedge.html for the parts twixt pyramid and full cylinder.
My idea on shapes involved :-
You are almost there.
As we were back here :-
https://www.diynot.com/diy/threads/heres-a-maths-one-for-ya.344532/
-0-