Hi
Welcome to the forums.
Sorry, but not seeing this behavior here at all. Have you expanded the time line enough to really work on the audio track?
The behavior here works.
- Right click on a handle, chose delete volume setting -- it is gone (alternate is to drag the handle to be deleted down and out of the audio track, - left mouse button, drag and drop.
- Moving a handle, does not create a new handle here.
If the time line is not expanded enough, then the handles are a little slippery

and one can easliy make a new one.