In https://github.com/WestHealth/pyvis/blob/master/pyvis/lib/bindings/utils.js there is a statement missing before the code runs (this has to be the first line):

let highlightActive = false;

This simple adding of a line is crucial. It took about three hours to figure this out...
There is a problem if this line is not added: When clicking on an empty part of the graph while "neighborhood_highlight" in the Network() options is enabled an exception in the underlying js library (visjs) is raised because this variable wasn't set before and the library tries to read it.
As a consequence the function "neighbourhoodHighlight" in the library crashes. This causes the graph to follow the mouse and then jump back to its original position when clicking again. This looks and feels really unnatural and definitely is not intended to be like this.
This only happens if no node was selected previously.
With this line added the variable is set, the exception doesn't raise and the behavior is as intended.

While this is not in the main branch this is what I did (for cdn_resources="in_line"):

@staticmethod
def fix_movement_before_first_highlight(original_file_content: str) -> str:
    line_that_need_to_be_added = "let highlightActive = false;"
    return original_file_content.replace("<script>", f"<script>{line_that_need_to_be_added}", 1)