Why does Snow Black not seem to have any problem with hooking up with Haruyuki despite his physical appearance? It's because of the fact that she's older than she looks thanks to Brain Burst, so she's developed past the point where she judges people based on looks or cares about what others think of her taste in romantic partners.
More than the Neurolinker itself, what could happen if it were misused.
The Neurolinker can effectively hack sight, hearing, touch, smell, taste and time perception. This only can happen while the Neurolinker is connected to the brain, but...
...they are ubiquitous, to the point that even ID checking, or photographies transfer, is done via Neurolinker. At least in daily life, people use them constantly, and rarely put them off or remove them.
One of them is explicitly noted in the Light Novel, volume 3, chapter 6. After Haruyuki's augmented vision is hacked via a virus-infected picture, he notes that he could have perfectly been killed that way, if the hacker decided to make him misread the traffic lights.
And then there was the backdoor program that was installed in Chiyu's Neurolinker, which could allow the intruder to not only route his network connection through it, but to see and hear everything Chiyu does.
Short after Haruyuki meets Aqua Current, it's shown that the NL can selectively delete memories, presumably permanently. Also, in the first chapters it's shown it can read and interpret the fears and obsessions of its owner, so it should be possible to read any memory and release it to the global net, including passwords, pincodes, and criminal evidence.
It's suspected, with the Chrome Disaster incident, that it can alter the personality of its owner.
With all that said, simply removing the Neurolinker to never put it on again buffers you against all those dangers. The trouble is if you have been using it practically since birth.
This is probably why the predecessor, Brain Chip, is illegal. At the very least you can remove and switch off a Neurolinker. And both Kuroyukihime and Noumi shows that you can limit your Neurolinker's exposure to the outside world if you're very careful. (Noumi's case might be a bad example, as he only needs his Neurolinker/Brain Chip when he requires Brain Burst, and can use the other to function normally in the Neurolinked society)
Even then, literally almost all of society is based around Neurolinker interaction, from school curriculum to bank transactions to even unlocking the door to your own house. Operating without a Neurolinker would be the equivalent of being physically handicapped.
Based on Chiyu's reaction in episode 14 when Seiji sadistically demands that she become his pet and earlier remarks that a direct NL-to-NL cable connection is regarded as lovers-level of intimate, it is completely possible to commit cyber-rape through an NL.