A 33% rate of developers just going offline after being "explained" the problem (through copy-pasting in instructions) seems pretty abysmal. I'd be really worried that you're losing quality candidates due to them being offended by having instructions pasted into a text file without any chance for a conversation.
I wouldn't be surprised that 33% of raw candidates for a dev position don't really know how to refactor properly, but in my experience, candidates who don't know what they're doing at least flounder around for a while doing the wrong thing, instead of suddenly leaving an interview.
It might be an easily solveable problem - I wonder how much the candidates were briefed about the blind nature of the screening, and why there wasn't a lot of interaction happening.
Another way to fix it would be to have a developer interact normally with the candidate - including voice chat, discussion, etc. but have the evaluation be done by another developer, who only has access to a recorded screen share without any audio. Seems like it gets most of the benefits without the drawback of seeming cold and impersonal to candidates.
You really shouldn't use percentages to talk about trends if your sample size is less than 100 - it's misleading.
But I wouldn't be surprised if a large amount of people who claim to be software developers can't refactor code. I've seen many of my peers (in a CS degree) become used to churning out code without looking back.
>> I'd be really worried that you're losing quality candidates due to them being offended by having instructions pasted into a text file without any chance for a conversation.
If a developer gets offended over something as simple as that AND goes offline without much word of an explanation as to why, I'd be relieved because I've just dodged a bullet.
That's probably true, but on the other hand - in my experience if you're really interested in attracting real talent, interviews are as much about selling your company to them as evaluating their skills. Building an interview process that doesn't allow interaction beyond pasting instructions might be a signal to the developer that they don't want to work there.
I am commenting based on my past experience. This sort of behaviour is more of a red flag against the developer rather than against the company. I knew someone who displayed duplicate behaviour in the past before I learned my lesson. Happy to let other companies train and deal with these types of people.
I wouldn't be surprised that 33% of raw candidates for a dev position don't really know how to refactor properly, but in my experience, candidates who don't know what they're doing at least flounder around for a while doing the wrong thing, instead of suddenly leaving an interview.
It might be an easily solveable problem - I wonder how much the candidates were briefed about the blind nature of the screening, and why there wasn't a lot of interaction happening.
Another way to fix it would be to have a developer interact normally with the candidate - including voice chat, discussion, etc. but have the evaluation be done by another developer, who only has access to a recorded screen share without any audio. Seems like it gets most of the benefits without the drawback of seeming cold and impersonal to candidates.