The network guessed some of the "correct" answers far to quickly. For example, with just an L shape (|_) it guessed suitcase. Feels like the model is suffering from overfitting.
For me a bit less extreme, but it guessed 'police car' when all i drew was a poor car doodle. there was nothing about that car the would make it a police car.
It's possible, but I'm not entirely convinced that this is honest, or at least not really accurate. It's far too eager to try to match it to something than it is to figure out what it has.
It could figure out most of my drawings but it would get them well in advance of me completing anything substantial (like others, I would be asked to draw a leg and draw just a curved line and it would guess leg before I finished).
Trying to draw what it asked for but with some unusual features (like lines or dot patterns around what it asked for before drawing it) and it gets extremely confused; it doesn't really seem to be good at filtering out any noise: http://imgur.com/a/oE1j2 (gallery of results and what it thought it saw.
Drawing things it didn't ask for just to see what it was guessing resulted in some really strange responses and fits. The answer set it has is extremely limited, so something like a hand giving the horns (\m/) was last guess a duck. A moose was a scorpion, then a duck, then a hand. Godzilla (or a bipedal dinosaur if you prefer) was a vase, then a scorpion, then a boat. My loaf of bread was a washing machine, an anvil, then a postcard. The Deathstar was a bandage, a helicopter, and a lighthouse. And a chainsaw was considered an aircraft.
Between the disruptive patterns and drawing things outside of it's vocabulary, the system seems really confused. Looking at the comparison results, I can see how when drawing some things it got it real fast. (Tennis Rackets were mostly defined by a crosshatch pattern, Harps by a series of parallel vertical lines). This makes sense. For other things, not as much.
It might be a more convincing presentation to give the user a list of items the machine knows (the full list) and tell the user to try to draw some, and then the computer could check it off as it gets them. That seems like a better way of presenting this than "Draw a box. hey! you drew a box! Isn't that cool?"
Huh, I had no clue what you meant until I looked at the picture then back at my scribbles and saw the accidental drawing of the face I made.
To me this is another interesting distinction on the NN recognition versus a human recognition - QuickDraw having a limited "vocabulary" to refer to really highlights this, as does my own lack of knowledge of Roobarb. Some of these things can really blindside us, and I suspect that it's going to require a lot of human hand-holding for awhile for the machines to get a strong vocabulary.
For some time I've been pondering how far you could take a machine's tabula rasa learning, for something like language, and how closely it would mimic a child's learning. (Language, color, math, etc).
Yeah, but it's cheating itself. If it's telling me what it recognizes before I'm done, it severely reduces the usefulness of my input for further training. Anything submitted will just reinforce existing patterns, and do comparatively little to improve recognition.
They just want you to provide annotated training samples. Their guess is visible to you, to gamify this for you and make it more interesting, it might also be used to force you to draw some distinctive features once it shows you some other category.
I wonder if it has a very small pool of potential drawings. Eg if it only has "police car" and no other cars it might be able to jump straight to police car after seeing a car.
It also seems to constantly have a "best guess" to some degree and if that happens to be correct it confirms pretty quickly.
I was asked to draw a "phone". Drew a telephone (the old ones with a rotary dialler, because that was quickest to draw), and it guessed "telephone", but it was incorrect :-(
EDIT: Drew a "cake", it guessed "birthday cake". Wrong answer apparently.
It's because it gets the most plausible answer, even if it's only a small chance of being correct. If there are not that many words, there are only a few objects that could be draw with a L shape like that. I mean, even if you draw something else starting with a L shape, it will probably be a slight different L shape (with a curve, for example). I think that's it, doesn't really mean it's overfitting, it would mean that if the probability was really high.
The ai keep throwing guesses in 20 seconds until he got the right one.
So if you draw a simple shape it start to go trough the list of things he recognize and end the game there.
It works great for this game because he can have very fast answer but only work win the cases when you actually have a feedback that eliminate all the wrong guesses.
Basically if it simply went trought the whole english dictionary fast enough he could get the same result without even looking at the pictures.
Yup, the minimalist Picassoesque renditions I was able to do on my laptop trackpad were still recognised as long as I gave a vague impression of the major shapes I figured other people would draw. The result was the kind of image where a human would frown at it until you said “kangaroo”.
Fun though.