Machine Learning: Why is overfitting bad?
This is basically how I explained it to my 6 year old.
Once there was a girl named Mel (“Get it? ML?” “Dad, you’re lame.”). And every day Mel played with a different friend, and every day she played it was a sunny, wonderful day.
Mel played with Jordan on Monday, Lily on Tuesday, Mimi no Wednesday, Olive on Thursday .. and then on Friday Mel played with Brianna, and it rained. It was a terrible thunderstorm!
More days, more friends! Mel played with Kwan on Saturday, Grayson on Sunday, Asa on Monday … and then on Tuesday Mel played with Brooke and it rained again, even worse than before!
Now Mel’s mom made all the playdates, so that night during dinner she starts telling Mel all about the new playdates she has lined up. “Luis on Wednesday, Ryan on Thursday, Jemini on Friday, Bianca on Saturday -”
Mel frowned.
Mel’s mom asked, “What’s the matter, Mel, don’t you like Bianca?”
Mel replied, “Oh, sure, she’s great, but every time I play with a friend whose name starts with B, it rains!”
What’s wrong with Mel’s answer?
Well, it might not rain on Saturday.
Well, I don’t know, I mean, Brianna came and it rained, Brooke came and it rained …
Yeah, I know, but rain doesn’t depend on your friends.