similarity - training and test data

[kkkddder]

In the Week 4-part 01 assignment there is a question - how to make the training and test sets more similar.

However, in the first chapter of textbook, it used hand writting of digital numbers from two population of entirely different writers as training and test data. That is, it required training and test at a certain degree of disimilarity.

I was wondering at which level traning and test data are desired to be similar? Thank you!

[danilnagy]

Great question @kkkddder. You are right that these two points talk about two different kinds of similarity. In the book example, they are using data from different people to make sure the models generalize to handwriting in general and don't overfit to just the handwriting represented in the training set. However, all the categories of digits are evenly represented within each of those sets (both training and test sets have an even distribution of 0s, 1s, etc.) However, in the week 4 exercise, when we import the data and create an arbitrary split between training and test data there is no guarantee that the categories are equally represented in both sets, which could effect how the model learns, and how it generalizes to the test set. Hope this helps!