your first learning algorithm (and the world's first!) that "draws the line" between yes and no by adaptively searching for a good line based on data

learning comes with many possibilities in different applications, with our focus being binary classification or regression from a batch of supervised data with concrete features

learning can be "probably approximately correct" when given enough statistical data and finite number of hypotheses

what we pay in choosing hypotheses during training: the growth function for representing effective number of choices

test error can approximate training error if there is enough data and growth function does not grow too fast

learning happens if there is finite model complexity (called VC dimension), enough data, and low training error

learning can still happen within a noisy environment and different error measures