You get half your genes from your mother and half from your father. How far back do you need to go until you have ancestors who did not contribute any genes to you?
Apparently (Science Magazine or Human Genome Project), humans have about 25,000 genes.
To get a number of ancestors higher than that, you need to take your family tree back 15 generations; whereupon you'll find 32,768 people as the 'terminal nodes'. How far back in time would that be?
Some branches of your family tree will be shorter than others e.g. there may be one route through the tree where the average generation is 20 years which would take you back 15x20 years = 300 years ago; other branches might have average generations of 25 years, making it 15x25 years = 375 years (presenting the possibility that, at that level in your family tree, some of them may be the grandparents of others at the same level and will therefore re-appear in your family tree at the 18 generation level).
So, there are people 300 or so years ago from whom you are descended but from whom you have inherited no genetic material. Theoretically, because of people marrying cousins (even very distant ones), many of the people in your family tree appear more than once and so are likely, even as far back as 15 generations, to have contributed more than the single gene that you'd expect.
What it all means, in effect, is that, once you've gone back enough generations, you're really not descended from individual people at all but from the gene pool corresponding to the cross-section of humanity that forms the nodes in the complicated graph that is your family tree.
Apparently (BBC News), we share 95% or more of our genes with chimps, which implies that only 1250 or less of your genes would matter in human terms, that would mean going back just 11 generations (2048 ancestors) - a couple of centuries.
So, if anyone ever tells you that they're descended from Nelson or Thomas Jefferson, just tell them:
"Perhaps you are, but not very much..."