Maybe in US, but elsewhere it has more to do than with races. We even have word for age racism. Also, for example racism against blacks has not that much to do with some genetic qualities, it is just because blacks have different skin color. It could be different age as well. Different hair style, different speaking style, different clothing, different religion, etc.
UN does not speak anything about age, but the definition is much broader:
"The UN does not define "racism"; however, it does define "racial discrimination": According to the United Nations Convention on the Elimination of All Forms of Racial Discrimination,
the term "racial discrimination" shall mean any distinction, exclusion, restriction, or preference based on race, colour, descent, or national or ethnic origin that has the purpose or effect of nullifying or impairing the recognition, enjoyment or exercise, on an equal footing, of human rights and fundamental freedoms in the political, economic, social, cultural or any other field of public life."
Racism - Wikipedia, the free encyclopedia