Write an application that takes an source file of the language
it is written in, counts and printes the number of "tokens" used
in that file.
A token is, in most languages, the smallest syntactical part
of a language that can't be split up using spaces without
changing the effect of the code.
Note that if your language has a preprocessor, includes, macroes etc,
these must NOT be expanded for the count (e.g. count "preprocessing tokens"),
What is tested: parsing
Contributed by Arthur J. O'Dwyer, ajo at nospam.andrew.cmu.edu