C Tokens Explained – BKNMU Junagadh | BCA Sem 1 Notes

BCA Sem 1  |  Unit 1  |  Computer Concepts & C Programming

C Tokens Explained — BKNMU Junagadh

Keywords, Identifiers, Constants, Strings, Special Symbols, and Operators in C Language

BKNMU Junagadh NEP 2020 C Tokens Lexical Analysis Building Blocks

In simple words: A C Token is the smallest individual unit in a C program that the compiler recognizes during compilation. Just as words, punctuation marks, and numbers are the fundamental units of a sentence, tokens are the basic building blocks used to compose C statements.

📖 The 6 Types of C Tokens

The C compiler breaks every line of source code into 6 distinct token categories during the lexical analysis phase:

int, return, if

1. Keywords

32 reserved words with predefined meanings to the C compiler.

total_marks, main

2. Identifiers

User-defined names for variables, functions, arrays, and structures.

100, 3.14, 'A'

3. Constants

Fixed values that do not change during program execution.

🎨 Tokens At a Glance

Examples of Every Token Category in Action

"Hello World" (String) + - * / % (Operators) ; { } [ ] (Special Symbols)

float, double (Keywords) _studentId (Identifiers)

1️⃣ Detailed Classification of C Tokens

Here is a complete breakdown of all 6 token types with examples and compiler rules:

Token Category	Definition & Role	Rules / Properties	Example Values
Keywords	Reserved system words in C library	Must be written in lowercase; cannot be used as variable names	int, void, while, struct, return
Identifiers	Names given to program elements	Must start with letter or _; case-sensitive; no spaces	main, rollNo, _count, avg_score
Constants	Fixed literal values	Include Integer, Real (Float), Character, and Enum constants	100, -45, 3.14159, 'Z'
Strings	Sequence of characters enclosed in double quotes	Automatically terminated by a null character (\0)	"BKNMU Junagadh", "BCA Sem 1"
Operators	Symbols that instruct CPU to perform operations	Arithmetic, Relational, Logical, Assignment, Bitwise	+, ==, &&, !=, =, ++
Special Symbols	Punctuation & structural syntax delimiters	Used to group statements, define blocks, and separate parameters	;, ,, {, }, (, ), [, ], #

ℹ️ Lexical Analysis Fact: When you compile a C file, the preprocessor and compiler's Lexer (Scanner) scans the source code character by character, strips out spaces and comments, and converts the code into a stream of tokens!

💻 Code Breakdown — Identifying Tokens in C

Let's dissect a single C statement into its individual tokens:

C Code Statement:

int total = 10 + 20;

Token Identification:

1. int

• Token Type: Keyword
• Specifies integer data type

2. total

• Token Type: Identifier
• User variable name

3. = and +

• Token Type: Operators
• Assignment & Addition

In total, the statement int total = 10 + 20; consists of 7 tokens: int, total, =, 10, +, 20, and ;.

💡 Exam Tip: In theory exams, a very common 5-mark question is: "What are C Tokens? Explain all types with examples." Be sure to list all 6 categories and provide a small code breakdown like above!

🔄 Rules for Constructing Valid Identifiers in C

1

First Character Rule

Must begin with an alphabet (A-Z, a-z) or an underscore (_). It CANNOT begin with a digit.

2

Allowed Characters

Can contain letters, digits (0-9), and underscores. No special symbols like $, @, or spaces allowed.

3

No Reserved Keywords

You cannot use any of the 32 C keywords (like for, if, float) as identifier names.

⚠️ Common Mistake: 1st_rank is INVALID because it starts with a number. first rank is INVALID because it contains a space. first_rank is VALID!

📝 Quick Revision — Key Exam Points

• Definition: Smallest individual element of a C program recognized by the compiler.
• 6 Token Categories: Keywords, Identifiers, Constants, Strings, Special Symbols, Operators.
• Keywords Count: C has 32 standard ANSI keywords (e.g., int, char, goto, switch).
• Identifiers: Names given to variables, functions, and arrays. Case-sensitive!
• Constants: Fixed values that cannot be modified during program execution.
• Strings: Character sequences in double quotes ending automatically with \0.
• Delimiters: Semicolon (;) terminates statements; curly braces ({ }) group code blocks.