# Lexer

`dfllexer.l` is the Flex specification for the netlist scanner, compiled into `Scanner::yylex()` (the `Scanner` class itself - and the mutable state these rules read and mutate - is declared in [`dflscanner.h`](dflscanner.md)). It runs as a state machine over a stack of Flex exclusive start conditions, used both for ordinary lexical context and to hand-roll a small sub-scanner for the `include`/library-section directive syntax, which doesn't fit the token-at-a-time shape the Bison [parser](dflparser.md) expects.

## Three entry modes

`Scanner::InputType` (`InputNetlist`, `InputExpression`, `InputParameters`) matches the [parser](dflparser.md)'s three top-level productions. The very first token produced (guarded by the `atBeginning` flag) is a synthetic `INNETLIST`/`INEXPR`/`INPARAMS` marker that tells the single generated grammar which top-level rule to expect, instead of needing three separate parsers.

## Start conditions: keyword recognition by position

`LINESTART` is the only state where line-leading keywords (`model`, `subckt`, `control`, `@if`, ...) are recognized; `BODY` is everything after that, where operators and values are tokenized. Keyword-vs-identifier ambiguity is resolved purely by position: a bareword at the start of a line is tried against the keyword list; the same word inside an expression is always an `IDENTIFIER`. A further layer applies to `analysis`, `sweep`, and `save`: they are only real keywords when `inControl` is set (inside a `control`/`endc` block) - outside it, they're ordinary identifiers, so a circuit-level model or instance can legally be named `save`.

`inParen`/`inBracket`/`inBrace` counters let a `NEWLINE` inside `(...)`/`[...]`/`{...}` be swallowed instead of ending a statement, so a parameter list, vector/list literal, or brace-delimited block can be split across multiple physical lines.

## The include/library-section sub-scanner

`include "file"` and `include "file" section=name` are handled entirely by the lexer, not by grammar rules - the file's own leading comment documents the full state diagram (`LINESTART -> INC -> QUOTED .. INCEND ...`, and the longer `... LIBSECTION .. LIBEND .. SECLOOK .. SECLOOKNAME .. SECLOOKEND` chain for the section form). Two shapes:

- **Plain `include "file"`**: on the closing quote, resolves and opens the file immediately via [`FileStack::addFile()`](filestack.md) and `Scanner::pushStream()`, then pushes a new Flex input buffer so scanning continues inside the included file - the include itself produces no token visible to the grammar.
- **`include "file" section=name`**: opens the target file the same way, then scans forward (`SECLOOK`/`SECLOOKNAME`) past its contents looking for a matching `section <name>` line before resuming normal lexing at `SECLOOKEND`. This is how one physical library file can hold several named sections (e.g. multiple device definitions in one Verilog-A library) with only the requested one actually scanned; a bare `section`/`endsection` appearing in an ordinary (non-library) file is a syntax error, since sections only make sense inside a file entered via a library include.

Long comments (`/* ... */`) and quoted-string scanning are pushed onto the *same* start-condition stack (`yy_push_state`/`yy_pop_state`) used by the include sub-scanner, so a comment or string can appear in the middle of an `include` directive - or anywhere else - and correctly resume whatever state was active before it, without every context needing its own copy of comment-handling rules.

## String literals

Ordinary `"..."` strings (`QUOTED` state) support C-style backslash escapes (`\n`, `\t`, `\r`, `\b`, `\f`, octal `\NNN`) and treat an unescaped literal newline, or a malformed escape, as a syntax error rather than continuing past it. Long-quoted strings (`<<<MARKER` ... `>>>MARKER`, `LONGQUOTED` state) instead run verbatim - no escaping at all - until a matching `>>>marker` reappears anywhere in the input; this form exists for embedding large blocks of foreign text (e.g. Verilog-A source passed to `embed`) that may itself contain quotes or backslashes.

## Numeric literals

Plain decimal/hex integers narrow a `std::stoul` result into the 32-bit `Int` by truncation - documented as intentional, matching OSDI's 32-bit integer parameters - rather than rejecting an oversized literal outright. Floating-point literals go through `std::stod`. A fourth numeric rule additionally recognizes an SI-magnitude suffix fused directly onto the number (`p`/`n`/`u`/`m`/`k`/`meg`/`mil`/`M`/`G`/`T`/...), scaling the value in the lexer itself before the token ever reaches the grammar - the case-sensitive distinction between lowercase `m`/`meg`/`mil` and uppercase `M`/`K`/`G`/`T` is deliberate (see the netlist number-format docs for the full SI-prefix table). `inf`/`nan` (case-insensitive) are recognized directly as floating-point literals.

## Identifiers

A single regex matches both plain (`[a-zA-Z_$][0-9a-zA-Z_$]*`) and quote-delimited identifier parts (`'...'`, with `''` as an escaped quote inside), and more than one part can be concatenated back to back - e.g. a plain prefix followed by a quoted segment. The matched text is unescaped into an `Id` (see [Identifiers](identifier.md)) directly inside the rule's action rather than through a separate lexer state, since however it's spelled, the whole thing is one token.

## End of file

`<<EOF>>` behaves differently depending on file-stack depth: inside an `include`d file, it pops back to the parent buffer/location and resumes there, just as invisibly to the grammar as the include itself; inside a library section that never reached `endsection`, it's a syntax error; at the true top level, it returns the ordinary `END` token. A handful of other states (`INC`, `QUOTED`, `LONGCOMMENT`, the section-lookup states) treat `<<EOF>>` as an unconditional syntax error, since reaching end of file mid-string or mid-comment can never be valid.

## Error reporting

Every lexer error path funnels through `Scanner::error()`, which does `status_.set(Status::Syntax, msg); status_.extend(loc)` (see [Status](status.md)) and then returns `token::YYerror` - a token Bison recognizes as a parse error, handing control to the [parser](dflparser.md)'s own (nonexistent) error-recovery machinery rather than having the lexer try to resynchronize on its own.
