用python编写Sebesda的词法分析器。不适用于输入文件中的最后一个词素

我必须将词法分析器的Sebesda编程语言概念（第4章，第2节）中的代码转换为python。这是我到目前为止的内容：

# Character classes #
LETTER = 0
DIGIT = 1
UNKNOWN = 99

# Token Codes #
INT_LIT = 10
IDENT = 11
ASSIGN_OP = 20
ADD_OP= 21
SUB_OP = 22
MULT_OP = 23
DIV_OP = 24
LEFT_PAREN = 25
RIGHT_PAREN = 26

charClass = ''
lexeme = ''
lexLen = 0
token = ''
nextToken = ''

### lookup - function to lookup operators and parentheses ###
###          and return the token                         ###
def lookup(ch):
    def left_paren():
        addChar()
        globals()['nextToken'] = LEFT_PAREN

    def right_paren():
        addChar()
        globals()['nextToken'] = RIGHT_PAREN

    def add():
        addChar()
        globals()['nextToken'] = ADD_OP

    def subtract():
        addChar()
        globals()['nextToken'] = SUB_OP

    def multiply():
        addChar()
        globals()['nextToken'] = MULT_OP

    def divide():
        addChar()
        globals()['nextToken'] = DIV_OP
    options = {')': right_paren, '(': left_paren, '+': add,
               '-': subtract, '*': multiply , '/': divide}

    if ch in options.keys():
        options[ch]()
    else:
        addChar()

### addchar- a function to add next char to lexeme ###
def addChar():
    #lexeme = globals()['lexeme']
    if(len(globals()['lexeme']) <=98):
        globals()['lexeme'] += nextChar
    else:
        print("Error. Lexeme is too long")

### getChar- a function to get the next Character of input and determine its character class ###
def getChar():
    globals()['nextChar'] = globals()['contents'][0]
    if nextChar.isalpha():
        globals()['charClass'] = LETTER
    elif nextChar.isdigit():
        globals()['charClass'] = DIGIT
    else:
        globals()['charClass'] = UNKNOWN
    globals()['contents'] = globals()['contents'][1:]


## getNonBlank() - function to call getChar() until it returns a non whitespace character ##
def getNonBlank():
    while nextChar.isspace():
        getChar()

## lex- simple lexical analyzer for arithmetic functions ##
def lex():
    globals()['lexLen'] = 0
    getNonBlank()
    def letterfunc():
        globals()['lexeme'] = ''
        addChar()
        getChar()
        while(globals()['charClass'] == LETTER or globals()['charClass'] == DIGIT):
            addChar()
            getChar()
        globals()['nextToken'] = IDENT

    def digitfunc():
        globals()['lexeme'] = ''
        addChar()
        getChar()
        while(globals()['charClass'] == DIGIT):
            addChar()
            getChar()
        globals()['nextToken'] = INT_LIT

    def unknownfunc():
        globals()['lexeme'] = ''
        lookup(nextChar)
        getChar()

    lexDict = {LETTER: letterfunc, DIGIT: digitfunc, UNKNOWN: unknownfunc}
    if charClass in lexDict.keys():
        lexDict[charClass]()
    print('The next token is: '+ str(globals()['nextToken']) + ' The next lexeme is: ' + globals()['lexeme'])

with open('input.txt') as input:
    contents = input.read()
    getChar()
    lex()
    while contents:
        lex()

我正在使用字符串sum + 1 / 33作为样本输入字符串。据我了解，在顶层对getChar（）的第一次调用将characterClass设置为LETTER，并设置contents为um + 1 / 33。

然后，程序进入while循环并调用lex()。lex()依次将单词sum累积到中lexeme。当里面的while循环letterfunc遇到第一个空格字符时，它将中断并退出lex()

由于contents不为空，因此程序将再次在顶级循环通过while循环。这次，getNonBlank()内部调用lex()“将空格填满，contents并且重复与以前相同的过程。

我遇到错误的地方是最后一个词素。有人告诉我，这globals()['contents'][0]超出了范围getChar()。我并不期望找到一个困难的错误，但是我尝试手动跟踪它，但似乎无法发现问题。任何帮助将不胜感激。