如何使用python中的mmap打开csv文件以进行阅读？

我知道您之前曾问过这个问题，但实际上我为自己创建了一个模块来执行此操作，因为我对大型CSV文件做了很多工作，有时我需要根据密钥将它们转换为字典。下面是我一直在使用的代码。请随时根据需要进行修改。

def MmapCsvFileIntoDict(csvFilePath, skipHeader = True, transform = lambda row: row, keySelector = lambda o: o):
    """
    Takes a CSV file path and uses mmap to open the file and return a dictionary of the contents keyed
    on the results of the keySelector.  The default key is the transformed object itself.  Mmap is used because it is
    a more efficient way to process large files.

    The transform method is used to convert the line (converted into a list) into something else.  Hence 'transform'.
    If you don't pass it in, the transform returns the list itself.
    """
    contents = {}
    firstline = False

    try:
        with open(csvFilePath, "r+b") as f:
            # memory-map the file, size 0 means whole file
            mm = mmap.mmap(f.fileno(), 0)
            for line in iter(mm.readline, b''):
                if firstline == False:
                    firstline = True

                    if skipHeader == True:
                        continue

                row = ''

                line = line.decode('utf-8')
                line = line.strip()

                row = next(csv.reader([line]), '')

                if transform != None and callable(transform):
                    if row == None or row == '':
                        continue

                    value = transform(row)
                else:
                    value = row

                if callable(keySelector):    
                    key = keySelector(value)
                else:
                    key = keySelector

                contents[key] = value

    except IOError as ie:
        PrintWithTs('Error decomposing the companies: {0}'.format(ie))
        return {}
    except:
        raise

    return contents

调用此方法时，有一些选择。

假设您有一个看起来像这样的文件：

Id, Name, PhoneNumber 
1, Joe, 7175551212 
2, Mary, 4125551212 
3, Vince, 2155551212 
4, Jane, 8145551212

调用它的最简单方法是这样的：

dict = MmapCsvFileIntoDict('/path/to/file.csv', keySelector = lambda row: row[0])

您得到的是一个看起来像这样的字典：

{ '1' : ['1', 'Joe', '7175551212'], '2' : ['2', 'Mary', '4125551212'] ...

我想做的一件事是创建一个类或一个命名元组来表示我的数据：

class CsvData:
    def __init__(self, row):
        self.Id = int(row[0])
        self.Name = row[1].upper()
        self.Phone = int(row[2])

然后，当我调用该方法时，我传入第二个lambda来将文件中的每一行转换为可以使用的对象：

dict = MmapCsvFileIntoDict('/path/to/file.csv', transform = lambda row: CsvData(row), keySelector = lambda o: o.Id)

那个时候我回来的样子是：

{ 1 : <object instance>, 2 : <object instance>...

我希望这有帮助！祝你好运