Haskell：向后读取二进制文件

debugcn 发表于 Dev

吉莉

我正在寻找使用Haskell在uInt32二进制转储中匹配特定模式的最后32位字。我可以使用完成任务last，但是代码必须遍历整个文件，因此效率很低。

是否有一种简单的方法可以使readfile文件反向操作？我相信这将以对当前代码的最小更改来解决该问题。

这是我当前的代码，以供参考。我这个周末才刚开始与Haskell交流，所以我确信它相当丑陋。它在MSB处寻找以0b10开头的最后32位字。

import System.Environment(getArgs)
import qualified Data.ByteString.Lazy as BL
import qualified Data.ByteString.Lazy.Internal as BL
import qualified Data.ByteString as BS
import Data.Binary.Get
import Data.Word
import Data.Bits
import Text.Printf(printf)

main = do
  args <- getArgs
  let file = args!!0
  putStrLn $ "Find last 0xCXXXXXXX in " ++ file

  content <- BL.readFile file

  let packets = getPackets content
  putStrLn . show . getValue . last . filterTimes $ packets

-- Data

type Packet = Word32

-- filter where first 2 bits are 10
filterTimes :: [Packet] -> [Packet]
filterTimes = filter ((== 0x2) . tag)

-- get the first 2 bits
tag :: Packet -> Packet
tag rp =
  let tagSize = 2
  in  shiftR rp (finiteBitSize rp - tagSize)

-- remove the tag bits
getValue :: Packet -> Packet
getValue =
  let tagSize = 2
      mask    = complement $ rotateR (2^tagSize - 1) tagSize
  in (.&.) mask

-- Input
-- Based on https://hackage.haskell.org/package/binary/docs/Data-Binary-Get.html

getPacket :: Get Packet
getPacket = do
  packet <- getWord32le
  return $! packet

getPackets :: BL.ByteString -> [Packet]
getPackets input0 = go decoder input0
  where
    decoder = runGetIncremental getPacket
    go :: Decoder Packet -> BL.ByteString -> [Packet]
    go (Done leftover _consumed packet) input =
      packet : go decoder (BL.chunk leftover input)
    go (Partial k) input                     =
      go (k . takeHeadChunk $ input) (dropHeadChunk input)
    go (Fail _leftover _consumed msg) _input =
      []

takeHeadChunk :: BL.ByteString -> Maybe BS.ByteString
takeHeadChunk lbs =
  case lbs of
    (BL.Chunk bs _) -> Just bs
    _ -> Nothing

dropHeadChunk :: BL.ByteString -> BL.ByteString
dropHeadChunk lbs =
  case lbs of
    (BL.Chunk _ lbs') -> lbs'
    _ -> BL.Empty

埃里克

关于您的代码的一些注释：

您正在使用last它可能会引发异常。您应该使用lastMayfromthe安全包返回也许。
由于您只是将文件视为Word32s的向量，因此我认为不值得使用Data.Binary.Get及其带来的相关开销和复杂性。只需将文件视为（可能是惰性的）ByteString并访问每第4个字节或将其分解为4个字节的子字符串即可。

您可以在这里查看使用ByteStrings的代码。它采用以下方法解决该问题：

以惰性ByteString的形式读取整个文件，并生成一个（延迟的）4字节子字符串列表。返回满足条件的最后一个子字符串。

intoWords :: BL.ByteString -> [ BL.ByteString ]
intoWords bs
  | BL.null a = []
  | otherwise = a : intoWords b
  where (a,b) = BL.splitAt 4 bs

-- find by breaking the file into 4-byte words
find_C0_v1 :: FilePath -> IO (Maybe BL.ByteString)
find_C0_v1 path = do
  contents <- BL.readFile path
  return $ lastMay . filter (\bs -> BL.index bs 0 == 0xC0) . intoWords $ contents

以惰性ByteString的形式读取整个文件，并访问每4个字节以寻找0xC0。返回最后一次出现。

-- find by looking at every 4th byte
find_C0_v2 :: FilePath -> IO (Maybe BL.ByteString)
find_C0_v2 path = do
  contents <- BL.readFile path
  size <- fmap fromIntegral $ withFile path ReadMode hFileSize
  let wordAt i = BL.take 4 . BL.drop i $ contents
  return $ fmap wordAt $ lastMay $ filter (\i -> BL.index contents i == 0xC0) [0,4..size-1]

向后读取文件，大小为64K。在每个块（严格的ByteString）中，每第4个字节访问一次，从块的末尾开始查找0xC0。返回第一个匹配项。

-- read a file backwords until a predicate returns a Just value
loopBlocks :: Int -> Handle -> Integer -> (BS.ByteString -> Integer -> Maybe a) -> IO (Maybe a)
loopBlocks blksize h top pred
  | top <= 0 = return Nothing
  | otherwise   = do
        let offset = top - fromIntegral blksize
        hSeek h AbsoluteSeek offset
        blk <- BS.hGet h blksize
        case pred blk offset of
          Nothing -> loopBlocks blksize h offset pred
          x       -> return x

-- find by reading backwords lookint at every 4th byte
find_C0_v3 :: FilePath -> IO (Maybe Integer)
find_C0_v3 path = do
  withFile path ReadMode $ \h -> do
    size <- hFileSize h
    let top = size - (mod size 4)
        blksize = 64*1024 :: Int
    loopBlocks blksize h top $ \blk offset ->
          fmap ( (+offset) . fromIntegral ) $ headMay $ filter (\i -> BS.index blk i == 0xC0) [blksize-4,blksize-8..0]

即使必须读取整个文件，第三种方法也是最快的。第一种方法实际上效果很好。我完全不建议使用第二个-随着文件大小的增加，它的性能会急剧下降。

本文收集自互联网，转载请注明来源。

如有侵权，请联系[email protected] 删除。