我有一个带有某种key -> Map(key1 -> Map(), key2 -> Map())
表示形式的嵌套映射,该映射基本上表示了发出的特定HTTP请求的路径结构。
root/twiki/bin/edit/Main/Double_bounce_sender root/twiki/bin/rdiff/TWiki/NewUserTemplate
我将它们存储在Map Maps中,这将给我路径的层次结构。使用解析器,我从服务器日志中读取数据,并获取所需的相应数据,然后在排序的映射中为数据建立索引。
val mainList: RDD[List[String]] = requesturl flatMap ( r => r.toString split("\\?") map (x => parser(x.split("/").filter(x => !x.contains("=")).toList).valuesIterator.toList))
def parser(list: List[String]): Map[Int, String]= {
val m = list.zipWithIndex.map(_.swap).toMap
val sM = SortedMap(m.toSeq:_*)
sM.+(0 -> "root")
}
在获得所需结构的数据后,我遍历整个集合以将数据结构化为如下所示的路径映射
root - twiki - bin - edit - Main - Double_bounce_sender -rdiff - TWiki - NewUserTemplate - oops - etc - local - getInterface
type innerMap = mutable.HashMap[String, Any]
def getData(input: RDD[List[String]]): mutable.HashMap[String, innerMap] ={
var mainMap = new mutable.HashMap[String, innerMap]
for(x <- input){
val z: mutable.HashMap[String, innerMap] = storeData(x.toIterator, mainMap ,x(0).toString)
mainMap = mainMap ++ z
}
mainMap
}
def storeData(list: Iterator[String], map: mutable.HashMap[String, innerMap], root: String): mutable.HashMap[String, innerMap]={
list.hasNext match {
case true =>
val v = list.next()
val y = map contains (root) match {
case true =>
println("Adding when exists: "+v)
val childMap = map.get(v).get match {
case _:HashMap[String, Any] => asInstanceOf[mutable.HashMap[String, innerMap]]
case _ => new mutable.HashMap[String, innerMap]
}
val x = map + (v -> storeData(list, childMap, v))
x
case false =>
val x = map + (v -> storeData(list, new mutable.HashMap[String, innerMap], v))
x
}
y.asInstanceOf[mutable.HashMap[String, innerMap]]
case false =>
new mutable.HashMap[String, innerMap]
}
}
get data方法调用每个输入列表,并将其发送到构建映射的storeData方法。
我被困在两个地方。
Map(root -> Map(twiki -> Map(bin -> Map(edit -> Map(Main -> Map(Double -> Map()))))))) Map(root -> Map(twiki -> Map(bin -> Map(rdiff -> Map(TWiki -> Map(NewUser -> Map())))))))
寻找有关如何实现此解决方案的建议,并获得一张最终地图,该地图在一个地图中包含服务器日志文件中存在的所有可能路径。
也许是这样的吗?
scala> type Node = Map[String, Any];
defined type alias Node
scala> def merge( me : Node, you : Node ) : Node = {
| val keySet = me.keySet ++ you.keySet;
| def nodeForKey( parent : Node, key : String ) : Node = parent.getOrElse( key, Map.empty ).asInstanceOf[Node]
| keySet.map( key => (key -> merge( nodeForKey( me, key ), nodeForKey( you, key ) ) ) ).toMap
| }
merge: (me: Node, you: Node)Node
scala> val path1 = Map( "root" -> Map("bin" -> Map("sh" -> Map.empty) ) )
path1: scala.collection.immutable.Map[String,scala.collection.immutable.Map[String,scala.collection.immutable.Map[String,scala.collection.immutable.Map[Nothing,Nothing]]]] = Map(root -> Map(bin -> Map(sh -> Map())))
scala> val path2 = Map( "root" -> Map( "bin" -> Map("csh" -> Map.empty), "usr" -> Map.empty ) )
path2: scala.collection.immutable.Map[String,scala.collection.immutable.Map[String,scala.collection.immutable.Map[_ <: String, scala.collection.immutable.Map[Nothing,Nothing]]]] = Map(root -> Map(bin -> Map(csh -> Map()), usr -> Map()))
scala> merge( path1, path2 )
res8: Node = Map(root -> Map(bin -> Map(sh -> Map(), csh -> Map()), usr -> Map()))
本文收集自互联网,转载请注明来源。
如有侵权,请联系[email protected] 删除。
我来说两句