【python】【OrderedDict】处理文本文件

论坛 期权论坛 脚本     
匿名技术用户   2020-12-27 03:20   583   0

讲真python在处理数据的时候还是很方便的,但是安装以及他的不兼容性觉得有点无奈

lit           lit_freq1    lit_freq2 lit_freq3   lit_freq4  lit_freq5     online    offline 
cpu0           11.08%      0.11%      0.24%      0.08%      5.83%     17.33%     82.67%
cpu1            9.13%      0.09%      0.20%      0.06%      4.81%     14.29%     85.71%
cpu2            5.28%      0.05%      0.12%      0.04%      2.78%      8.26%     91.74%
cpu3            5.13%      0.05%      0.11%      0.04%      2.70%      8.03%     91.97%
xxxxx          63.89%      0.62%      1.41%      0.45%     33.63%


big           big_freq1   big_freq2   big_freq3  big_freq4  big_freq5  big_freq6  online    offline 
cpu4            5.30%      0.15%      0.16%      0.13%      0.04%      0.16%      5.94%     94.06%
cpu5            0.84%      0.02%      0.03%      0.02%      0.01%      0.02%      0.95%     99.05%
cpu6            1.92%      0.06%      0.06%      0.05%      0.01%      0.06%      2.15%     97.85%
cpu7            0.00%      0.00%      0.00%      0.00%      0.00%      0.00%      0.00%    100.00%
yyyyy          89.17%      2.59%      2.76%      2.25%      0.60%      2.63%

这里考虑的CPU分为两个cluster,其中CPU0\CPU1\CPU2\CPU3属于cluster0

CPU4\CPU5\CPU6\CPU7属于cluster1

比如我现在需要计算每个cluster横行和每个纵行的数据的和

#!/usr/bin/python
# coding=utf-8

from collections import OrderedDict

import os
import sys

file_path = sys.argv[1]

def open_file_get_time_data(file_name, dic_data,order_dic_freq={}):

 lit_freq = [lit_freq1, lit_freq2, lit_freq3, lit_freq4, lit_freq5]
 big_freq = [big_freq1, big_freq2, big_freq3, big_freq5, big_freq5, big_freq6]
 try:
  with open ( file_name, "r", encoding="UTF-8", errors='ignore' ) as file_obj:
   lines = file_obj.readlines()
   for content in lines:
    if(content.split()):
     list = content.split()
     #print(list)
     if(list[0]=="cpu0" or list[0]=="cpu1" or\
        list[0]=="cpu2" or list[0]=="cpu3"):
      i = 1
      for freq in lit_freq:
       dic_data["lit_"+list[0]][freq] = float(list[i].replace("%",""))
       #print(dic_data["lit_"+list[0]][freq])
       i = i+1
     if (list[0] == "cpu4" or list[0] == "cpu5" or \
         list[0] == "cpu6" or list[0] == "cpu7"):
         i = 1
         for freq in big_freq:
          dic_data["big_"+list[0]][freq] = float(list[i].replace("%",""))
          #print ( dic_data["lit"][freq] )
          i = i + 1
 except:
  print ( "there is some wrong during open" + file_name )


if __name__ == '__main__':
 cpu_result = file_path
 order_dic_cpu_result = OrderedDict()

 dic_sum = OrderedDict()
 dic_sum["lit"]=0
 dic_sum["big"]=0

 order_dic_cpu_result["lit_cpu0"] = OrderedDict ()
 order_dic_cpu_result["lit_cpu1"] = OrderedDict ()
 order_dic_cpu_result["lit_cpu2"] = OrderedDict ()
 order_dic_cpu_result["lit_cpu3"] = OrderedDict ()
 order_dic_cpu_result["big_cpu4"] = OrderedDict ()
 order_dic_cpu_result["big_cpu5"] = OrderedDict ()
 order_dic_cpu_result["big_cpu6"] = OrderedDict ()
 order_dic_cpu_result["big_cpu7"] = OrderedDict ()
 open_file_get_time_data ( cpu_result, order_dic_cpu_result)

 total = 0
 for cluster in dic_sum:
  print(cluster)
  print("%-10s"%"SUM",end="")
  for dict_cluster in order_dic_cpu_result:
   if cluster in dict_cluster:
    for freq in order_dic_cpu_result[dict_cluster]:
     dic_sum[cluster] = dic_sum[cluster]+order_dic_cpu_result[dict_cluster][freq]/100*freq
  print(dic_sum[cluster])
  total = total +dic_sum[cluster]
 print("%-10s"%"TOTAL",end="")
 print(total)

计算显示结果如下

lit
SUM x
big
SUM y

TOTAL x+y

具体可以参考:https://github.com/feifeiyuan/features/tree/master/python/calculate_sum

同时也有相关代码如下:

https://github.com/feifeiyuan/features/blob/master/python/time_in_state%E8%AE%A1%E7%AE%97cpu_loading/tool/cpu_data_compute.py

分享到 :
0 人收藏
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

积分:7942463
帖子:1588486
精华:0
期权论坛 期权论坛
发布
内容

下载期权论坛手机APP