Skip to content

Instantly share code, notes, and snippets.

@smileart

smileart/parse_fields.rb

Last active February 3, 2021 16:22
Show Gist options
  • Save smileart/01e32f4ae48911c35099ab98b58f6903 to your computer and use it in GitHub Desktop.
Save smileart/01e32f4ae48911c35099ab98b58f6903 to your computer and use it in GitHub Desktop.
Download ZIP
A little AWK-like method to parse fields in a string of text
Raw
parse_fields.rb
text = <<~DOC
ID Name Type Size Date
735557379 Mitch Murder - Kung Fury (Lost Tapes) - 2015 FOLDER 38.37 MB 8 months ago
589243897 A Game Of Thrones ~ Books 1-5 EPUB,MOBI & PDF [GrYff0N] FOLDER 83.36 MB 2 years ago
735557525 VA - Kung Fury (2015) FOLDER 109.84 MB 8 months ago
819683233 19000 FOLDER 1.77 GB 2 days ago
819682329 18000 FOLDER 3.12 GB 2 days ago
819681592 17000 FOLDER 4.23 GB 2 days ago
784060241 Doctor.Who.2005.S12.1080p.BluRay.x264-SHORTBREHD FOLDER 31.73 GB 2 months ago
601795129 chill.institute FOLDER 51.66 GB 2 days ago
DOC
def parse_fields(str:, hash_result: false, row_sep: "\n")
rows = str.split(row_sep)
header = rows.shift.scan(/\w+\s+/)
col_widths = header.map(&:length)
header.map!(&:strip)
acc_index_memo = {}
rows.map! do |r|
result = []
col_widths.each_with_index { |w, i|
start_index = i > 0 ? acc_index_memo[i] ||= col_widths.slice(0, i).inject(:+) : 0
result << r[start_index, w].strip
}
result
end
hash_result ? rows.map {|r| Hash[header.zip(r)] } : rows
end
parse_fields(str: text)
parse_fields(str: text, hash_result: true)
parse_fields(str: text, hash_result: true, row_sep: /\R/)
Raw
parse_fields_perf.rb
# gem 'benchmark-ips'
require 'benchmark/ips'
def slice_inject(data, i)
# to emulate the difference on "bigger inputs" by increasing
# the cost of operation using STDOUT output or delay
puts "Called with #{i}" if ENV['DEBUG']
sleep 0.1 unless ENV['DEBUG']
data.slice(0, i).inject(:+)
end
def parse_fields_memo(str:, hash_result: false, row_sep: "\n")
rows = str.split(row_sep)
header = rows.shift.scan(/\w+\s+/)
col_widths = header.map(&:length)
header.map!(&:strip)
acc_index_memo = {}
rows.map! do |r|
result = []
col_widths.each_with_index { |w, i|
start_index = i > 0 ? acc_index_memo[i] ||= slice_inject(col_widths, i) : 0
result << r[start_index, w].strip
}
result
end
hash_result ? rows.map {|r| Hash[header.zip(r)] } : rows
end
def parse_fields(str:, hash_result: false, row_sep: "\n")
rows = str.split(row_sep)
header = rows.shift.scan(/\w+\s+/)
col_widths = header.map(&:length)
header.map!(&:strip)
rows.map! do |r|
result = []
col_widths.each_with_index { |w, i|
start_index = i > 0 ? slice_inject(col_widths, i) : 0
result << r[start_index, w].strip
}
result
end
hash_result ? rows.map {|r| Hash[header.zip(r)] } : rows
end
text = <<~DOC
ID Name Type Size Date
735557379 Mitch Murder - Kung Fury (Lost Tapes) - 2015 FOLDER 38.37 MB 8 months ago
589243897 A Game Of Thrones ~ Books 1-5 EPUB,MOBI & PDF [GrYff0N] FOLDER 83.36 MB 2 years ago
735557525 VA - Kung Fury (2015) FOLDER 109.84 MB 8 months ago
819683233 19000 FOLDER 1.77 GB 2 days ago
819682329 18000 FOLDER 3.12 GB 2 days ago
819681592 17000 FOLDER 4.23 GB 2 days ago
784060241 Doctor.Who.2005.S12.1080p.BluRay.x264-SHORTBREHD FOLDER 31.73 GB 2 months ago
601795129 chill.institute FOLDER 51.66 GB 2 days ago
DOC
text_top = <<~DOC
PID COMMAND %CPU TIME #TH #WQ #PORT MEM PURG CMPRS PGRP PPID STATE BOOSTS %CPU_ME %CPU_OTHRS UID FAULTS COW MSGSENT MSGRECV SYSBSD SYSMACH CSW PAGEINS IDLEW POWE INST CYCL USER #MRE RPRV VPRV
98509 QuickLookUIS 0.0 00:02.75 3 1 136 3900K 0B 2460K 98509 1 sleeping 0[7025] 0.00000 0.00000 501 24188 251 44888 15347 60625 124496 84970 27 925 0.0 0 0 smileart N/A N/A N/A
98508 com.apple.ap 0.0 00:09.54 3 1 300 43M 0B 33M 98508 1 sleeping 0[7504] 0.00000 0.00000 501 97408 450 71065 24606 142017 160802 124853 281 1094 0.0 0 0 smileart N/A N/A N/A
98393 Marked 0.0 00:24.85 8 1 365 166M 0B 104M 98393 1 sleeping 0[2258] 0.00000 0.00000 501 664387 626 120428 38004 1176054 277211 435284 3112 22004 0.0 0 0 smileart N/A N/A N/A
95910 Code Helper 0.0 00:19.90 10 1 55 23M 0B 22M 32593 1 sleeping *0[1] 0.00000 0.00000 501 29502 1488 5941 5618 450732 49776 699169 0 129171 0.0 0 0 smileart N/A N/A N/A
95799 Code Helper 0.0 00:02.45 10 1 57 23M 0B 22M 28395 1 sleeping *0[1] 0.00000 0.00000 501 14863 1491 1016 1010 45432 5829 55146 0 7257 0.0 0 0 smileart N/A N/A N/A
94850 MTLCompilerS 0.0 00:00.08 2 1 24 5764K 0B 4232K 94850 1 sleeping *0[2] 0.00000 0.00000 501 3973 557 133 44 533 295 367 36 0 0.0 0 0 smileart N/A N/A N/A
94849 MTLCompilerS 0.0 00:00.05 2 1 32 3224K 0B 2580K 94849 1 sleeping *0[2] 0.00000 0.00000 501 1592 160 136 40 575 283 257 51 0 0.0 0 0 smileart N/A N/A N/A
94237 MTLCompilerS 0.0 00:00.14 2 1 24 6892K 0B 6876K 94237 1 sleeping *0[1] 0.00000 0.00000 501 4547 557 127 42 536 287 689 36 0 0.0 0 0 smileart N/A N/A N/A
94236 MTLCompilerS 0.0 00:00.09 3 2 32 3596K 0B 3564K 94236 1 sleeping *0[2] 0.00000 0.00000 501 1858 160 144 42 606 292 349 49 2 0.0 0 0 smileart N/A N/A N/A
93746 Telegram 0.0 03:43.02 27 4 938 160M 7980K 69M 93746 1 sleeping *0[1930] 0.00000 0.00000 501 290142 1795 436274 166615 3776744 1086625 1662775 888 15471 0.0 0 0 smileart N/A N/A N/A
93691 MTLCompilerS 0.0 00:00.18 2 1 24 7244K 0B 7236K 93691 1 sleeping *0[6] 0.00000 0.00000 501 8293 557 157 51 575 322 629 35 0 0.0 0 0 smileart N/A N/A N/A
93686 MTLCompilerS 0.0 00:00.30 2 1 32 6328K 0B 6320K 93686 1 sleeping *0[10] 0.00000 0.00000 501 5806 160 194 54 1065 351 1175 63 0 0.0 0 0 smileart N/A N/A N/A
93614 com.apple.We 0.0 00:05.82 5 1 101 15M 0B 9328K 93614 1 sleeping *0[982] 0.00000 0.00000 501 85468 327 13526 9366 196447 49178 164810 3793 23132 0.0 0 0 smileart N/A N/A N/A
93613 com.apple.We 0.0 00:52.58 5 1 129 92M 808K 20M 93613 1 sleeping 0[20765] 0.00000 0.00000 501 339820 371 325486 156493 408189 662518 575214 1644 9900 0.0 0 0 smileart N/A N/A N/A
93574 DevDocs 0.0 00:33.92 5 1 397 56M 0B 19M 93574 1 sleeping 0[2327] 0.00000 0.00000 501 305038 824 381534 138222 335774 1273338 709039 1496 22731 0.0 0 0 smileart N/A N/A N/A
93145 Code Helper 0.0 00:15.35 10 1 55 23M 0B 21M 28395 1 sleeping *0[1] 0.00000 0.00000 501 25412 1485 4759 4500 339138 38619 537621 0 99828 0.0 0 0 smileart N/A N/A N/A
92767 PerfPowerSer 0.0 00:13.15 3 1 293 7164K 256K 1656K 92767 1 sleeping *1[17] 0.00000 0.00000 0 40602 261 90501 44774 265120 150627 136970 147 701 0.0 0 0 root N/A N/A N/A
92327 MTLCompilerS 0.0 00:00.28 2 1 24 11M 0B 11M 92327 1 sleeping *0[2] 0.00000 0.00000 501 5878 557 138 43 620 309 872 0 0 0.0 0 0 smileart N/A N/A N/A
91869 com.apple.Di 0.0 00:00.02 2 1 24 656K 0B 644K 91869 1 sleeping *0[1] 0.00000 0.00000 501 890 107 129 41 504 255 167 0 3 0.0 0 0 smileart N/A N/A N/A
91858 Code Helper 0.0 00:01.61 10 1 54 22M 0B 10M 91048 1 sleeping *0[1] 0.00000 0.00000 501 14155 1487 290 207 21670 1725 25421 1279 3677 0.0 0 0 smileart N/A N/A N/A
91563 FinderSyncEx 0.0 00:00.38 3 1 160 4680K 0B 1244K 91563 1 sleeping *0[47] 0.00000 0.00000 501 5668 587 1297 496 9125 3176 2832 2 49 0.0 0 0 smileart N/A N/A N/A
91542 SetappLaunch 0.0 00:16.28 6 1 185 16M 0B 3960K 91542 1 sleeping *0[1] 0.00000 0.00000 501 11625 682 101777 22245 112939 156823 124410 11 254 0.0 0 0 smileart N/A N/A N/A
91456 SetappAgent 0.0 01:52.36 14 6 309 116M 8192K 3016K 91456 1 sleeping *0[1] 0.00000 0.00000 501 338624 894 90279 41239 2782518 396616 880982 7502 3433 0.0 0 0 smileart N/A N/A N/A
91434 Code Helper 0.0 00:29.51 21 1 151 34M 0B 7352K 91048 91048 sleeping *0[1] 0.00000 0.00000 501 75340 1507 54815 10718 727001 402600 460512 1 7099 0.0 0 0 smileart N/A N/A N/A
91247 Code Helper 0.0 00:01.25 7 1 90 11M 0B 7560K 91048 91048 sleeping *0[2] 0.00000 0.00000 501 7693 1455 13056 7408 23620 17874 13512 156 605 0.0 0 0 smileart N/A N/A N/A
91234 VTDecoderXPC 0.0 00:00.23 2 1 64 9096K 0B 8420K 91234 1 sleeping *0[2] 0.00000 0.00000 501 4068 167 1744 797 4902 2217 1959 1 40 0.0 0 0 smileart N/A N/A N/A
91149 Code Helper 0.0 01:44.62 8 1 175 72M 256K 9176K 91048 91048 sleeping *1[5] 0.00000 0.00000 501 193736 1508 1907778 852976 360384 2258210 1137651 75 23773 0.0 0 0 smileart N/A N/A N/A
91143 chrome_crash 0.0 00:00.03 4 1 33 1044K 0B 576K 91141 1 sleeping *0[1] 0.00000 0.00000 501 1249 146 58 21 637 146 206 12 5 0.0 0 0 smileart N/A N/A N/A
91048 Electron 0.0 01:21.79 31 1 445 76M 64K 11M 91048 1 sleeping *0[1600] 0.00000 0.00000 501 441981 3514 1206298 338417 802733 3076145 1556945 179 6831 0.0 0 0 smileart N/A N/A N/A
90976 Setapp 0.0 01:20.34 8 1 366 222M 0B 55M 90976 1 sleeping *0[1] 0.00000 0.00000 501 250631 942 622936 113170 1201344 1521622 1096383 40 2541 0.0 0 0 smileart N/A N/A N/A
90490 Code Helper 0.0 28:46.47 15 1 70 128M 0B 126M 28395 1 sleeping *0[1] 0.00000 0.00000 501 76434 1497 2317 2172 167206671 17763 14978287 0 182456 0.0 0 0 smileart N/A N/A N/A
90349 Code Helper 0.0 00:19.44 10 1 57 23M 0B 22M 32593 1 sleeping *0[1] 0.00000 0.00000 501 30985 1484 5821 5508 440856 48701 692375 12 131712 0.0 0 0 smileart N/A N/A N/A
90197 Code Helper 0.0 00:06.60 10 1 56 24M 0B 22M 28395 1 sleeping *0[1] 0.00000 0.00000 501 19067 1489 2244 2159 154274 17411 222523 0 36844 0.0 0 0 smileart N/A N/A N/A
89811 com.apple.au 0.0 00:00.01 2 1 21 772K 0B 764K 89811 1 sleeping *0[2] 0.00000 0.00000 501 1035 121 116 40 560 233 91 0 0 0.0 0 0 smileart N/A N/A N/A
89368 universalAcc 0.0 00:02.64 3 1 233 40M 0B 33M 89368 1 sleeping *0[960] 0.00000 0.00000 501 55122 357 23921 10123 52809 59178 62206 27 1106 0.0 0 0 smileart N/A N/A N/A
88718 Python 0.0 00:11.48 1 0 14 13M 0B 13M 88718 84403 sleeping *0[1] 0.00000 0.00000 501 6267 382 50 22 353427 190 115906 59 7 0.0 0 0 smileart N/A N/A N/A
88510 PlugInLibrar 0.0 00:00.01 2 1 25 620K 0B 608K 88510 1 sleeping *0[4] 0.00000 0.00000 501 839 99 132 41 438 238 71 0 0 0.0 0 0 smileart N/A N/A N/A
88415 MTLCompilerS 0.0 00:00.21 2 1 32 6436K 0B 6408K 88415 1 sleeping *0[1] 0.00000 0.00000 501 3859 162 131 39 691 281 512 493 1 0.0 0 0 smileart N/A N/A N/A
88384 XprotectServ 0.0 00:00.18 2 1 44 3364K 0B 3344K 88384 1 sleeping *0[18] 0.00000 0.00000 501 9514 162 449 184 1946 814 933 6 10 0.0 0 0 smileart N/A N/A N/A
87832 VTDecoderXPC 0.0 00:07.08 2 1 77 27M 0B 26M 87832 1 sleeping *0[813] 0.00000 0.00000 501 173182 174 56776 26906 259457 66563 65841 218 1045 0.0 0 0 smileart N/A N/A N/A
87811 PlugInLibrar 0.0 00:00.08 2 1 24 624K 0B 244K 87811 1 sleeping *0[205] 0.00000 0.00000 501 1840 99 939 243 2054 844 624 0 0 0.0 0 0 smileart N/A N/A N/A
87686 MTLCompilerS 0.0 00:00.17 2 1 24 8464K 0B 8452K 87686 1 sleeping *0[3] 0.00000 0.00000 501 5111 557 136 45 590 303 477 36 0 0.0 0 0 smileart N/A N/A N/A
87667 MTLCompilerS 0.0 00:00.28 2 1 32 8760K 0B 8724K 87667 1 sleeping *0[5] 0.00000 0.00000 501 4608 160 156 45 1232 325 973 60 1 0.0 0 0 smileart N/A N/A N/A
87621 MTLCompilerS 0.0 00:00.12 2 1 27 7240K 0B 7232K 87621 1 sleeping *0[2] 0.00000 0.00000 501 4633 557 136 44 554 301 297 0 3 0.0 0 0 smileart N/A N/A N/A
87620 MTLCompilerS 0.0 00:00.09 2 1 32 3644K 0B 3636K 87620 1 sleeping *0[3] 0.00000 0.00000 501 1902 160 140 42 625 282 368 1 0 0.0 0 0 smileart N/A N/A N/A
87421 com.apple.sp 0.0 00:03.97 2 1 72 13M 0B 8468K 87421 1 sleeping *0[1] 0.00000 0.00000 501 253222 174 2734 1078 472775 6008 19896 912 36 0.0 0 0 smileart N/A N/A N/A
87327 UserNotifica 0.0 00:01.88 3 1 193 6848K 0B 5636K 87327 1 sleeping *0[1] 0.00000 0.00000 501 26946 279 16746 7667 41449 50865 56115 4 962 0.0 0 0 smileart N/A N/A N/A
87255 Skype Helper 0.0 00:10.54 8 1 178 7632K 0B 6444K 86550 86550 sleeping *0[1] 0.00000 0.00000 501 115005 1422 246400 89683 371951 516239 427127 3 14707 0.0 0 0 smileart N/A N/A N/A
DOC
# To visualise the difference:
# parse_fields(str: text)
# parse_fields_memo(str: text)
Benchmark.ips do |x|
x.report('Without memo') {
parse_fields(str: text)
}
x.report('With memo') {
parse_fields_memo(str: text)
}
x.compare!
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment