Last active
September 18, 2018 08:05
-
-
Save akbargumbira/5d331cd931b79cf1e2ac98b917bc7f8d to your computer and use it in GitHub Desktop.
Official vs custom tensorflow
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 4 cpus on AI1 | |
| # Our inhouse built tensorflow - without XLA | |
| 1st run: | |
| ``` | |
| Running 1m test @ http://0.0.0.0:5642/detect | |
| 1 threads and 1 connections | |
| Thread Stats Avg Stdev Max +/- Stdev | |
| Latency 1.47s 114.37ms 1.91s 77.50% | |
| Req/Sec 0.00 0.00 0.00 100.00% | |
| 40 requests in 1.00m, 21.95KB read | |
| Requests/sec: 0.67 | |
| ``` | |
| 2nd run: | |
| ``` | |
| Transfer/sec: 374.61B | |
| akbar@debian:~/dev/yoloist/benchmark$ sh run.sh | |
| Running 1m test @ http://0.0.0.0:5642/detect | |
| 1 threads and 1 connections | |
| Thread Stats Avg Stdev Max +/- Stdev | |
| Latency 1.48s 119.42ms 1.87s 77.50% | |
| Req/Sec 0.00 0.00 0.00 100.00% | |
| 40 requests in 1.00m, 21.95KB read | |
| Requests/sec: 0.67 | |
| Transfer/sec: 374.65B | |
| ``` | |
| with XLA: | |
| 1st run: | |
| ``` | |
| Running 1m test @ http://0.0.0.0:5642/detect | |
| 1 threads and 1 connections | |
| Thread Stats Avg Stdev Max +/- Stdev | |
| Latency 1.48s 101.21ms 1.74s 80.00% | |
| Req/Sec 0.00 0.00 0.00 100.00% | |
| 40 requests in 1.00m, 21.95KB read | |
| Requests/sec: 0.67 | |
| Transfer/sec: 374.18B | |
| ``` | |
| 2nd run: | |
| ``` | |
| unning 1m test @ http://0.0.0.0:5642/detect | |
| 1 threads and 1 connections | |
| Thread Stats Avg Stdev Max +/- Stdev | |
| Latency 1.46s 89.41ms 1.68s 65.85% | |
| Req/Sec 0.00 0.00 0.00 100.00% | |
| 41 requests in 1.00m, 22.50KB read | |
| Requests/sec: 0.68 | |
| Transfer/sec: 383.76B | |
| ``` | |
| # Custom tensorflow | |
| 1st run: | |
| ``` | |
| Running 1m test @ http://0.0.0.0:5642/detect | |
| 1 threads and 1 connections | |
| Thread Stats Avg Stdev Max +/- Stdev | |
| Latency 1.65s 117.00ms 2.04s 69.44% | |
| Req/Sec 0.00 0.00 0.00 100.00% | |
| 36 requests in 1.00m, 19.76KB read | |
| Requests/sec: 0.60 | |
| Transfer/sec: 336.72B | |
| ``` | |
| 2nd run: | |
| ``` | |
| Running 1m test @ http://0.0.0.0:5642/detect | |
| 1 threads and 1 connections | |
| Thread Stats Avg Stdev Max +/- Stdev | |
| Latency 1.69s 187.05ms 2.31s 85.71% | |
| Req/Sec 0.00 0.00 0.00 100.00% | |
| 35 requests in 1.00m, 19.21KB read | |
| Requests/sec: 0.58 | |
| Transfer/sec: 327.39B | |
| ``` | |
| # Official tensorflow | |
| 1st run: | |
| ``` | |
| Running 1m test @ http://0.0.0.0:5642/detect | |
| 1 threads and 1 connections | |
| Thread Stats Avg Stdev Max +/- Stdev | |
| Latency 2.14s 195.81ms 2.81s 82.14% | |
| Req/Sec 0.00 0.00 0.00 100.00% | |
| 28 requests in 1.00m, 15.31KB read | |
| Requests/sec: 0.47 | |
| Transfer/sec: 260.96B | |
| ``` | |
| 2nd run: | |
| ``` | |
| Running 1m test @ http://0.0.0.0:5642/detect | |
| 1 threads and 1 connections | |
| Thread Stats Avg Stdev Max +/- Stdev | |
| Latency 2.15s 178.86ms 2.71s 81.48% | |
| Req/Sec 0.00 0.00 0.00 100.00% | |
| 27 requests in 1.00m, 14.77KB read | |
| Requests/sec: 0.45 | |
| Transfer/sec: 251.63B | |
| ``` |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment