Hey - 基于Golang开发的一款类似Ab的压测工具

2019/08/04 共 1161 字 4 分钟阅读
EFFICIENT • SOFTWARE-ENGINEERING tools hey stress

Table of Content

AI 摘要: Hey是一款类似于ab的工具，基于golang开发，代码较为精简，对代码简要分析。可以用于压测、巡检以及请求数据分析。

仓库地址：https://github.com/rakyll/hey
Hey是一款类似于ab的工具，基于golang开发，代码较为精简，对代码简要分析。
最开始项目命名为(boom)，后来改名为hey，再次表明取名的重要性！

1. 安装

基于golang编译安装：

1
go get -u github.com/rakyll/hey

仓库地址中也支持二进制安装

2. 运行效果

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
$ hey -n 1000 -c 100 http://10.40.2.181

Summary:
  Total:    0.3404 secs
  Slowest:  0.0683 secs
  Fastest:  0.0165 secs
  Average:  0.0324 secs
  Requests/sec: 2938.0702


Response time histogram:
  0.017 [1] |
  0.022 [13]    |■
  0.027 [36]    |■■
  0.032 [579]   |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.037 [249]   |■■■■■■■■■■■■■■■■■
  0.042 [29]    |■■
  0.048 [46]    |■■■
  0.053 [22]    |■■
  0.058 [19]    |■
  0.063 [4] |
  0.068 [2] |


Latency distribution:
  10% in 0.0278 secs
  25% in 0.0289 secs
  50% in 0.0303 secs
  75% in 0.0336 secs
  90% in 0.0413 secs
  95% in 0.0472 secs
  99% in 0.0558 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0011 secs, 0.0165 secs, 0.0683 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0000 secs, 0.0010 secs
  resp wait:    0.0309 secs, 0.0070 secs, 0.0519 secs
  resp read:    0.0004 secs, 0.0000 secs, 0.0092 secs

Status code distribution:
  [200] 1000 responses

3. 核心流程

3.1. 拎出请求包的Work主流程

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
func (b *Work) Run() {
    b.Init()
    b.start = now()
    b.report = newReport(b.writer(), b.results, b.Output, b.N)
    // Run the reporter first, it polls the result channel until it is closed.
    go func() {
        runReporter(b.report)
    }()
    b.runWorkers()
    b.Finish()
}

3.2. runWorkers() - 基于sync.WaitGroup做多goroutine同步

这里client被多个goroutine复用，

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
func (b *Work) runWorkers() {
    var wg sync.WaitGroup
    wg.Add(b.C)

    tr := &http.Transport{
        TLSClientConfig: &tls.Config{
            InsecureSkipVerify: true,
            ServerName:         b.Request.Host,
        },
        MaxIdleConnsPerHost: min(b.C, maxIdleConn),
        DisableCompression:  b.DisableCompression,
        DisableKeepAlives:   b.DisableKeepAlives,
        Proxy:               http.ProxyURL(b.ProxyAddr),
    }
    if b.H2 {
        http2.ConfigureTransport(tr)
    } else {
        tr.TLSNextProto = make(map[string]func(string, *tls.Conn) http.RoundTripper)
    }
    client := &http.Client{Transport: tr, Timeout: time.Duration(b.Timeout) * time.Second}

    // Ignore the case where b.N % b.C != 0.
    for i := 0; i < b.C; i++ {
        go func() {
            b.runWorker(client, b.N/b.C)
            wg.Done()
        }()
    }
    wg.Wait()
}

3.3. runWorker() - 分配给单个worker运行n次http请求

runWorker()基于非阻塞方式makeRequest()，执行client.Do()http请求

b.stopCh为程序基于运行时间设置-z或者发送Interupt中断信号给到hey进程时候，会从通道接收数据，并退出

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
func (b *Work) runWorker(client *http.Client, n int) {
    var throttle <-chan time.Time
    if b.QPS > 0 {
        throttle = time.Tick(time.Duration(1e6/(b.QPS)) * time.Microsecond)
    }

    if b.DisableRedirects {
        client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
            return http.ErrUseLastResponse
        }
    }
    for i := 0; i < n; i++ {
        // Check if application is stopped. Do not send into a closed channel.
        select {
        case <-b.stopCh:
            return
        default:
            if b.QPS > 0 {
                <-throttle
            }
            b.makeRequest(client)
        }
    }

3.4. makeRequest() - 具体的http请求和trace跟踪

基于net/http/httptrace，对HTTP请求的几个关键节点进行了跟踪：trace := &httptrace.ClientTrace{..}
- DNS开始、结束
- ConnTCP连接开始结束
- 请求发送、首字节响应获取
trace需要放入到http请求对象req中，以上下文req.WithContext进行设置： req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
请求的body部分丢弃：io.Copy(ioutil.Discard, resp.Body)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
func (b *Work) makeRequest(c *http.Client) {
    s := now()
    var size int64
    var code int
    var dnsStart, connStart, resStart, reqStart, delayStart time.Duration
    var dnsDuration, connDuration, resDuration, reqDuration, delayDuration time.Duration
    req := cloneRequest(b.Request, b.RequestBody)
    trace := &httptrace.ClientTrace{
        DNSStart: func(info httptrace.DNSStartInfo) {
            dnsStart = now()
        },
        DNSDone: func(dnsInfo httptrace.DNSDoneInfo) {
            dnsDuration = now() - dnsStart
        },
        GetConn: func(h string) {
            connStart = now()
        },
        GotConn: func(connInfo httptrace.GotConnInfo) {
            if !connInfo.Reused {
                connDuration = now() - connStart
            }
            reqStart = now()
        },
        WroteRequest: func(w httptrace.WroteRequestInfo) {
            reqDuration = now() - reqStart
            delayStart = now()
        },
        GotFirstResponseByte: func() {
            delayDuration = now() - delayStart
            resStart = now()
        },
    }
    req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
    resp, err := c.Do(req)
    if err == nil {
        size = resp.ContentLength
        code = resp.StatusCode
        io.Copy(ioutil.Discard, resp.Body)
        resp.Body.Close()
    }
    t := now()
    resDuration = t - resStart
    finish := t - s
    b.results <- &result{
        offset:        s,
        statusCode:    code,
        duration:      finish,
        err:           err,
        contentLength: size,
        connDuration:  connDuration,
        dnsDuration:   dnsDuration,
        reqDuration:   reqDuration,
        resDuration:   resDuration,
        delayDuration: delayDuration,
    }
}

4. 其他

Usage的编写方式，基于单一的usage方法设置，相对于flag的默认提升，可以做到help的统一
清晰的包文件、以及结构方法，先骨架，再细节，另外一点，导出的方法往前写：
- type Work struct {}
- func (b *Work) writer() io.Writer
- func (b *Work) Init()
- func (b *Work) Run()
- func (b *Work) Stop()
- func (b *Work) Finish()
- func (b *Work) makeRequest(c *http.Client)
- func (b *Work) runWorker(client *http.Client, n int)
- func (b *Work) runWorkers()
- func cloneRequest(r *http.Request, body []byte) *http.Request
- func min(a, b int) int
信号量的捕获、防止竟态+单次实例的sync.Once.Do方法、sync.WaitGroup

4.1. 模板的呈现方式、模板函数的定义

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
func newTemplate(output string) *template.Template {
    outputTmpl := output
    switch outputTmpl {
    case "":
        outputTmpl = defaultTmpl
    case "csv":
        outputTmpl = csvTmpl
    }
    return template.Must(template.New("tmpl").Funcs(tmplFuncMap).Parse(outputTmpl))
}

var tmplFuncMap = template.FuncMap{
    "formatNumber":    formatNumber,
    "formatNumberInt": formatNumberInt,
    "histogram":       histogram,
    "jsonify":         jsonify,
}

5. 后续可以做的事情

常规的压测工具替换，相比于ab，hey可以接收定制化的开发：

基于requester包思想，可以自己组装http请求，实现一个全功能的http并发请求模拟
基于C端的巡检相关工作
基于C端的请求数据分析（类似statusok) & 收集，上报到influxDB，基于grafana图显