Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

데이터는 조회 시간과 각 항목별 수치만 저장되며, /show, /watch 또는 /report 명령 실행시 데이터 파싱 후 포멧에 맞춰 출력합니다.

...

후 포멧에 맞춰 출력합니다.

Code Block
2021-07-20_05:55:21.443 955 955 244480 244480 1042 1042 266752 266752

/show [/t {types[,...]|all}] [/r {resource[,...]|all}]

전체 리소스에 대한 모든 항목의 성능 데이터를 한번에 출력합니다.

  • 출력 대상은 성능 파일에 마지막으로 기록된 데이터이며, 특정 성능 항목 및 리소스를 지정하여 출력할 수 있는 옵션을 제공합니다.

    • 성능 항목 및 리소스 명은 ','로 구분하며 공백 없이 입력합니다.

  • ex) 모든 리소스의 모든 항목 출력

Expand
titlebsrmon /show
  • memory, resource 별로 timestamp 가 출력됩니다.

  • iostat, ioclat, reqstat, peer_reqstat, al_stat 은 vnr 로 구분하여 출력됩니다.

  • memory 성능 데이터는 os(windows/linux)에 따라 출력되는 항목이 다릅니다.

Code Block
[root@cent79_01 bsr-utils]# bsrmon /show
bsrmon {
    memory {
        system {
            total_memory        7990028; # kbytes
            used_memory 1843756; # kbytes
            free_memory 2511096; # kbytes
            buff/cache  3635176; # kbytes
        }
        module {
            slab {
                bsr_req 15982; # kbytes
                bsr_al  1020; # kbytes
                bsr_bm  7880; # kbytes
                bsr_ee  2384; # kbytes
                total_bio_set   520; # kbytes
                kmalloc 157; # kbytes
            }
            vmalloc     0; # kbytes
            total_page_pool     33344; # kbytes
        }
        user {
            top_process {
                name    gnome-shell;
                pid     2482;
                rsz     250212; # kbytes
                vsz     4082020; # kbytes
            }
            bsr_process {
                name    bsrmon;
                pid     19370;
                rsz     1364; # kbytes
                vsz     12780; # kbytes
            }
        }
        timestamp 2022-10-04_00:10:55.805;
    }
    resource r0 {
        vnr 0 {
            iostat {
                read_iops       0;
                read_iocnt      0;
                read_kbs        0; # kbytes/second
                read_kb 0; # kbytes
                write_iops      0;
                write_iocnt     0;
                write_kbs       0; # kbytes/second
                write_kb        0; # kbytes
            }
            ioclat {
                local_min       0; # usec
                local_max       0; # usec
                local_avg       0; # usec
                master_min      0; # usec
                master_max      0; # usec
                master_avg      0; # usec
            }
            reqstat {
                requests {
                    count       0;
                    before_queue_min    0; # usec
                    before_queue_max    0; # usec
                    before_queue_avg    0; # usec
                    before_al_begin_min 0; # usec
                    before_al_begin_max 0; # usec
                    before_al_begin_avg 0; # usec
                    in_actlog_min       0; # usec
                    in_actlog_max       0; # usec
                    in_actlog_avg       0; # usec
                    submit_min  0; # usec
                    submit_max  0; # usec
                    submit_avg  0; # usec
                    bio_endio_min       0; # usec
                    bio_endio_max       0; # usec
                    bio_endio_avg       0; # usec
                    destroy_min 0; # usec
                    destroy_max 0; # usec
                    destroy_avg 0; # usec
                }
                al_update {
                    count       0;
                    before_bm_write_min 0; # usec
                    before_bm_write_max 0; # usec
                    before_bm_write_avg 0; # usec
                    after_bm_write_min  0; # usec
                    after_bm_write_max  0; # usec
                    after_bm_write_avg  0; # usec
                    after_sync_page_min 0; # usec
                    after_sync_page_max 0; # usec
                    after_sync_page_avg 0; # usec
                }
                peer cent79_03 {
                    pre_send_min        0; # usec
                    pre_send_max        0; # usec
                    pre_send_avg        0; # usec
                    acked_min   0; # usec
                    acked_max   0; # usec
                    acked_avg   0; # usec
                    net_done_min        0; # usec
                    net_done_max        0; # usec
                    net_done_avg        0; # usec
                }
            }
            peer_reqstat {
                peer cent79_03 {
                    count       0;
                    submit_min  0; # usec
                    submit_max  0; # usec
                    submit_avg  0; # usec
                    bio_endio_min       0; # usec
                    bio_endio_max       0; # usec
                    bio_endio_avg       0; # usec
                    destroy_min 0; # usec
                    destroy_max 0; # usec
                    destroy_avg 0; # usec
                }
            }
            al_stat {
                al-extents      6001;
                al_used 0;
                al_used_max     0;
                hits    0;
                hits_total      2064;
                misses  0;
                misses_total    1998;
                starving        0;
                starving_total  0;
                locked  0;
                locked_total    0;
                changed 0;
                changed_total   249;
                al_wait_retry_cnt       0;
                al_wait_total_retry_cnt 0;
                al_wait_max_retry_cnt   0;
                pending_changes 0;
                max_pending_changes     64;
                error {
                    nobufs_starving     0;
                    nobufs_pending_slot 0;
                    nobufs_used_slot    0;
                    busy        0;
                    wouldblock  0;
                }
                flags   __LC_DIRTY,__LC_LOCKED;
            }
        }
        vnr 1 {
            iostat {
                read_iops       0;
                read_iocnt      0;
                read_kbs        0; # kbytes/second
                read_kb 0; # kbytes
                write_iops      0;
                write_iocnt     0;
                write_kbs       0; # kbytes/second
                write_kb        0; # kbytes
            }
            ioclat {
                local_min       0; # usec
                local_max       0; # usec
                local_avg       0; # usec
                master_min      0; # usec
                master_max      0; # usec
                master_avg      0; # usec
            }
            reqstat {
                requests {
                    count       0;
                    before_queue_min    0; # usec
                    before_queue_max    0; # usec
                    before_queue_avg    0; # usec
                    before_al_begin_min 0; # usec
                    before_al_begin_max 0; # usec
                    before_al_begin_avg 0; # usec
                    in_actlog_min       0; # usec
                    in_actlog_max       0; # usec
                    in_actlog_avg       0; # usec
                    submit_min  0; # usec
                    submit_max  0; # usec
                    submit_avg  0; # usec
                    bio_endio_min       0; # usec
                    bio_endio_max       0; # usec
                    bio_endio_avg       0; # usec
                    destroy_min 0; # usec
                    destroy_max 0; # usec
                    destroy_avg 0; # usec
                }
                al_update {
                    count       0;
                    before_bm_write_min 0; # usec
                    before_bm_write_max 0; # usec
                    before_bm_write_avg 0; # usec
                    after_bm_write_min  0; # usec
                    after_bm_write_max  0; # usec
                    after_bm_write_avg  0; # usec
                    after_sync_page_min 0; # usec
                    after_sync_page_max 0; # usec
                    after_sync_page_avg 0; # usec
                }
                peer cent79_03 {
                    pre_send_min        0; # usec
                    pre_send_max        0; # usec
                    pre_send_avg        0; # usec
                    acked_min   0; # usec
                    acked_max   0; # usec
                    acked_avg   0; # usec
                    net_done_min        0; # usec
                    net_done_max        0; # usec
                    net_done_avg        0; # usec
                }
            }
            peer_reqstat {
                peer cent79_03 {
                    count       0;
                    submit_min  0; # usec
                    submit_max  0; # usec
                    submit_avg  0; # usec
                    bio_endio_min       0; # usec
                    bio_endio_max       0; # usec
                    bio_endio_avg       0; # usec
                    destroy_min 0; # usec
                    destroy_max 0; # usec
                    destroy_avg 0; # usec
                }
            }
            al_stat {
                al-extents      6001;
                al_used 0;
                al_used_max     0;
                hits    0;
                hits_total      0;
                misses  0;
                misses_total    0;
                starving        0;
                starving_total  0;
                locked  0;
                locked_total    0;
                changed 0;
                changed_total   0;
                al_wait_retry_cnt       0;
                al_wait_total_retry_cnt 0;
                al_wait_max_retry_cnt   0;
                pending_changes 0;
                max_pending_changes     64;
                error {
                    nobufs_starving     0;
                    nobufs_pending_slot 0;
                    nobufs_used_slot    0;
                    busy        0;
                    wouldblock  0;
                }
                flags   NONE;
            }
        }
        network {
            peer cent79_02 {
                send    180; # byte/second
                recv    384; # byte/second
            }
            peer cent79_03 {
                send    0; # byte/second
                recv    0; # byte/second
            }
        }
        sendbuf {
            peer cent79_02 {
                ap_in_flight {
                    size        17301504; # bytes
                    count       33;
                }
                rs_in_flight {
                    size        0; # bytes
                    count       0;
                }
                highwater       33;
                fill    17301504; # bytes
                data_stream {
                    size        20971520; # bytes
                    used        0; # bytes
                    packet {
                        name    P_DATA
                        count   1;
                        size    0; # bytes
                    }
                }
                control_stream {
                    size        5242880; # bytes
                    used        0; # bytes
                }
            }
            peer cent79_03 {
                ap_in_flight {
                    size        0; # bytes
                    count       0;
                }
                rs_in_flight {
                    size        0; # bytes
                    count       0;
                }
                highwater       0;
                fill    0; # bytes
                data_stream {
                    size        20971520; # bytes
                    used        0; # bytes
                }
                control_stream {
                    size        5242880; # bytes
                    used        0; # bytes
                }
            }
        }
        resync_ratio {
            peer cent79_02 {
                replication     0; # byte/second
                resync  0; # byte/second
                resync_ratio    0; # percent
            }
            peer 0 {
                replication     0; # byte/second
                resync  0; # byte/second
                resync_ratio    cent79_03; # percent
            }
        }
        timestamp 2022-09-29_22:54:25.064;
    }
    resource r1 {
        ...
        timestamp 2022-10-04_00:10:55.805;
    }
}
  • ex) 모든 리소스의 특정 항목 출력

Expand
titlebsrmon /show /t iostat
Code Block
[root@cent79_01 bsr-utils]# bsrmon /show /t iostat
bsrmon {
    resource r0 {
        vnr 0 {
            iostat {
                read_iops       0;
                read_iocnt      0;
                read_kbs        0; # kbytes/second
                read_kb 0; # kbytes
                write_iops      0;
                write_iocnt     0;
                write_kbs       0; # kbytes/second
                write_kb        0; # kbytes
            }
        }
        vnr 1 {
            iostat {
                read_iops       0;
                read_iocnt      0;
                read_kbs        0; # kbytes/second
                read_kb 0; # kbytes
                write_iops      0;
                write_iocnt     0;
                write_kbs       0; # kbytes/second
                write_kb        0; # kbytes
            }
        }
        timestamp 2022-09-29_22:54:25.064;
    }
    resource r1 {
        vnr 0 {
            iostat {
                read_iops       0;
                read_iocnt      0;
                read_kbs        0; # kbytes/second
                read_kb 0; # kbytes
                write_iops      0;
                write_iocnt     0;
                write_kbs       0; # kbytes/second
                write_kb        0; # kbytes
            }
        }
        timestamp 2022-10-04_00:17:52.249;
    }
}
  • ex) 특정 리소스의 특정 항목 출력

Expand
titlebsrmon /show /t network,sendbuf /r r0,r1
Code Block
[root@cent79_01 bsr-utils]# bsrmon /show /t network,sendbuf /r r0,r1
bsrmon {
    resource r0 {
        network {
            peer cent79_02 {
                send    180; # byte/second
                recv    384; # byte/second
            }
            peer cent79_03 {
                send    0; # byte/second
                recv    0; # byte/second
            }
        }
        sendbuf {
            peer cent79_02 {
                ap_in_flight {
                    size        17301504; # bytes
                    count       33;
                }
                rs_in_flight {
                    size        0; # bytes
                    count       0;
                }
                highwater       33;
                fill    17301504; # bytes
                data_stream {
                    size        20971520; # bytes
                    used        0; # bytes
                    packet {
                        name    P_DATA
                        count   1;
                        size    0; # bytes
                    }
                }
                control_stream {
                    size        5242880; # bytes
                    used        0; # bytes
                }
            }
            peer cent79_03 {
                ap_in_flight {
                    size        0; # bytes
                    count       0;
                }
                rs_in_flight {
                    size        0; # bytes
                    count       0;
                }
                highwater       0;
                fill    0; # bytes
                data_stream {
                    size        20971520; # bytes
                    used        0; # bytes
                }
                control_stream {
                    size        5242880; # bytes
                    used        0; # bytes
                }
            }
        }
        timestamp 2022-09-29_22:54:24.021;
    }
    resource r1 {
        network {
            peer cent79_02 {
                send    0; # byte/second
                recv    0; # byte/second
            }
            peer cent79_03 {
                send    0; # byte/second
                recv    0; # byte/second
            }
        }
        sendbuf {
            peer cent79_02 {
                ap_in_flight {
                    size        0; # bytes
                    count       0;
                }
                rs_in_flight {
                    size        0; # bytes
                    count       0;
                }
                highwater       0;
                fill    0; # bytes
                data_stream {
                    size        0; # bytes
                    used        0; # bytes
                }
                control_stream {
                    size        0; # bytes
                    used        0; # bytes
                }
            }
            peer cent79_03 {
                ap_in_flight {
                    size        0; # bytes
                    count       0;
                }
                rs_in_flight {
                    size        0; # bytes
                    count       0;
                }
                highwater       0;
                fill    0; # bytes
                data_stream {
                    size        0; # bytes
                    used        0; # bytes
                }
                control_stream {
                    size        0; # bytes
                    used        0; # bytes
                }
            }
        }
        timestamp 2022-10-04_00:19:53.026;
    }
}

/watch {types} [/scroll]

types 별로 집계되고 있는 데이터를 실시간으로 출력합니다. /scroll 옵션 사용시 출력을 줄넘김 방식으로 표기합니다.

...

복제 볼륨의 읽기, 쓰기 I/O 에 대한 집계 데이터 (iops, iosio count, kbs, kb)

Expand
title사용 예
  • I/O 성능 데이터 실시간 모니터링

Code Block
> bsrmon /watch iostat r0 0
2020-12-06_08:22:36.223
  read : IOPS=0 (IOs=0), BW=0kb/s (0KB)
  write: IOPS=216 (IOs=432), BW=110208kb/s (220416KB)
2020-12-06_08:22:37.640
  read : IOPS=0 (IOs=0), BW=0kb/s (0KB)
  write: IOPS=34 (IOs=34), BW=16896kb/s (16896KB)
2020-12-06_08:22:39.32
  read : IOPS=0 (IOs=0), BW=0kb/s (0KB)
  write: IOPS=326 (IOs=326), BW=155176kb/s (155176KB)
2020-12-06_08:22:40.106
  read : IOPS=0 (IOs=0), BW=0kb/s (0KB)
  write: IOPS=330 (IOs=330), BW=164624kb/s (164624KB)
  ...

→ write I/O 가 지속적으로 발생하고 있는 상황

  • I/O 성능 데이터의 통계 출력

Code Block
> bsrmon /report iostat r0 0
Report r0 [IO STAT - vnr0]
 Run: 2021-07-20_08:14:27.405 - 2021-08-04_18:59:53.573
  read : ios=24855, bw=6325194kbyte
    IOPS        : min=10, max=1113, avg=972, samples=24
    BW (kbyte/s): min=40, max=284928, avg=248578, samples=24
  write: ios=8360345, bw=2139182512kbyte
    IOPS        : min=3, max=1568, avg=1252, samples=6670
    BW (kbyte/s): min=12, max=401408, avg=320256, samples=6671

→ 2021-07-20_08:14:27.405 - 2021-08-04_18:59:53.573 기간 동안 집계된 데이터

→ 24개의 read I/O samples. 평균 972 iops, 최소 10 iops, 최대 1113 iops로 측정됨

→ 6670개의 write I/O samples. 평균 1252 iops, 최소 3 iops, 최대 1568 iops로 측정됨

  • 백업 파일을 사용하여 통계 계산

Code Block
> bsrmon /report iostat r0 0 /f vnr0_IO_STAT_2021-07-20_081427.005 
Report r0 [vnr0_IO_STAT_2021-07-20_081427.005]
 Run: 2021-07-07_21:24:24.383 - 2021-07-20_08:14:26.371
  read : ios=6332219, bw=1595360917kbyte
    IOPS        : min=1, max=1233, avg=980, samples=6452
    BW (kbyte/s): min=4, max=428032, avg=247033, samples=6452
  write: ios=22550577, bw=3961426883kbyte
    IOPS        : min=1, max=6478, avg=786, samples=28468
    BW (kbyte/s): min=4, max=1085440, avg=138461, samples=28484

→ vnr0_IO_STAT_2021-07-20_081427.005 파일로 부터 데이터 추출

→ 2021-07-07_21:24:24.383 - 2021-07-20_08:14:26.371 기간 동안 집계된 데이터

→ 6452 개의 read I/O samples. 평균 980 iops, 최소 1 iops, 최대 1233 iops 로 측정됨

→ 28468 개의 write I/O samples. 평균 786iops, 최소 1 iops, 최대 6478 iops 로 측정됨

...