Нагрузочное тестирование Atlassian Jira, Confluence, Bitbucket Часть 3

Part 2.

В этой части мы поговорим об отчетах и графиках, которые Вы получаете после запуска теста из коробки.

Но сначала я бы хотел рассказать о важном изменении, которое произошло за то время, пока я писал эту часть. Теперь при запуске теста Вы можете мониторить процесс выполнения теста в Вашем терминале.

Этот экран называется Taurus console reporter. Экран разделен на три части: информация о прохождении теста в графиках, информация о прохождении теста текстом и системная информация.

Системная информация очень важная секция. Она показывает загрузку компьютера, с которого Вы запустили тест. Вам необходимо следить, чтобы использование ресурсов не превышало 80% от имеющихся, потому что если, например, загрузка процессора будет 100%, то это может означать, что Ваша система уже настолько загружена, что не может подавать нагрузку, которую Вы задали в конфигурации теста.

Результаты тестирования

Я поговорю о результатах тестирования на примере Confluence.

После того, как тесты были выполнены, то Вы увидите следующие файлы в директории app/results/confluence:

Давайте пройдемся по каждому файлу. Я буду приводить содержимое файлов для того, чтобы Вы смоги посмотреть, что Вы получите после выполнения тестов без необходимости запускать тест самостоятельно.

test_1_selenium_create_comment-2020-07-24-14-37-36.853895.html

Директория error_artifacts folder содержит информацию об ошибках в тестах Selenium. Вот файл test_1_selenium_create_comment-2020-07-24-14-37-36.853895.html:

<html><head>
                            <title>Selenium - UobuX2FuIO - TES - Product Team Confluence</title>
<meta http-equiv="X-UA-Compatible" content="IE=EDGE,chrome=IE7">
<meta charset="UTF-8">
<meta id="confluence-context-path" name="confluence-context-path" content="/confluence">
<meta id="confluence-base-url" name="confluence-base-url" content="https://my.domain.com/confluence">

    <meta id="atlassian-token" name="atlassian-token" content="f83db95dc7c654e23f789178b7ed6b7e98d1077b">

<meta id="confluence-cluster-node-id" name="confluence-cluster-node-id" value="37863810">
<meta id="confluence-cluster-node-name" name="confluence-cluster-node-name" value="node3">

<meta id="confluence-space-key" name="confluence-space-key" content="TES">
<script type="text/javascript">
        var contextPath = '/confluence';
</script>

    

    <meta name="robots" content="noindex,nofollow">
    <meta name="robots" content="noarchive">
    <meta name="confluence-request-time" content="1595590631816">
        
    ..........

Как Вы видите, в этом файле содержится html код страницы, который вызвал ошибку. А в файле test_1_selenium_create_comment-2020-07-24-14-37-36.853895.png мы видим скриншот этой страницы:

У нас произошла ошибка “Unable to communicate with server”, что означает, что Confluence перестал отвечать. Посмотрев в логи видно, что у нас была вот такая ошибка: java.lang.OutOfMemoryError: Java heap space. Значит нужно смотреть, чем у нас был забит Heap.

bzt.log

Файл содержит информацию, которую вы видите в консоли, после запуска теста:

[2020-07-24 14:32:11,645 INFO root] Taurus CLI Tool v1.14.2
[2020-07-24 14:32:11,646 DEBUG root] Command-line options: {'log': '/var/folders/4r/w8hkd4w51x787tw4ynb29v2w0000gn/T/bzt_deodyr86.log', 'option': None, 'quiet': None, 'verbose': None, 'no_system_configs': None, 'aliases': []}
[2020-07-24 14:32:11,646 DEBUG root] Python: CPython 3.8.5
[2020-07-24 14:32:11,646 DEBUG root] OS: uname_result(system='Darwin', node='Alexeys-MacBook-Pro-2.local', release='19.6.0', version='Darwin Kernel Version 19.6.0: Sun Jul  5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64', machine='x86_64', processor='i386')
[2020-07-24 14:32:11,646 INFO root] Starting with configs: ['confluence.yml']
[2020-07-24 14:32:11,649 DEBUG root] Using personal config: /Users/alexm/.bzt-rc
[2020-07-24 14:32:11,649 INFO Engine] Configuring...
[2020-07-24 14:32:11,718 DEBUG Engine] Base configs list: ['/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/venv/lib/python3.8/site-packages/bzt/resources/10-base-config.yml']
[2020-07-24 14:32:11,719 DEBUG Engine.Configuration] Configs: ['/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/venv/lib/python3.8/site-packages/bzt/resources/10-base-config.yml']
[2020-07-24 14:32:11,719 DEBUG Engine.Configuration] Reading /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/venv/lib/python3.8/site-packages/bzt/resources/10-base-config.yml as YAML
[2020-07-24 14:32:11,742 DEBUG Engine] User configs list: ['/Users/alexm/.bzt-rc', 'confluence.yml']
[2020-07-24 14:32:11,742 DEBUG Engine.Configuration] Configs: ['/Users/alexm/.bzt-rc', 'confluence.yml']
[2020-07-24 14:32:11,743 DEBUG Engine.Configuration] Reading /Users/alexm/.bzt-rc as YAML
[2020-07-24 14:32:11,747 DEBUG Engine.Configuration] Reading confluence.yml as YAML
[2020-07-24 14:32:11,762 DEBUG Engine.Configuration] Configs: ['/Users/alexm/.bzt-rc', 'confluence.yml']
[2020-07-24 14:32:11,762 DEBUG Engine.Configuration] Reading /Users/alexm/.bzt-rc as YAML
[2020-07-24 14:32:11,766 DEBUG Engine.Configuration] Reading confluence.yml as YAML
[2020-07-24 14:32:11,781 DEBUG Engine] Requesting updates info: https://gettaurus.org/updates/?version=1.14.2&installID=macos-acde48001122
[2020-07-24 14:32:11,781 DEBUG HTTPClient] Making HTTP request GET https://gettaurus.org/updates/?version=1.14.2&installID=macos-acde48001122
[2020-07-24 14:32:11,781 INFO Engine] Artifacts dir: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11
[2020-07-24 14:32:11,783 DEBUG Engine] New artifact filename: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11/effective
[2020-07-24 14:32:11,785 DEBUG Engine.Configuration] Dumping YAML config into /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11/effective.yml
[2020-07-24 14:32:11,807 DEBUG Engine.Configuration] Dumping JSON config into /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11/effective.json
[2020-07-24 14:32:11,808 DEBUG Engine] New artifact filename: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11/merged.yml
[2020-07-24 14:32:11,809 DEBUG Engine.Configuration] Dumping YAML config into /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11/merged.yml
[2020-07-24 14:32:11,816 DEBUG Engine] New artifact filename: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11/merged.json
[2020-07-24 14:32:11,817 DEBUG Engine.Configuration] Dumping JSON config into /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11/merged.json
[2020-07-24 14:32:11,817 DEBUG Engine] Add existing artifact (move=False): confluence.yml
[2020-07-24 14:32:11,818 DEBUG Engine] Copying confluence.yml to /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11/confluence.yml
[2020-07-24 14:32:11,819 DEBUG root] Closing log handler: /var/folders/4r/w8hkd4w51x787tw4ynb29v2w0000gn/T/bzt_deodyr86.log
[2020-07-24 14:32:11,819 DEBUG root] Switched writing logs to /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_14-32-11/bzt.log
[2020-07-24 14:32:11,819 DEBUG root] Linting config
[2020-07-24 14:32:11,823 INFO Engine] Preparing...
........................

Если Вам нужно посмотреть, что происходило на фазах подготовки, запуска и после запуска тестирования, то Вы можете это увидеть именно здесь.

confluence.yml

Этот файл является копией файла confluence.yml, который Вы использовали при запуске теста командой bzt confluence.yml. Здесь Вы всегда можете посмотреть параметры, которые Вы использовали при запуске теста.

effective.json/effective.yml

Эти два файла это копии файла confluence.yml в форматах yml и json.

error.jtl

Вы можете исползовать этот файл для дебага скриптов Jmeter. О том как это сделать, Вы можете почитать вот здесь docs/confluence/README.md.

jmeter.err

Файл ошибок Jmeter. В моем случае этот файл пуст.

jmeter.log

Лог файл Jmeter, в котором Jmeter логирует все свои сообщения во время выполнения тестов.

2020-07-24 15:10:00,962 INFO o.a.j.u.JMeterUtils: Setting Locale to en_RU
2020-07-24 15:10:00,978 INFO o.a.j.JMeter: Loading user properties from: /Users/alexm/.bzt/jmeter-taurus/5.2.1/bin/user.properties
2020-07-24 15:10:00,979 INFO o.a.j.JMeter: Loading system properties from: /Users/alexm/.bzt/jmeter-taurus/5.2.1/bin/system.properties
2020-07-24 15:10:00,979 INFO o.a.j.JMeter: Loading additional properties from: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_15-07-19/jmeter-bzt.properties
2020-07-24 15:10:00,979 INFO o.a.j.JMeter: Setting System properties from file: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_15-07-19/system.properties
2020-07-24 15:10:00,980 INFO o.a.j.JMeter: Copyright (c) 1998-2019 The Apache Software Foundation
2020-07-24 15:10:00,980 INFO o.a.j.JMeter: Version 5.2.1
2020-07-24 15:10:00,980 INFO o.a.j.JMeter: java.version=1.8.0_252
2020-07-24 15:10:00,980 INFO o.a.j.JMeter: java.vm.name=OpenJDK 64-Bit Server VM
2020-07-24 15:10:00,980 INFO o.a.j.JMeter: os.name=Mac OS X
2020-07-24 15:10:00,980 INFO o.a.j.JMeter: os.arch=x86_64
2020-07-24 15:10:00,980 INFO o.a.j.JMeter: os.version=10.15.6
2020-07-24 15:10:00,980 INFO o.a.j.JMeter: file.encoding=UTF-8
2020-07-24 15:10:00,981 INFO o.a.j.JMeter: java.awt.headless=true
2020-07-24 15:10:00,981 INFO o.a.j.JMeter: Max memory     =7635730432
2020-07-24 15:10:00,981 INFO o.a.j.JMeter: Available Processors =8
2020-07-24 15:10:00,984 INFO o.a.j.JMeter: Default Locale=English (Russia)
2020-07-24 15:10:00,984 INFO o.a.j.JMeter: JMeter  Locale=English (Russia)
2020-07-24 15:10:00,984 INFO o.a.j.JMeter: JMeterHome=/Users/alexm/.bzt/jmeter-taurus/5.2.1
2020-07-24 15:10:00,984 INFO o.a.j.JMeter: user.dir  =/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app
2020-07-24 15:10:00,984 INFO o.a.j.JMeter: PWD       =/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app
2020-07-24 15:10:05,994 INFO o.a.j.JMeter: IP: 192.168.0.33 Name: Alexeys-MacBook-Pro-2.local FullName: 192.168.0.33
2020-07-24 15:10:05,995 INFO o.a.j.JMeter: user.classpath=/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_15-07-19:/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/jmeter
2020-07-24 15:10:05,995 INFO o.a.j.JMeter: Adding to classpath and loader: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_15-07-19
2020-07-24 15:10:05,996 INFO o.a.j.JMeter: Adding to classpath and loader: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/jmeter
2020-07-24 15:10:06,002 INFO o.a.j.s.FileServer: Default base='/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app'
2020-07-24 15:10:06,006 INFO o.a.j.s.FileServer: Set new base='/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/jmeter'
2020-07-24 15:10:06,118 INFO o.a.j.s.SaveService: Testplan (JMX) version: 2.2. Testlog (JTL) version: 2.2
2020-07-24 15:10:06,126 INFO o.a.j.s.SaveService: Using SaveService properties file encoding UTF-8
2020-07-24 15:10:06,130 INFO o.a.j.s.SaveService: Using SaveService properties version 5.0
................

jmeter-bzt.properties

eter.save.saveservice.autoflush=true
jmeter.save.saveservice.connect_time=true
jmeterengine.force.system.exit=true
summariser.name=
application_hostname=my.domain.io
application_protocol=https
application_port=443
application_postfix=/confluence
total_actions_per_hr=200
perc_view_page=54
perc_view_dashboard=6
perc_view_blog=8
perc_search_cql=7
perc_create_blog=3
perc_create_and_edit_page=6
perc_comment_page=5
perc_view_attachment=3
perc_upload_attachment=5
perc_like_page=3
perc_standalone_extension=0
jmeterengine.nongui.port=4445
jmeterengine.nongui.maxport=4445
jmeter.save.saveservice.timestamp_format=ms
sampleresult.default.encoding=UTF-8
user.classpath=/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2020-07-24_15-07-19:/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/jmeter

Этот файл содержит параметры, которые использовал Jmeter во время выполнения тестов.

kpi.jtl

timeStamp,elapsed,label,responseCode,responseMessage,threadName,success,bytes,grpThreads,allThreads,Latency,Hostname,Connect
1595592651980,13982,jmeter_login_and_view_dashboard,200,"Number of samples in transaction : 8, number of failing samples : 0",Confluence 1-1,true,20910,1,1,0,Alexeys-MacBook-Pro-2.local,0
1595592666301,20109,jmeter_view_page,200,"Number of samples in transaction : 13, number of failing samples : 0",Confluence 1-1,true,34575,1,1,0,Alexeys-MacBook-Pro-2.local,0
1595592686489,983,jmeter_view_page_tree,404,,Confluence 1-1,false,32539,1,1,392,Alexeys-MacBook-Pro-2.local,0
1595592738274,16580,jmeter_view_page,200,"Number of samples in transaction : 13, number of failing samples : 0",Confluence 1-1,true,34564,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592754868,805,jmeter_view_page_tree,404,,Confluence 1-1,false,32539,2,2,411,Alexeys-MacBook-Pro-2.local,0
1595592742430,15346,jmeter_login_and_view_dashboard,200,"Number of samples in transaction : 8, number of failing samples : 0",Confluence 1-2,true,20958,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592774276,36189,jmeter_view_page,200,"Number of samples in transaction : 13, number of failing samples : 0",Confluence 1-1,true,34581,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592810477,30908,jmeter_view_page_tree,Non HTTP response code: java.net.SocketTimeoutException,Non HTTP response message: Read timed out,Confluence 1-1,false,2267,2,2,10712,Alexeys-MacBook-Pro-2.local,590
1595592829809,20611,jmeter_view_page,Non HTTP response code: java.net.SocketTimeoutException,,Confluence 1-2,false,2905,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592841398,20907,jmeter_recently_viewed,Non HTTP response code: java.net.SocketTimeoutException,"Number of samples in transaction : 1, number of failing samples : 1",Confluence 1-1,false,2905,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592862307,19672,jmeter_search_results,200,"Number of samples in transaction : 2, number of failing samples : 0",Confluence 1-1,true,2091,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592850434,32843,jmeter_login_and_view_dashboard,Non HTTP response code: java.net.SocketTimeoutException,"Number of samples in transaction : 8, number of failing samples : 3",Confluence 1-3,false,20394,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592883305,10416,jmeter_login_and_view_dashboard,200,"Number of samples in transaction : 8, number of failing samples : 0",Confluence 1-4,true,20932,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592882057,15549,jmeter_view_page,200,"Number of samples in transaction : 13, number of failing samples : 0",Confluence 1-1,true,34577,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592897621,699,jmeter_view_page_tree,404,,Confluence 1-1,false,32539,2,2,306,Alexeys-MacBook-Pro-2.local,0
1595592898329,3847,jmeter_view_dashboard,200,"Number of samples in transaction : 7, number of failing samples : 0",Confluence 1-1,true,19843,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592902229,16529,jmeter_create_page_editor,404,"Number of samples in transaction : 11, number of failing samples : 1",Confluence 1-1,false,58991,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592918772,8326,jmeter_create_page,200,"Number of samples in transaction : 17, number of failing samples : 0",Confluence 1-1,true,40246,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592927111,4573,jmeter_open_editor,404,"Number of samples in transaction : 9, number of failing samples : 1",Confluence 1-1,false,56905,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592929738,7926,jmeter_view_page,200,"Number of samples in transaction : 13, number of failing samples : 0",Confluence 1-4,true,34561,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592931695,6455,jmeter_edit_page,200,"Number of samples in transaction : 17, number of failing samples : 0",Confluence 1-1,true,40055,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592937682,745,jmeter_view_page_tree,404,,Confluence 1-4,false,32539,2,2,339,Alexeys-MacBook-Pro-2.local,0
1595592938438,942,jmeter_comment,200,"Number of samples in transaction : 1, number of failing samples : 0",Confluence 1-4,true,1316,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592939441,3717,jmeter_upload_attachment,200,"Number of samples in transaction : 1, number of failing samples : 0",Confluence 1-4,true,13991,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592965749,5643,jmeter_view_page,200,"Number of samples in transaction : 13, number of failing samples : 0",Confluence 1-4,true,34558,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592971405,674,jmeter_view_page_tree,404,,Confluence 1-4,false,32539,2,2,270,Alexeys-MacBook-Pro-2.local,0
1595592974241,7707,jmeter_view_page,200,"Number of samples in transaction : 13, number of failing samples : 0",Confluence 1-1,true,34558,2,2,0,Alexeys-MacBook-Pro-2.local,0
1595592981957,649,jmeter_view_page_tree,404,,Confluence 1-1,false,32539,2,2,252,Alexeys-MacBook-Pro-2.local,0
1595593010250,5336,jmeter_view_page,200,"Number of samples in transaction : 13, number of failing samples : 0",Confluence 1-1,true,34576,2,2,0,Alexeys-MacBook-Pro-2.local,0
...............................

Файл содержит информацию о все запросах, которые делал Jmeter во время тестирования. Позже на основе этого файла будет построена сводная информация.

merged.yml/merged.json

Taurus может быть запущен с несколькими файлами конфигурации. Merged.yml/merged.json содержат сводную информацию из всех файлов конфигурации. В нашем случае мы запускаем Taurus с одним файлом конфигурации, поэтому содержимое файлов эквивалентно файлам effective.json/effective.yml.

modified_confluence.jmx

Это файл проекта JMeter, который был использован для запуска тестов из JMeter.

pytest.err/pytest.out

Файлы pytest. Pytest используется для тестов Selenium.

results.csv

Сводный файл результатов тетсриования. Файл содержит сводную информацию о выполненных тестах JMeter и Selenium:

Label,# Samples,Average,Median,90% Line,95% Line,99% Line,Min,Max,Error %
jmeter_login_and_view_dashboard,4,15996,12216,29332,31667,33536,5548,34003,25.0
jmeter_view_page,18,9950,7290,21767,25701,31178,1041,32547,11.0
jmeter_view_page_tree,16,3422,1001,10405,13115,19515,700,21115,100.0
jmeter_recently_viewed,2,5068,5068,8533,8966,9312,738,9399,0.0
jmeter_search_results,2,21082,21082,36756,38716,40283,1488,40675,50.0
jmeter_view_blog,2,21786,21786,30568,31666,32544,10807,32764,50.0
jmeter_view_dashboard,2,6070,6070,6424,6468,6503,5629,6512,0.0
jmeter_create_page_editor,2,6514,6514,7286,7382,7460,5549,7479,100.0
jmeter_create_page,2,12844,12844,17641,18240,18720,6848,18840,0.0
jmeter_open_editor,2,20412,20412,32523,34037,35248,5272,35551,100.0
jmeter_edit_page,2,45840,45840,76536,80373,83443,7470,84210,50.0
jmeter_comment,2,1176,1176,1293,1307,1319,1029,1322,0.0
jmeter_upload_attachment,2,3639,3639,3922,3958,3986,3285,3993,0.0
jmeter_create_blog_editor,1,7213,7213,7213,7213,7213,7213,7213,100.0
jmeter_create_blog,1,15801,15801,15801,15801,15801,15801,15801,0.0
jmeter_view_attachment,1,1000,1000,1000,1000,1000,1000,1000,0.0
jmeter_like_page,1,2020,2020,2020,2020,2020,2020,2020,0.0
selenium_login:open_login_page,17,1355,538,2404,5970,9466,342,10340,0.0
selenium_login:login_and_view_dashboard,17,1977,1660,2917,3929,7006,1027,7775,0.0
selenium_login,17,3456,2317,5472,10064,16635,1520,18278,0.0
selenium_view_page,17,2310,734,3046,8832,19338,599,21964,0.0
selenium_create_page:open_create_page_editor,17,5715,1623,7635,23837,50605,1341,57297,0.0
selenium_create_page:save_created_page,17,5462,3789,10068,17413,19791,3056,20385,0.0
selenium_create_page,17,12186,6244,23697,33888,66310,5331,74415,0.0
selenium_edit_page:open_create_page_editor,16,4595,3460,8714,13476,14733,2,15047,0.0
selenium_edit_page:save_edited_page,15,3365,2567,4053,6707,10897,1975,11945,0.0
selenium_edit_page,16,8029,6406,15966,16456,17122,2,17288,0.0
selenium_create_comment:write_comment,15,2522,2147,4070,4833,4875,1813,4886,0.0
selenium_create_comment:save_comment,15,1251,1146,1652,1669,1685,1120,1689,0.0
selenium_create_comment,16,4187,3982,6027,6661,7047,2,7143,0.0
selenium_view_blog,16,7152,4360,13916,16767,23327,2,24967,0.0
selenium_view_dashboard,16,5155,3646,5286,12210,27374,1,31165,0.0
selenium_log_out,16,5115,862,1096,18352,59433,1,69703,0.0

results_summary.log

Файл со сводной информацией о проведенном тесте.

Summary run status                                FAIL
                                             
Artifacts dir                                     2020-07-24_14-32-11                               
OS                                                macOS                                             
DC Apps Performance Toolkit version               3.0.1                                             
Application                                       confluence 7.4.1                                  
Dataset info                                      975425 pages                                      
Application nodes count                           2                                                 
Concurrency                                       2                                                 
Expected test run duration from yml file          2700 sec                                          
Actual test run duration                          793 sec                                           
Finished                                          (False, 'Actual test duration 793 sec < than expected test_duration 2700 sec.')
Compliant                                         (False, 'Test run duration 793 sec < than minimum test duration 2700 sec. Test run concurrency 2 < than minimum test concurrency 200.')
Success                                           (False, 'One or more actions have success rate < 95.0 %.')

Action                                            Success Rate                                      Status                                            
jmeter_comment                                    100.0                                             OK                                                
jmeter_create_blog                                100.0                                             OK                                                
jmeter_create_blog_editor                         0.0                                               Fail                                              
jmeter_create_page                                100.0                                             OK                                                
jmeter_create_page_editor                         0.0                                               Fail                                              
jmeter_edit_page                                  50.0                                              Fail                                              
jmeter_like_page                                  100.0                                             OK                                                
jmeter_login_and_view_dashboard                   75.0                                              Fail                                              
jmeter_open_editor                                0.0                                               Fail                                              
jmeter_recently_viewed                            100.0                                             OK                                                
jmeter_search_results                             50.0                                              Fail                                              
jmeter_upload_attachment                          100.0                                             OK                                                
jmeter_view_attachment                            100.0                                             OK                                                
jmeter_view_blog                                  50.0                                              Fail                                              
jmeter_view_dashboard                             100.0                                             OK                                                
jmeter_view_page                                  88.89                                             Fail                                              
jmeter_view_page_tree                             0.0                                               Fail                                              
selenium_a_login                                  100.0                                             OK                                                
selenium_create_comment                           93.75                                             Fail                                              
selenium_create_page                              94.12                                             Fail                                              
selenium_edit_page                                93.75                                             Fail                                              
selenium_view_blog                                93.75                                             Fail                                              
selenium_view_dashboard                           87.5                                              Fail                                              
selenium_view_page                                100.0                                             OK                                                
selenium_z_log_out                                93.75                                             Fail                                              

selenium.err

Этот файл содержит информацию обо всех ошибках, которые были зафиксированы Selenium webdriver:

Action: test_1_selenium_create_page, Error: webdriver = <selenium.webdriver.chrome.webdriver.WebDriver (session="a14471438bdc54e10792c212a59928c0")>
confluence_datasets = {'blog_id': '52215665', 'blogs': [['52377239', 'TES'], ['52215590', 'TES'], ['52215591', 'TES'], ['52540654', 'TES'], ...'], ['46780039', 'TES'], ['46777426', 'TES'], ['50383897', 'TES'], ['47037799', 'TES'], ['47037475', 'TES'], ...], ...}
confluence_screen_shots = None

    def test_1_selenium_create_page(webdriver, confluence_datasets, confluence_screen_shots):
>       modules.create_confluence_page(webdriver, confluence_datasets)

selenium_ui/confluence_ui.py:15: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
selenium_ui/confluence/modules.py:100: in create_confluence_page
    measure()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    @functools.wraps(func)
    def wrapper():
        if LOGIN_ACTION_NAME in interaction:
            globals.login_failed = False
        if globals.login_failed:
            pytest.skip(f"login is failed")
        start = time.time()
        error_msg = 'Success'
        full_exception = ''
        try:
            func()
            success = True
        except Exception:
            success = False
            # https://docs.python.org/2/library/sys.html#sys.exc_info
            exc_type, full_exception = sys.exc_info()[:2]
            error_msg = f"Failed measure: {interaction} - {exc_type.__name__}"
        end = time.time()
        timing = str(int((end - start) * 1000))
    
        with open(selenium_results_file, "a+") as jtl_file:
            timestamp = round(time.time() * 1000)
            jtl_file.write(f"{timestamp},{timing},{interaction},,{error_msg},,{success},0,0,0,0,,0\n")
    
        print(f"{timestamp},{timing},{interaction},{error_msg},{success}")
    
        if not success:
            if LOGIN_ACTION_NAME in interaction:
                globals.login_failed = True
>           raise Exception(error_msg, full_exception)
E           Exception: ('Failed measure: selenium_create_page - Exception', Exception('Failed measure: selenium_create_page:save_created_page - TimeoutException', TimeoutException("Error in wait_until: Timed out after 20 sec waiting for <selenium.webdriver.support.expected_conditions.visibility_of_element_located object at 0x10eb8f7f0>. \nLocator: ('id', 'title-text')", None, None)))

selenium_ui/conftest.py:118: Exception

selenium.jtl

Этот файл содержит иформацию о всех запросах, которые делал Selenium в ходе выполнения теста.

timeStamp,elapsed,label,responseCode,responseMessage,threadName,success,bytes,grpThreads,allThreads,Latency,Hostname,Connect
1595590485432,4877,selenium_login:open_login_page,,Success,,True,0,0,0,0,,0
1595590488565,2967,selenium_login:login_and_view_dashboard,,Success,,True,0,0,0,0,,0
1595590488565,8011,selenium_login,,Success,,True,0,0,0,0,,0
1595590494133,5549,selenium_view_page,,Success,,True,0,0,0,0,,0
1595590509607,15472,selenium_create_page:open_create_page_editor,,Success,,True,0,0,0,0,,0
1595590517792,5666,selenium_create_page:save_created_page,,Success,,True,0,0,0,0,,0
1595590517793,23658,selenium_create_page,,Success,,True,0,0,0,0,,0

После того, как тест завершился, Вы должны посмотреть файлы results.csv и results_summary.log files для того, чтобы понять насколько успешным был тест. Если Вы обнаружили, что в тесте были ошибки, то уже в этом случае можете начинать просматривать файлы с ошибками в тестах Selenium и JMeter.

Reports

Помимо отчетов в текстовом виде Вы можете получить и отчеты в виде графиков. Эти отчеты изначально нужны для сертификации своих приложений для Data Center.

Performance Report

Этот отчет позволяет Вам сравнить два теста. В рамках сертификации Вы должны указывать путь к тесту без Вашего приложения и к тесту с Вашим приложением. Пути указываются в файле performance_profile.yml:

# Defines which column from test runs is used for aggregated report. Default is "90% Line"
column_name: "90% Line"
runs:
  # fullPath should contain a full path to the directory with run results. E.g. /home/$USER/dc-app-performance-toolkit/jira/results/2019-08-06_17-41-08
  - runName: "without app"
    fullPath: "/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/noplugin"
  - runName: "with app"
    fullPath: "/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/4_nodes"

# Chart generation config
index_col: "Action"
title: "DCAPT Performance Testing"
image_height_px: 1000
image_width_px: 1200

Для генерации отчета Вы запускаете следующую команду:

python3 csv_chart_generator.py performance_profile.yml

Отчет содержит три файла: performance_profile_summary.log, performance_profile.csv, performance_profile.png

performance_profile_summary.log содержит сводную информацию о двух тестах:

cenario status: OK
========================================================================================================================
Run name: without app

Summary run status                                OK
                                             
Artifacts dir                                     2020-07-24_13-29-26                               
OS                                                macOS                                             
DC Apps Performance Toolkit version               3.0.1                                             
Application                                       confluence 7.4.1                                  
Dataset info                                      975407 pages                                      
Application nodes count                           4
Concurrency                                       200
Expected test run duration from yml file          2700 sec                                          
Actual test run duration                          2710 sec
Finished                                          (True, 'OK')
Compliant                                         (True, 'OK')
Success                                           (True, 'OK')

Action                                            Success Rate                                      Status                                            
jmeter_comment                                    100.0                                             OK                                                
jmeter_create_blog                                100.0                                             OK                                                
jmeter_create_blog_editor                         100.0                                             OK
jmeter_create_page                                100.0                                             OK                                                
jmeter_create_page_editor                         100.0                                             OK
jmeter_edit_page                                  100.0                                             OK                                                
jmeter_like_page                                  100.0                                             OK                                                
jmeter_login_and_view_dashboard                   100.0                                             OK
jmeter_open_editor                                100.0                                             OK
jmeter_recently_viewed                            100.0                                             OK                                                
jmeter_search_results                             100.0                                             OK
jmeter_upload_attachment                          100.0                                             OK                                                
jmeter_view_attachment                            100.0                                             OK                                                
jmeter_view_blog                                  100.0                                             OK                                                
jmeter_view_dashboard                             100.0                                             OK                                                
jmeter_view_page                                  100.0                                             OK
jmeter_view_page_tree                             100.0                                             OK
selenium_a_login                                  100.0                                             OK                                                
selenium_create_comment                           100.0                                             OK                                                
selenium_create_page                              100.0                                             OK                                                
selenium_edit_page                                100.0                                             OK
selenium_view_blog                                100.0                                             OK                                                
selenium_view_dashboard                           100.0                                             OK                                                
selenium_view_page                                100.0                                             OK                                                
selenium_z_log_out                                100.0                                             OK                                                

========================================================================================================================
Run name: with app

Summary run status                                OK
                                             
Artifacts dir                                     2020-07-24_13-29-26                               
OS                                                macOS                                             
DC Apps Performance Toolkit version               3.0.1                                             
Application                                       confluence 7.4.1                                  
Dataset info                                      975407 pages                                      
Application nodes count                           4
Concurrency                                       200
Expected test run duration from yml file          2700 sec                                          
Actual test run duration                          2710 sec
Finished                                          (True, 'OK')
Compliant                                         (True, 'OK')
Success                                           (True, 'OK')

Action                                            Success Rate                                      Status                                            
jmeter_comment                                    100.0                                             OK                                                
jmeter_create_blog                                100.0                                             OK                                                
jmeter_create_blog_editor                         100.0                                             OK
jmeter_create_page                                100.0                                             OK                                                
jmeter_create_page_editor                         100.0                                             OK
jmeter_edit_page                                  100.0                                             OK                                                
jmeter_like_page                                  100.0                                             OK                                                
jmeter_login_and_view_dashboard                   100.0                                             OK
jmeter_open_editor                                100.0                                             OK
jmeter_recently_viewed                            100.0                                             OK                                                
jmeter_search_results                             100.0                                             OK
jmeter_upload_attachment                          100.0                                             OK                                                
jmeter_view_attachment                            100.0                                             OK                                                
jmeter_view_blog                                  100.0                                             OK                                                
jmeter_view_dashboard                             100.0                                             OK                                                
jmeter_view_page                                  100.0                                             OK
jmeter_view_page_tree                             100.0                                             OK
selenium_a_login                                  100.0                                             OK                                                
selenium_create_comment                           100.0                                             OK                                                
selenium_create_page                              100.0                                             OK                                                
selenium_edit_page                                100.0                                             OK
selenium_view_blog                                100.0                                             OK                                                
selenium_view_dashboard                           100.0                                             OK                                                
selenium_view_page                                100.0                                             OK                                                
selenium_z_log_out                                100.0                                             OK                                                

========================================================================================================================

performance_profile.csv содержит сводную информацию о времени ответа запросов:

Action,without app,with app
jmeter_login_and_view_dashboard,92764,90969
jmeter_view_page,20174,21212
jmeter_view_page_tree,1027,1182
jmeter_recently_viewed,6537,6309
jmeter_search_results,20103,20091
jmeter_view_blog,20552,19301
jmeter_comment,1497,1581
jmeter_upload_attachment,3758,3895
jmeter_create_blog_editor,6201,6335
jmeter_create_blog,12925,11319
jmeter_view_attachment,1394,1281
jmeter_like_page,798,866
jmeter_view_dashboard,4971,4775
jmeter_create_page_editor,8401,8508
jmeter_create_page,8620,8801
jmeter_open_editor,4513,4541
jmeter_edit_page,9528,9225
selenium_login:open_login_page,703,790
selenium_login:login_and_view_dashboard,2115,2057
selenium_login,3007,3132
selenium_view_page,1065,1269
selenium_create_page:open_create_page_editor,9562,9378
selenium_create_page:save_created_page,5603,5202
selenium_create_page,14663,14980
selenium_edit_page:open_create_page_editor,4059,3950
selenium_edit_page:save_edited_page,2965,2870
selenium_edit_page,6393,6959
selenium_create_comment:write_comment,4763,4575
selenium_create_comment:save_comment,1378,1178
selenium_create_comment,6284,6698
selenium_view_blog,9723,9937
selenium_view_dashboard,5365,5007
selenium_log_out,1543,1110

performance_profile.png представляет данные о времени ответа запроса для тестов в виде графика:

Scale Report

Этот отчет позволяет Вам сравнить три теста. В рамках сертификации на Data Center, Вы указывает ссылки на тесты, которые проводились на 4-х нодах, 2-х нодах и 1-й ноде в файле scale_profile.yml.

# Defines which column from test runs is used for aggregated report. Default is "90% Line"
column_name: "90% Line"
runs:
  # fullPath should contain a full path to the directory with run results. E.g. /home/$USER/dc-app-performance-toolkit/jira/results/2019-08-06_18-41-08
  - runName: "Node 1"
    fullPath: "/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/1_node"
  - runName: "Node 2"
    fullPath: "/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/2_nodes"
  - runName: "Node 4"
    fullPath: "/Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/confluence/4_nodes"

# Chart generation configs
index_col: "Action"
title: "DCAPT Scale Testing"
image_height_px: 1000
image_width_px: 1200

В резултате получаются все те же три файла: scale_profile_summary.log, scale_profile.cvs и scale_profile.png.

На этом мы завершили цикл статей о тулките для нагрузочного тестирования dc-app-performance-toolkit от Atlassian. Теперь Вы обладаете достаточной информацией, чтобы успешно пользоваться этим тулкитом.

If you have found a spelling error, please, notify us by selecting that text and pressing Ctrl+Enter.

Leave a Reply

%d bloggers like this:

Spelling error report

The following text will be sent to our editors: