Jira, Confluence, Bitbucket performance testing Part 2

You can read Part 1 here.

In Part 2 we will talk about preparing test data for dc-app-performance-toolkit.

Prepare test data

There are two options to prepare data:

  • use backups prepared by Atlassian. This option can be used if you develop your own app and you want to test your app for performance.
  • use your own data. This option can be used if you have your own Jira, Confluence or Bitbucket instance which you modified with out of the box features or using scripts, apps or any other way of modification.

Atlassian backups

Atlassian provides backups for Jira, Confluence and Bitbucket which can be used for testing your instance.

Move to app-dc-performance-toolkit/app/util/jira if you test Atlassian Jira, app-dc-performance-toolkit/app/util/confluence if you test Atlassian Confluence or app-dc-performance-toolkit/app/util/bitbucket if you test Atlassian Bitbucket. In these folders you will find files which are needed for restoring Atlassian backups.

Here are the files:

  • index-sync.sh (for Jira and Confluence only) – this script searches atlassian-jira.log (atlassian-confluence.log) file for phrases like “indexes – 100%” just to make sure that indexing was done on Jira/Confluence. It makes sense to run this script after you restored Atlassian backup and ran reindex.
  • populate_db.sh – this script downloads postgres dump with data and recreates Jira/Confluence/Bitbucket database with the backup.
  • upload_attachments.sh – this script downloads attachments prepared by Atlassian and moves these attachments to the data folder.
  • index-snapshot.sh (Confluence only) – this script checks index snapshot generation.

All these scripts must be run on the same virtual machine where your Jira, Confluence or Bitbucket runs. These scripts define the version of your Jira/Confluence/Bitbucket and download the backup data and attachments for your version.

Well, what should I do if I do not use postgres and the populate_db.sh works only for Postgres?

In this case for Jira you can download the XML backup and restore it with the Jira restore backup feature. Here is the url for downloading:

https://centaurus-datasets.s3.amazonaws.com/jira/${Jira_Verson}/large/xml_backup.zip – supported versions are 8.0.3, 7.13.6, 8.5.0.

There are no xml backups for Confluence and Bitbucket. You need to use the populate_db.sh script.

Use your own data

if you have your own Jira, Confluence or Bitbucket instance, there is a good chance that you have your own data and you want to test Jira, Confluence or Bitbucket on your own dataset. It is possible. All you need to do is to make sure that all entities which will be used for testing by dc-app-performance-toolkit are available in your instance.

dc-app-performance-toolkit runs prepare-data.py script during the Prepare stage of Taurus lifecycle. prepare-data.py creates csv files which will be used later during testing.

In order to understand which data you need in your Jira/Confluence/Bitbucket to run tests without errors you need to explore the prepare-data.py files for the product which you intend to test.

Let’s explore these files.

Jira

You can find the prepare-data.py file for Jira in the dc-app-performance-toolkit/blob/master/app/util/data_preparation/jira/ folder.

The script selects data from Jira and create csv files with this data. This data will be used later for testing. Here is the list of the files:

  • issues.csv – contains issues.
  • jqls.csv – contains jql queries.
  • kanban-boards.scv – contains kanban boards.
  • project-keys.csv – contains project keys.
  • scrum-boards.csv- contains scrum boards.
  • users.scv – contains users.

We will not explore the whole prepare-data.py file, but just this function:

def __create_data_set(jira_api):
    dataset = dict()
    dataset[USERS] = __get_users(jira_api)
    software_project_keys = __get_software_project_keys(jira_api, PROJECTS_COUNT_LIMIT)
    dataset[PROJECT_KEYS] = software_project_keys
    dataset[ISSUES] = __get_issues(jira_api, software_project_keys)
    dataset[SCRUM_BOARDS] = __get_boards(jira_api, 'scrum')
    dataset[KANBAN_BOARDS] = __get_boards(jira_api, 'kanban')
    dataset[JQLS] = __generate_jqls(count=150)

    return dataset

As you can see this function selects data which are needed for all these files:

dataset[USERS] = __get_users(jira_api)

We select users from the Jira instance. The name of the users must start with the “performance_” prefix. We need as many users as the concurrency parameter in the jira.yml file. If no users are found or not enough users, then new users will be created in your Jira with “performance_” prefix and password: password.

software_project_keys = __get_software_project_keys(jira_api, PROJECTS_COUNT_LIMIT)
dataset[PROJECT_KEYS] = software_project_keys

We select project keys for software projects.

__get_issues(jira_api, software_project_keys)

We select not more than 8000 issues from software projects which are not in the Closed status.

dataset[SCRUM_BOARDS] = __get_boards(jira_api, 'scrum')

We select not more than 250 scrum boards.

dataset[KANBAN_BOARDS] = __get_boards(jira_api, 'kanban')

We select not more than 250 kanban boards.

dataset[JQLS] = __generate_jqls(count=150)

We generate jqls like ‘text ~ “abc*”‘.

It means that in order the tests worked you need to have a software project with issues, a scrum and a kanban board.

That is why I created Scrum Software Development and Kanban Software Development projects in my empty Jira and ran again:

bzt jira.yml

This time the script ran successfully. Here is the output:

18:15:24 INFO: Taurus CLI Tool v1.14.0
18:15:24 INFO: Starting with configs: ['jira.yml']
18:15:24 INFO: Configuring...
18:15:24 INFO: Artifacts dir: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/jira/2020-05-02_18-15-24
18:15:24 INFO: Preparing...
18:15:25 INFO: Starting shell command: python util/environment_checker.py
18:15:25 INFO: Starting shell command: python util/data_preparation/jira/prepare-data.py
18:15:26 WARNING: There is newer version of Taurus 1.14.2 available, consider upgrading. What's new: http://gettaurus.org/docs/Changelog/
18:15:28 INFO: Will install JMeter into /Users/alexm/.bzt/jmeter-taurus/5.2.1
18:15:28 INFO: Downloading: https://apache-mirror.rbc.ru/pub/apache//jmeter/binaries/apache-jmeter-5.2.1.zip
100% [===========================================================] Time: 0:00:04
18:15:33 INFO: Unzipping /var/folders/4r/w8hkd4w51x787tw4ynb29v2w0000gn/T/tmp9g5rzqs1.zip to /Users/alexm/.bzt/jmeter-taurus/5.2.1
18:15:50 INFO: Downloading jmeter-plugins-manager-1.3.jar from https://search.maven.org/remotecontent?filepath=kg/apc/jmeter-plugins-manager/1.3/jmeter-plugins-manager-1.3.jar
18:15:51 INFO: Downloading cmdrunner-2.2.jar from https://search.maven.org/remotecontent?filepath=kg/apc/cmdrunner/2.2/cmdrunner-2.2.jar
100% [===========================================================] Time: 0:00:01
18:15:52 INFO: Installing JMeter plugins: jpgc-casutg,jpgc-dummy,jpgc-ffw,jpgc-fifo,jpgc-functions,jpgc-json,jpgc-perfmon,jpgc-prmctl,jpgc-tst,bzm-parallel=0.4,bzm-random-csv=0.6,jpgc-casutg=2.5,jpgc-dummy=0.2,jpgc-ffw=2.0,jpgc-fifo=0.2,jpgc-functions=2.1,jpgc-json=2.6,jpgc-perfmon=2.1,jpgc-prmctl=0.4,jpgc-tst=2.4,jpgc-wsc=0.3,tilln-sshmon=1.0,jpgc-cmd=2.2,jpgc-synthesis=2.2
18:16:38 INFO: 1 obsolete CookieManagers are found and fixed
18:16:56 INFO: Installing ChromeDriver...
18:16:56 INFO: Will install ChromeDriver into /Users/alexm/.bzt/selenium-taurus/tools/chromedriver/80.0.3987.106
18:16:56 INFO: Downloading: https://chromedriver.storage.googleapis.com/80.0.3987.106/chromedriver_mac64.zip
100% [===========================================================] Time: 0:00:01
18:16:57 INFO: Unzipping /var/folders/4r/w8hkd4w51x787tw4ynb29v2w0000gn/T/tmpurpp_7tz.zip to /Users/alexm/.bzt/selenium-taurus/tools/chromedriver/80.0.3987.106
18:16:57 INFO: Installing GeckoDriver...
18:16:57 INFO: Will install GeckoDriver into /Users/alexm/.bzt/selenium-taurus/tools/geckodriver/0.23.0
18:16:57 INFO: Downloading: https://github.com/mozilla/geckodriver/releases/download/v0.23.0/geckodriver-v0.23.0-macos.tar.gz
100% [===========================================================] Time: 0:00:02
18:17:00 INFO: Untaring /var/folders/4r/w8hkd4w51x787tw4ynb29v2w0000gn/T/tmpiurqzzt4.zip to /Users/alexm/.bzt/selenium-taurus/tools/geckodriver/0.23.0
18:17:00 WARNING: You are using Python 3, make sure that your scripts are able to run in Python 3
18:17:00 INFO: Starting...
18:17:00 INFO: Waiting for results...
18:17:00 INFO: Waiting for finish...
18:17:00 ERROR: Console screen failure: ord() expected string of length 1, but int found
18:25:22 WARNING: Please wait for graceful shutdown...
18:25:22 INFO: Shutting down...
18:25:22 INFO: Starting shell command: python util/jmeter_post_check.py
18:25:22 INFO: Starting shell command: python util/jtl_convertor/jtls-to-csv.py kpi.jtl selenium.jtl
18:25:59 INFO: Post-processing...
18:25:59 INFO: Test duration: 0:08:22
18:25:59 INFO: Samples count: 3272, 0.06% failures
18:25:59 INFO: Average times: total 0.595, latency 0.000, connect 0.000
18:25:59 INFO: Percentiles:
┌───────────────┬───────────────┐
│ Percentile, % │ Resp. Time, s │
└───────────────┴───────────────┘
18:25:59 INFO: Request label stats:
┌───────────────────────────────────────┬────────┬─────────┬────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ label                                 │ status │    succ │ avg_rt │ error                                                                                                                                           │
├───────────────────────────────────────┼────────┼─────────┼────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ jmeter_browse_boards                  │   OK   │ 100.00% │  0.186 │                                                                                                                                                 │
│ jmeter_browse_projects                │   OK   │ 100.00% │  0.149 │                                                                                                                                                 │
│ jmeter_create_issue                   │   OK   │ 100.00% │  0.135 │                                                                                                                                                 │
│ jmeter_login_and_view_dashboard       │   OK   │ 100.00% │  0.497 │                                                                                                                                                 │
│ jmeter_open_comment                   │   OK   │ 100.00% │  0.210 │                                                                                                                                                 │
│ jmeter_open_editor                    │   OK   │ 100.00% │  0.287 │                                                                                                                                                 │
│ jmeter_open_quick_create              │   OK   │ 100.00% │  0.063 │                                                                                                                                                 │
│ jmeter_save_comment                   │   OK   │ 100.00% │  0.521 │                                                                                                                                                 │
│ jmeter_save_edit                      │   OK   │ 100.00% │  0.900 │                                                                                                                                                 │
│ jmeter_search_jql                     │   OK   │ 100.00% │  0.610 │                                                                                                                                                 │
│ jmeter_view_backlog                   │   OK   │ 100.00% │  0.387 │                                                                                                                                                 │
│ jmeter_view_dashboard                 │   OK   │ 100.00% │  0.267 │                                                                                                                                                 │
│ jmeter_view_issue                     │   OK   │ 100.00% │  0.593 │                                                                                                                                                 │
│ jmeter_view_kanban_board              │   OK   │ 100.00% │  0.344 │                                                                                                                                                 │
│ jmeter_view_project_summary           │   OK   │ 100.00% │  0.611 │                                                                                                                                                 │
│ jmeter_view_scrum_board               │   OK   │ 100.00% │  0.347 │                                                                                                                                                 │
│ selenium_a_login                      │   OK   │ 100.00% │ 28.460 │                                                                                                                                                 │
│ selenium_browse_boards_list           │   OK   │ 100.00% │  9.871 │                                                                                                                                                 │
│ selenium_browse_projects_list         │   OK   │ 100.00% │  6.376 │                                                                                                                                                 │
│ selenium_create_issue                 │   OK   │ 100.00% │  5.143 │                                                                                                                                                 │
│ selenium_edit_issue                   │   OK   │ 100.00% │ 16.305 │                                                                                                                                                 │
│ selenium_save_comment                 │   OK   │ 100.00% │ 19.909 │                                                                                                                                                 │
│ selenium_search_jql                   │   OK   │ 100.00% │ 32.249 │                                                                                                                                  │
│ selenium_view_backlog_for_scrum_board │   OK   │ 100.00% │ 14.223 │                                                                                                                                                 │
│ selenium_view_dashboard               │   OK   │ 100.00% │ 13.805 │                                                                                                                                                 │
│ selenium_view_issue                   │   OK   │ 100.00% │ 12.332 │                                                                                                                                                 │
│ selenium_view_kanban_board            │   OK   │ 100.00% │ 14.675 │                                                                                                                                                 │
│ selenium_view_project_summary         │   OK   │ 100.00% │ 12.710 │                                                                                                                                                 │
│ selenium_view_scrum_board             │   OK   │ 100.00% │ 14.893 │                                                                                                                                                 │
│ selenium_z_log_out                    │   OK   │ 100.00% │ 14.889 │                                                                                                                                                 │
└───────────────────────────────────────┴────────┴─────────┴────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
18:26:00 INFO: Writing JUnit XML report into: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/jira/2020-05-02_18-15-24/xunit.xml
18:26:00 INFO: Starting shell command: python util/analytics.py jira
18:26:06 INFO: Starting shell command: python util/cleanup_results_dir.py
18:26:06 INFO: Artifacts dir: /Users/alexm/PycharmProjects/easymigration/dc-app-performance-toolkit/app/results/jira/2020-05-02_18-15-24
18:26:06 INFO: Done performing with code: 0

You can see that Taurus ran prepare-data.py and created all csv files. Here are the files:

issues.csv

KAN-1,10023,KAN
KAN-2,10024,KAN
KAN-3,10025,KAN
KAN-4,10026,KAN
KAN-5,10027,KAN
...........

jqls.csv

text ~ "vqz*" order by key
text ~ "fpl*" order by key
text ~ "zjs*" order by key
text ~ "wph*" order by key
text ~ "xhx*" order by key
text ~ "pxt*" order by key
text ~ "deh*" order by key
text ~ "iri*" order by key
text ~ "jbr*" order by key
text ~ "snt*" order by key
...........

kanban-boards.csv

2

project_keys.csv

KAN
SCRUM

scrum-boards.csv

1

users.csv

performance_kqgdayqwfj,password
performance_zyqnwihaxr,password
performance_onfxfovior,password
performance_rrujzwrxlt,password
performance_agjbcxxeqy,password

Then Taurus installed JMeter and Chrome web driver, ran tests and produced results.

Next I will explain the prepare-data.py file for Confluence and Bitbucket.

Confluence

Now let’s have a look at the prepare-data.py for Confluence.

Here is the function for getting test data from Confluence:

def __create_data_set(rest_client, rpc_client):
    dataset = dict()
    dataset[USERS] = __get_users(rest_client, rpc_client, CONFLUENCE_SETTINGS.concurrency)
    dataset[PAGES] = __get_pages(rest_client, 5000)
    dataset[BLOGS] = __get_blogs(rest_client, 5000)
    return dataset
 dataset[USERS] = __get_users(rest_client, rpc_client, CONFLUENCE_SETTINGS.concurrency)

We create users for performance testing with the same logic as for Jira.

dataset[PAGES] = __get_pages(rest_client, 5000)

We select pages from Confluence with the following cql: “type=page and title !~ JMeter and title !~ Selenium and title !~ Home”.

dataset[BLOGS] = __get_blogs(rest_client, 5000)

We select blogs from Confluence with the following cql: “type=blogpost and title !~ Performance”.

It means that we need to have pages and blogs in Confluence to run tests.

Bitbucket

Let’s have a look at the prepare-data.py file for Bitbucket.

Here is the function to get test data for Bitbucket:

def __create_data_set(bitbucket_api):
    dataset = dict()
    dataset[USERS] = __get_users(bitbucket_api)
    dataset[PROJECTS] = __get_projects(bitbucket_api)
    dataset[REPOS] = __get_repos(bitbucket_api)
    dataset[PULL_REQUESTS] = __get_prs(bitbucket_api)
    return dataset
dataset[USERS] = __get_users(bitbucket_api)

We create Bitbucket users. This time users are created with the “dcapt-perf-user” prefix and the password of the user is the user name.

dataset[PROJECTS] = __get_projects(bitbucket_api)

we select Bitbucket projects.

dataset[REPOS] = __get_repos(bitbucket_api)

We select Bitbucket repositories.

dataset[PULL_REQUESTS] = __get_prs(bitbucket_api)

We select pull requests. The number of pull requests must be not less than the concurrency parameter in the bitbucket.yml.

It means in order to run tests on your Bitbucket instance you need to have at least one project and one repository and as many pull requests as the concurrency parameter in the bitbucket.yml file.

In this part I described how to prepare test data. Let’s sum it up.

There are two options to create test data:

  • use backups which were prepared by Atlassian
  • use your own data but you need to make sure that your data contain information which is selected by the prepare-data.py file

In the next part we will talk about test results and charts which are provided out of the box.

Part 3

If you have found a spelling error, please, notify us by selecting that text and pressing Ctrl+Enter.

Leave a Reply

%d bloggers like this:

Spelling error report

The following text will be sent to our editors: