Load data in memory - work with arrays in RPE

A lot of times when building a particular report (document) we are forced to query the same collection of data multiple times because we want to have different representations of that collection in the document.

For example, in one document, we want to have the first section with all Effective use cases and at the end of the document, we want to have the list of all use cases that are still in Draft state.

App Versions Used:

  • DNG version: 6.0.6.1
  • RPE/RRDG version: 6.0.6.1

Challenge: DNG Reportable API is quite limited

The biggest challenge in DNG Reportable API is that there is not always the possibility to filter a particular collection of artifacts based on some attribute or group of attributes. That’s why we need to perform the filtering on the RPE side.

Link to DNG Reportable API: https://jazz.net/wiki/bin/view/Main/DNGReportableRestAPI

 

1. The traditional way - querying data multiple times

The common approach to build such a document will be to query the same collection multiple times and to implement a different filter each time. As I’ve mentioned earlier, the filter will have to be implemented in RPE as a script expression because DNG Reportable API does not provide the ?fields= filtering param;

Here is how the Use Case looks like in DNG (DOORS Next Generation). The custom attribute that we need to use for filtering UCs is State (UC)

Fig 1 - Use Case artifact in DNG

Build the first section (Effective UCs)

What we need to do first in RPE is to query the collection of UCs and first to display only those UCs that are effective ( State(UC)==” Effective” ).

Fig 2 - Query UC artifacts

For each UC artifact, we need to get the custom attribute State (UC) and to check if the value of this attribute is “Effective”.

Fig 3 - Use javascript to determine if a UC is in an Effective state

The last thing that we need to do is to use a condition and to check if the UC is Effective or not. For this we’ll use the value saved in the isUCEffective variable (boolean value);

Fig 4 - RPE condition to filter only the Effective use cases

In the selected container we will display all the information that is requested in the report for that UC. We know that because of the condition that is placed on the container, only the UC with State(UC)==” Effective” will be processed.

Build the second section (Draft UCs)

To build the second section of the document with all the UCs that are in State (UC)==” Draft” all we need to do is replicate the same template structure, to query again the collection but in this case to build a different condition.

Fig 5 - Use javascript to determine if a UC is in Draft state

The second section in the RPE template will look like this:

Fig 6 - Iterate Draft UCs

This way, we can obtain 2 different representations of the same collection of artifacts by iterating twice the same data but using a different filter each time.

Performance

To be able to determine the performance of this report, we’ll simulate a real-case scenario:


Number of artifacts in collection: 500 UCs

Number of requests to server: 500x2=1000 (1 req for each UC)

Avg req time: 0.4s

Total execution time: 1000x0.4s =400s

Note: Now think even further, and imagine that you need to add another section in the document with all UCs that are under review and so on. Each new iteration will add an extra 200s to the final generation time.


1. Using arrays - load data in memory and process multiple times

There is anything that we can do to decrease the execution time in this case? This is what I’ll try to solve in the second part of this article.

As we know RPE is using javascript to build the logic inside the template (filters, conditions, text processing, etc), but can be also used to store the data in memory for later processing. I’ll use this second scenario, to request only once all UCs from the server and to store them in memory (in one big array). Then I’ll iterate internally the UC array and filter the UCs based on the State (UC) attribute. 

Instead of requesting the same collection of UCs multiple times from the server (bad performance because of the slow request time from the server), I’ll read the UCs once and I’ll process the collection from memory multiple times.

Load UCs collection in memory

The first part of the RPE template will be identical to what we've built in the first scenario. We need to iterate all UCs, but this time instead of filtering them we are saving all UCs in an internal array.

Note: Each UC object will have to contain all the information that will be displayed in the final document. Because we need to filter the collection based on the State (UC) attribute, we’ll have to add this attribute also.

 So in the end we are looking to save for each UC the title, content, and state attributes.

To store the UCs we need to define an internal variable in RPE that will act like an array.

Fig 7 - Define _ucArray internal variable

Before starting to iterate through all UCs we’ll have to initialize our array.

Fig 8 - Array initialization

Now the _ucArray is initialized, so we can store UC objects in it. When we are iterating the UCs, for each UC artifact we are creating a UC object that is pushed into the array. At the end of the iteration, our array will contain all UCs from the collection.

Fig 9 - Create a UC object and push it to the array

Iterate arrays

Now to recreate the Effective and Draft UC sections will have to iterate the _ucArray twice and use the state property to filter the content.

Fig 10 - Iteration element in the RPE Palette

To iterate an array of elements, RPE provides a special element called Iteration (Palette->Data->Iteration). This Iteration element is acting like a DO/WHILE block, where we’ll have a re-iteration as long as the condition is still true.

To iterate our _ucArray, we insert the Iteration element and we check in the condition that the number of processed elements is less or equal to the array size. For this, we are using an internal variable that acts like a counter (to count the processed elements). After each iteration, the counter is increased by +1.

Fig 11 - DO/WHILE condition

All we need to do now is to filter the UCs based on the ucState attribute.

Fig 12 - Using UC object property in a condition

To implement the other sections, we need to reiterate the _ucArray (which is load in memory) and to change the condition. This way no other requests are done to the server and the execution time will decrease considerably.

Performance

Because the UC collection was loaded in memory we have a reduced number of requests performed to the server


Number of artifacts in collection: 500 UCs

Number of requests to server: 500=500 (1 req for each UC)

Avg req time: 0.4s

Extra javascript processing time: 5s

Total execution time: 500x0.4s +5s =205s


 

Image

Softacus AG

Löwenstrasse 20
8001 Zürich
Switzerland
E-Mail: info@softacus.com
Tel.: +41 43 5087081
Fax: +41 43 344 6075 

VAT: CHE-108.817.809 MWST
D-U-N-S® Number 486800618

Image

Softacus GmbH

Westendstrasse 28
60325 Frankfurt am Main
Germany
E-Mail: info@softacus.com
Tel.: +49 69 34876544
Fax: +49 69 5830 35709

VAT: DE301903892
D-U-N-S® Number 313482703

Image

Softacus s.r.o.

Křídlovická 351/47A
603 00 Brno
Czech Republic
E-Mail: info@softacus.com
Tel.: +420 530333482
Fax: +41 43 344 6075

VAT: CZ07286333
D-U-N-S® Number 496165108

Image

Softacus s.r.o.

Tatranské nám. 3
058 01 Poprad
Slovakia
E-Mail: info@softacus.com
Tel: +421 911 083 612
Fax: +41 43 344 6075

VAT: SK53507070
D-U-N-S® Number  2121388148