The Dataware data type

Storga reports allow you to list sequentially, after filtering and sorting, a set of files, or to build the elementary graph corresponding to the sequence. On the other hand, as soon as one seeks to obtain a more sophisticated representation, for example in dimension 2, one will prefer to use the type 'Dataware'. We will also do this to avoid creating a new report (which consumes storage) just to display a pie chart for example.

The classic course of the process of presenting a summary element is:
1. collect the data
2.form them
The 'Dataware' type serves as a receptacle during the data collection part. It's a bit like the pivot tables in spreadsheets.
Then, for formatting, you can either use one of Storga's 'built in' representations (table, bar graph or pie chart), or browse through the dataware content again and use the vector drawing instructions.

In this article, we expose the data filling phase (and possible consultation).
The shaping phase is described in two other articles:

→ Setting up table and graph representations

→ Vector drawing instructions

The Dataware type is rich in functionality, however, basic usage only requires four instructions which are 'dataware' 'dw_bin' 'dw_category' and 'dw_add'. Here is an example of a program that populates the dataware 'd' field of a form from the 'sales_list' state of invoices. Dimension 1 (bin) is the month, in the form 'yyyy-mm', and dimension 2 (category) is the customer:

var Date from_date := ...
var Date to_date := ...
dataware d
  report @sales_list
    field Date invoice_date ; field Str client ; field Float tax_excluded
    if invoice_date>=from_date and invoice_date<=to_date
      var Str b := string:invoice_date 0 7 # Les 7 premiers caractères sont le mois sous la forme yyyy-mm      
      dw_bin b # on défini la clé sur la première dimension
      dw_category client # on défini la clé sur la deuxième dimension
      dw_add b client tax_excluded # on ajoute le montant de la facture dans la cellule correspondante

Let us now see in detail the features of the Dataware type:

General statement

You can either create a Dataware as a simple local variable of a program:

var Dataware d
dataware d
  ...

or fill in the content of a dataware type block which is part of the form. Suppose that at the block level we have chosen 'd' as the variable name for the dataware block, then the code to fill the content will be:

dataware d
  ...

You can reset the dataware at any time via the instruction, but the use of this instruction is rare because the dataware data is already initialized at the start of program execution:

dw_reset

Si un programme Storga doit utiliser deux datawares en même temps, on peut indiquer auquel on fait référence via l'instruction dw_use :

dw_use d
  ...

On pourra donc avoir un algorithme du genre :

var Dataware d1 d2
... # remplir d1
dataware d2 # déduire d2 par filtrage à partir du contenu de d1

    each_dw_bin b # pour toutes les ligne de d1
      var Float v := dw_value b "c" Float # on extrait une information de d1
      if ... # condition de filtrage
        var Str k := ... # on regroupe, en calculant k à partir de b
        dw_use d2
          dw_bin k # on rempli d2
          dw_add k "c" v

Declaration and use of dimension 1

Dimension 1 of a dataware is called 'bin' ('corbeille' in French). Each entry (recycle bin) is identified by a character string key, and creates the 'dw_bin' instruction:

dw_bin "A"

When declaring an entry, you can optionally associate one or more properties with it:

dw_bin "A" label "Première" order "001" color hsl 60 100 90 button "foo" link "http://www.fullpliant.org/" help "Message d'aide" align center fixed 2 group "abc"

The label allows you to specify the name of the row in a table view or a CSV export of the dataware.
The option 'order' allows to force the order of the entries, when the order of the keys is not the right one.
The 'color' option allows you to force the background color of the row in the table view.
The 'button' option associates a virtual button with the table view cell that indicates the name of the column, and the 'help' option allows you to specify the help message displayed when the mouse pointer passes over it. .
L'option 'link' permet de spécifier un lien web qui sera activé si l'on clic sur la cellule. Cette option ne peut pas être utilisée en même temps que l'option 'button'.
L'option 'align' specifie l'alignement horizontal en representation table, qui peut être 'left' 'center' ou 'right'. Cette option concerne 'dw_category', sauf si l'option 'horizontal_align' a été fournie au niveau des options générales du dataware.
L'option 'fixed' specifie le nombre de chiffres décimaux à afficher.
L'option 'group' permet de faire disparaître le trait horizontal entre deux lignes successives (ou le trait vertical entre deux colonnes successives) quand celles-ci appartiennent au même groupe.

The 'order' option accepts string values, but also integer, float or even date.

For the color, we can use several notations:

dw_bin "A" color rgb 255 128 128
dw_bin "A" color hsl 0 50 50
var Color c := color rgb 255 128 128
var Color c := color rgba 255 128 128
var Color c := color hsl 0 50 50
var Color c := color hsl 0 50 50 opacity 50
dw_bin "A" color c

At the level of the 'hsl' notation, 'h' means 'hue', and denotes the hue, which represents an angle on the chromatic circle, and therefore ranges from 0 to 360 (0 \u003d red, 60 \u003d yellow, 120 \u003d green , 240 \u003d blue). Then 's' denotes the saturation, which ranges from 0 to 100 (0 \u003d gray, 100 \u003d pure color). Finally, 'l' designates the brightness, which goes from 0 to 100 (0 \u003d black, 100 \u003d white)

Calling the 'dw_bin' instruction several times with the same key is not a problem: only one entry will be created.

You can remove an entry by providing its key to the 'dw_bin_undefine' instruction:

dw_bin_undefine "A"

You can browse the different entries, respecting the lexicographic order provided via the 'order' option or, failing that, that of the keys. The variable, here 'b' will successively contain the key associated with each entry:

each_dw_bin b
  ...

At the very beginning, before starting to record the entries, we can indicate that the instruction 'each_dw_bin', and the display, will have to perform the browse in reverse order:

dw_bin_reverse

The number of entries is obtained using the 'dw_bin_size' function:

var Int n := dw_bin_size

You can get the index corresponding to each key in the browse order. The key of the first entry is zero, as with arrays:

var Int i := dw_bin_index "A"

Or conversely, obtain the key corresponding to an index:

var Str k := dw_bin 3

In fact, the following program:

each_dw_bin b
  var Int i := dw_bin_index b
  ...

is equivalent to this second version:

for (var Int i) 0 dw_bin_size-1
  var Str b := dw_bin i
  ...

Finally, we can reread the label that was assigned to an entry (via the 'label' option):

var Str l := dw_bin_label "A"

When viewing dataware in table view, or exporting it as a CSV file, the first column provides the label, or failing that the key associated with each entry. The name of this column is specified via the 'dw_bin_column' instruction

dw_bin_column "Mois"

Declaration and use of dimension 2

It's the same as for dimension 1, but we use 'category' instead of 'bin'. For example :

dw_category "A" order "001"

Toutes les fonctions 'dw_category_undefine', 'each_dw_category', 'dw_category_reverse', 'dw_category_size', 'dw_category_index', 'dw_category', 'dw_category_label' sont utilisables, à l'exception de l'instruction 'dw_bin_column' qui n'a pas d'équivalent pour les catégories.

Fill in and access values

You can associate a numerical value with an entry:

dw_value "A" 12

where if we use both dimensions:

dw_value "A" "2020-03" 12

Often, dataware is used to total the values \u200b\u200bfrom a scan. For example, we browse the list of invoices, and we fill in the entries corresponding to the different months. We therefore prefer to use the 'dw_add' instruction:

var Str m := ...
var Float f := ...
dw_add m f

which is equivalent to:

var Float f := ...
dw_value m (shunt (dw_value m Float)<>undefined (dw_value m Float) 0)+f

You can also associate a character string value with an entry:

dw_value "A" "Hello"

or, if two dimensions are used:

dw_value "A" "2020-30" "Hello"

In practice, a value of character string type is only used to be read a little later by the program, or in the case of a representation of the dataware in the form of a table. On the other hand, it is useless in the case of a graphic representation.

When you save a value, whether it is numeric or text, whether the dimension is 1 or 2, and whether you use the 'dw_value' instruction or the 'dw_add' instruction, you can add a whole series of options to assign properties to the cell corresponding to this value:

dw_value "A" "Hello" color hsl 60 100 90 foreground hsl 0 50 50 button "foo" link "http://www.fullpliant.org/" help "Message d'aide"

The 'color' option specifies the background color of the cell in the case of a table-type representation, or of the bar in the case of a graphical representation in the form of bars.
The 'foreground' option specifies the color of the text, always in the context of a table-type representation.
The 'button' 'link' and 'help' options make it possible to make the cell clickable in the case of a table-type representation, and the clickable bar in the case of a graphical representation in the form of bars.

To read a value, we use:

var Float f := dw_value "A" Float

or, if two dimensions are used:

var Float f := dw_value "A" "2020-03" Float

You can also read a value of type string, using 'Str' instead of 'Float'.

Fonctions statistiques

La fonction 'dw_average' retourne la moyenne des valeurs d'une des catégories :

var Float a := dw_average "A"

et la fonction 'dw_standard_deviation' retourne l'écart-type de l'une des catégories :

var Float d := dw_standard_deviation "A"

La fonction 'dw_regression' calcul les paramètres de la droite de régression linéaire (méthode des moindres carrés) :

dw_regression "X" "Y" (var Float a) (var Float b)

Dans le cas où l'axe Y est l'une des catégories 'A', mais qu'il n'y a pas de catégorie de l'axe X parce-que les valeurs de la catégories 'A' sont implicitement associées à 0, 1, 2, etc, alors on peut utiliser la syntaxe suivante :

dw_regression "A" (var Float a) (var Float b)

Enfin, la fonction 'dw_correlation' retourne le niveau de corrélation entre deux catégories :

var Float c := dw_correlation "X" "Y"

ou

var Float c := dw_correlation "A"

Examples

Here is a new version of the example at the beginning, in which we chose to have a cool month label, in the form 'Jan 2020' instead of '2020-01':

var Date from_date := ...
var Date to_date := ...
dataware d
  dw_bin_column "Mois" # ne sert que pour un éventuel export CSV
  report @sales_list
    field Date invoice_date ; field Str client ; field Float tax_excluded
    if invoice_date>=from_date and invoice_date<=to_date
      var Str b := string:invoice_date 0 7 # Les 7 premiers caractères sont le mois sous la forme yyyy-mm
      var Str l := ("   JanFevMarAvrMaiJunJuiAoûSepOctNovDec" invoice_date:month*3 3)+" "+(string invoice_date:year) # calcul du libellé sympa
      dw_bin b label l # on défini la clé sur la première dimension
      dw_category client # on défini la clé sur la deuxième dimension
      dw_add b client tax_excluded # on ajoute le montant de la facture dans la cellule correspondante

Lastly, details that the 'Dataware' type can be seen as a more sophisticated version, and possibly in dimension 2, of the (Index Str Float) and (Index Str Str) types. To illustrate this, consider the following code that exposes the basic functions of using an index:

var (Index Str Str) idx
idx "A" := "abc"
var Str s := idx "B"
var Str t := ""
each v idx getkey k
  t += " "+k+":"+v

here is now its equivalent with the Dataware type:

var Dataware d
dataware d # on utilise le Dataware 'd', et donc le 'd' ne sera pas précisé dans les instructions suivantes
  dw_bin "A" # avec le type datware, il faut créer explicitement l'entrée, avant ou après l'avoir remplie
  dw_value "A" "abc"
  var Str s := dw_value "B" Str
  var Str t := ""
  each_dw_bin b
    t += " "+b+" "+(dw_value b Str)