Page Comparison

What is the easiest way to analyse multiple data sources?

How can I rename a variable?

To rename a variable:

1. Open your project and click the Data sources icon in the sidebar.
2. Open the data source file that contains the variable and click Edit.
3. Rename the variable in the Variable definition section.
4. Click Load to update the variable name used throughout the project.

Where can I learn more about Lisp programming?

Lisp is an established programming language that is simple to learn. To learn the basics, you can visit LISP Tutorial or LISP for Beginners. For help writing specific expressions for your project, contact 70457104.

How can I share my DATAmaestro project with colleagues?

This functionality is restricted to platform administrators. Please contact your local administrator or Pepite support team with authorisation to create a folder.

• To navigate between folders:
  • From within a project: click the DATAmaestro logo to return to the home page.
  • From the home page, select the “eject” button at the top of the list of projects to view all available folders.  
• To move tasks/projects to another folder:
  • This functionality is restricted to platform administrators. Please contact your local administrator or Pepite support team with authorisation to change a folder.

There are two options to create a new project:

• In the main DmA page, enter the New project name in the Create new project field.
• In the project main menu select Project and then, New Project. The page is redirected to the main DmA page. Enter the new project name in the Create new project field.

There are two options to open a project:

• In the main DmA page search a project in an existing folders in the Open an existing project.
• In a project, select Project from the top menu bar and then, Open Project. The page is redirected to the main DmA page.

In the project main menu select Project and enter Make project copy. The new project’s name can be edited (optional), then click Ok.

In the project main menu select Project and then enter Delete project.

All actions are live within DATAmaestro, however there are several options:

• The standard browser forward/back buttons allow navigation between saved tasks.
• While editing a task, before clicking save/view/run, the user may click back or click away from the editor screen to avoid saving changes.
• Contact the Pepite support team to recover a backup.

You can associate a color with a symbol and customize the color to be used for a Symbol. Otherwise, it is automatically selected by the browser.

• On the top menu, in DATAmaestro Analytics project, click on “Project” then “Preferences”.
• Click on “Add Entry”.
• Enter “Symbol” name. Note: Be careful not to type a space (Enter) just after the symbol name).
• Click on color bar to choose a color (Basic or Custom colors).
• Click “Save”.

DmA accepts all Comma-Separated Values (.csv) file formats.

From within Excel, select Save As and from the list of FIle format options select “CSV UTF-8 (Comma Delimited) (.csv)” (recommended) or “Comma Separated Values (.csv)”

There are several options to upload a file:

• To upload data directly into DATAmaestro Analytics (data directly ready for analysis), in the DmA main menu select the Data menu and then select Upload file. For more information, refer to the DmA User Guide.
• To upload data into DATAmaestro Lake (for merging data or data aggregation), refer to DmL User Guide Import.
• Using an Excel plugin. More information please contact Pepite support team. 

Yes, it is possible to create a report. In the main menu, select Explore and choose Standard Report. In the editor you can create a Trend and several scatter plots. For more information check DmA User’s Guide - Standard Report .

Edit the data source, locate the date/time variable and in the Units column enter one of the following according to the temporal units:

• excelTime
• millisUnixTime
• unixTime

• You can do this using the Trends option in Explore menu. Keep in mind that the X axis must be a time related variable available in your database (typically, a numerical timestamp variable or an index variable).
• The Trends feature allows multiple variables on the first or second Y axes.

The type of model depends on the analytics objective:

• Supervised or unsupervised learning (Prediction, exploration, monitoring)
• Type of input parameters (numerical or symbolic)
• For supervised learning, the type of output function to predict (regression or classification)

For more information, please contact the Pepite support team.

Several tools are available in DATAmaestro that may help:

• Dendrogram tool (via the Explore menu): This tool will produce both the Dendrogram view (a hierarchical clustering tree view) and the Correlation matrix (a table giving all the correlation factor values, one for each pair of variables). Note that this Dendrogram tool only works for numerical variables. With a specific KPI variable in mind, the Dendrogram outputs will teach you which variables seem to be linked to your KPI (that is, the variables that may influence it). Tip: Always confirm/infirm the physical reason of the highest correlation factor values obtained by drawing the Scatter Plot graphs for the corresponding pairs of variables. For more information, refer to the DmA User’s Guide - Dendrograms .
• Extra trees tool (found in the Models menu) is an interesting alternative. It produces a Pareto graph of the most significant variables regarding your KPI or output variable. For more information, refer to the DmA User’s Guide Models .

DATAmaestro proposes two interesting tools to help do this verification: PCA and K-Means.

• The Principal components analysis tool (reached via the Transform menu) highlights the variability present in your database. Creating 2 components and drawing them via a Scatter Plot graph is an easy way to confirm the homogeneous nature of your records (records typically form an unique quite compact cloud of points in the Scatter Plot) or, conversely, the heterogeneous nature of your records (records form a quite extended cloud of points and/or form distinct clouds of points). For more information, refer to the DmA User’s Guide - Transform.
• The K-Means tool (in the Models menu) tries to split your database into several (2, 3 or more) groups of homogeneous points named clusters. For more information, refer to the DmA User’s Guide - Clustering models .
• Note that both PCA and K-Means only work for numerical variables.

Users can edit the tree, force decisions to node or vice versa.

• Export the function and adapt the rules based on constraints.
• Create a record set to remove data not satisfying constraints.
• Remember this Decision Tree is based on historical data so only covers events that have really occurred, therefore, if constraints always existed, the data should not differ from those constraints.

Yes, there is an advanced feature for Admin users to calculate “Computed tags”. For more information, contact your system administrator or the Pepite Support team.

What does “View data” do?

The View tab gives access to 3 features that are handy to quickly explore the data in DmLake :

• Trends - allows the user to visualise trends of previously.
• Raw data - allows the user to view the recorded values of up to 16 different tags. The values displayed corresponding to the lowest frequency of the selected.
• Statistics - provides useful statistical information about the data stored in the DmLake, such as the time period a tag has been recorded, the number of recorded values, the number of missing values, mean, min, max, etc.

• Pepite account (default): account created by Pepite unique to DATAmaestro. Passwords are managed by Pepite.
• Google/Microsoft: account based on an existing email account. The user must be logged into Google/Microsoft in order to access DATAmaestro via this authentication method. Passwords are managed by Google/Microsoft.

How do I create a new account?

• Pepite account: From the login page select Pepite Account, enter your username and existing password, then below select “Change Password”. Two fields will appear: New Password and Repeat New Password. Enter your new password in both fields and select login. You have now updated your password.
• Google account: Please follow the standard process to update your Google Account or Gmail password. Once completed, login to Gmail or Google Account with this new password and you’ll be able to login to DATAmaestro.    
• Microsoft account: Please follow the standard process to update your Microsoft password. Once completed, login to Microsoft Account with this new password and you’ll be able to login to DATAmaestro.    

Please contact the Pepite Support team.

• From within Excel, select Save As and from the list of file format options select “CSV UTF-8 (Comma Delimited) (.csv)” (recommended) or “Comma Separated Values (.csv)”
• It is recommended to remove all formula from your Excel sheet before saving as a CSV file.

How do I upload data files to DmLake?

Once your file has been saved in CSV format, connect to the DmLake using your DATAmaestro account (Pepite, Google or Microsoft Account). Then go to the Import menu. For more information, see Import data in the DmLake User's Guide Import.

In what format does my date column need to be?

The following date formats are currently supported:

• Excel time
• Excel time for Mac
• Unix (s)
• Unix time (ms)
• Text time (eg. 10/12/2017 10:25)

Note : if you use “Excel time” or “Excel time for mac”, it is recommended to ensure sufficient decimal places are displayed in the cells before savings as a .csv file (12 decimal places recommended) so that the precise and correct time will be uploaded into the DmLake.

DATAmaestro Lake is a temporal database which allows data to be consolidated based on date and time. To consolidate data when each file corresponds to a different month, upload the data in chronological order into the same DM Lake folder, ensuring that the variable names match across files. DmLake will ensure that the data from each month is merged into one complete dataset. Refer to the DmLake User Guide for more information on uploading data.

• File is saved as a Comma Separated Values (CSV) file
• Check your date format - refer to “In what format does my date column need to be?”
• Check chronological order - ensure data is ordered in chronological order
• Activate “skip existing values” in case you could not upload data in chronological order

What is the charset?

Charset or Character Set or Character Encoding indicates to the web browser which text format to use, in order to display an HTML page correctly.

The delimiter is the symbol used to separate columns in a .csv file. Generally, a comma is used, however other common delimiters include semicolons, forward-slash, etc.

• Europe/Brussels is an example of a Region time zone. This format should be used when your data comes from a system that shows the local clock time, including daylight savings changes.
• +02:00 is an example of a Fixed Offset. This format should be used when data comes from a system not taking into account daylight savings changes, not displaying the wall-clock time.
• NB: These options are equivalent for time zones where there is no daylight savings time changes.

• Append mode will attach new data to the folder (new rows or new columns) without replacing previously uploaded information.
• Reset will replace existing information (for any given tag) as well as entering new data.

Example: if you have already uploaded your production for January 2016, then decide to upload production data for all of 2016. The reset mode will first delete the January 2016 data and then upload all production data from 2016.

A DM Lake folder is equivalent to a folder on your computer. To keep things organised, it is recommended to upload your data into directories based on:

1/ Location (Site, Country, Process Unit, Equipment)

2/ Data source (Historian, LIMS, etc.)

3/ Data type (Manual upload, DmCollector automatic data uploads)

A “Base folder” is an existing “folder” within DmLake. A “New folder” simply indicates sub-folder within that “base folder” location.

On the bottom-left, select the “+ Add trends button” to select variables. For more information refer to the User Guide.

The sampling method defines the aggregation (if any) to perform on the data before extracting. The table below outlines each method type:

For more information, please check Lake - Export. 

• All the data can be uploaded to DmL into the same or different directories.
• Then the DmL export function can be used to extract data with different frequencies and from different sources for analysis.

Please contact the Pepite support team with authorisation from the folder owner.

• This functionality is restricted to platform administrators. Please contact your local administrator or Pepite support team with authorisation from the folder owner to delete data.
• If you are an administrator, delete data via the Admin -> Tag management menu and refer to the DmL User's Guide Tag Management for more information.

This functionality is restricted. Please contact the Pepite support team with authorisation from the folder owner to move variables.

• This functionality is restricted to platform administrators. Please contact your local administrator or the Pepite support team.
• If you are an administrator, create recurring exports via the Admin -> Exports to DATAmaestro Analytics menu and refer to the /wiki/spaces/DATMA/pages/70660093 for more information.

Several options are available to export to third party platforms (Excel, Python, R, etc.) Please contact the Pepite support team for more information.

The download button will download a .dmff file of your selected data and date ranges on your local computer drive.

The period, also know as sampling rate, depends on the frequency of your selected variables. If your selected variables have daily values, a period less than one day (1d) will not be useful. The period also depends on the dynamics and inertia of the process to analyse. If your selected variables are measured every second but the process changes occur slowly over several hours, a period of seconds or minutes will not be appropriate. Select the period that best suits your variables and process dynamics.   

There are two options to select the period:

• Enter the period value in milliseconds (eg. 5 minutes = 300000)
• Enter the short-hand format (eg. 5 minutes = 5m) 
• Refer to the DmL User's Guide for more information.