Matlab Connector (VisploreMatlab v1.4.0)




Overview and Use-Cases


VisploreMatlab is a Matlab toolbox to integrate Visplore cockpits into existing Matlab workflows. It offers a bidirectional link between the Matlab workspace and the interactive visual exploration features of Visplore. This supports multiple use-cases:

Installation


Requirements:

Installation:

VisploreMatlab comes as a toolbox, shipped with the Visplore installation. There are two ways of installing it:
  1. From a running Visplore instance: in the startup dialog of Visplore, choose the "Connect to Matlab" option, and select the "Install VisploreMatlab integration package" checkmark. When confirming the dialog, a code snippet is produced that, when executed in Matlab, installs the toolbox and immediately connects Matlab with that Visplore instance.


  2. Alternatively, you can install the toolbox via a double click in Matlab after navigating to the VisploreMatlab folder of your Visplore installation, or double clicking it in your file explorer and choosing the right Matlab version.

  3. Note: After installing the toolbox, Matlab requires a restart for the toolbox to become operational.

    Step by Step Guide


    Once installed, this section describes an example workflow to demonstrate its basic usage.

    You can open the demo file "visplore_demo.m" shipped with Visplore in the VisploreMatlab directory of your Visplore installation, and try out all the examples there.

    Alternatively, you can type the commands in your Matlab environment as described in the following.

    1) Connect Matlab and Visplore


    There are two ways to connect Matlab and Visplore:

    a) Start a new Visplore instance from Matlab and connect to it, by typing the following:

    v = visplore()

    This opens a new Visplore instance, and stores a handle to it in a variable "v". Any number of Visplore instances can be created and used at the same time. The distinct handles are used to communicate with any particular instance.
    Note: This way of starting Visplore is ideal when Visplore is installed via Installer locally on the same machine as Matlab . In other cases, refer to option (b), described in the following.

    b) Connecting to an already running Visplore instance (locally, or on another machine):

    This alternative is to start a new Visplore instance by hand first (e.g. via Start menu or desktop icon). In the welcome dialog of Visplore, choose the option "Connect to Matlab". Confirming the dialog copies a code snippet to the clipboard, which you can paste and run in Matlab. It looks like this:

    v = visplore('connect', true, 'ip', '127.0.0.1', 'port', 33113)

    After pasting and running this snippet in Matlab, the variable "v" is a handle to the existing Visplore instance you connected to. The tools communicate via TCP network protocol, which is why IP and port are specified as parameters here . Please leave these parameters as they are produced by Visplore. The Visplore instance waits for incoming connections on that port, and the "visplore('connect',..)" call establishes the connection.

    2) Prepare a dataframe in Matlab


    A typical workflow starts with a data frame that is either already available in the Matlab workspace, or loaded, generated, or accessed at this point. In our example, we use a timetable, loaded from an open photovoltaic and weather dataset shipped with Visplore as csv file:

    tt = visplore.load_demo_data('data', 'openpv_sampled');


    3) Send dataframe to Visplore


    We can now send this (or any other) data table to Visplore:
    v.send_data(tt)

    To see if your data successfully arrived at the Visplore instance, you can take a look at the statusbar at the bottom of your Visplore instance, and see if it shows any received records:

     

    If something went wrong, it still shows:
     

    4) Start a cockpit to explore the data


    Once the data is available in Visplore, we can start a “Cockpit”, i.e., an entry point for a certain task or analysis question.
    This can be done in one of two ways: either by a double click on the respective cockpit icon in Visplore, for example “Trends and Distributions”:


    Alternatively, the cockpit can be started from Matlab via the following command:
    v.start_cockpit("Trends and Distributions")
    As the parameter, you can simply use the name of the cockpit as shown in the user interface.


    5) Interactively select data in Visplore, and make this information available in Matlab


    A key feature in Visplore is the selection of data records using interactive selection mechanisms (selection rectangles, Lasso brushes, etc.). This allows the user to select interesting subsets, like outliers, patterns, clusters, or training data for modeling interactively in Visplore. These subsets can be made available in Matlab for downstream processing.

    For example, you can drag an interval or rectangle with the left mouse in a “Time Series View” or “Histogram”, to select some data records, and then type the following command in Matlab:
    boolean_mask = v.get_selection();
    This returns a binary mask that states for every row of the data table, whether it is currently selected (=true) or not (=false). This can be used for performing tasks with these records in Matlab.


    6) Label selected data in Visplore, and get labels in Matlab


    After selecting records in Visplore like in the previous step, the selected records can be labeled by clicking "create" in the "Focus" bar then selecting "Label" (see Visplore documentation). This creates a named condition, that can be retrieved as a binary mask from Matlab using the following command. Say you selected outliers, and labeled them as a condition "Outliers", retrieve them with:
    outliers = v.get_query("Outliers");
    num_outliers = sum(outliers) %optionally print how many records are selected
    plot(outliers) %optionally plot them


    7) Clean data values in Visplore, and retrieve the cleaned data table as data frame


    Visplore allows editing data records interactively, such as imputing outliers or missing values by interpolating neighbour values (see Visplore documentation).
    In particular after peforming edits, but basically at any time, you can retrieve the current Visplore data table using:
    new_table = v.get_data()
    This returns a new Matlab table, holding the data as received from Visplore. Optionally, you can limit the received rows to the selected data records by calling v.get_data('LimitToFocus', true).

    8) Get data from a Visplore visualization as data frame


    Many table-like visualizations support retrieving the displayed data as new pandas data frame. Examples are statistical tables, bar charts, heat maps, and raw value tables.
    my_statistics = v.get_view_data("Statistics")
    Note:The parameter is the displayed title of the visualization. Only views that are currently visible in the current cockpit can be retrieved. Not all view types can be retrieved, see v.get_view_properties().

    8) Get names of selected data attributes in Visplore, or set the selection of data attributes


    When you interactively select data attributes in Visplore (e.g., in the "Statistics" view), retrieve the names of the selected attributes with:
    v.get_selected_columns() #returns names as strings
    Conversely, you can programmatically select data attributes in Visplore using set_selected_columns(), like this:
    v.set_selected_columns({"A_Phase_Voltage_FlowerVillage_PV", "A_Phase_Voltage_Happyville_PV"})
    Note: You may need to adapt the column names of the example, if you load other data, of course.


    9) Compute new data columns in Matlab, and make them available in Visplore


    In this last example, a new data column is created in Matlab by multiplying an existing column that is selected in Visplore by 2. Use the send_data command with the "merge" parameter, to add the new column to the existing Visplore table:
    selected_columns = v.get_selected_columns();
    first_selected_column = selected_columns{1};

    subtable = tt(:,first_selected_column); %new timetable that contains just the new column, and matching time keys

    subtable.(first_selected_column) = subtable.(first_selected_column) * 2; %alter the values
    subtable.Properties.VariableNames = strcat(subtable.Properties.VariableNames,'_times_two'); %rename the new column

    v.send_data(subtable, 'mode', 'merge') % and send back to visplore (by merging)
    Note: This addition of a new column in Visplore does not immediately show effect. At the moment, the cockpit needs to be restarted (see Visplore documentation) for the new column to show up.

    But: When you update the values of an already displayed Visplore variable, e.g., running the snippet again with a different rolling window parameter, the values update immediately, allowing for efficient visual tweaking of parameters:
    %% Tune parameters and immediately replace values in Visplore
    new_varname = subtable.Properties.VariableNames{1}
    subtable.(new_varname) = subtable.(new_varname) * 10; %alter the values
    v.send_data(subtable, 'mode', 'merge') % and send back to visplore (by merging)



    API Documentation


    close(...)

       Closes the connected instance of Visplore
       Usage:
           close()
       Example:
           % v is a visplore handle received by v = visplore():
           %
           v.close()

    get_classification(...)

        Returns the classification(= categorical column) for the given name, if existing, as data frame.
        Usage:
            get_classifcation(name)
        Input parameters:
            name - the name of the classification as string
        Return type:
            array
        Example:
            % v is a visplore handle received by v = visplore():
            % Assumes, that a classification column called 'Classification Nr. 1' has been created in Visplore.
            %
            c = v.get_classification('Classification Nr. 1')
            % returns pandas dataframe with the length of the Visplore table, and
            % the classification as categorical column.

    export_story(...)

       Exports the visplore story to a pdf file.
       Usage:
            export_story_to_pdf(filepath)
       Input parameters:
    	filepath (mandatory, string): full path of image to create, including the file extension
    	logopath (optional, string): full path of image to use as logo on each page (incl. extension)
       Example:
            % v is a visplore handle received by v = visplore():
            % Assumes, that a story has been created in Visplore.
            %
            v.export_story_to_pdf("C:/Users/vs/Documents/Images/matlab.png")

    get_columns(...)

       Returns the names of the columns in the current visplore table
       Usage:
           get_columns()
       Return type:
           cell array
       Example:
           % v is a visplore handle received by v = visplore():
           %
           c = v.get_columns()
           % returns for example {'A_Phase_Voltage_BrightCounty_PV', 'A_Phase_Voltage_Cloudington_PV', ...}

    get_condition(...)

       Returns a list of boolean values, stating for each record of the Visplore table
       if it is part of the named condition (true), or not (false).
       Usage:
           get_condition(name)
       Input parameters:
           name (mandatory, string): the name of a condition whose membership is returned
       Return type:
           logical array
       Examples:
           % v is a visplore handle received by v = visplore():
           %
           s = v.get_condition('Outliers')
           % returns for example [false; true; false; true], stating which records belong to
           % an existing named condition 'Outliers'

    get_data(...)

       Returns the data table of Visplore as pandas data frame.
       Usage:
               get_data()
       Input parameters:
               SelectionName(optional, string)
                       Uses the given selection for the basis of the returned data.
               LimitToFocus(optional, boolean)
                       Determines whether only data records that are in focus should be returned.
               ReplacementStyle(optional, char)
                       Determines how unsupported characters are replaced in
                       Matlab. options: 'hex', 'underscore', 'delete'
                       Default: 'underscore'
       Return value:
               table
       Example:
           % v is a visplore handle received by v = visplore() with data:
           %
           new_data_table = v.get_data()
           %
           % Get data limited to current focus
           limited_data_table = v.get_data('LimitToFocus', false)

    get_filter(...)

       Returns a list of boolean values, stating for each record of the Visplore table
       if it passes the current Visplore filter (true) or not (false).
       Usage:
           get_filter()
       Return type:
           logical array
       Example:
           % v is a visplore handle received by v = visplore():
           %
           c = v.get_filter()
           % returns for example [true; true; true; false; true; ... true;]

    get_selected_columns(...)

       Returns the names of the currently selected columns in the current visplore table
       Usage:
           get_selected_columns()
       Input parameters:
           fullNames (optional, boolean, default: false): if true, the full Visplore attribute hierarchy path is used
       Example:
           % v is a visplore handle received by v = visplore():
           %
           c = v.get_selected_columns()
           % returns for example [{'A_Phase_Voltage_BrightCounty_PV'}, {'A_Phase_Voltage_Cloudington_PV'}, ...]

    get_selection(...)

       Returns a list of boolean values, stating for each record of the Visplore table
       if it is part of (1) the current Focus or (2) a named condition, if name is specified.
       Usage:
           get_selection()
       Input parameters:
           name (optional, string, default: None): name of a condition whose membership is returned.
                 If not specified, membership of the current Focus is returned. .
                 Is equivalent to calling get_condition(name).
       Return type:
           logical array
       Examples:
           % v is a visplore handle received by v = visplore():
           %
           s = v.get_selection()
           % returns for example [true; false; false; false], stating that only the
           %  first record is currently in Focus.
           %
           s = v.get_selection('Outliers')
           % returns for example [false; true; false; true], stating which records belong to
           % an existing named condition 'Outliers'

    get_view_data(...)

       Returns the contents of a particular Visplore view as data frame.
       Is not applicable for all views. Call get_view_properties() to see,
       which views of the current Cockpit support this.
       Supported examples : 'Statistics', 'Focus data records', 'Calendar', 'Histogram'
       Attention : at the moment, get_view_data is only possible for views that are visible
       in Visplore, not views hidden in tabs. Open them first.
       Usage:
           get_view_data(view_id)
       Input parameters:
           view_id(mandatory): the ID of the view whose contents are to be retrieved.
                               In most cases, this is the name displayed as view title in Visplore.
                               See get_view_properties() to see the View IDs, and if a view supports this.
       Return type:
           table
       Example:
           % v is a visplore handle received by v = visplore():
           stats = v.get_view_data('Statistics')
           % or
           stats = v.get_view_data('Focus data records')

    get_view_properties(...)

       Returns a table stating for each view
       - the view ID needed for API calls to it
       - if the view is exportable via API calls
       Usage:
           get_view_properties()
       Return type:
           table
       Example:
           % v is a visplore handle received by v = visplore():
           props = get_view_properties()
           %returns for example
           %  DisplayedName                           Identifier                      SupportsDataExport    SupportsImageExport
           % ________________________    _____________________________________________    __________________    ___________________
           %
           % {'Statistics'          }    {'Statistics'                               }          true                   true
           % {'Heatmaps'            }    {'Temporal Overview'                        }          true                   true
           % {'Histograms'          }    {'Histograms'                               }          true                   false
           % {'Horizon Graph View'  }    {'Horizon Graphs'                           }          false                  true

    visplore.load_demo_data(...)

    Loads an already prepared data set which can be used for Matlab testing.
    Only works if Visplore has been installed using the installer.
       Usage:
         visplore.load_demo_data(data)
       Input parameters:
           data(optional, string) - The identifier of a packaged data set.
                                       Possible sets are:
                                       openpv -         The full Open Photovoltaic
                                                        and Weather data set
                                       openpv_sampled - A sampled subset of
                                                        the openpv data set
       Examples:
           % Load the sampled data set
           data_sampled = visplore.load_demo_data();
           % Load the full data set
           data = visplore.load_demo_data('data', 'openpv');

    load_session(...)

       Loads a.visplore session with the given filename(full path)
       Usage:
           load_session(filename)
       Input parameters:
           filename(mandatory) : string specifying full name and path of visplore file
           mode(optional, string): string specifying the mode on how data is restored when loading a session
                                   support modes are: auto, askUser, fromEmbedded, fromLink, fromLinkWithDialog, no
       Example:
           % v is a visplore handle received by v = visplore():
           %
           v.load_session("C:/data/visplore_sessions/my_dashboard.visplore")
           v.load_session("C:/data/visplore_sessions/my_dashboard.visplore",'mode','askUser')

    save_image(...)

       Saves a view as an image to the given location and filename
       Usage:
           save_image(viewID, filename)
       Input parameters:
           viewID(string, mandatory) - the ID of the view to export
           fileName(string, mandatory) - the file name and location for the image
           parameters(struct, optional) - in this optional struct, you can customize
                                          the image export, using any of the
                                          following parameters:
                                          width, height (numbers): the total width/height of the image in pixels.
                                          colwidth, rowheight (numbers): alternatives to total width and height, for
                                                                         views that have rows or columns, for example:
                                                                         Statistics, Heatmaps, Bar Chart, Heatmap,...
                                                                         - colwidth defines the width per column in px,
                                                                         - rowheight defines the height per row in px.
                                          font (string): defines font type and size for all text in the image,
                                                         for example, "Arial 16", or "Tahoma 20"
                                          xaxislabel (boolean or string): set to False to hide the x axis label,
                                                                     or specify as string to show a custom x axis label.
                                          yaxislabel (boolean or string): set to False to hide the y axis label,
                                                                     or specify as string to show a custom y axis label.
                                          labelfont (string): format for the label's font type and size, see below.
                                          colorlegend (boolean or string): set to False to hide color legends,
                                                                           or specify as string to show a custom legend label
                                          visiblerangebars (boolean): set to False to hide scrollbars.
                                          missingdatabar (boolean): set to False to hide info-bars above some plots.
                                          context (boolean): set to False to show only data records that are in Focus.
                                          selectionhandles (boolean): set to False to hide orange selection tool handles.
                                          title (boolean or string): set to True to show the view name as title,
                                                                     or specify as string to show a custom title.
                                          titlefont (string): format for title's font type and size, see below.
                                          description (boolean or string): set to True to show the Focus description in image,
                                                                           or specify a string to show a custom description text.
                                          descriptionfont (string): format string for description's font type/size, see below.
                                          arealabel (boolean or string): Only available in some(aggregated) views.
                                                                         set to False to hide the area label
                                                                         or specify as string to show a custom area label.
                                          rightyaxislabel (boolean or string): Only available in some(aggregated) views.
                                                                               set to False to hide the right y axis label
                                                                               or specify as string to show a right y axis label.
                                          topxaxislabel (boolean or string): Only available in some(aggregated) views.
                                                                             set to False to hide the top x axis label
                                                                             or specify as string to show a top x axis label.
        Example:
            % v is a visplore handle received by v = visplore():
            % exports the Time Series view with a hidden x-axis and a
            % custom labeled y-axis
            v.save_image("Time Series","C:/Users/vs/Documents/Images/matlab.png", struct('xaxis', false,'yaxis','Values'))

    send_data(...)

       Sends data to visplore, either replacing the current visplore table, or extending it.
       Usage:
           send_data(data)
       Input parameters:
           data(mandatory) - as a table.
               supported column types are numerical, categorical, or timestamp attributes.
               An additional type supported by some Visplore versions are 1D curves,
               given as a column holding lists in the cells:
                  Each curve is a cell containing a double array like this:
                  Example: {[0,1,2; 100,-10,30]}
                           {[0,1,2; -20, -30, -40]} in line 2, etc.
           mode(string, optional, default: 'auto')
                  a string that defines how the data is combined with a
                  possibly existing table in visplore.
                  Options : ('replace', 'merge', 'append', 'auto')
           tableDescription(string, optional, default: None)
                  an optional table description that is shown in the title bar of Visplore.
                  If not specified, the data type or variable name is shown
       Example:
           % v is a visplore handle received by v = visplore():
           %
           v.send_data(df) % df is any table

    set_selected_columns(...)

       Selects the columns in Visplore that are specified by a list column names(strings).
       Usage:
           set_selected_columns(column_names)
       Input paramters:
           column_names: a list of strings specifying exact names of columns to select in Visplore.
                         In the same format, as retrieved by get_selected_columns()
       Example:
           % v is a visplore handle received by v = visplore():
           %
           li = [{'A_Phase_Voltage_BrightCounty_PV'}; {'A_Phase_Voltage_Cloudington_PV'}]
           v.set_selected_columns(li)

    set_selection(...)

       Sets the current Focus of data records to those records marked in the provided sequence of boolean values.
       Optionally also stores these records as a named condition.
       Usage:
           set_selection(sequence)
       Input parameters:
           sequence(boolean, mandatory) - the records to select(where true). Must be length of the Visplore table.
           description(optional, string) - a selection description, optional
           name(optional, string) is the name of a named condition to create and set from these records.
       Examples:
           % v is a visplore handle received by v = visplore():
           %
           v.set_selection([true; false; false])
           % sets the Focus to contain only the 1st record of a 3-record table.
           %
           v.set_selection([true; false; false], 'name', 'MyCondition')
           % does the same, and creates a named condition called MyCondition that holds these records.

    start_cockpit(...)

       Starts a cockpit with the given name, if such a cockpit exists.
       Usage:
           start_cockpit(cockpit).
       Input parameters:
           cockpit(string, mandatory) - is the name of the cockpit that should be started
           skipRolesDialog(bool, optional) - if True: signals to Visplore to attempt to automatically assign roles
                                             The role assignment dialog is shown if no automatic role assignment is possible
       Examples:
           % v is a visplore handle received by v = visplore():
           %
           v.start_cockpit('Trends and Distributions')

    visplore(...)

       Starts a new instance of visplore and returns a handle to it.
       The location of visplore.exe is assumed to be found in an environment variable,
       called VISPLORE_LOCATION, which is set by the Visplore installer, or by you.
       Alternatively, you can specify a different visplore location by a parameter.
       As a third option, the VisploreStarter application can be used (legacy support).
       Further optional parameters allow specify a cockpit to start, and passing data.
       Usage:
           visplore()
       Input parameters:
           local_visplore(optional, boolean) - whether a local Visplore installation is started.
                                           Is possible if Visplore was installed by installer, or
                                           when you set VISPLORE_LOCATION to path/visplore.exe manually.
                                           Default: true. If false, the VisploreStarter app is used.
           path(optional, char)              - if specified, and the local_visplore parameter is set to true, a local
                                           visplore.exe is started from this location instead of VISPLORE_LOCATION.
           cockpit(optional, char)           - specifies the name of a cockpit to start with visplore
           data(optional)                    - initial data sent to the new visplore instance
                                           (see send_data for a description of the expected format)
           connect(optional, boolean)        - Connects to an already running Visplore instance instead of starting a new one
                                           Uses TCP IP connection. By specifying the IP + port pair, you can either connect to
                                           a local Visplore, or to a Visplore running on another machine.
           ip(optional, char)              - ip address of the Visplore instance, e.g. '127.0.0.1'
           port(optional, int)               - the port that Visplore is listening to, e.g. 50123
                                           you find the port in the 'Connect to Matlab' option in Visplore's welcome dialog.
       Examples:
           % Start a new visplore instance from a local installation
           v = visplore()
           % Start a new visplore instance from an installer-less Visplore in a different directory
           v = visplore('local_visplore',true, 'path','C:/temp/Visplore/Visplore/visplore.exe')