Python Connector (VisplorePy v1.2.0)




Overview and Use-Cases



  Video available: The easiest way to get acquainted with the data connector is to watch the video tutorial in our Video Academy

VisplorePy is a Python 3 extension to integrate Visplore cockpits into existing Python workflows. It offers a bidirectional link between the Python workspace and the interactive visual exploration features of Visplore. This supports multiple use-cases:

Installation


Requirements:

Installation:

VisplorePy comes as a python wheel, shipped with the Visplore installation. There are two ways of installing it. Option 1 is straightforward if you use Jupyter, or the plain python console. If you use Spyder as environment, we recommend going with option 2.
  1. From a running Visplore instance: in the startup dialog of Visplore, choose the "Connect to Python" option, and select the "Install VisplorePy integration package" checkmark. When confirming the dialog, a code snippet is produced that, when executed in your Python environment, installs the Python wheel and immediately connects Python with that Visplore instance:


    Note for Spyder-users: it is not recommended to run pip commands like the snippet above directly in Spyder (see here). Instead, we recommend option 2 for you.

  2. Alternative: using pip to install it in your terminal, e.g. Anaconda prompt (but outside of graphical shells like Spyder).
    pip install --upgrade --force-reinstall "C:/Program Files/Visplore/VisplorePy/visplorepy-0.0.1-py3-none-any.whl"
    Note: You may need to adapt the path to match your Visplore installation and exact name of the wheel file. To find out where your Visplore is installed, click "Help" in a running Visplore, and "Where is my Visplore version installed?".


  3. Step by Step Guide


    Once installed, this section describes how to start VisplorePy, and sketches an example workflow to demonstrate its basic usage.

    In case you use Jupyter notebook, you can load the accompanying demo notebook “Pandas_OpenPV_Demo.ipynb” from the VisplorePy subfolder or your Visplore installation, which contains several examples ready to try out.

    Alternatively, you can can type the example commands in any shell/console, as described in the following.

    1) Connect Python and Visplore

    There are two ways to connect Python and Visplore:

    a) Start a new Visplore instance from Python and connect to it, by typing the following:

    import visplorepy

    visplore = visplorepy.start_visplore()

    This opens a new Visplore instance, and stores a handle to it in a variable "visplore". Any number of Visplore instances can be created and used at the same time. The distinct handles are used to communicate with any particular instance.
    Note: This way of starting Visplore is ideal when Visplore is installed via Installer locally on the same machine as the Python environment (see start_visplore() in the API documentation for details). In other cases, refer to option (b), described in the following.

    b) Connecting to an already running Visplore instance (locally, or on another machine):

    The alternative is to start a new Visplore instance by hand first (e.g. via Start menu or desktop icon). In the welcome dialog of Visplore, choose the option "Connect to Python". Confirming the dialog copies a code snippet to the clipboard, which you can paste and run in your Python environment. It looks like this:

    import visplorepy

    visplore = visplorepy.connect("127.0.0.1",50200)

    After pasting and running this snippet in Python, the variable "visplore" is a handle to the existing Visplore instance you connected to. The tools communicate via TCP network protocol, which is why IP and port are specified as parameters here (see API documentation for details). Please leave these parameters as they are produced by Visplore. The Visplore instance waits for incoming connections on that port, and the "connect()" call establishes the connection.

    2) Prepare a dataframe in Python


    A typical workflow starts with a data frame that is either already available in the Python workspace, or loaded, generated, or accessed at this point. In our example, we use a pandas data frame, loaded from an open photovoltaic and weather dataset shipped with Visplore, using this built-in command:

    data = visplorepy.load_demo_data('openpv_sampled')

    3) Send dataframe to Visplore


    We can now send this (or any other) data table to Visplore:
    visplore.send_data(data)

    To see if your data successfully arrived at the Visplore instance, you can take a look at the statusbar at the bottom of your Visplore instance, and see if it shows any received records:

     

    If something went wrong, it still shows:
     

    4) Start a cockpit to explore the data


    Once the data is available in Visplore, we can start a “Cockpit”, i.e., an entry point for a certain task or analysis question.
    This can be done in one of two ways: either by a double click on the respective cockpit icon in Visplore, for example “Trends and Distributions”:


    Alternatively, the cockpit can be started from Python via the following command:
    visplore.start_cockpit("Trends and Distributions")
    As the parameter, you can simply use the name of the cockpit as shown in the user interface.


    5) Interactively select data in Visplore, and make this information available in Python


    A key feature in Visplore is the selection of data records using interactive selection mechanisms (selection rectangles, Lasso brushes, etc.). This allows the user to select interesting subsets, like outliers, patterns, clusters, or training data for modeling interactively in Visplore. These subsets can be made available in Python for downstream processing.

    For example, you can drag an interval or rectangle with the left mouse in a “Time Series View” or “Histogram”, to select some data records, and then type the following command in Python:
    selection_mask = visplore.get_selection()
    This returns a binary mask that states for every row of the data table, whether it is currently selected (=True) or not (=False). This can be used for performing tasks with these records in Python.


    6) Label selected data in Visplore, and get labels in Python


    After selecting records in Visplore like in the previous step, the selected records can be labeled by clicking "Name" in the "Focus" bar (see Visplore documentation). This creates a named condition, that can be retrieved as a binary mask from Python using the following command. Say you selected outliers, and labeled them as a condition "Outliers", retrieve them with:
    outlier_mask = visplore.get_condition("Outliers")


    7) Clean data values in Visplore, and retrieve the cleaned data table as data frame


    Visplore allows editing data records interactively, such as imputing outliers or missing values by interpolating neighbour values (see Visplore documentation).
    In particular after peforming edits, but basically at any time, you can retrieve the current Visplore data table using:
    new_data_frame = visplore.get_data()
    This returns a new pandas data frame, holding the data as received from Visplore. Optionally, you can limit the received rows to the selected data records, see the API documentation.

    8) Get data from a Visplore visualization as data frame


    Many table-like visualizations support retrieving the displayed data as new pandas data frame. Examples are statistical tables, bar charts, heat maps, and raw value tables.
    my_statistics = visplore.get_view_data("Statistics")
    Note:The parameter is the displayed title of the visualization. Only views that are currently visible in the current cockpit can be retrieved. Not all view types can be retrieved, see visplore.get_view_properties() in the the API documentation.

    8) Compute new data columns in Python, and make them available in Visplore


    In this last example, a new data column is created in Python by computing a moving average from an existing data column. Select any data column in Visplore, then call the following snippet to compute the moving average for that selected column:
    # compute smoothed variant of selected column and send to Visplore
    first_selected_timeseries = visplore.get_selected_columns()[0]

    new_column = first_selected_timeseries + "_smoothed" data[new_column] = data[first_selected_timeseries].rolling(100, center=True, min_periods=1).mean()

    visplore.send_data(data[['DateTime', new_column]], "merge")
    Let's look the individual commands involved in this. First, visplore.get_selected_columns() returns the names of the currently selected variables in Visplore as a list. The next line concatenates the name of the first selected column with the word "smoothed", preparing the name for the new column. The next line computes the moving average using python functionality, by indexing into the original data frame using the name of the currently selected column. The result is appended as new column to the dataframe. Finally, the last line transfers a part of the extended data frame to Visplore. Specifically, it transfers the columns "DateTime" and the newly created column, while merging the records based on "DateTime" as a join index.

    Note:If you run the snippet again with a different rolling window parameter, the values update immediately, allowing for efficient visual tweaking of parameters.



    API Documentation


    Use help(visplorepy) in Python to produce this documentation:
    
        

    close(...)

            Closes the visplore application for this handle.
    
            Usage:
                close()
            Example:
                #v is a visplore handle received by v = visplorepy.start_visplore():
                #
                v.close()

    connect(...)

            Connects to an already running Visplore instance.
    
            Uses TCP IP connection. By specifying the IP + port pair, you can either connect to
            a local Visplore, or to a Visplore running on another machine.
    
            Usage:
                connect(ip, port)
            Input parameters:
                ip(mandatory, string) - ip address of the Visplore instance (e.g. "127.0.0.1")
                port(mandatory, int) - the port that Visplore is listening to (e.g. 50123)
                 - you find the port in the 'Connect to Python' option in Visplore's welcome dialog.
            Example:
                visplore = visplorepy.connect("127.0.0.1", 70010)

    connect_local(...)

            Connects to a local, already running Visplore instance.
    
            Usage:
                connect(port)
            Input parameters:
                port(optional, int) - the port that Visplore is listening to (e.g. 50123);
                                      If not given, tries to find the last started Visplore instance.
                 - you find the port in the 'Connect to Python' option in Visplore's welcome dialog.
            Example:
                visplore = visplorepy.connect_local()

    get_classification(...)

            Returns the classification(= categorical column) for the given name, if existing, as data frame.
    
            Usage:
                get_classifcation(name)
            Input parameters:
                name - the name of the classification as string
            Return type:
                Pandas data frame
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                # Assumes, that a classification column called 'Classification Nr. 1' has been created in Visplore.
                #
                c = v.get_classification('Classification Nr. 1')
                # returns pandas dataframe with the length of the Visplore table, and
                # the classification as categorical column.

    get_columns(...)

            Returns the names of the columns in the current visplore table (optionally limited to the selected ones)
    
            Usage:
                get_columns(selectedOnly=False, fullNames=False)
            Input parameters:
                selectedOnly: (optional, boolean, default: False) - if only the selected columns,
                                                                    or all columns are returned
                fullNames (optional, boolean, default: False): if True, the full Visplore attribute hierarchy path is used
            Return type:
                list
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                c = v.get_columns(selectedOnly=True)
                # returns for example ['A_Phase_Voltage_BrightCounty_PV', 'A_Phase_Voltage_Cloudington_PV', ...]

    get_condition(...)

            Returns a list of boolean values, stating for each record of the Visplore table
            if it is part of the named condition (True), or not (False).
    
            Usage:
                get_condition(name)
            Input parameters:
                name (mandatory, string): the name of a condition whose membership is returned
            Return type:
                list
            Examples:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                s = v.get_condition('Outliers')
                # returns for example [False, True, False, True], stating which records belong to
                # an existing named condition 'Outliers'

    get_data(...)

            Returns the data table of Visplore as pandas data frame.
    
            Usage:
                    get_data(selectionName = "", limitToFocus=False)
            Input parameters:
                    selectionName(optional, string)
                            Uses the given selection for the basis of the returned data.
                    limitToFocus(optional, boolean)
                            Determines whether only data records that are in focus should be returned.
            Return value:
                    a Pandas data frame
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore() with data:
                #
                new_data_table = v.get_data()

    get_filter(...)

            Returns a list of boolean values, stating for each record of the Visplore table
            if it passes the current Visplore filter (True) or not (False).
    
            Usage:
                get_filter()
            Return type:
                list
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                c = v.get_filter()
                # returns for example [True, True, True, False, True, ..., True]

    get_image(...)

            Retrieves the view image data and returns it to Python.
    
            Usage:
                    get_image(viewID, parameters)
            Input parameters:
                viewID(mandatory, string): the ID of the view to export as image.
                                           In most cases, this is the name displayed as
                                           view title in Visplore.
                                           See get_view_properties() to see the view IDs,
                                           and if a view supports this.
                parameters(optional, dictionary): in this optional dictionary, you can
                                                  customize the image export, using any
                                                  of the following parameters:
    
                - width, height (numbers): the total width/height of the image in pixels.
                - colwidth, rowheight (numbers): alternatives to total width and height, for
                                                 views that have rows or columns, for example:
                                                 Statistics, Heatmaps, Bar Chart, Heatmap,...
                                                 - colwidth defines the width per column in px,
                                                 - rowheight defines the height per row in px.
                - font (string): defines font type and size for all text in the image,
                                 for example, "Arial 16", or "Tahoma 20"
                - xaxislabel (boolean or string): set to False to hide the x axis label
                                             or specify as string to show a custom x axis label.
                - yaxislabel (boolean or string): set to False to hide the y axis label
                                             or specify as string to show a custom y axis label.
                - labelfont (string): format for the label's font type and size, see below.
                - colorlegend (boolean or string): set to False to hide color legends
                                                   or specify as string to show a custom legend label
                - visiblerangebars (boolean): set to False to hide scrollbars.
                - missingdatabar (boolean): set to False to hide info-bars above some plots.
                - context (boolean): set to False to show only data records that are in Focus.
                - selectionhandles (boolean): set to False to hide orange selection tool handles.
                - title (boolean or string): set to True to show the view name as title,
                                             or specify as string to show a custom title.
                - titlefont (string): format for title's font type and size, see below.
                - description (boolean or string): set to True to show the Focus description in image,
                                                   or specify a string to show a custom description text.
                - descriptionfont (string): format string for description's font type/size, see below.
                - arealabel (boolean or string): Only available in some(aggregated) views.
                                             set to False to hide the area label
                                             or specify as string to show a custom area label.
                - rightyaxislabel (boolean or string): Only available in some(aggregated) views.
                                             set to False to hide the right y axis label
                                             or specify as string to show a right y axis label.
                - topxaxislabel (boolean or string): Only available in some(aggregated) views.
                                             set to False to hide the top x axis label
                                             or specify as string to show a top x axis label.
            Example:
                # v is a visplore handle to a Visplore instance.:
    
                # Export a view in default size and settings:
                     v.get_image("Time Series")
    
                # Export in specific size:
                     v.get_image("Time Series", {"width": 1000, "height": 400})
    
                # Export with specific font:
                     v.get_image("Time Series", {"font": "Arial 16", "title": "My Plot"})

    get_selected_columns(...)

            Returns the names of the currently selected columns in the current visplore table
    
            Usage:
                get_selected_columns()
            Input parameters:
                fullNames (optional, boolean, default: False): if True, the full Visplore attribute hierarchy path is used
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                c = v.get_selected_columns()
                # returns for example ['A_Phase_Voltage_BrightCounty_PV', 'A_Phase_Voltage_Cloudington_PV', ...]

    get_selection(...)

            Returns a list of boolean values, stating for each record of the Visplore table
            if it is part of (1) the current Focus or (2) a named condition, if name is specified.
    
            Usage:
                get_selection(name = None)
            Input parameters:
                name (optional, string, default: None): name of a condition whose membership is returned.
                      If not specified, membership of the current Focus is returned. .
                      Is equivalent to calling get_condition(name).
            Return type:
                list
            Examples:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                s = v.get_selection()
                # returns for example [True, False, False, False], stating that only the
                #  first record is currently in Focus.
                #
                s = v.get_selection('Outliers')
                # returns for example [False, True, False, True], stating which records belong to
                # an existing named condition 'Outliers'

    get_view_data(...)

            Returns the contents of a particular Visplore view as data frame.
    
            Is not applicable for all views. Call get_view_properties() to see,
             which views of the current Cockpit support this.
            Supported examples : 'Statistics', 'Focus data records', 'Calendar', 'Histogram'
            Attention : at the moment, get_view_data is only possible for views that are visible
            in Visplore, not views hidden in tabs. Open them first.
    
            Usage:
                get_view_data(view_id)
            Input parameters:
                view_id(mandatory): the ID of the view whose contents are to be retrieved.
                                    In most cases, this is the name displayed as view title in Visplore.
                                    See get_view_properties() to see the View IDs, and if a view supports this.
            Return type:
                pandas data frame
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                stats = v.get_view_data('Statistics')
                # or
                stats = v.get_view_data('Focus data records')

    get_view_properties(...)

            Returns a table stating for each view
            - the view ID needed for API calls to it
            - if the view is exportable via API calls
    
            Usage:
                get_view_properties()
            Return type:
                pandas data frame
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                props = get_view_properties()
                #returns for example
                #     Displayed Name    Identifier         Supports Data Export  Supports Image Export
                # 0   Statistics        Statistics         True                  True
                # 1   Heatmaps          Heatmaps           True                  True
                # 2   Histograms        Histograms         True                  False

    load_demo_data(...)

            Loads demo data and returns it as a pandas.DataFrame.
    
            Usage:
                load_demo_data(dataset)
            Input parameters:
                dataset(optional, string) - the identifier of the demo set.
                                            Currently packaged data sets are:
                                              'openpv_sampled' - A sampled version of the
                                                                 Open Photovoltaic and Weather data set. (DEFAULT)
                                              'openpv' - The full Open Photovoltaic and Weather data set.
            Example:
                #loads the openpv_sampled data
                pvData = visplorepy.load_demo_data()

    load_session(...)

            Loads a.visplore session with the given filename(full path)
    
            Usage:
                load_session(filename)
            Input parameters:
                filename(mandatory): string specifying full name and path of.visplore file
                restoreData(optional, string): string specifying the mode on how data is restored when loading a session.
                                               supported modes are:
                                                   - "auto":
                                                   - "askUser":
                                                   - "fromEmbedded":
                                                   - "fromLink":
                                                   - "fromLinkWithDialog":
                                                   - "no":
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                v.load_session('C:/data/visplore_sessions/my_dashboard.visplore')

    save_image(...)

            Exports a view as image, with optional parameters, and saves it as PNG file to a location.
    
            Usage:
                save_image(viewID, filepath, {parameters})
            Input parameters:
                viewID(mandatory, string): the ID of the view to export as image.
                                           In most cases, this is the name displayed as
                                           view title in Visplore.
                                           See get_view_properties() to see the view IDs,
                                           and if a view supports this.
                filename(mandatory, string): full path of image to create, including the
                                             file extension
                parameters(optional, dictionary): in this optional dictionary, you can
                                                  customize the image export, using any
                                                  of the following parameters:
    
                - width, height (numbers): the total width/height of the image in pixels.
                - colwidth, rowheight (numbers): alternatives to total width and height, for
                                                 views that have rows or columns, for example:
                                                 Statistics, Heatmaps, Bar Chart, Heatmap,...
                                                 - colwidth defines the width per column in px,
                                                 - rowheight defines the height per row in px.
                - font (string): defines font type and size for all text in the image,
                                 for example, "Arial 16", or "Tahoma 20"
                - xaxislabel (boolean or string): set to False to hide the x axis label
                                             or specify as string to show a custom x axis label.
                - yaxislabel (boolean or string): set to False to hide the y axis label
                                             or specify as string to show a custom y axis label.
                - labelfont (string): format for the label's font type and size, see below.
                - colorlegend (boolean or string): set to False to hide color legends
                                                   or specify as string to show a custom legend label
                - visiblerangebars (boolean): set to False to hide scrollbars.
                - missingdatabar (boolean): set to False to hide info-bars above some plots.
                - context (boolean): set to False to show only data records that are in Focus.
                - selectionhandles (boolean): set to False to hide orange selection tool handles.
                - title (boolean or string): set to True to show the view name as title,
                                             or specify as string to show a custom title.
                - titlefont (string): format for title's font type and size, see below.
                - description (boolean or string): set to True to show the Focus description in image,
                                                   or specify a string to show a custom description text.
                - descriptionfont (string): format string for description's font type/size, see below.
                - arealabel (boolean or string): Only available in some(aggregated) views.
                                             set to False to hide the area label
                                             or specify as string to show a custom area label.
                - rightyaxislabel (boolean or string): Only available in some(aggregated) views.
                                             set to False to hide the right y axis label
                                             or specify as string to show a right y axis label.
                - topxaxislabel (boolean or string): Only available in some(aggregated) views.
                                             set to False to hide the top x axis label
                                             or specify as string to show a top x axis label.
            Examples:
                # v is a visplore handle to a Visplore instance.:
    
                # Export a view in default size and settings:
                     v.save_image("Time Series", "C:/temp/image.png")
    
                # Export in specific size:
                     v.save_image("Time Series", "C:/temp/image.png", {"width": 1000, "height": 400})
    
                # Export with specific font:
                     v.save_image("Time Series", "C:/temp/image.png", {"font": "Arial 16", "title": "My Plot"})

    send_data(...)

            Sends data to visplore, either replacing the current visplore table, or extending it.
    
            Usage:
                send_data(data, mode)
            Input parameters:
                data(mandatory) - in PANDAS dataframe format.
                    supported column types are numerical, categorical, or timestamp attributes.
                    An additional type supported by some Visplore versions are 1D curves,
                    given as a column holding lists in the cells:
                       Each curve is a list of two lists, (1) the axis values and (2) curve values
                       Example: [[0,1,2],[100,-10,30]] in line 1,
                                [[0,1,2],[-20, -30, -40]] in line 2, etc.
                mode(string, optional, default: 'auto')
                       a string that defines how the data is combined with a
                       possibly existing table in visplore.
                            Options : ('replace', 'merge', 'append', 'auto')
                tableDescription(string, optional, default: None)
                       an optional table description that is shown in the title bar of Visplore.
                       If not specified, the data type or variable name is shown
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                v.send_data(df) # df is any dataframe

    set_selected_columns(...)

            Selects the columns in Visplore that are specified by a list column names(strings).
    
            Usage:
                set_selected_columns(column_names)
            Input paramters:
                column_names: a list of strings specifying exact names of columns to select in Visplore.
                              In the same format, as retrieved by get_selected_columns()
            Example:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                li = ['A_Phase_Voltage_BrightCounty_PV', 'A_Phase_Voltage_Cloudington_PV']
                v.set_selected_columns(li)

    set_selection(...)

            Sets the current Focus of data records to those records
            marked in the provided sequence of boolean values.
            Optionally also stores these records as a named condition.
    
            Usage:
                set_selection(sequence, description = '', name = '')
            Input parameters:
                sequence(boolean, mandatory) - the records to select(where True).
                                               Must be length of the Visplore table.
                description(optional, string) - a selection description, optional
                name(optional, string) is the name of a named condition to create and set from these records.
            Examples:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                v.set_selection([True, False, False])
                # sets the Focus to contain only the 1st record of a 3-record table.
                #
                v.set_selection([True, False, False], name = 'MyCondition')
                # does the same, and creates a named condition called MyCondition that holds these records.

    start_cockpit(...)

            Starts a cockpit with the given name, if such a cockpit exists.
    
            Usage:
                start_cockpit(cockpit).
            Input parameters:
                cockpit(string, mandatory) - is the name of the cockpit that should be started
            Examples:
                # v is a visplore handle received by v = visplorepy.start_visplore():
                #
                v.start_cockpit('Trends and Distributions')

    start_visplore(...)

            Starts a new instance of visplore and returns a handle to it.
    
            The location of visplore.exe is assumed to be found in an environment variable,
            called VISPLORE_LOCATION, which is set by the Visplore installer, or by you.
            Alternatively, you can specify a different visplore location by a parameter.
            As a third option, the VisploreStarter application can be used (legacy support).
            Further optional parameters allow specify a cockpit to start, and passing data.
    
            Usage:
                start_visplore(local_visplore=True, path=None, cockpit=None, data=None)
            Input parameters:
                local_visplore(optional, boolean) - whether a local Visplore installation is started.
                                                    Is possible if Visplore was installed by installer, or
                                                    when you set VISPLORE_LOCATION to path/visplore.exe manually.
                                                    Default: True. If False, the VisploreStarter app is used.
                path(optional, string) - if specified, and the local_visplore parameter is set to True, a local
                                         visplore.exe is started from this location instead of VISPLORE_LOCATION.
                cockpit(optional, string) - specifies the name of a cockpit to start with visplore
                data(optional) - initial data sent to the new visplore instance
                                 (see send_data for a description of the expected format)
            Examples:
                # Start a new visplore instance from a local installation
                # v = visplorepy.start_visplore()
    
                # Start a new visplore instance from an installer-less Visplore in a different directory
                # v = visplorepy.start_visplore(local_visplore=True, path="C:/temp/Visplore/Visplore/visplore.exe")